[Worklog] The Beast Slayer

[hiigaran]

Parts have arrived! I only ended up paying 1% duty on the parts, and I didn't have to pay the $30 per day holding fee that they originally told me I'd need to pay. Some good news there.

Now things are going to be full tilt around here in my spare time. I quickly hooked up my personal rig to the monitor to test basic functionality. Since there is no OS installed on it, I could only bring up the BIOS, but it's a start. While I obviously plan on installing an OS, it is actually low priority. Right now, my focus will be on the other two systems.

At present, I don't have the time to test any of the cards. I'm still on minimum rest between flights, and I've already cut into a significant portion of my sleep after skipping my last sleep, since I had to be awake long enough for the customs guy to clock in to work. Took an hour to get things sorted, but I brought the box home with all the parts. After the quick test of the personal rig, I continued on the casing for the other two systems.

I'll post images later, but basically, I cut the threaded rods to length, secured nuts below and above the PSU mounting plate, then secured the front rods with nuts in a similar fashion, and the rear rods by inserting the screws in their centers.

Of course, I just had to drill the plate in the wrong way around. I think I can work with it though. Probably shouldn't be doing this while tired. Also, the screws need to be countersunk, so that there is more of a gap between them and the boards.

[BestPony]

Celestia be praised! Let's take our spot back above Reddit PCMR 

1% is a really good deal, almost makes up for how long they took to process everything. Are you going to have any time over the next two weeks to build any of this or are you on flights pretty much the whole time? Hearthswarming is a busy time after all.

[hiigaran]

After this next flight, my 7 day block of constant flights will come to an end, and I will have two days off, a London, then I think another two days off, then a 6 day block to Sydney and Christchurch. I'm hoping that the first two days will be sufficient to get things operational. Thankfully, my next flight is just a quick turnaround to Pakistan. Fly out, and be back home 9 or 10 hours later.

Though I just woke up now and got a call from crew scheduling, telling me that the flight has been rescheduled for 4 hours later. Now that I'm awake, I think I'll try and work on the casing a little more, wherever possible, without waking anyone up.

EDIT:



The video cards and my personal system finally clear customs, after 13 days. Due to the delay, I didn't have to pay for the storage fee, which would have been about $30 a day, and the duty was only 1%. So I paid the duty, and carried the box home.

I guess I got a little too excited, because I only remembered to take this photo after I had pulled the first few cards out. Yes, those are paper towels that were used as packing material. Bit ridiculous that this was what I was provided with as packing material, but after a thorough inspection of the PCBs on the back of each card, things look intact.

At this point, I did not test any cards, due to time constraints, but my personal system was working, so there's that.



You can see how everything is starting to take shape now. The threaded rods were cut to length, and nuts secure them in place on the PSU plate. Additional nuts were added to the front two rods, and the motherboard plate was lowered in place. The rods in the back have the screw holes in them, so while the plate was lowered, these screws were inserted through the plate, with a nut on each, and then screwed into the rods.

While the PSU plate is immediately aligned due to the nuts being of uniform height and with no rods protruding, the motherboard plate needs to be leveled. This will be done by adjusting the appropriate nuts. Using a level, I can ensure that everything will be perfectly straight.



PSUs fit perfectly. At the moment, there is just enough height clearance for the hardware to slide in easily. Later when the hardware is mounted for the finished setup, the nuts will be tightened to the point where a small compressive force will hold the PSUs in place.

You can see the right rod is also misaligned. Not sure if I didn't measure correctly, drilled an angled hole, or a combination of both, but thankfully that won't be visible, as I don't plan on leaving the rods visible.

[hiigaran]

Been working non-stop to finish the mounting plates. Forgot to eat twice, but the plates are finally finished, and I must say that the simplicity of it looks great. I'll post photos a little later.

Of course, all that had to be ruined by the cables. There are so many of the damn things!

Anyway, at the point of this post, I have verified that the hardware in the first system works. BS1 is currently in the final stages of installing the OS. BS2 will hopefully follow in half an hour or so.

[BestPony]

What you're doing is awesome but please don't forget to take care of yourself first! You need to eat and sleep, and don't forget to stay hydrated.

[hiigaran]

I used the time during installation of programs to take care of most of that!

I'm an hour late, but currently installing the OS for BS2.

The fan noise from BS1 at 100% is LOUD. Necessary though. Unlike the single 1080 in my personal rig, which has plenty of unrestricted airflow, the first three cards obviously have very little space to draw air in from. The fourth one sitting right at the end is chugging along happily though!

BS2 is currently with only three video cards. Don't worry, the fourth one is fine! It's just that the positioning of the power button built on the mobo is covered by the fourth GPU, and while it was barely possible to get it pressed with a knife on BS1 (I know, I know...metal near mobo = bad time), it just can't be done on BS2, because BS1 blocks the only access I could have with 4 GPUs in. I'm going to look into Wake On LAN to get around the issue.

Anyway, during the time it took to write this post, the OS on BS2 has installed.

Onwards! FOR SCIENCE!

[hiigaran]

Updates!



To hide the unsightly threaded rods and nuts, I cut lengths of shower curtain rods (found some thrown away, so I took them). They were just the right diameter to fit around the nuts. Plus they are shiny!



Curtain rods in place over the threaded rods. At this point, I began to do some leveling. With the mounting places secure, I tightened the top nuts and screws, until everything was reasonably level longitudinally and laterally.



I changed my mind about securing the PSUs with compressive force. Instead, I used strips of rubber bands to prevent motion by friction. Also started peeling off the protective acrylic coverings. More satisfying than peeling the protective plastic off electronics because you can put a little force into it!



PSUs in place, and acrylic goes all naked and lewd on us!




Ladies and gentlemen, I give you the Clusterfuck! Without cable management, the simplicity and elegance that the acrylic and hardware had just moments prior simply vanished. I'll need to fix that at some point. Still plenty of work to do, but everything is running at this point.

You'll notice that the second system is running only three cards. That's because while I was able to reach the power button on the motherboard itself on the first system, the second system is blocked by the first. I can enable Wake On LAN and turn it on remotely, but I need to set it up first.

At this point, the first system was already operational, outputting just short of 3 million PPD on Folding@Home. Fans are LOUD at 100%, but necessary, as the hottest card runs at 82 degrees. The one sitting right on the end with unrestricted airflow is chugging away nicely at 58 degrees though.

[RoBorg]

Amazing stuff Hiig - so great to see you get something up and running 

Hope the cooling is OK - mine has two right next to each other, and the one with the worse airflow died after a few months :(
My PC has just been a bit of a nightmare though - with any luck you won't have to go through that.

[BestPony]

Great work dude!

I do share RoBorg's concern about the temperature, 82 degrees is too hot for my liking, I hope your eventual cooling solution is effective enough to get that temperature down. Maybe you can position this under the air conditioning outlet for the time being?

Also, the proximity of the GPU nearest to the CPU heatsink scares me, what if it wobbles and the solder points touch the metal? Will there eventually be some kind of PCI bracket arrangement to keep the GPUs in place?

[PoNyePiC]

I think all GPUs should have backplates.  Even a cheap plastic one that wouldn't cost them anything to add during manufacturing    I usually add a little electrical tape to potential contact points during clusterfuck builds.  

Luckily for me, I don't need to worry myself with temps.  My house is frigid cold now.  I just cleaned the ice out of the shower so I could use it, the floor has a sheet of ice from the dripping water.  Dubai looks like a hot summer day by the looks of it, 19°C there and -18°C here today.  Gotta roll my car down a hill to start it  

[hiigaran]

Yeah, I'm not a fan of 80+ temperatures myself, so it would be best if I started planning the cooling. As a temporary measure, I could try rearranging the cards to see which one has the weakest cooling, and place that at the end where the airflow is least restricted. The cards throttle themselves around the 82 degree point as well.

The angle of the photos makes it look like the GPU is closer to the heatsink than it actually is. There is about two or three cm space between them. The PCI-e slot will break before the card could tilt far enough to touch the heatsink. That said, a support is the next thing planned. It would consist of a simple thin strip of acrylic that would span the left edge of the left motherboard, to the right edge of the right motherboard. I'll position this acrylic bar right where the video card brackets would normally be secured to a standard case, then mark the holes where the screws would go. From there, it's just a bunch of drilling, tapping, and screwing.

This alone would keep all the GPUs upright, as they would support each other. However, I will actually secure the support bar to the rest of the acrylic.

[BestPony]

That's a good amount of distance between the GPU and the CPU heatsink, it was hard to make out in the photo lol

Is anyone living/sleeping anywhere near these GPUs? If not then maybe use a tool like EVGA Precision XOC to adjust the fan speeds all the way up. It'll be even louder but you'll squeeze out a few more PPD until you get the final cooling setup sorted.

[hiigaran]

I use that exact program, and the fans are already set at 100%. And it is pretty loud. Luckily you can't hear it from the bedrooms.

[hiigaran]

I've had some thoughts about what I'm going to do for cable management, but I'll need a bit of input for opinions, or perhaps if anyone has better ideas.

I currently have about 15 meters of cable trunking. I'm thinking I could use this to hide the majority of the cables, and make the upper section much neater. I tried looking into 90 degree connectors for PCI-e and ATX power cables, but the products I saw were either meant to be soldered on to a PCB, or they come with cables. I'd rather not connect cables to cables.

I don't have the trunking dimensions with me, but if I recall, the cross section is about 6cm by 3cm. I'd guess that's enough room to fit 6 PCI-e cables in on a straight run.

So what I was thinking of doing with the trunking is to first have a length running across the top of every single card across both motherboards. The cables run along inside, then I cut out holes right above where each video card's PCI-e power port would be. The hole would obviously be slightly larger than needed, since the cables need to curve.

The direction the cables take from GPU to PSU would be inwards. That is, the left side cables head right, and the right side cables head left, where they meet in the middle, then head down towards the PSU.

That should solve a significant amount of the cable management issues. I'd then need to figure out what I'm going to do with the 24 pin and 8 pin ATX cables, the supplementary molex connectors for the PCI-e devices, and the SATA cables. Haven't thought that far ahead though.

[PoNyePiC]

I've had some thoughts about what I'm going to do for cable management, but I'll need a bit of input for opinions, or perhaps if anyone has better ideas.

I currently have about 15 meters of cable trunking. I'm thinking I could use this to hide the majority of the cables, and make the upper section much neater. I tried looking into 90 degree connectors for PCI-e and ATX power cables, but the products I saw were either meant to be soldered on to a PCB, or they come with cables. I'd rather not connect cables to cables.

I don't have the trunking dimensions with me, but if I recall, the cross section is about 6cm by 3cm. I'd guess that's enough room to fit 6 PCI-e cables in on a straight run.

So what I was thinking of doing with the trunking is to first have a length running across the top of every single card across both motherboards. The cables run along inside, then I cut out holes right above where each video card's PCI-e power port would be. The hole would obviously be slightly larger than needed, since the cables need to curve.

The direction the cables take from GPU to PSU would be inwards. That is, the left side cables head right, and the right side cables head left, where they meet in the middle, then head down towards the PSU.

That should solve a significant amount of the cable management issues. I'd then need to figure out what I'm going to do with the 24 pin and 8 pin ATX cables, the supplementary molex connectors for the PCI-e devices, and the SATA cables. Haven't thought that far ahead though.

For my build, I'm planning on using some 5cm PVC pipe wrapped in carbon fiber vinyl for my cable management.  I will glue all the pieces together then dremel a slit along the back to insert cables.  Hard to explain, but that's the jist.