My big mining rig project
-
It’s pulling 320 Watts with one card right now at intensity 20. Still fiddling with the settings. So far seems like intensity 20 is the way to go hands down. Ordering more power supplies now.
About the ducting, yes, that’s a general pet peeve whenever I see rigs. People just point normal room fans at them. That mitigates the problem lightly but not very effectively. Room fans are made for a broad effect, to move large masses of air, so it moves, which has a cooling effect all in itself. They are very bad at cooling a very specific point. Just run one and place your hand in front of it then feel where the stream drops off to the sides and then move a step back and do it again.
I’ll try to explain what I mean with a single GPU. Say you have a fan with a radius of 30 centimeters. Now in mid air in front of the fans direction hovers our GPU. Sure, somehow it is kept there, but let’s just assume it magically floats there for the sake of keeping it simple and to the important aspects. The tail is facing the fan, the front with the mounting bracket away from it. Most GPUs are designed to be cooled in that fashion, old ones even had a fan at the tail end that then blew over the entire card length. If the space around all of this is just empty, say nothing around it for a meter or so, then this will cool the card more than no fan, but not very efficiently. If the card is too close, the center of the fan creates a dead zone and it will even be less efficient than a bit away from the fan. Go too far and the effect is again almost nothing. You have to find the butterzone. Or you use a duct. Introduce a pipe, 30 cms diameter so the fan sits in it and the GPU behind it, still inside the pipe. Now the power of the fan is more evenly spread, even if the distance is a bit bigger. The moving air can’t diffuse and escape to the sides, there is pipe there as well. The increase in power (still the same fan, nothing changed there) is noticeable even if you had it in the butterzone before, in the open space. Now we take a huge vice and squeeze the pipe after the fan towards the card. The diameter is now half as much, the card just barely fits in there. The pressure form the fan is now bundled and cools the card much more efficiently. All that air around the card didn’t need to be moved around, it was wasted effort from the fan. Now all the fans power is directed towards cooling that card.
My example is a bit high level, but you can do simple things to reach similar goals. The main setup I see is a bunch of cards lined up next to one another. Now simply add some cardboard as a test (I would use something more solid, wood or metal) Just above and just below the are where the gpus are mounted to whatever frame you use. It will look like a V shape, with the opening away from the cards, getting narrower towards them. Make it large enough so your fan blows at them directly and isn’t able to blow over or below them. You could also tape the top and the bottom of the cardboard to the fan itself, which would increase the effect a bit. You could add sides to that ducting and through that have more air pressed into the area that counts.
If you fee very adventurous you could put a sheet below the cards so the air does get lined up more directly towards the mounting bracket. Above you could put one too, but keep two things in mind: Heat rises, so keep a bit of spacing and for a fire you only need air, fuel and heat, no spark. That cardboard is really bad for anything that gets too close to the GPUs.
Should you try it, please post some before and after numbers of the heat level :)
-
I understand completely and you are correct.
In my case though the issue comes from a space/fabrication/time/cost perspective. If i were to duct the 12" fan into a cone that is 6" there needs to be sufficient pressure to force the air through at a the higher velocity else you’ll just get a smaller radius of air. With high enough pressure of course you’ll double/triple your velocity and it would work a treat.
You’ll notice that the fans are placed 18" to 2ft away from the rigs as that seems to be the “butterzone” and avoids the dead spots in the center of the fan. Also 1 fan can cool 2 full rigs so noise levels are lower as are power costs. 8)
Just out of frame on that pic, i have a couple of 6" extractors which suck the hot air up blow it out of the shed (its all in my Log Cabin). They were originally rigged to duct cold air from outside to directly “through” the cards pretty much as you describe but even then the velocity wasn’t high enough to shift the hot air from one card away from the intakes on the next.
The other issue of course is the stoopid coolers on the windforce cards that just blow heat everywhere so it’s really difficult to move it away effectively and efficiently.
-
Yes, the on card fans are usually pretty dumb. I intend to build a metal duct that will mitigate the trouble there. Also will act as a heat sink in a fashion :)
Room fans are horrible for cooling. You would be better off getting better cooling fans that can get higher pressure, which would be in the same wattage area, though made to run 24/7. But even then, those fans are enough strong (usually, depends on what you have there) for small ducting. A 3:1 ratio is usually doable and you also have a “wide” spot to cool, not just a single spot. With that you will probably end up in a 2:1 ratio tops (fan area to back area of one rig).
-
Well, the room fans are currently doing a sterling job of keeping the cards at between 60 & 75 degrees so i’m happy with that.
look forward to seeing pics of your creation. :D
-
I keep my 3 x 6950’s outside in a metal shed, I have the warmest shed in Britain :D
It is often 20oC above ambient temps in there, but have not seen my cards break 70oC yet, the longterm heatsoak to mobo is astonishing though so I have a temp controlled extractor fan setup @ 30oC to the outside world, I use 2x PSU’s and am planning to tri-psu the rig and purchase 2 more 6950’s to help with the current hashrate. -
[quote name=“UKMark” post=“2489” timestamp=“1368275179”]
I keep my 3 x 6950’s outside in a metal shed, I have the warmest shed in Britain :D
[/quote]LOL - my brother is adamant that i’ll get raided by the cops coz the chopper will think i’m growing weed in there!
-
[quote name=“Nutnut” post=“2490” timestamp=“1368276831”]
[quote author=UKMark link=topic=387.msg2489#msg2489 date=1368275179]
I keep my 3 x 6950’s outside in a metal shed, I have the warmest shed in Britain :D
[/quote]LOL - my brother is adamant that i’ll get raided by the cops coz the chopper will think i’m growing weed in there!
[/quote]Let them raid me, I would love to see their faces when I have finished with my extrovertly geeky explanation :)
I am arresting you for (errrrrrr). You do not have to say anything, but it may harm your defence if you do not mention when questioned something which you later rely on in court. Anything you say may be given in evidence…
[i]…Satoshi made me do it. [/i] :D -
[quote name=“UKMark” post=“2496” timestamp=“1368278498”]
[quote author=Nutnut link=topic=387.msg2490#msg2490 date=1368276831]
[quote author=UKMark link=topic=387.msg2489#msg2489 date=1368275179]
I keep my 3 x 6950’s outside in a metal shed, I have the warmest shed in Britain :D
[/quote]LOL - my brother is adamant that i’ll get raided by the cops coz the chopper will think i’m growing weed in there!
[/quote]Let them raid me, I would love to see their faces when I have finished with my extrovertly geeky explanation :)
I am arresting you for (errrrrrr). You do not have to say anything, but it may harm your defence if you do not mention when questioned something which you later rely on in court. Anything you say may be given in evidence…
[i]…Satoshi made me do it. [/i] :D
[/quote]Haha, I think if you explained it into that sort of length they’d probably take you in on the grounds of ‘computer hacker’. Gotta love the UK police. =w=;
I’ll be running my 7970s in the outhouse (probably Tuesday onwards as my case/mobo haven’t been delivered today D: ) and when I ran just 1 6950 down there for BTC mining it became rather, cosy-warm. :P -
[quote name=“Nutnut” post=“2490” timestamp=“1368276831”]
LOL - my brother is adamant that i’ll get raided by the cops coz the chopper will think i’m growing weed in there!
[/quote]Was only thinking about this yesterday ;D
Some great ideas in these posts that I can use. My thanks to all.
-
Note to self: Use your own damn start scripts, you made them for a reason!
I was plagued with tons of HW errors. Turns out I simply didn’t run these lines after my reboots:
export DISPLAY=:0
export GPU_USE_SYNC_OBJECTS=1
export GPU_MAX_ALLOC_PERCENT=100*double facepalm*
-
So, some 3D Models here for your viewing pleasure.
[img]http://imageshack.us/a/img716/9792/miningrig01.jpg[/img]
A first general view. The blocks on the right are the power supplies, the one on the left the GPUS. The motherboard is behind the GPUs and those strange things at the bottom are two 230mm fans. There are these two metal sheets that angle towards the GPUs, that’s some ducting for airflow.
[img]http://imageshack.us/a/img560/5007/miningrig02.jpg[/img]
From the other side. The PSU on the left sucks in air through the bottom (large hole in the upper one) and pushes it out towards the back of the PSU (the lower one is in a standard position, the upper blows the hot air upwards). Also missing here is the back plate that holds all of this together. The mainboard is floating, as is the upper PSU. There is a plate that will cover the space from where the gpus are towards the back. I just removed it for displaying purposes.
[img]http://imageshack.us/a/img9/2857/miningrig03.jpg[/img]
Last but not least a close up of the ducting. The air gets pushed towards the cards. On each side (now open) there is another piece that stops air from going sideways, again removed to see everything easier. This setup also shields the mainboard from some extra heat. I was thinking about adding a side fan that just blows over the mainboard for a bit extra cooling of that area and to make sure not too much heat form the card gets to the mainboard.
Overall the rig is now 42 cm tall, 70 wide and 28 cm deep.
Not perfectly happy with it though. The cards are very close imho and that can lead to troubles. The cables will reach, they are at the most 16 cm from connector to plug, but it’s really stretching the outer limits of 19 cm that the cables can do. I still don’t have them in my hands, so that makes it a bit harder to fully plan this out.
I will store this version and start drawing up an alternative. Maybe I can come up with something even better.
-
Would placing a PSU on each side help?
Also to space the cards out more, you could use 2 pci riser cables for per card for cards furtherest away from the motherboard. -
Yes, placing one on each side was one of the modifications I had in mind. The cables seem to be rather short otherwise. Stacking risers is something I try to avoid, though I might not really need to, since we are talking about PCIE x1 here and the data that goes through there is rather low.
On another note: Powered risers seem to be a hand made thing, nobody sells them outright except cablesaurus and they don’t seem to do that anymore either. Looking at their picture, you can see very clear that it’s soldered on as an afterthought with only giving it some extra 12V juice. I have to say that sounds a bit odd to me, since GPUs draw the majority of their 12V power already from the external connectors. I maybe will run it on my other motherboard that has some nice tabs to read the voltage on the pcie bus easy. But that is for another time, once I have risers and maybe a powered one too. Either way, if you can’t buy one, it seems rather trivial to make one.
-
The journey continues.
To make everything a bit more stable I put the whole thing int an ATX case for now and added a second GPU. With two 7950’s running at 560 Kh/s each the system draws 580 - 590 Watts. Adding a third card would probably still be possible though pushing the PSU. Card 4 could mean disaster.
So I ordered some risers and cables to make powered risers out of them. I already have simple 1x risers on the way, which I could abuse (and destroy) for this task, since the x1 to x16 risers are just unconnected at the back part anyways.
A curious discovery on heat has happened though. Directly connected to the board, there is one free slot between these dual slot cards. Now the card whose fans are sucking in air from the other cards back seem to emit heat out back to the card below as well. Something I noticed in the single card setup, where there is heat leaking out around the fan, not just down and up. My design idea is now to create a heat barrier between cards.
-
Since I have the cards now in my hands and a better understanding and feel for them, I decided to redo the model for them in 3D. I also decided to leave off the bracket as it just hinders airflow with my new plans. Here a peek at how the model looks like:
[img]http://img198.imageshack.us/img198/3909/club3damdradeonhd7950no.jpg[/img]
Now for the next part, the heat shield.
-
[quote name=“ChristianRiesen” post=“2974” timestamp=“1368433750”]
My design idea is now to create a heat barrier between cards.
[/quote]
I have been thinking about that also, would ceramic fibre be a possible material to use as a heat shield, or would that be overkill? -
Really awesome man. Good luck with your project. Please post some pics once it’s up.
-
Pretty cool! But do you think it will ever pay for itself at this point? :-\
-
[quote name=“atariguy” post=“3075” timestamp=“1368474544”]
Pretty cool! But do you think it will ever pay for itself at this point? :-\
[/quote]
This is mainly an engineering challenge for me, something for fun, but also trying out things which might be useful to others. If I got a good design, I could create multiple of these nodes and I would put the plans for it public so anyone could build them.So far the actual money I invested is very little. I put a lot of time into it to get it right and then I will put more time into it to make it better :)
-
[quote name=“pyxis” post=“3069” timestamp=“1368472326”]
[quote author=ChristianRiesen link=topic=387.msg2974#msg2974 date=1368433750]
My design idea is now to create a heat barrier between cards.
[/quote]
I have been thinking about that also, would ceramic fibre be a possible material to use as a heat shield, or would that be overkill?
[/quote]
I was more thinking about metal sheets. They themselves will act as heatsink in a fashion. I don’t want the casing to deflect or worse, reflect heat, I want it to make sure it doesn’t get blown hard on neighboring components. That way, and with a good fan, I probably could reduce the heat significantly. That in turn means longer lifetime of the components.