We are eyeballing a new farm next year (about that time again) and this thread has been great. I do have a few questions as we are currently using arnold for everything and don't have much experience with Redshift.

1. everyone keeps chiming in on how you want the graphics card to have
   as much memory as possible. What happens when you run out of VRAM
   during a render? Does it just crap out? Or get really slow?
2. Some folks were mentioning they assign one render job per GPU rather
   than per node? Does your farm software have to be set up to handle
   that? (we use Royal Render).

Chad Briggs
Element X

On 8/6/2015 5:59 PM, Tim Crowson wrote:
Looks like you guys have gone into all the relevant details already, so I don't have much to add.

I will only strongly suggest that you get cards with as much RAM as you can afford. Personally, if I were building a small farm, I wouldn't even bother with the 970s, simply because of their ram limitations: Redshift can only use 3.5GB of its 4GB, because of the crap Nvidia pulled with the memory on that card. And frankly, I can't recommend 4GB cards anyway. I really recommend 6GB or higher. This makes the 980Ti a very good buy. Because of diminishing returns in performance for multiple GPUs, I would't put more than 3-4 in a machine. Here at Magnetic we stopped at 3, and decided to parallelize after that.

Just remember that even though RS can go out-of-core, it REALLY likes as much memory as you can give it, and your render times will reflect it.

If you wind up getting mixed cards, you'll be glad to know that the RS devs are working on a smarter memory management system that will adjust its use of available memory at render time. This should really help newcomers, but also with mixed-hardware scenarios.

-Tim C.

On 8/6/2015 1:27 PM, Mirko Jankovic wrote:
as much as I know those risers and extensions can't really support full speed PCIE, it would lower PCI speed significantly for each card, now not sure about other render engines but for Redshift it is good to keep at least at speed of PCIE3.0@x8 or PCIE2.0@x16

On Thu, Aug 6, 2015 at 7:55 PM, Angus Davidson <angus.david...@wits.ac.za <mailto:angus.david...@wits.ac.za>> wrote:

    Or use something like this

    http://amfeltec.com/products/gpu-oriented-cluster/

    Kind regards

    Angus

    *From:*Mirko Jankovic [mailto:mirkoj.anima...@gmail.com]
    *Sent:* 06 August 2015 02:55 PM
    *To:* softimage@listproc.autodesk.com
    <mailto:softimage@listproc.autodesk.com>
    *Subject:* Re: OT'ish: Redshift renderfarm with Softimage setup?

    yep exactly like the one on the image :) a lot of nice space and
    rad placeholder down at the bottom. also on the back of the case
    same position you can place even more fans for fresh air intake
    and to go straight over those rads and out.

    but this is going more into workstation build area now

    On Thu, Aug 6, 2015 at 2:52 PM, Matt Morris <matt...@gmail.com
    <mailto:matt...@gmail.com>> wrote:

    The cheapest option would be any brand with the reference design
    coolers, which are pretty much straight from nvidia. They push
    the heat out of the rear of the case, you can set a custom fan
    profile to try to keep them from throttling.

    The water-cooled version looks great, and the temps are
    fantastic. £100 price premium isn't too bad for a cooler like
    that. The major problem is where you put the fans, if you go for
    4 in a case. in most normal cases 2 of them at least would be
    directing warm air into the case... Might be worth looking at
    cases which are built to have a watercooling radiator along the
    base, where you could put all 4 fans pointing one way, something
    like the corsair 900D.

    On 6 August 2015 at 12:42, Morten Bartholdy <x...@colorshopvfx.dk>
    wrote:

    Along these lines - which 980ti would you guys recommend? I found
    this an EVGA GeForce GTX 980 Ti Hybrid which has built-in
     watercooling and a separate fan to mount in the back of the
    cabinet - pretty cool.

    
http://www.proshop.dk/Grafikkort/EVGA-GeForce-GTX-980-Ti-Hybrid-6GB-2493470.html
    (danish site)

    Morten


    Den 6. august 2015 kl. 11:16 skrev Tim Leydecker <bauero...@gmx.de>:

        Would you guys find the 980Ti hitting the sweetspot between
        price and performance?

        How about connectors and power supply?

        The 970 is running on 2x6pin, e.g. a maximum of 150 Watts
        plus the 75 Watts from the slot, a 225 Watts total.

        The 980ti is mostly 1x6pin and 1x8pin, the 1x8pin offering
        150Watts compared to a 1x6pin offering 75 Watts.

        In my case, I find it already hard to provide more than one
        1x8pin and 1x6pin via connectors.
        How do you guys provide reliable power to more than 1 or 2
        graphics cards without melting your power lines?

        Here in Germany, it is rare to have more than around 1 kW
        sustained drain per average wall plug supported by a great
        many home installations.
        There is always loads of headroom of course but technically,
        constantly draining a lot more from  such a wall plug can
        get, uhmmm, hot.

        That´s a few of the reasons I suggested to start out with
        just 1 card, like a Titan X (or a GTX980ti), case power
        supply connection, wall plugs, electrical limits.

        Cheers,

        tim






        Am 05.08.2015 um 16:10 schrieb Mirko Jankovic:

            agree. 980ti is just a bit above 2 970s price wise,
            performance wise it realyl dpends on scenes you are
            working on. but I plan to upgrade my 4x970 with 980ti as
            soon as possible, even if it means replacing 1 by  1

            On Wed, Aug 5, 2015 at 3:36 PM, Matt Morris <
            <mailto:matt...@gmail.com>matt...@gmail.com > wrote:

            The 970 is the most cost efficient only with scenes that
            fit into its memory - which using redshift is limited to
            3.5Gb because of the internal memory architecture. I'd
            recommend looking at gpus with 6Gb or higher. The 980ti
            is a great card for the money, and the extra vram will
            help performance even on small scenes as you can utilise
            memory optimisation settings. Because you're limited to 4
            gpus (risers don't work too well and limited by number
            and speed of pci-e lanes as mirko said) you want to make
            the most of that space. Per card electricity usage and
            heat output isn't that much more for the 980ti.

            On 5 August 2015 at 14:04, Tim Leydecker <
            <mailto:bauero...@gmx.de>bauero...@gmx.de > wrote:

            Thanks for the clarification, Dan.

            I think I mixed this up with the download section of the
            forum for customers?

            Whatever, good that the registered user forum is
            accessible to interested parties.

            Cheers,

            tim

            P.S: For Hair, Shave&Haircut is supported (I don´t have
            personal experience with it).

            Am 05.08.2015 um 14:17 schrieb Dan Yargici:

                " you may find it helpful to register in the
                Redshift3D.com forums, afaik you´ll need to have
                at least one registered license to get access to the
                "Registered users only" forum area."

                Just to clear this up.  I'm pretty sure you don't
                need to have a license to access the Registered Users
                section of the Redshift forums.

                DAN

                On Wed, Aug 5, 2015 at 2:58 PM, Rob Chapman <
                <mailto:tekano....@gmail.com>tekano....@gmail.com >
                wrote:

                A lot of good and informed points by all, just wanted
                to add, this guy here, Sven, at
                http://www.render4you.de/renderfarm.html recently
                became the first official Redshift GPU render farm
                and have used him already on a few jobs with very
                tight deadlines. Essentially he has a rack of 7x
                Tesla K40st - so 1 node is the equivalent of a 6x
                single 980gtx which I find is pretty cost effective
                solution of adding a decent online GPU render node,
                that works with hardly any setup if you have a
                redshift scene ready to go

                best

                Rob

                On 5 August 2015 at 11:56, Tim Leydecker <
                <mailto:bauero...@gmx.de>bauero...@gmx.de > wrote:

                Hi Morten,

                you may find it helpful to register in the
                Redshift3D.com forums, afaik you´ll need to have
                at least one registered license to get access to the
                "Registered users only" forum area.

                There´s a few threads there about Hardware, multiple
                GPU systems and some user cases
                of testing single gpu vs. multi gpu rendering plus
                some Developer info about roadmaps and such.

                Personally, I´m a big fan of Redshift 3D.

                Still, here´s a few things to consider you may find
                useful:

                - Compared to Arnold, there is no HtoA or C4DtoA
                equivalent, e.g. no direct C4D or Houdini support
                - Compared to Arnold, rendering Yeti is not yet
                supported in Redshift3D - it´s looked at, no ETA.
                - Maya Fluids, Volumerendering, FumeFX e.g.
                Fire&Smoke&Dust&such isn´t in Redshift3D sofar

                - Multitasking, compared to CPU based multitasking
                and task switching (e.g. switching between
                  rendering in Maya, Softimage while simultaneously
                comping in Nuke and painting Textures in Photoshop
                  or Mari) may pose GPU specific limitations with
                multiple applications fighting for a very limited GPU
                VRAM.
                 Redshift3D can utilize system RAM for VRAM but there
                can be headache when other, "dumber" apps go ahead
                 and just block VRAM for their caching. It´s well
                worth running a good few hard tests in typical
                workflow scenarios.
                 Maya, Substance Painter/Designer, Nuke, Photoshop,
                they all offer one type or another of GPU caching or GPU
                 acceleration option. My personal feeling is, such
                stuff never gets tested in real-world,
                multiple-applications-running scenarios.

                At a glance, it would sound easy enough to have
                separate, dedicated GPUs run headless for rendering
                and reserving one GPU
                for viewport display and other apps but to be honest,
                all this stuff is so new, even thought it´s great,
                it´s still pushing grown
                legacy workflows and boundaries and in doing so, it
                may sometimes hurt.

                My very personal suggestion is:

                - a starter kit is just one GPU, optimally a Titan X
                with 12GB VRAM.
                - step 2, adding a second GPU, running headless,
                reserved for rendering
                - step 3, adding a third GPU, comparing speed to step 2
                - step 4, price/performance balancing, comparing a
                1-2-3 GPU GTX970 render rig with the above

                Could be you find out you like to run 1 Titan X for
                viewport display and multi-apps, and 2 GTX970 for a
                render job.


                Another thing.

                Multi-socket CPU boards and PCIe slots. It seems
                easier to get solid single socket CPU boards with
                lot´s of PCIe slots.

                Again, from my personal experience running a current
                generation dual socket Xeon rig, it is annoying how
                many CPU
                cycles I see wasted away in idle in most of my daily
                chores, except for pure rendering with Arnold or the
                likes, I find
                myself mostly having one CPU and even most of the
                other CPU´s cores just not used properly by software.

                I think a good sweetspot would have been to just go
                for one fast, solid 6-core(budget) or 8core (current)
                CPU, unless of course for a dedicated render slave...


                Cheers,

                tim











                Am 05.08.2015 um 12:05 schrieb Morten Bartholdy:

                    I know several of you are using Redshift
                    extensively or only now. We are looking in to
                    expanding our permanent render license pool and
                    are considering the pros and cons of Arnold, Vray
                    and Redshift. I believe Redshift will provide the
                    most bang for the buck, but at a cost of some
                    production functionality we are used to with
                    Arnold and Vray. Also, it will likely require an
                    initial investment in new hardware as Redshift
                    will not run on our Pizzabox render units, so
                    that cost has to be counted in as well.

                    It looks like the most priceefficient Redshift
                    setup would be to make a few machines with as
                    many GPUs in them as physically possible, but how
                    have you guys set up your Redshift renderfarms?

                    I am thinking a large cabinet with a huge PSU,
                    lots of cooling, as much memory as possible on
                    the motherboard and perhaps 8 GPUs in each. GTX
                    970 is probably the most power per pricepoint
                    while Titans would make sense if more memory for
                    rendering is required.

                    Any thoughts and pointers will be much appreciated.

                    Morten



--
            www.matinai.com <http://www.matinai.com>




--
    www.matinai.com

    This communication is intended for the addressee only. It is
    confidential. If you have received this communication in error,
    please notify us immediately and destroy the original message.
    You may not copy or disseminate this communication without the
    permission of the University. Only authorised signatories are
    competent to enter into agreements on behalf of the University
    and recipients are thus advised that the content of this message
    may not be legally binding on the University and may contain the
    personal views and opinions of the author, which are not
    necessarily the views and opinions of The University of the
    Witwatersrand, Johannesburg. All agreements between the
    University and outsiders are subject to South African Law unless
    the University agrees in writing to the contrary.



--
Signature


Reply via email to