On 9/10/17 2:31 PM, Alex Lennon wrote: > > > On 10/09/2017 19:17, Mark Hatle wrote: >> On 9/10/17 11:14 AM, Alex Lennon wrote: >>> >>> On 10/09/2017 17:06, Mark Hatle wrote: >>>> On 9/10/17 2:00 AM, Usman Haider wrote: >>>>> Hi, >>>>> >>>>> Can someone please recommend some good machine for yocto environment and >>>>> building sdks. I am interested in RAM, hard disk space, processor. >>>> You want fast I/O, as much RAM and as many (fast) cores as you can afford. >>>> I >>>> don't think there is a single answer as what is 'best'. It also depends on >>>> which Yocto Project versions, and which layers you are using as to which >>>> combination is best. >>>> >>>> I run builds on my laptop, 4-core/8-thread & SSD and 16 GB of ram from a >>>> few >>>> years ago. It's fast, but I wouldn't want to do all of my development on >>>> it. >>>> >>>> I've had 8-core/16-thread (32GB ram/standard disk), 16-core/32-thread (72GB >>>> ram/SAS-3 RAID), 24-core/48-thread (64GB ram/SATA - software RAID), >>>> 72-core/144 >>>> thread (256 GB ram/hardware raid/SAS-3), and recently upgraded to >>>> 96-core/192-thread (256 GB ram/hardware raid/SAS-3). >>>> >>>> I would not go below quad-core (8-thread) myself. You can get a quad >>>> core, good >>>> quality machine for $1000 or less these day. If you move up to the larger >>>> machines, you can even be able to get to a 24-core for less then $5000. >>>> By the >>>> time you get to 96-core and all of the googles you are likely talking >>>> $50000 or >>>> more. >>>> >>>> By clock raid, the 24-core machine is the fastest.. While the 96-core >>>> monster >>>> can do the builds the quickest. But when you figure out >>>> cost/performance/etc.. >>>> the 24-core is probably the best performance per dollar, and with adequate >>>> RAM >>>> (I'd say at least 64GB if not 128GB), and fast I/O you'll probably get the >>>> lowest price for the best performance in that category. >>>> >>>> If you need sheer speed and price is no option, then the (4 CPU w/ 24 core >>>> each) >>>> 96-core monster (or even better) is what you want to go with. 256GB ram >>>> would >>>> be a minimum with that configuration (I'm not sure if more is actually >>>> helpful, >>>> I rarely end up in swap -- but I go get into situations where more then >>>> 50% of >>>> ram is used.) With that many cores, disk I/O starts to become obvious. So >>>> faster the better... SSDs would be the fastest, but of course the most >>>> expensive. >>>> >>>> If your employer is paying for the machine, you may be able to get a >>>> better then >>>> normal machine by explaining how much time a faster machine will save and >>>> how >>>> comparing to your salary a machine is inexpensive. (If you are a >>>> contractor or >>>> student, that changes of course.) :) >>>> >>>> So my point is really, figure out how much money you have to spend. My >>>> rule of >>>> thumb is roughly: >>>> >>>> 1) Buy as many cores as you can. Try to get a CPU that has Hyperthreading >>>> or >>>> equivalent to double the effective core count. Fastest processing speed >>>> helps >>>> in repetitive cases vs full system builds. >>>> >>>> So if the choice is a 24 core @ 2.2GHz vs 22 core @ 2.5, I'd probably go >>>> with >>>> the 22-core. While if it was 24 core @ 2.2GHz vs 8 core @ 4.2 GHz, I'd go >>>> with >>>> the 24 core. >>>> >>>> 2) Try to get at least 1 GB of ram per thread (2 GB per core..) You can >>>> cut >>>> back on the ram (if necessary) once you hit 72 threads or so. (72 threads >>>> right now seems to cover most of the parallelization in a full system >>>> build. >>>> There are points in the system where it can parallelize MUCH more, but >>>> they are >>>> fairly rare.) >>>> >>>> 3) You need fast disks. Software RAID works fine, but you likely need to >>>> buy at >>>> least a couple of disk to boost performance. SSDs are fast, but lots of >>>> builds >>>> take space, so fast SATA or even better SAS drives are the best >>>> performance per >>>> cost. >>> This brings to mind a related question I keep coming back to as to the >>> economics of a docker (or similar) image running a fast Yocto build in >>> the cloud. >>> >>> i.e. set config params -> bring up server image on plaform A/B/C -> >>> perform build taking time X/Y/Z -> store output images -> bring down >>> serverĀ == $ ? >>> >>> I find myself asking what the optimal cost per-build would be using this >>> approach... >> I helped someone do some very -preliminary- figured a few years ago. The >> processing was 'cheap', but between storage and network transfer costs.. it >> was >> cheaper to buy a reasonable machine.. payback time was only a few months. >> >> (cloud 'storage' is often very slow as well, because there are expectations >> of >> migration and things like that.) >> >> So as of a few years ago at least, the economics didn't factory the cloud -- >> yet. >> >> > > Yeah that is more or less my thinking. Although I do imagine that if a > source mirror on an internal network leg could be set up then that might > significantly reduce chargeable network b/w...
premirror for downloads and sstate-cache shared between multiple customers could reduce the network b/w far enough to make it economical. But you would need to trust other vendors since the sstate-cache can be manipulated to provide malicious software (in some cases). (And strengthening the sstate-cache checking can cause enough extra overhead to make it 'cheaper' to build then use sstate-cache in many cases.) --Mark > Cheers, > > Alex > -- _______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto