[android-building] Cuttlefish and userdata partition image
Hi all, I'm trying to boot up cuttlefish with a premade userdata image (loaded with instrumentation, unit and tradefed tests). I'm on Android 11, and I've noticed that launch_cvd doesn't pass on the --data_image to the cvd assembler, it keeps picking up userdata.img from --system_image_dir. Mounting the default userdata.img, it looks empty to me. When I replace it with my own F2FS image, cuttlefish doesn't boot. :( Has anyone experienced similar problems with cuttlefish? Any hints or point out on how to deal with usedata images would be helpful. Thanks, Chris -- -- You received this message because you are subscribed to the "Android Building" mailing list. To post to this group, send email to android-building@googlegroups.com To unsubscribe from this group, send email to android-building+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-building?hl=en --- You received this message because you are subscribed to the Google Groups "Android Building" group. To unsubscribe from this group and stop receiving emails from it, send an email to android-building+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CABxGUTjaB40_kgSm6F3s23-N0NbZ2zhOu0gfv645Lz6ag5GefA%40mail.gmail.com.
Re: [android-building] AOSP build time vs CPU core count (up to 56)
Hi Dan, Thanks a lot for your detailed answers, it is really helpful. The machine i used for benchmarking build time has indeed less than ideal mass storage configuration. I will do more benchmarking on another machine that has nvme. I as well started to look into RBE (see my other email). Thanks again, Chris On Tuesday, 7 June 2022 at 07:44:11 UTC+12 Dan Willemsen wrote: > This is probably one of two things: > > 1. Disk Speed. Once you saturate your CPUs (and have the associated memory > required -- somewhere around 2GB/core is usually decent), the next thing > you'll often hit is disk bandwidth. This is where ccache tends to hurt more > than it helps, unless the ccache is on a completely separate [also fast] > disk (full cache hits may get better, but writing the cache ends up being > 2x the disk bandwidth). Remote cache/execution plus Bazel's Build without > the bytes <https://blog.bazel.build/2019/05/07/builds-without-bytes.html> > concept > is a more ideal caching solution for this case, but that's not available > for Android at this point. > > 2. That -j isn't a great limiter. It worked "okay" when every build action > was single threaded, but more and more tools are multithreaded these days. > So if we have 64 cpu threads, and use -j64, if you get really unlucky with > a tool that spawns a process thread for every CPU, we could end up with > 4096 (64*64) process threads running, well over the 64 specified by `-j`. > In a more limited case, this may mean that on average you're still using a > good portion of your CPU, even with smaller -j values. > > Anyone knows how to disable some of the build tasks? For example i've >> noticed that there is a lot of java source processing and wonder if they >> are all needed. I'm pretty sure that some documentation is generated, in >> our case we don't need that. >> > > You're probably seeing metalava/etc. While that is used to generate > documentation, it is also used to generate the API stubs used later in the > build, so it can't just be turned off. > > If you haven't seen it yet, Soong's Performance doc > <https://android.googlesource.com/platform/build/soong/+/master/docs/perf.md> > is a good overview on how some performance issues can be diagnosed. > > Hopefully you're doing many more incremental builds than full builds, but > I know that gets really complicated with some use cases. > > --- > > For reference, the newer desktops at Google end up being 2x 18-core Intel > Xeon Gold 6154 (so 72 threads) or the Ryzen 3995WX (128 threads), at least > with NVME, but often extended with SSDs. We also run builds on a variety of > GCE-based machines, from 32 to 128 vCPUs, and the difference there between > pd-standard and pd-ssd can be substantial. > > On my Intel desktop (w/NVME), without ccache, I'm seeing about a 10% > improvement from -j36 -> -j74, so a little better than your comparison, but > not a ton: > > `lunch aosp_arm64-userdebug; m` (effectively -j74) takes 32m46s > `lunch aosp_arm64-userdebug; m -j54` takes 34m00s > `lunch aosp_arm64-userdebug; m -j36` takes 36m19s > > With RBE (Google's implementation of > https://bazel.build/community/remote-execution-services): > > `lunch aosp_arm64-userdebug; USE_RBE=true m` takes 18m46s > > - Dan > > On Wed, Jun 1, 2022 at 4:21 PM Christian Gagneraud > wrote: > >> Hi all, >> >> We're looking into reducing our build time. >> The first benchmark we did was to measure build time of aosp_arm64 versus >> number of parallel jobs. >> We have a machine with 28 cores (56 threads), so i ran a script to build >> aosp from 4 jobs up to 56 jobs with an increment of 4. >> >> The results surprised me, in a nutshell, with ccache enabled and 100% hit >> rate >> 4 cores: 1h >> 8 cores: 42m >> 12 cores: 35m >> 16+ cores: 32m >> >> Throwing more than 16 cores doesn't influence build time anymore, the >> curve is flat! >> >> I just can't explain what is going on here. Does anyone know why the >> build process doesn't scale? >> >> I'm currently running the same benchmark with ccache disabled. Will post >> an update once i have the data. >> >> Anyone knows how to disable some of the build tasks? For example i've >> noticed that there is a lot of java source processing and wonder if they >> are all needed. I'm pretty sure that some documentation is generated, in >> our case we don't need that. >> >> As well, any tips or feedback around reducing build times would be more >> than welcome. >> >> Thanks, >> Chris >> >> -- >> -- >> You receiv
[android-building] Reducing AOSP build time, RBE? (was Re: AOSP build time vs CPU core count (up to 56))
On Thursday, 2 June 2022 at 08:22:01 UTC+12 Christian Gagneraud wrote: > Hi all, > > We're looking into reducing our build time. > The first benchmark we did was to measure build time of aosp_arm64 versus > number of parallel jobs. > We have a machine with 28 cores (56 threads), so i ran a script to build > aosp from 4 jobs up to 56 jobs with an increment of 4. > > The results surprised me, in a nutshell, with ccache enabled and 100% hit > rate > 4 cores: 1h > 8 cores: 42m > 12 cores: 35m > 16+ cores: 32m > Here are my results with ccache disabled. These benchmarks were done on android 11 branch. 4 cores: 4h15m 8 cores: 2h20m 12 cores: 1h40m 16 cores: 1h25m 20+ cores: 1h15m down to 55 minutes Again, the curve flatten with increasing cores. I am now exploring RBE (Remote build execution), i found some information on the web (joined the magic group to access the docs), i have enabled the API on gcloud. I haven't try a build yet, because i'm having a hard time to understand how to setup all of these. Ultimately, we're interesting in reducing our build time in our CI. Our builds are done in AWS, but we'll consider moving to gcloud if it's worth it. In another thread, i found this table: [image: Screenshot_20220605_104826.png] Timing of local builds don't really match my benchmarking, odd. Does anyone knows more about what should be a typical CI setup? eg. How many cores is needed for the machine that run ninja, and how to size the worker pool? eg. number of workers and worker machine spec. Looking at logs from ci.android.com, i can see NINJA_REMOTE_NUM_JOBS="500". I wonder if the worker pool can be auto-scaled or not. Thanks, Chris -- -- You received this message because you are subscribed to the "Android Building" mailing list. To post to this group, send email to android-building@googlegroups.com To unsubscribe from this group, send email to android-building+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-building?hl=en --- You received this message because you are subscribed to the Google Groups "Android Building" group. To unsubscribe from this group and stop receiving emails from it, send an email to android-building+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/0443622c-1ba6-4466-a8e4-51bd9261cbecn%40googlegroups.com.
[android-building] AOSP build time vs CPU core count (up to 56)
Hi all, We're looking into reducing our build time. The first benchmark we did was to measure build time of aosp_arm64 versus number of parallel jobs. We have a machine with 28 cores (56 threads), so i ran a script to build aosp from 4 jobs up to 56 jobs with an increment of 4. The results surprised me, in a nutshell, with ccache enabled and 100% hit rate 4 cores: 1h 8 cores: 42m 12 cores: 35m 16+ cores: 32m Throwing more than 16 cores doesn't influence build time anymore, the curve is flat! I just can't explain what is going on here. Does anyone know why the build process doesn't scale? I'm currently running the same benchmark with ccache disabled. Will post an update once i have the data. Anyone knows how to disable some of the build tasks? For example i've noticed that there is a lot of java source processing and wonder if they are all needed. I'm pretty sure that some documentation is generated, in our case we don't need that. As well, any tips or feedback around reducing build times would be more than welcome. Thanks, Chris -- -- You received this message because you are subscribed to the "Android Building" mailing list. To post to this group, send email to android-building@googlegroups.com To unsubscribe from this group, send email to android-building+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-building?hl=en --- You received this message because you are subscribed to the Google Groups "Android Building" group. To unsubscribe from this group and stop receiving emails from it, send an email to android-building+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/a5b97997-311e-4913-a829-111d0433d227n%40googlegroups.com.
Re: [android-building] Understanding flow Sequence in Android Build system
On Tue, 25 May 2021 at 06:15, Sumit Kumar wrote: > > HI all , > I am attempting to understand the flow sequence for Android Build system > using Andorid 11 as the base, as a part of my work. I would like to know > from you guys which would be the best indenxing tool for this purpose , as > the build system consists of many different file formats, ".bp" ,".mk" > ".ninja" ".sh" ".go" to name some of them? > Further if any one has ever attempted any thing similar to what ever extent > please share some docs/links/references as it would be a big stepping stone > for me to start. Sounds like a good idea, tho i don't know if this is actually feasible. I use "mgrep" when i'm digging into mk/bp. I have to say that there are a lot of global variables, and most of them are undocumented. This is very frustrating. The build system of AOSP is very obscure, that was a big disappointment for me when I started. Magic stuff everywhere! "make product-graph" is a very useful tool, I think soong/bp is fairly documented. The Makefiles are the pain point IMO. Chris > > The following links gave me a fair understanding of syntax for kati, ninja > and soong:- > https://android.googlesource.com/platform/build/soong/+/refs/heads/master/README.md > https://github.com/google/kati/blob/master/README.md > https://ninja-build.org/manual.html#_introduction > > Regards, > Sumit Kumar > > -- > -- > You received this message because you are subscribed to the "Android > Building" mailing list. > To post to this group, send email to android-building@googlegroups.com > To unsubscribe from this group, send email to > android-building+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/android-building?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "Android Building" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to android-building+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/android-building/4566d7a3-9434-4731-80e6-56a97348707bn%40googlegroups.com. -- -- You received this message because you are subscribed to the "Android Building" mailing list. To post to this group, send email to android-building@googlegroups.com To unsubscribe from this group, send email to android-building+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/android-building?hl=en --- You received this message because you are subscribed to the Google Groups "Android Building" group. To unsubscribe from this group and stop receiving emails from it, send an email to android-building+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CABxGUThovxfg7S2u6s-v8jk4uY0tPsEqBjjrpSorCjhuXwYDAw%40mail.gmail.com.
