thinking about this a bit more, i can already do the multi .config thing: if i remove the checked-in generated/ directory, add system and vendor sub-directories, each containing a .config file, and then run your "make" twice, with .config a symlink to each real .config file in turn, and moving the generated directory into the corresponding sub-directory. then just modify the include path at build time.
what i don't like about this is that it makes me have two .config files, which i don't really want. getting back to "what does an ideal solution look like?", i still prefer a way to have a "main" configuration and then just describe the differences from it. but i guess i can just make my shell script smart enough to apply a patch rather than actually use two .config files. if we go that route, the only useful thing would be a way to run your "make" in a way that doesn't actually try to build toybox (which will always fail on the host anyway), instead just stopping after all the generated/ stuff is done. On Fri, Jun 30, 2017 at 3:13 PM, enh <e...@google.com> wrote: > On Fri, Jun 30, 2017 at 12:52 PM, Rob Landley <r...@landley.net> wrote: >> On 06/30/2017 01:13 AM, enh wrote: >>> i've mentioned this a few times, but i have a concrete example now. >>> with treble >>> [https://android-developers.googleblog.com/2017/05/here-comes-treble-modular-base-for.html] >>> there will be a copy of mksh and toybox on the /vendor partition in >>> addition to the copies on the /system partition. this lets us upgrade >>> /system's copy at will without worrying about breaking any of the code >>> on /vendor (because it'll only be allowed to access the /vendor shell >>> and utilities). >>> >>> so far so good. and ideally you'd want them to be the same. but the >>> functionality and libraries available to /vendor code differ from >>> /system code. in particular, some of the Android extensions to SELinux >>> aren't available for /vendor. which basically means getprop -Z and >>> restorecon can't be built. >> >> Which sounds like the "building with the NDK" problem all over again. > > yes and no. the NDK problem is fixable within your current system > (it's just a low priority for me because i'm assuming what we _really_ > want is to be able to build a toybox binary that works as well as > possible even when run on older devices, and doing that right requires > NDK work that we're not likely to get to before 2018). > > this is more like "what if you needed to build for the NDK and glibc > in a world where you have to just run the scripts to generate files > once". > >>> short term we could provide dummies in portability.h: >>> >>> #if __ANDROID_VENDOR__ >>> static inline int f(...) { error_exit("f unimplemented on /vendor"); } >>> static inline int g(...) { ... } >>> #endif >>> >>> and then -D__ANDROID_VENDOR__ when we build the /vendor copy. >>> >>> but i think it would be better if we just don't have restorecon and getprop >>> -Z. >>> >>> i can send a patch that makes getprop -Z require a new "has Android >>> SELinux extensions" configure variable, but there's still no easy way >>> for me to override stuff in the .config file (which would let me >>> control both this and the availability of restorecon). >> >> I can add a compile time probe, but you're not using my build scripts. > > you know this already, but just to be clear for those following along > at home --- i am using your scripts; i'm just not using them at build > time. i run them once every time i sync, and then check in the > resulting generated files. > >> I'd also point you at miniconfig >> (http://landley.net/aboriginal/FAQ.html#dev_miniconfig) but again... >> >> (Sigh. I designed infrastructure for this sort of thing.) >> >>> if config.h instead of >>> >>> #define CFG_RESTORECON 1 >>> #define USE_RESTORECON(...) __VA_ARGS__ >>> >>> said >>> >>> #ifndef CFG_RESTORECON >>> #define CFG_RESTORECON 1 >>> #endif >>> #if CFG_RESTORECON == 1 >>> #define USE_RESTORECON(...) __VA_ARGS__ >>> #else >>> #define USE_RESTORECON(...) >>> #endif >>> >>> and global_union could have >>> >>> #if CFG_RESTORECON >>> struct restorecon_data restorecon; >>> #endif >>> >>> i could -DCFG_RESTORECON=0 (and not have restorecon.c in the files >>> list) when building the vendor copy. >> >> True. Not an ideal solution. And roughly equivalent to hitting config.h >> with sed. There's no dependency resolution going on there, so might as >> well just: >> >> sed -E '/ (CFG|USE)_RESTORECON /{s/1$/0/;s/__VA_ARGS__//}' \ >> generated/config.h >> >> But again, not an ideal solution. :) >> >>> i have another couple of configurations lurking in the wings too. >>> they're likely to not have any SELinux, and might want to support >>> different subsets of commands. >> >> Grrr. I thought you could feed a KCONFIG_ALLCONFIG to defconfig, but >> looking at the kconfig source (copied from 2.6.12 back in the day, I've >> meant to replace it _forever_) it's only reading it for >> allno/yes/mod/randomconfig. Sigh. >> >> And even if that did work it wouldn't help you because you've >> snapshotted the generated files. > > and because i might be building on a Mac host (don't blame me), even > if your scripts did let us say "dump the generated stuff in directory > X" so we could do it as part of the build, that just seems like asking > for other [Mac portability] problems. > >> Right, I need to redo my build so it's something you can actually _use_, >> and step 1 of that is getting the existing Ninja build to run the >> Android.mk file you've got. But "apt-get install ninja-build" in Ubuntu >> 14.04 was followed by: >> >> $ ninja -f Android.mk >> ninja: error: Android.mk:17: expected '=', got ':' >> LOCAL_PATH := $(call my-dir) >> ^ near here >> >> Which put it back on the todo list. :) >> >> Then again Ubuntu 14.04's "aptitude show ninja-build" says: >> >> Homepage: http://martine.github.com/ninja/ >> >> Which is 404 (it appears to be ninja-build.org now) so that's possibly a >> really old version. I should try building it from source... > > i doubt you'll get very far without a full AOSP tree. as we transition > from make to soong, there's a lot of chewing gum and string holding > everything together. > >>> i think previously you've been thinking along the lines of making it >>> easier to work with multiple generated/ directories, but i'm wondering >>> whether it might be more convenient to just generate something that >>> can be overridden up front? (it would certainly be a good fit for my >>> uses.) >> >> Both need to happen, but you've got a use case sitting in front of you >> right now so let's fix that. >> >>> i haven't actually tried this yet, so maybe there's something that >>> won't work. but if this seems like a reasonable thing to try i can >>> have a look. i'll probably have a look anyway, because having multiple >>> copies in the tree or having local #if __ANDROID_VENDOR__ patches seem >>> like significantly worse choices and i don't want to be without an >>> alternative when the music stops and we have to sit somewhere... >>> though dummy implementations is probably the smallest hack if it comes >>> to it. >> >> 90% of my toybox work is on the weekends these days. (Startup $DAYJOB >> has missed 2 of the last 3 paychecks so I'm currently doing a consulting >> gig on the side of that, which means my weekdays are pretty thoroughly >> shot right now. I'm just by fiat saying "weekends are for toybox!" and >> ignoring 'em both during that). >> >> Which means I get 2 days to poke at this now, if you think it's worth >> trying to fix it up. >> >> The current problem seems to be that your build system is using a >> snapshot of the generated directory checked into your repo, and now you >> need multiple build contexts with different configurations. We could >> hack up those saved configs with more #ifdeffery, or hit them with sed, >> but long-term getting the generated/ stuff generating under your build >> seems like a better solution? I'd like to at least get _started_ on that. > > well, like i said... macOS makes this likely to just be a different headache. > > but, yes, if there was a script that i could say "take .config file X > and write all the generated stuff to directory D", i think we can > plumb the rest in. > >> Which means I probably need to reproduce your build context. I could >> reinstall AOSP on the big machine (which is nontrivial because if I >> recall I need to clear off something like 250 gigabytes of disk space, >> and it currently has 4 gigs free). > > i don't think you do. just have two .config files in a regular toybox > checkout, and be able to build them both. > > you can see what i currently have at > https://android.googlesource.com/platform/external/toybox/+/master/Android.mk > > i'll still have to manage multiple lists of files in any multi-config > world, but that's fine: it just gives every instance motivation to > minimize unnecessary differences. and realistically i don't see it > ever being anything other than stuff like "do i have SELinux?", "do i > have the Android extensions to SELinux?", which is why i think letting > me -D in my cflags is probably the best for me. > > (as i've said, macOS gives me mixed feelings about generating the > generated files at runtime, plus it feels like that road leads to us > duplicating more of your infrastructure in our build system [as we > have to do things like build the C help generator first and so on].) > >> But the BIGGER problem is last I poked at that it was a giant monolith >> that was non-obvious how to focus on individual bits of it. (I had a >> series of web pages to read about that, and a youtube presentation. It's >> on the todo list. But those resources were for "M" and we're up to "O" >> now, so I expect a lot's changed anyway...) >> >> Ideally I'd like to reproduce the android build context in an otherwise >> minimal chroot, but I have no idea how to _start_ going about that. >> Other than installing, successfully running, and then reverse >> engineering AOSP. >> >> Rob > > i did experiment this afternoon with making the getprop -Z > configuration more specific (that there are Android SELinux > extensions), and i changed generated/config.h to be more like this: > > #ifndef CFG_GETPROP_Z > #define CFG_GETPROP_Z 1 > #endif > #if CFG_GETPROP_Z > #warning with getprop_z > #define USE_GETPROP_Z(...) __VA_ARGS__ > #else > #warning without getprop_z > #define USE_GETPROP_Z(...) > #endif > > the only other thing i had to do (which i did manually because it > wasn't as obvious how to fix this) was generated/flags.h: > > #if CFG_GETPROP_Z > #define OPTSTR_getprop ">2Z" > #else > #define OPTSTR_getprop ">2" > #endif > > unlike config.h and globals.h which are trivial, it wasn't obvious to > me how to fix mkflags.c. > > but with the generated files modified as described, i could just add > -DCFG_GETPROP_Z=0 in the vendor toybox's cflags and otherwise reuse > everything. > > -- > Elliott Hughes - http://who/enh - http://jessies.org/~enh/ > Android native code/tools questions? Mail me/drop by/add me as a reviewer. -- Elliott Hughes - http://who/enh - http://jessies.org/~enh/ Android native code/tools questions? Mail me/drop by/add me as a reviewer. _______________________________________________ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
