Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
Jonas Meurer wrote: > > i guess the best solution would be to improve the module dependency > system. cipher meta-modules like 'aes' should depend on all available > implementations. same for ivciphers, hashs, rng implementations etc. We already do that for algorithms at least, they should all have aliases of the form "ALG" or "ALG-all". Things like chainiv/eseqiv should just be included always. There are unlikely to be any new algorithms of that type. Cheers, -- Visit Openswan at http://www.openswan.org/ Email: Herbert Xu ~{PmV>HI~} Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt -- To unsubscribe from this list: send the line "unsubscribe linux-crypto" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
* Jonas Meurer | 2009-09-01 02:51:20 [+0200]: >as you already discovered, that's because the initramfs hook script for >cryptroot adds some crypto modules manually in debian. currently these >are dm-mod, dm-crypt, cbc, aes, sha256 and all the modules they depend >on. You could add something like |for dev in $rootdev $resumedevs; do | info="$(cryptsetup luksDump $rootdev)" | if [ $? -ne 0]; then | # Partition is now encrypted | continue | fi | cipher="$(cryptsetup luksDump /dev/md1 |grep "Cipher name:" | \ |sed 's...@^cipher name:\s\+@@')" | add_crypto_modules $cipher | mode="$(cryptsetup luksDump /dev/md1 |grep "Cipher name:")" | echo "$mode" | sed \ | -e 's...@^cipher name:\s\+@@') \ | -e 's...@-@ @' \ | -e 's@:@ @' | # mode is now " " | blk_mode=$(echo $mode | awk '{print $1}' | hash=$(echo $mode | awk '{print $3})' | add_crypto_modules $blk_mode | add_crypto_modules $hash |done instead of hardcoding cbc, aes & sha256 >> So this brings me to the following question: Why are you so picky and you >> don't take all of the modules? sh 2.6.30-1 on various debian archs: >> du -sh x86_64 i386 powerpc s390x mips(r5k-ip32) >> crypto 720K636K 1.1M 804K968K >> arch//crypto 72K 36K 092K 0 >> driver/crypto 32K 60K 60K 0 52K >> Total 824K732K 1.2M 896K1020 >> >> So all modules together have an average footprint of 941.0K. This isn't >> that much. There is no size limit on initrd unless you are on a system >> with has 32MiB or something like that :) >> With *all* those crypto modules which are selected and built you are >> able to cover all corner cases. >> I took a look at hooks/cryptroot and really and only a few modules are >> included. Right now you don't include HW cipher e.g. VIA's padlock or >> AMD's geode. If you don't load those module while openning the root >> partition (the first request), then the module is never loaded / used. >> The other unhandled case are non default ciphers like xts(aes) which one >> might use. The brand new aes implementation aesni-intel is also not be >> considered. Most modules have _generic in their name if there is >> another implementation available and -arch if there is an assembly >> variant but I would prefer that this does not become part of an ABI >> where one can rely on. >> >> Would it be okay for you to change the way the crypto modules are picked >> and include the full set? > >i don't like the idea to add _all_ crypto modules into the initramfs per >default. the size of crypto and arch//crypto will keep on growing >for kernels which do have ship all available modules for ciphers and >implementations. i think that the initramfs should be a _minimal_ >system, and detecting the required crypto modules should be possible. Why is a minimal initramsfs here a good thing here? 1 MiB isn't that huge and with gzip it does not grow much. Here is an example on my amd64 with current initramfs as of 2.6.30-1-amd64: picky : 10511917 bytes / 11 MiB full crypto : 10729154 bytes / 11 MiB / + 2.1% So. On AMD64 we are talking here about 200KiB or 2% of the whole image size. If arch/*/crypto grows than just because there is more support and this is a good thing not bad. >alternative crypto cipher and blockcipher modules are already detected >if required, as information about these are available from the cryptroot >configuration (either luks header or cmdline options given to cryptsetup). ah so my little script was not required. What about d-i? AFAIK that one runs from initramfs and relies on modules which are selected here. Isn't this the case? >i added cryptomgr, chainiv and krng to the list of required modules in >/usr/share/initramfs-tools/hooks/cryptroot for now. And in .32 you have work again because you need eseqiv. >still i see the point that this solution has it's drawbacks. modules for >crypto hardware and cipher implementations not following the convention to >call modules _generic and -alias aren't detected by the >current system. exactly. Plus runtime dependency. >i guess the best solution would be to improve the module dependency >system. cipher meta-modules like 'aes' should depend on all available >implementations. same for ivciphers, hashs, rng implementations etc. > >what do you think about it? I don't the like extra work for 2%. Even if someone does this, you still have lets say the geode, via-padlock and the hfin driver on a 486 system which can't use all of them and relies on pure SW. No benefit there. Okay. You put the crypto modules in initramfs in case the root partition and or swap is encrypted. In case just /data is encrypted you don't need those modules at all. So d-i needs all them in case the user might want to xts(twofish) and not just the standard thing. The user-initramfs s
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
reopen 541835 thanks hey, On 31/08/2009 Sebastian Andrzej Siewior wrote: > * Jonas Meurer | 2009-08-31 17:52:00 [+0200]: > > >> Could someone please look at initramfs to figure out why those three > >> modules are not copied in this reduced setup? > > > >the reason is simply that no other crypto modules define depends on the > >listed ones: > Well yes, but aes cbc is placed in the new initrd. I took a look, read > below. as you already discovered, that's because the initramfs hook script for cryptroot adds some crypto modules manually in debian. currently these are dm-mod, dm-crypt, cbc, aes, sha256 and all the modules they depend on. > >so the following depends should be added/changed: > > > >- chainiv should depend o 'krng' instead of 'rng' at least >rng is a subsubsystem within crypto. krng is one implementation which >provides random numbers. So does ansi_cprng. Another implementation >may come. > > >- maybe cipher modules like aes,serpent,... should depend on 'cryptomgr' > > instead of 'crypto_algapi' >This isn't possible without dummy variables because they don't need >each other in first place. We could move code but crypto_algapi can >live without cryptomgr. However cryptomgr doesn't make sense without >crypto_algapi. >Lets say you have a VIA CPU which provides cbc(aes) in HW. So you >don't need cryptomgr because cbc(aes) is allready available. You need >aes_generic but that's another story :) > > >- crypto_algapi should depend on chainiv >chainiv is one possible implementation eseqiv is another one. The >later is used on async-hw and in 2.6.32+ on SMP afaik. > > >these changes are pure guesses, i don't know the details. but at least > >additional depends need to be defined for crypto modules, don't you > >think so? > > We have them in Kconfig. All required modules are built and are > available. However the things aren't that easy. The crypto subsystem is > very flexible and undocumented and this is the problem here I think. ok, i see the point that it's not enough to simply add some additional dependencies in order to fix the problem. seems like an easy solution doesn't exist. > So this brings me to the following question: Why are you so picky and you > don't take all of the modules? sh 2.6.30-1 on various debian archs: > du -sh x86_64 i386 powerpc s390x mips(r5k-ip32) > crypto 720K636K 1.1M 804K968K > arch//crypto 72K 36K 092K 0 > driver/crypto 32K 60K 60K 0 52K > Total 824K732K 1.2M 896K1020 > > So all modules together have an average footprint of 941.0K. This isn't > that much. There is no size limit on initrd unless you are on a system > with has 32MiB or something like that :) > With *all* those crypto modules which are selected and built you are > able to cover all corner cases. > I took a look at hooks/cryptroot and really and only a few modules are > included. Right now you don't include HW cipher e.g. VIA's padlock or > AMD's geode. If you don't load those module while openning the root > partition (the first request), then the module is never loaded / used. > The other unhandled case are non default ciphers like xts(aes) which one > might use. The brand new aes implementation aesni-intel is also not be > considered. Most modules have _generic in their name if there is > another implementation available and -arch if there is an assembly > variant but I would prefer that this does not become part of an ABI > where one can rely on. > > Would it be okay for you to change the way the crypto modules are picked > and include the full set? i don't like the idea to add _all_ crypto modules into the initramfs per default. the size of crypto and arch//crypto will keep on growing for kernels which do have ship all available modules for ciphers and implementations. i think that the initramfs should be a _minimal_ system, and detecting the required crypto modules should be possible. alternative crypto cipher and blockcipher modules are already detected if required, as information about these are available from the cryptroot configuration (either luks header or cmdline options given to cryptsetup). i added cryptomgr, chainiv and krng to the list of required modules in /usr/share/initramfs-tools/hooks/cryptroot for now. still i see the point that this solution has it's drawbacks. modules for crypto hardware and cipher implementations not following the convention to call modules _generic and -alias aren't detected by the current system. i guess the best solution would be to improve the module dependency system. cipher meta-modules like 'aes' should depend on all available implementations. same for ivciphers, hashs, rng implementations etc. what do you think about it? greetings, jonas signature.asc Description: Digital signature
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
* Jonas Meurer | 2009-08-31 17:52:00 [+0200]: >> Could someone please look at initramfs to figure out why those three >> modules are not copied in this reduced setup? > >the reason is simply that no other crypto modules define depends on the >listed ones: Well yes, but aes cbc is placed in the new initrd. I took a look, read below. >so the following depends should be added/changed: > >- chainiv should depend o 'krng' instead of 'rng' at least rng is a subsubsystem within crypto. krng is one implementation which provides random numbers. So does ansi_cprng. Another implementation may come. >- maybe cipher modules like aes,serpent,... should depend on 'cryptomgr' > instead of 'crypto_algapi' This isn't possible without dummy variables because they don't need each other in first place. We could move code but crypto_algapi can live without cryptomgr. However cryptomgr doesn't make sense without crypto_algapi. Lets say you have a VIA CPU which provides cbc(aes) in HW. So you don't need cryptomgr because cbc(aes) is allready available. You need aes_generic but that's another story :) >- crypto_algapi should depend on chainiv chainiv is one possible implementation eseqiv is another one. The later is used on async-hw and in 2.6.32+ on SMP afaik. >these changes are pure guesses, i don't know the details. but at least >additional depends need to be defined for crypto modules, don't you >think so? We have them in Kconfig. All required modules are built and are available. However the things aren't that easy. The crypto subsystem is very flexible and undocumented and this is the problem here I think. Another example: The geode aes HW/driver can handle cbc(aes) requests as long as its keysize is 128bits. Larger keys can't be handled by HW so we fallback to the next implementation. If you just copy the geode module then the geode module won't work because an alternative implementation is missing. So this brings me to the following question: Why are you so picky and you don't take all of the modules? sh 2.6.30-1 on various debian archs: du -sh x86_64 i386 powerpc s390x mips(r5k-ip32) crypto 720K636K 1.1M 804K968K arch//crypto 72K 36K 092K 0 driver/crypto 32K 60K 60K 0 52K Total 824K732K 1.2M 896K1020 So all modules together have an average footprint of 941.0K. This isn't that much. There is no size limit on initrd unless you are on a system with has 32MiB or something like that :) With *all* those crypto modules which are selected and built you are able to cover all corner cases. I took a look at hooks/cryptroot and really and only a few modules are included. Right now you don't include HW cipher e.g. VIA's padlock or AMD's geode. If you don't load those module while openning the root partition (the first request), then the module is never loaded / used. The other unhandled case are non default ciphers like xts(aes) which one might use. The brand new aes implementation aesni-intel is also not be considered. Most modules have _generic in their name if there is another implementation available and -arch if there is an assembly variant but I would prefer that this does not become part of an ABI where one can rely on. Would it be okay for you to change the way the crypto modules are picked and include the full set? >greetings, > jonas Sebastian -- To unsubscribe from this list: send the line "unsubscribe linux-crypto" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
hey, On 30/08/2009 Sebastian Andrzej Siewior wrote: > * Sebastian Andrzej Siewior | 2009-08-28 10:00:56 [+0200]: > > >>> the problem is not reproducible with a debian/unstable 2.6.30.6 kernel, > >>> even though it has cbc compiled as module as well. but if I recompile > >>> the same kernel sources with Celejars kernel .config, the problem > >>> occurs. thus it must be related to the kernel config in some way. > >It must be the kernel confing since I run .30.stable and it works. I try > >to look at it later. > > Your kernel config is fine, the problem is that the initramfs tools do > not copy all of the required modules into the initramfs. The missing > modles are: > - cryptomgr: that one is responsible to load the cbc and aes module and > bind them to cbc(aes) > - chainiv: that one creates IVs if the "user" does not specify one. > dm-crypt probably does not use that one but is required due to the way > crypto works atm. > - krng: provides random numbers and is required by chainiv. > > If you add those three to /etc/initramfs/modules than it should work. > > Could someone please look at initramfs to figure out why those three > modules are not copied in this reduced setup? the reason is simply that no other crypto modules define depends on the listed ones: # modinfo -F depends dm-crypt dm-mod,crypto_blkcipher # modinfo -F aes_generic sha256_generic cbc crypto_algapi crypto_hash crypto_algapi,crypto_blkcipher # modinfo -F crypto_blkcipher crypto_hash crypto_algapi crypto_algapi and even the new modules don't depend on each other: # modinfo -F cryptomgr crypto_hash,crypto_algapi,crypto_blkcipher,aead,pcompress # modinfo -F chainiv crypto_algapi,rng,crypto_wq,crypto_blkcipher # modinfo -F krng rng,crypto_algapi so the following depends should be added/changed: - chainiv should depend o 'krng' instead of 'rng' at least - maybe cipher modules like aes,serpent,... should depend on 'cryptomgr' instead of 'crypto_algapi' - crypto_algapi should depend on chainiv these changes are pure guesses, i don't know the details. but at least additional depends need to be defined for crypto modules, don't you think so? greetings, jonas -- To unsubscribe from this list: send the line "unsubscribe linux-crypto" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
On Sun, 30 Aug 2009 17:37:22 +0200 Sebastian Andrzej Siewior wrote: > * Sebastian Andrzej Siewior | 2009-08-28 10:00:56 [+0200]: > > >>> the problem is not reproducible with a debian/unstable 2.6.30.6 kernel, > >>> even though it has cbc compiled as module as well. but if I recompile > >>> the same kernel sources with Celejars kernel .config, the problem > >>> occurs. thus it must be related to the kernel config in some way. > >It must be the kernel confing since I run .30.stable and it works. I try > >to look at it later. > > Your kernel config is fine, the problem is that the initramfs tools do > not copy all of the required modules into the initramfs. The missing > modles are: > - cryptomgr: that one is responsible to load the cbc and aes module and > bind them to cbc(aes) > - chainiv: that one creates IVs if the "user" does not specify one. > dm-crypt probably does not use that one but is required due to the way > crypto works atm. > - krng: provides random numbers and is required by chainiv. > > If you add those three to /etc/initramfs/modules than it should work. Right your are! It now works, thanks very much. I guess the catch was that the working (stock Debian) kernel config built cryptomgr (CRYPTO_MANAGER and CRYPTO_MANAGER2) and krng (CRYPTO_RNG2, although not CRYPTO_RNG) and chainiv (CRYPTO_BLKCIPHER2, although not CRYPTO_BLKCIPHER - I have the impression that chainiv is now controlled by the BLKCIPHER options?) into the kernel, not as modules, so I wasn't hit by the initramfs failure until I built them as modules (I have a bit of a fetish for building everything as a module, especially when using initrd's :/). > Could someone please look at initramfs to figure out why those three > modules are not copied in this reduced setup? Celejar -- mailmin.sourceforge.net - remote access via secure (OpenPGP) email ssuds.sourceforge.net - A Simple Sudoku Solver and Generator -- To unsubscribe from this list: send the line "unsubscribe linux-crypto" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
* Sebastian Andrzej Siewior | 2009-08-28 10:00:56 [+0200]: >>> the problem is not reproducible with a debian/unstable 2.6.30.6 kernel, >>> even though it has cbc compiled as module as well. but if I recompile >>> the same kernel sources with Celejars kernel .config, the problem >>> occurs. thus it must be related to the kernel config in some way. >It must be the kernel confing since I run .30.stable and it works. I try >to look at it later. Your kernel config is fine, the problem is that the initramfs tools do not copy all of the required modules into the initramfs. The missing modles are: - cryptomgr: that one is responsible to load the cbc and aes module and bind them to cbc(aes) - chainiv: that one creates IVs if the "user" does not specify one. dm-crypt probably does not use that one but is required due to the way crypto works atm. - krng: provides random numbers and is required by chainiv. If you add those three to /etc/initramfs/modules than it should work. Could someone please look at initramfs to figure out why those three modules are not copied in this reduced setup? >Sebastian -- To unsubscribe from this list: send the line "unsubscribe linux-crypto" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
* Randy Dunlap | 2009-08-27 12:34:01 [-0700]: >> On 27/08/2009 Randy Dunlap wrote: >> > On Tue, 25 Aug 2009 19:58:52 -0400 Celejar wrote: >> > >> > > I'm having a pretty bizarre problem with my kernel crypto >> > > configuration. I need support for a bog standard LUKS (aes / >> > > cbc-essiv:sha256) / cryptsetup installation, but even after I enable >> > > virtually everything in the crypto section of the kernel configs, cbc >> > > fails to load. All the relevant modules are exist (dm-mod, dm-crypt, >> > > crypto_blkcipher, crypto_algapi, crypto_hash, aes_generic, >> > > sha256_generic), but even after modprobing / insmoding >> > > everything, /proc/crypto shows that aes and sha is there, but not cbc. >> > > >> > > The problem has been reproduced (using my kernel config) by Jonas >> > > Meurer, the Debian cryptsetup maintainer, so it's not just me ;). >> > > We've tried numerous different kernel versions in the .30 / .31 range. >> > > >> > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=541835 >> > > >> > > Does this mean that something else somewhere in the kernel needs to be >> > > configured but isn't, and the necessary dependency isn't properly >> > > declared? >> > > >> > > My config is at: >> > > http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=45;filename=config-2.6.31-rc6-lizzie-00042-gb2add73;att=1;bug=541835 >> > > >> > > [I'm not subscribed to lkml; please cc me on responses] >> the problem rather is that loading the cbc blockcipher module simply >> does nothing. the module is listed in /proc/modules, but the blockcipher >> is still missing from /proc/crypto. Yes. cbc(aes) is auto-generated once someone requests such a cipher. The block modes are not listed. >> the problem is not reproducible with a debian/unstable 2.6.30.6 kernel, >> even though it has cbc compiled as module as well. but if I recompile >> the same kernel sources with Celejars kernel .config, the problem >> occurs. thus it must be related to the kernel config in some way. It must be the kernel confing since I run .30.stable and it works. I try to look at it later. >--- >~Randy Sebastian -- To unsubscribe from this list: send the line "unsubscribe linux-crypto" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [pkg-cryptsetup-devel] Bug#541835: crypto configuration / dependencies broken
On Thu, 27 Aug 2009 20:35:01 +0200 Jonas Meurer wrote: > hey, > > On 27/08/2009 Randy Dunlap wrote: > > On Tue, 25 Aug 2009 19:58:52 -0400 Celejar wrote: > > > > > I'm having a pretty bizarre problem with my kernel crypto > > > configuration. I need support for a bog standard LUKS (aes / > > > cbc-essiv:sha256) / cryptsetup installation, but even after I enable > > > virtually everything in the crypto section of the kernel configs, cbc > > > fails to load. All the relevant modules are exist (dm-mod, dm-crypt, > > > crypto_blkcipher, crypto_algapi, crypto_hash, aes_generic, > > > sha256_generic), but even after modprobing / insmoding > > > everything, /proc/crypto shows that aes and sha is there, but not cbc. > > > > > > The problem has been reproduced (using my kernel config) by Jonas > > > Meurer, the Debian cryptsetup maintainer, so it's not just me ;). > > > We've tried numerous different kernel versions in the .30 / .31 range. > > > > > > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=541835 > > > > > > Does this mean that something else somewhere in the kernel needs to be > > > configured but isn't, and the necessary dependency isn't properly > > > declared? > > > > > > My config is at: > > > http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=45;filename=config-2.6.31-rc6-lizzie-00042-gb2add73;att=1;bug=541835 > > > > > > [I'm not subscribed to lkml; please cc me on responses] > > > > > > You could try/test a patch that was just posted: > > http://lkml.org/lkml/2009/8/27/249 > > that patch seems to be for ecryptfs only, while Celejar uses dm-crypt. > second, the patch just ensures that CRYPTO_ECB and CRYPTO_CBC are > selected along with ECRYPT_FS. but Celejar does have both CRYPTO_ECB and > CRYPTO_CBC selected. ack. > the problem rather is that loading the cbc blockcipher module simply > does nothing. the module is listed in /proc/modules, but the blockcipher > is still missing from /proc/crypto. > > the problem is not reproducible with a debian/unstable 2.6.30.6 kernel, > even though it has cbc compiled as module as well. but if I recompile > the same kernel sources with Celejars kernel .config, the problem > occurs. thus it must be related to the kernel config in some way. add cc: linux-crypto@vger.kernel.org -- maybe someone there can help out. --- ~Randy *** Remember to use Documentation/SubmitChecklist when testing your code *** -- To unsubscribe from this list: send the line "unsubscribe linux-crypto" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html