Re: [cmake-developers] [CPack] CPackDeb and fakeroot
Hi, I see that I'm a bit late to the party... Wasn't at my computer for the past few days. Now if you find a way to set root ownership in archive created by CPackDeb then every deb package will have those right. Again two solutions: - using system tar with the proper uid/gid options - using and tweaking libarchive directly I haven't used uid/gid options in libarchive so I'm not certain what it's capabilities are but if possible I'd definitely go with libarchive. My opinion (from the various bug report related to deb ownership) is that is OK since creating a deb including whatever non-root user in it is a mistake. I agree. So the proper way to go may be to use libarchive directly in CPackDeb to create tar in order to better control ownership of the created bits. This is definitely more work, but this looks the proper way to me. Ok, I just had a look to the cmlibarchive, cmakelib and cpackdeb sources, it's not such a big deal neither to support that functionality. One question remains though: in case of lzma or xy compression, cmake -E tar is not used, and system tar is used instead (lines 414 in cmCPackDebGenerator.cxx). This looks different from what you mentioned about the BSD ar format using ar_append at the end of the same function. LZMA and xy are supported by libarchive, so it should be possible to avoid any call to tar or cmake -E in all cases. Since lzma and xz are also using fakeroot and for consistency I'd go with with libarchive here as well. I'm guessing that the only reason these two compression formats were treated as special cases was because of lack of support in CMake in the past: http://www.cmake.org/Bug/view.php?id=13072 but this was resolved more than a year ago. Regards, Domen -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] [CPack] CPackDeb and fakeroot
Le 26/08/15 01:07, Eric Noulard a écrit : Right, but I am more concerned about the proper way of doing it and not the difficulty. From all this discussion, using fakeroot directly does not look to me as the right solution for having root in the tar, in the first place. So if we are also able to get rid of the fakeroot machinery in cpack, maybe it would be a better solution. I agree, then the question is should every deb package built by CPack being owned by root? Currently people making deb with CPack without having fakeroot installed get their package with current user owning. i.e. fakeroot is not ALWAYS used. Ok, but the created packages are not debian compliant, so we cannot really say that the generated artifacts are Debian packages. Now if you find a way to set root ownership in archive created by CPackDeb then every deb package will have those right. Again two solutions: - using system tar with the proper uid/gid options - using and tweaking libarchive directly My opinion (from the various bug report related to deb ownership) is that is OK since creating a deb including whatever non-root user in it is a mistake. I agree. So the proper way to go may be to use libarchive directly in CPackDeb to create tar in order to better control ownership of the created bits. This is definitely more work, but this looks the proper way to me. Ok, I just had a look to the cmlibarchive, cmakelib and cpackdeb sources, it's not such a big deal neither to support that functionality. One question remains though: in case of lzma or xy compression, cmake -E tar is not used, and system tar is used instead (lines 414 in cmCPackDebGenerator.cxx). This looks different from what you mentioned about the BSD ar format using ar_append at the end of the same function. LZMA and xy are supported by libarchive, so it should be possible to avoid any call to tar or cmake -E in all cases. Raffi -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] [CPack] CPackDeb and fakeroot
Le 25/08/15 16:48, Eric Noulard a écrit : Hi guys, Some of my thoughts about this one. Hi, thanks you for your quick reply! 2015-08-25 15:42 GMT+02:00 Raffi Enficiaud [snip] Yes but the may be interesting part should be in: /«BUILDDIR»/build_dir/_CPack_Packages/Linux/DEB/Deb.log which is not displayed in the previous file. Sorry for that, I forgot to tell that everything works ok without pbuilder. So I believe this is the issue, I will check next time I build this. Basically in my case, the debian/rule file calls cmake and cpack to generate the deb packages. pbuilder mimics a clean build environment that is used on Launchpad for isolating the build, and I believe (not sure) it calls fakeroot under the hood. pbuilder calls fakeroot for sure: In your log one can see: make[1]: Leaving directory `/«BUILDDIR»/build_dir' fakeroot debian/rules binary-arch touch debian/files cpack --version cpack version 3.3.20150824 [snip] The introduction of fakeroot usage for CPackDeb was made for that issue: http://public.kitware.com/Bug/view.php?id=10325 with the small follow-up you cited: http://public.kitware.com/Bug/view.php?id=13118 Right, but again the main idea seems to be able to set the uid/guid properly in the tar. fakeroot is not used unless it is detected as installed on the system. The question in your case is, why is fakeroot installed AND not working when used in pbuilder. My guess is that pbuilder is ALREADY calling fakeroot and that fakeroot cannot be called from within fakerootED environnement (nested fakeroot is unsupported). I also think this is the issue, nesting fakeroot is not supported. I would suggest checking (in CPackDeb generator code) whether if CPack has been called in a fakerootED environment. This can be done by checking whether if FAKEROOTKEY env var is defined or not. If fakeroot is already in action then one should not call fakeroot again (whatever the fakerootED user is). I did not know about this key, that would be a nice/clean resolution to the problem. The #13700 bugs referred hereafter confirms this kind of issue: http://public.kitware.com/Bug/view.php?id=13700 - unifying the tool used for creating the tar: from the code in cmCPackDebGenerator.cxx, some compression schemes use the native tar while some others use the cmake tar. This is explained in Source/CPack/cmCPackDebGenerator.cxx // NOTE: // A debian package .deb is simply an 'ar' archive. The only subtle difference // is that debian uses the BSD ar style archive whereas most Linux distro have // a GNU ar. // See http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=161593 for more info // Therefore we provide our own implementation of a BSD-ar: static int ar_append(const char*archive,const std::vectorstd::string files); I don't really known whether if this argument still apply nowadays and I let you read the discussion on bugs.debian.org http://bugs.debian.org. Ok. Let's not burn our fingers on this direction then :) What would be the best option for you? I let Domen answer for that, I was just jumping in to let you all know the bits of the story I remember. I believe it would address this issues already in the backlog (and help ppl deploy things on Launchpad with cmake): - http://public.kitware.com/Bug/view.php?id=13700 - http://public.kitware.com/Bug/view.php?id=11766 Like I said 13700 should be easy to fix by avoiding nested fakeroot call. 11766 is another story. I think that adding ownership option to cmake -E tar may be useful but not that urgent considering that most of these issues can be fixed in another easy mean. If I understand the issue well, their concern https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=1863 was about doing some experiments for having root/root uid/guid + some fancy block size. root/root is ok now with fakeroot, and they abandoned the block_size fancy options. The thing is that root/root implementation today is not fully ok due to the bug exposed on this thread (nested fakeroot). That said libarchive now seems to support BSD tar https://github.com/libarchive/libarchive/tree/master/tar and ownership using set_ownership(struct archive_write_disk *) API. so may be it's possible to get rid of the BSD tar implementation embedded in cmCPackDebGenerator.cxx and add appropriate calls to libarchive. That would be a solution, but it is hard for me to see the gain of touching that many files instead of having the nice FAKEROOTKEY detection. Thanks again! Raffi -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses:
Re: [cmake-developers] [CPack] CPackDeb and fakeroot
Le 25/08/15 16:48, Eric Noulard a écrit : I would suggest checking (in CPackDeb generator code) whether if CPack has been called in a fakerootED environment. This can be done by checking whether if FAKEROOTKEY env var is defined or not. If fakeroot is already in action then one should not call fakeroot again (whatever the fakerootED user is). Hmmm, unfortunately it does not work. This variable does not exist on the fakerooted shell. I think there is no good way to detect that we are in a fakerooted environment because their purpose is to mimic the commands as if the real root user is entering them (and introspection should not be possible by design). I see two technical solutions: - either using the system tar: this would not work in the case of cross compilation (and in some circumstances only) but would work in the other case. I believe that all the debian packaging tools are just using the system provided tar, so this should closely mimic the debian-packaging machinery better, - or using a CPACK variable to avoid having fakeroot executed I see several workarounds/hacks: - executing with fakeroot in cpack first and in case of failure falling back to run tar without fakeroot - detect if the current user is root already Raffi -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] [CPack] CPackDeb and fakeroot
Le 26/08/15 00:45, Eric Noulard a écrit : [snip] I see two technical solutions: - either using the system tar: this would not work in the case of cross compilation (and in some circumstances only) but would work in the other case. I believe that all the debian packaging tools are just using the system provided tar, so this should closely mimic the debian-packaging machinery better, Cross-compiling is one thing, cross packaging is another. I think trying to cross-package is a very hard task (unless you simply create a bare archive (tar, zip, etc...) Sorry I wanted to say cross-packaging. Still I think using the system default tar program is a good way to go. - or using a CPACK variable to avoid having fakeroot executed seems awkward to disable black magic with another black magic spell. I agree. I see several workarounds/hacks: - executing with fakeroot in cpack first and in case of failure falling back to run tar without fakeroot - detect if the current user is root already Detecting if user is already root doesn't seems to be such a big hack it should even be robust as well and should be a 2 line modfication in CPackDeb.cmake protecting it even detecting FAKEROOT alltogether. Right, but I am more concerned about the proper way of doing it and not the difficulty. From all this discussion, using fakeroot directly does not look to me as the right solution for having root in the tar, in the first place. So if we are also able to get rid of the fakeroot machinery in cpack, maybe it would be a better solution. Raffi -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] [CPack] CPackDeb and fakeroot
2015-08-25 23:52 GMT+02:00 Raffi Enficiaud raffi.enfici...@mines-paris.org : Le 25/08/15 16:48, Eric Noulard a écrit : I would suggest checking (in CPackDeb generator code) whether if CPack has been called in a fakerootED environment. This can be done by checking whether if FAKEROOTKEY env var is defined or not. If fakeroot is already in action then one should not call fakeroot again (whatever the fakerootED user is). Hmmm, unfortunately it does not work. This variable does not exist on the fakerooted shell. I think there is no good way to detect that we are in a fakerooted environment because their purpose is to mimic the commands as if the real root user is entering them (and introspection should not be possible by design). You are right the design of fakeroot is to mimic... I tested it using erk@capitaine:tmp$ fakeroot bash root@capitaine:tmp# echo $FAKEROOTKEY 489393277 but it does not seem to be set when fakerooting a simple command (outside a shell). I see two technical solutions: - either using the system tar: this would not work in the case of cross compilation (and in some circumstances only) but would work in the other case. I believe that all the debian packaging tools are just using the system provided tar, so this should closely mimic the debian-packaging machinery better, Cross-compiling is one thing, cross packaging is another. I think trying to cross-package is a very hard task (unless you simply create a bare archive (tar, zip, etc...) - or using a CPACK variable to avoid having fakeroot executed seems awkward to disable black magic with another black magic spell. I see several workarounds/hacks: - executing with fakeroot in cpack first and in case of failure falling back to run tar without fakeroot - detect if the current user is root already Detecting if user is already root doesn't seems to be such a big hack it should even be robust as well and should be a 2 line modfication in CPackDeb.cmake protecting it even detecting FAKEROOT alltogether. -- Eric -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] [CPack] CPackDeb and fakeroot
2015-08-26 0:56 GMT+02:00 Raffi Enficiaud raffi.enfici...@mines-paris.org: [...] I see several workarounds/hacks: - executing with fakeroot in cpack first and in case of failure falling back to run tar without fakeroot - detect if the current user is root already Detecting if user is already root doesn't seems to be such a big hack it should even be robust as well and should be a 2 line modfication in CPackDeb.cmake protecting it even detecting FAKEROOT alltogether. Right, but I am more concerned about the proper way of doing it and not the difficulty. From all this discussion, using fakeroot directly does not look to me as the right solution for having root in the tar, in the first place. So if we are also able to get rid of the fakeroot machinery in cpack, maybe it would be a better solution. I agree, then the question is should every deb package built by CPack being owned by root? Currently people making deb with CPack without having fakeroot installed get their package with current user owning. i.e. fakeroot is not ALWAYS used. Now if you find a way to set root ownership in archive created by CPackDeb then every deb package will have those right. My opinion (from the various bug report related to deb ownership) is that is OK since creating a deb including whatever non-root user in it is a mistake. So the proper way to go may be to use libarchive directly in CPackDeb to create tar in order to better control ownership of the created bits. This is definitely more work, but this looks the proper way to me. -- Eric -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
[cmake-developers] [CPack] CPackDeb and fakeroot
Hi Domen, Brad and CMake developers, I am trying to create my packages in Launchpad, which uses a pbuilder environment for building the packages, directly using CMake=3.3 (which contains the nice recent fixes). I got the following error when I execute the build of a source package from pbuilder: CPack Error: /build/cmake-AHTLnO/cmake-3.3travisci1/Source/CPack/cmCPackDebGenerator.cxx:483 Problem running tar command: /usr/bin/fakeroot /usr/bin/cmake -E tar czf data.tar.gz Please check /«BUILDDIR»/build_dir/_CPack_Packages/Linux/DEB/Deb.log for errors The full logs for the curious ppl may be found here: https://launchpadlibrarian.net/215456337/buildlog_ubuntu-trusty-amd64.yayi_0.8.8.1_BUILDING.txt.gz Basically in my case, the debian/rule file calls cmake and cpack to generate the deb packages. pbuilder mimics a clean build environment that is used on Launchpad for isolating the build, and I believe (not sure) it calls fakeroot under the hood. To my understanding fakeroot is a workaround to have the proper credentials in the tar of the .deb packages since the libarchive used in cmake was not allowing that at that time: - http://public.kitware.com/Bug/view.php?id=11766 - http://public.kitware.com/Bug/view.php?id=13118 - http://public.kitware.com/Bug/view.php?id=12901 I am wondering if, instead of always using fakeroot: - there is now the possibility to set the user in libarchive (even if it is the case, this would mean that cmake -E tar is able to understand those options) - if we can get rid of the fakeroot in some circumstances? eg. a variable that would avoid using the fakeroot or detecting that the user is already root - unifying the tool used for creating the tar: from the code in cmCPackDebGenerator.cxx, some compression schemes use the native tar while some others use the cmake tar. What would be the best option for you? I believe it would address this issues already in the backlog (and help ppl deploy things on Launchpad with cmake): - http://public.kitware.com/Bug/view.php?id=13700 - http://public.kitware.com/Bug/view.php?id=11766 Thanks for your feedbacks, Best, Raffi Enficiaud -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
