Re: [sage-devel] Re: Package installation/uninstallation confusion
On Tue, Nov 12, 2019 at 4:44 PM Simon King wrote: > > Dear Erik, > > On 2019-11-12, E. Madison Bray wrote: > > On Fri, Nov 8, 2019 at 11:06 PM Simon King wrote: > >> While we are at it: Currently, for building the package's documentation, > >> I use "$MAKE html" followed by some "cp -r" to copy the resulting html > >> files to $SAGE_SHARE/doc/p_group_cohomology. I guess this needs to be > >> fixed (in the sense of "rather use sdh_*) as well > > > > You can also use sdh_install which is a (partial) replacement for the > > "install" program, which does a few things in addition to just "cp -r" > > and tries to take care of platform-specific differences, if any. > > Concerning "$MAKE html": Is "sdh_make html" the right thing, or did I > miss something? And I'll have a look into sdh_install. I don't know what `make html` does in this case so it's hard to say for certain, but as you can see sdh_make doesn't do much other than call $MAKE and then call sdh_die if it fails :) -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAOTD34aaabx6i2SU4mANj9qg%3DYBwCO94j_x3h6JSEX1jbNyfDA%40mail.gmail.com.
[sage-devel] Re: Package installation/uninstallation confusion
Dear Erik, On 2019-11-12, E. Madison Bray wrote: > On Fri, Nov 8, 2019 at 11:06 PM Simon King wrote: >> While we are at it: Currently, for building the package's documentation, >> I use "$MAKE html" followed by some "cp -r" to copy the resulting html >> files to $SAGE_SHARE/doc/p_group_cohomology. I guess this needs to be >> fixed (in the sense of "rather use sdh_*) as well > > You can also use sdh_install which is a (partial) replacement for the > "install" program, which does a few things in addition to just "cp -r" > and tries to take care of platform-specific differences, if any. Concerning "$MAKE html": Is "sdh_make html" the right thing, or did I miss something? And I'll have a look into sdh_install. Thank you! Simon -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/qqek1e%2454e%241%40blaine.gmane.org.
Re: [sage-devel] Re: Package installation/uninstallation confusion
On Fri, Nov 8, 2019 at 11:06 PM Simon King wrote: > > Hi Erik, > > On 2019-11-08, E. Madison Bray wrote: > > Just to clarify, once more (and please reread my earlier message in > > this thread regarding what a "DESTDIR install" is and why that is > > used), > > This does not clarify it to me at all, I'm afraid. To start with, > in my very simple mind, I can not intuitively grasp that an > spkg-install script can possibly support a DESTDIR install if the > word "DESTDIR" does not appear in the spkg-install script. I don't think you have a very simple mind; I agree it's not obvious. But if you look in most spkg-install scripts you can see (as it appears you have below) that there are helper functions with names like "sdh_make_install", so I would think it would follow to look at what those do and how they are implemented if it's apparent that DESTDIR is not referenced explicitly in most spkg-install scripts. > > ... This allows sage-spkg, the script that actually > > handles building/installing SPKGs, to look in the $SAGE_DESTDIR > > directory that is created as a temporary installation root for the > > package, and generate a list of files that is installed by the package > > [2]. > > Do I understand correctly that sage-spkg creates a temporary directory > and assigns the environment variable DESTDIR to it, then spkg-install > does its job, and *if* all installation steps done in spkg-install take > the DESTDIR environment variable into account then the installation > is correctly staged there before it is transferred from there to its > final destination? And the content of the json file is obtained from > the contents of the stage area before it is erased? > > If that's the case then I'm confident that it should work to replace "$MAKE > install" with "sdh_make_install". Yep, you've understood correctly. > While we are at it: Currently, for building the package's documentation, > I use "$MAKE html" followed by some "cp -r" to copy the resulting html > files to $SAGE_SHARE/doc/p_group_cohomology. I guess this needs to be > fixed (in the sense of "rather use sdh_*) as well You can also use sdh_install which is a (partial) replacement for the "install" program, which does a few things in addition to just "cp -r" and tries to take care of platform-specific differences, if any. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAOTD34bTaRaGdtD%3D0TpgLShTkDf-mJ11-EzsONpFnOJPmqF4Ug%40mail.gmail.com.
[sage-devel] Re: Package installation/uninstallation confusion
I created https://trac.sagemath.org/ticket/28721 to add documentation for these to the developer's guide. Needs review. John On Thursday, November 7, 2019 at 8:27:32 AM UTC-8, John H Palmieri wrote: > > These shell functions are defined and documented in > SAGE_ROOT/build/bin/sage-dist-helpers. In my opinion they should also be > documented in the developer's guide, but they don't seem to be. > > > On Thursday, November 7, 2019 at 6:35:05 AM UTC-8, Simon King wrote: >> >> Hi Dima, >> >> On 2019-11-06, Dima Pasechnik wrote: >> > if you look at its spkg-install, you'd see that shd_install is not >> > used at some place, instead >> > >> > make install >> > >> > is done, something that won't write any JSON files, IMHO... >> >> Good that I leave comments in my code... >> >> The comment says that I used $MAKE install, because "sdh_make install >> got broken by trac ticket #24106". >> >> However, I was unaware that apparently "sdh_make install" ought to be >> replaced by "sdh_install". So, I guess I should test this. >> >> Or not? I see that far more packages use sdh_make_install (note the >> difference: What got broken apparently was sdh_make install with a blank >> space) than sdh_install. >> >> Can you point me to man pages, where sdh_install versus >> sdh_make_install is discussed? >> >> Best regards, >> Simon >> >> -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/7ccd2619-78e0-42f7-b4af-98ff6166ee21%40googlegroups.com.
[sage-devel] Re: Package installation/uninstallation confusion
Hi Erik, On 2019-11-08, E. Madison Bray wrote: > Just to clarify, once more (and please reread my earlier message in > this thread regarding what a "DESTDIR install" is and why that is > used), This does not clarify it to me at all, I'm afraid. To start with, in my very simple mind, I can not intuitively grasp that an spkg-install script can possibly support a DESTDIR install if the word "DESTDIR" does not appear in the spkg-install script. > ... This allows sage-spkg, the script that actually > handles building/installing SPKGs, to look in the $SAGE_DESTDIR > directory that is created as a temporary installation root for the > package, and generate a list of files that is installed by the package > [2]. Do I understand correctly that sage-spkg creates a temporary directory and assigns the environment variable DESTDIR to it, then spkg-install does its job, and *if* all installation steps done in spkg-install take the DESTDIR environment variable into account then the installation is correctly staged there before it is transferred from there to its final destination? And the content of the json file is obtained from the contents of the stage area before it is erased? If that's the case then I'm confident that it should work to replace "$MAKE install" with "sdh_make_install". While we are at it: Currently, for building the package's documentation, I use "$MAKE html" followed by some "cp -r" to copy the resulting html files to $SAGE_SHARE/doc/p_group_cohomology. I guess this needs to be fixed (in the sense of "rather use sdh_*) as well I opened #28711 for it. Best regards, Simon -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/qq4ot1%2462r3%241%40blaine.gmane.org.
Re: [sage-devel] Re: Package installation/uninstallation confusion
On Thu, Nov 7, 2019 at 6:26 PM Simon King wrote: > > If sdh_make_install and sdh_pip_install can share a single json file, > then there would be no need to split the package. I.e., the question is > whether "sdh_pip_install" will override the json file created by > "sdh_make_install", or whether it will append to it. Just to clarify, once more (and please reread my earlier message in this thread regarding what a "DESTDIR install" is and why that is used), these commands do not write *anything* to any JSON file. Their purpose is to wrap commands like "make" and "pip" with additional flags that are needed for building/installing Sage SPKGs. In the case of sdh_make_install, for example, as you can see in its definition [1]. This allows sage-spkg, the script that actually handles building/installing SPKGs, to look in the $SAGE_DESTDIR directory that is created as a temporary installation root for the package, and generate a list of files that is installed by the package [2]. This happens after the package's spkg-install script has been run, so this is entirely agnostic as to how the files got into $SAGE_DESTDIR, whether it was via some `make install`, `pip install`, `install`, `cp`, or some arbitrary combination of all of the above and more. I do believe this should be better documented and I don't appreciate being told "read the code" anymore than anyone else does. But as there is a lack of documentation currently that's the best thing one can do rather than speculate. Best, Erik [1] https://gitlab.com/sagemath/sage/blob/b90b558fc93ab78b4736d73535baf1f8053999df/build/bin/sage-dist-helpers#L190 [2] https://gitlab.com/sagemath/sage/blob/b90b558fc93ab78b4736d73535baf1f8053999df/build/bin/sage-spkg#L886 -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAOTD34am1T%3DC%3DJYaGAxMQzQ6c7H-K4_o9widcW_AOL3iBVUmWw%40mail.gmail.com.
Re: [sage-devel] Re: Package installation/uninstallation confusion
On Wed, Nov 6, 2019 at 6:33 PM Dima Pasechnik wrote: > > sdh_install installs a json file into > SAGE_LOCAL/var/lib/sage/installed/ > which are then used by unistallaller > > e.g. "make meataxe" produces SAGE_LOCAL/var/lib/sage/installed/meataxe-1.0.p0 > { > "package_name": "meataxe", > "package_version": "1.0.p0", > "install_date": "Wed Nov 6 17:24:57 GMT 2019", > "system_uname": "Linux hilbert 5.1.12-gentoo #1 SMP Fri Jun 21 > 19:27:38 BST 2019 x86_64 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz > GenuineIntel GNU/Linux", > "sage_version": "SageMath version 9.0.beta4, Release Date: 2019-11-02", > "test_result": "", > "files": [ > "bin/cfcomp", > "bin/chop", > ... > } > > I guess sdh_pip_install does not do this, I don't know. It does. As I wrote in my previous message all SPKGs should be installed using a DESTDIR scheme, and pip supports this (it's part of what sdh_pip_install does--ensuring that the correct DESTDIR scheme is used). > Anyhow, it would make perfect sense to split the package into two, as > you propose. I'm not sure. It depends on what "package" means in this case. From the perspective of Sage it shouldn't matter. > On Wed, Nov 6, 2019 at 3:55 PM Simon King wrote: > > > > Hi Eric, > > > > On 2019-11-06, E. Madison Bray wrote: > > >> However, I heard rumours that in order to make a Sage optional package > > >> uninstallable, one needs some new script analogous to spkg-install. > > >> > > >> Can someone give me a pointer on what should be done in that script and > > >> what tools (sdh_*) are available for it? > > > > > > That's not quite accurate. In general you do *not* need any > > > additional scripts, with some exception. > > > These packaging guidelines still need better documentation, but > > > essentially you need to make sure the package is built for what is > > > referred in GNU standards as a "DESTDIR install" [1]. This means that > > > the package is built for installation to some particular prefix (e.g. > > > `./configure --prefix=$SAGE_LOCAL` in autoconf terms), but can then be > > > *installed* with some additional path (the "DESTDIR") prepended to the > > > prefix. This is similar to providing an alternate root (e.g. prefix > > > is appended to DESTDIR instead of just "/"). > > > > Then what to do with p_group_cohomology? Its spkg-install script > > installs two sub-packages. Each of them, I think, follows the above > > scheme. However, Sage has no way of knowing that uninstalling > > p_group_cohomology means uninstalling two sub-packages. > > > > Perhaps one possibility is to split the current p_group_cohomology-3.3 > > into its two sub-packages, and make the first (which is autotoolized) > > a dependency for the second (which is a bunch of cython and python > > modules linked against the first sub-package. > > > > Say: > > - p_group_cohomology-3.4 shall only comprise what currently is the > > p/cython part of p_group_cohomology-3.3, and have modular_resolution > > as a dependency. > > - modular_resolution-1.0 shall only comprise what currently is the > > autotoolized c-library part of p_group_cohomology-3.3 > > > > Advantage: "make p_group_cohomology" would also install the dependency, > > hence, both packages get installed. And both packages can also be > > un-installed, I suppose. > > > > Disadvantage: The only purpose of modular_resolution-1.0 would be to > > serve as a dependency, it is (to my knowledge at least) of no > > independent use. > > > > Best regards, > > Simon > > > > -- > > You received this message because you are subscribed to the Google Groups > > "sage-devel" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to sage-devel+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/sage-devel/qpuqe5%247598%241%40blaine.gmane.org. > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sage-devel/CAAWYfq22%3DK%2Bss9jhCj2L57iBOe5i9_5iUE9Kdg-ZQO_SxwM50Q%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAOTD34Zftk%3Dkm-PaVtuw2_H1rozfcME%3DziVRG18t1netfgq_LA%40mail.gmail.com.
[sage-devel] Re: Package installation/uninstallation confusion
Hi Dima, On 2019-11-06, Dima Pasechnik wrote: > sdh_install installs a json file into > SAGE_LOCAL/var/lib/sage/installed/ > which are then used by unistallaller > ... > I guess sdh_pip_install does not do this, I don't know. No, I just checked: Those files of p_group_cohomology that got installed by sdh_pip_install are listed in the json file. Those installed by "$MAKE install" of course aren't. > Anyhow, it would make perfect sense to split the package into two, as > you propose. If sdh_make_install and sdh_pip_install can share a single json file, then there would be no need to split the package. I.e., the question is whether "sdh_pip_install" will override the json file created by "sdh_make_install", or whether it will append to it. Best regards, Simon -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/qq1k35%245jm3%241%40blaine.gmane.org.
[sage-devel] Re: Package installation/uninstallation confusion
Hi John, thank you! So, I'll try sdh_make_install (sdh_install just copies some files), keep using sdh_pip_install, and check what jason file results. Best regards, Simon On 2019-11-07, John H Palmieri wrote: > These shell functions are defined and documented in > SAGE_ROOT/build/bin/sage-dist-helpers. In my opinion they should also be > documented in the developer's guide, but they don't seem to be. > > > On Thursday, November 7, 2019 at 6:35:05 AM UTC-8, Simon King wrote: >> >> Hi Dima, >> >> On 2019-11-06, Dima Pasechnik > wrote: >> > if you look at its spkg-install, you'd see that shd_install is not >> > used at some place, instead >> > >> > make install >> > >> > is done, something that won't write any JSON files, IMHO... >> >> Good that I leave comments in my code... >> >> The comment says that I used $MAKE install, because "sdh_make install >> got broken by trac ticket #24106". >> >> However, I was unaware that apparently "sdh_make install" ought to be >> replaced by "sdh_install". So, I guess I should test this. >> >> Or not? I see that far more packages use sdh_make_install (note the >> difference: What got broken apparently was sdh_make install with a blank >> space) than sdh_install. >> >> Can you point me to man pages, where sdh_install versus >> sdh_make_install is discussed? >> >> Best regards, >> Simon >> >> > -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/qq1hv5%2413u7%241%40blaine.gmane.org.
[sage-devel] Re: Package installation/uninstallation confusion
These shell functions are defined and documented in SAGE_ROOT/build/bin/sage-dist-helpers. In my opinion they should also be documented in the developer's guide, but they don't seem to be. On Thursday, November 7, 2019 at 6:35:05 AM UTC-8, Simon King wrote: > > Hi Dima, > > On 2019-11-06, Dima Pasechnik > wrote: > > if you look at its spkg-install, you'd see that shd_install is not > > used at some place, instead > > > > make install > > > > is done, something that won't write any JSON files, IMHO... > > Good that I leave comments in my code... > > The comment says that I used $MAKE install, because "sdh_make install > got broken by trac ticket #24106". > > However, I was unaware that apparently "sdh_make install" ought to be > replaced by "sdh_install". So, I guess I should test this. > > Or not? I see that far more packages use sdh_make_install (note the > difference: What got broken apparently was sdh_make install with a blank > space) than sdh_install. > > Can you point me to man pages, where sdh_install versus > sdh_make_install is discussed? > > Best regards, > Simon > > -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/10e06ff0-55d0-425e-b03a-68096f688cb2%40googlegroups.com.
[sage-devel] Re: Package installation/uninstallation confusion
Hi Dima, On 2019-11-06, Dima Pasechnik wrote: > if you look at its spkg-install, you'd see that shd_install is not > used at some place, instead > > make install > > is done, something that won't write any JSON files, IMHO... Good that I leave comments in my code... The comment says that I used $MAKE install, because "sdh_make install got broken by trac ticket #24106". However, I was unaware that apparently "sdh_make install" ought to be replaced by "sdh_install". So, I guess I should test this. Or not? I see that far more packages use sdh_make_install (note the difference: What got broken apparently was sdh_make install with a blank space) than sdh_install. Can you point me to man pages, where sdh_install versus sdh_make_install is discussed? Best regards, Simon -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/qq1a2f%24865l%241%40blaine.gmane.org.
Re: [sage-devel] Re: Package installation/uninstallation confusion
On Wed, Nov 6, 2019 at 7:17 PM John H Palmieri wrote: > > Simon, you should look at the json file for p_group_cohomology to see if it > contains all of the installed files, or if indeed some are not listed. If > everything is there, there is no need to split it into two packages, unless > I'm missing something. if you look at its spkg-install, you'd see that shd_install is not used at some place, instead make install is done, something that won't write any JSON files, IMHO... > > > > On Wednesday, November 6, 2019 at 9:33:04 AM UTC-8, Dima Pasechnik wrote: >> >> sdh_install installs a json file into >> SAGE_LOCAL/var/lib/sage/installed/ >> which are then used by unistallaller >> >> e.g. "make meataxe" produces SAGE_LOCAL/var/lib/sage/installed/meataxe-1.0.p0 >> { >> "package_name": "meataxe", >> "package_version": "1.0.p0", >> "install_date": "Wed Nov 6 17:24:57 GMT 2019", >> "system_uname": "Linux hilbert 5.1.12-gentoo #1 SMP Fri Jun 21 >> 19:27:38 BST 2019 x86_64 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz >> GenuineIntel GNU/Linux", >> "sage_version": "SageMath version 9.0.beta4, Release Date: 2019-11-02", >> "test_result": "", >> "files": [ >> "bin/cfcomp", >> "bin/chop", >> ... >> } >> >> I guess sdh_pip_install does not do this, I don't know. >> >> Anyhow, it would make perfect sense to split the package into two, as >> you propose. >> >> On Wed, Nov 6, 2019 at 3:55 PM Simon King wrote: >> > >> > Hi Eric, >> > >> > On 2019-11-06, E. Madison Bray wrote: >> > >> However, I heard rumours that in order to make a Sage optional package >> > >> uninstallable, one needs some new script analogous to spkg-install. >> > >> >> > >> Can someone give me a pointer on what should be done in that script and >> > >> what tools (sdh_*) are available for it? >> > > >> > > That's not quite accurate. In general you do *not* need any >> > > additional scripts, with some exception. >> > > These packaging guidelines still need better documentation, but >> > > essentially you need to make sure the package is built for what is >> > > referred in GNU standards as a "DESTDIR install" [1]. This means that >> > > the package is built for installation to some particular prefix (e.g. >> > > `./configure --prefix=$SAGE_LOCAL` in autoconf terms), but can then be >> > > *installed* with some additional path (the "DESTDIR") prepended to the >> > > prefix. This is similar to providing an alternate root (e.g. prefix >> > > is appended to DESTDIR instead of just "/"). >> > >> > Then what to do with p_group_cohomology? Its spkg-install script >> > installs two sub-packages. Each of them, I think, follows the above >> > scheme. However, Sage has no way of knowing that uninstalling >> > p_group_cohomology means uninstalling two sub-packages. >> > >> > Perhaps one possibility is to split the current p_group_cohomology-3.3 >> > into its two sub-packages, and make the first (which is autotoolized) >> > a dependency for the second (which is a bunch of cython and python >> > modules linked against the first sub-package. >> > >> > Say: >> > - p_group_cohomology-3.4 shall only comprise what currently is the >> > p/cython part of p_group_cohomology-3.3, and have modular_resolution >> > as a dependency. >> > - modular_resolution-1.0 shall only comprise what currently is the >> > autotoolized c-library part of p_group_cohomology-3.3 >> > >> > Advantage: "make p_group_cohomology" would also install the dependency, >> > hence, both packages get installed. And both packages can also be >> > un-installed, I suppose. >> > >> > Disadvantage: The only purpose of modular_resolution-1.0 would be to >> > serve as a dependency, it is (to my knowledge at least) of no >> > independent use. >> > >> > Best regards, >> > Simon >> > >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "sage-devel" group. >> > To unsubscribe from this group and stop receiving emails from it, send an >> > email to sage-...@googlegroups.com. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msgid/sage-devel/qpuqe5%247598%241%40blaine.gmane.org. > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sage-devel/f30617cc-9e63-49b2-b097-b10682596c1b%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq3R-vdqXwBo9j5EvZJ%2BBMFycwrzGs7QQBpiM5JuGAbkew%40mail.gmail.com.
Re: [sage-devel] Re: Package installation/uninstallation confusion
Simon, you should look at the json file for p_group_cohomology to see if it contains all of the installed files, or if indeed some are not listed. If everything is there, there is no need to split it into two packages, unless I'm missing something. On Wednesday, November 6, 2019 at 9:33:04 AM UTC-8, Dima Pasechnik wrote: > > sdh_install installs a json file into > SAGE_LOCAL/var/lib/sage/installed/ > which are then used by unistallaller > > e.g. "make meataxe" produces > SAGE_LOCAL/var/lib/sage/installed/meataxe-1.0.p0 > { > "package_name": "meataxe", > "package_version": "1.0.p0", > "install_date": "Wed Nov 6 17:24:57 GMT 2019", > "system_uname": "Linux hilbert 5.1.12-gentoo #1 SMP Fri Jun 21 > 19:27:38 BST 2019 x86_64 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz > GenuineIntel GNU/Linux", > "sage_version": "SageMath version 9.0.beta4, Release Date: > 2019-11-02", > "test_result": "", > "files": [ > "bin/cfcomp", > "bin/chop", > ... > } > > I guess sdh_pip_install does not do this, I don't know. > > Anyhow, it would make perfect sense to split the package into two, as > you propose. > > On Wed, Nov 6, 2019 at 3:55 PM Simon King > wrote: > > > > Hi Eric, > > > > On 2019-11-06, E. Madison Bray > wrote: > > >> However, I heard rumours that in order to make a Sage optional > package > > >> uninstallable, one needs some new script analogous to spkg-install. > > >> > > >> Can someone give me a pointer on what should be done in that script > and > > >> what tools (sdh_*) are available for it? > > > > > > That's not quite accurate. In general you do *not* need any > > > additional scripts, with some exception. > > > These packaging guidelines still need better documentation, but > > > essentially you need to make sure the package is built for what is > > > referred in GNU standards as a "DESTDIR install" [1]. This means that > > > the package is built for installation to some particular prefix (e.g. > > > `./configure --prefix=$SAGE_LOCAL` in autoconf terms), but can then be > > > *installed* with some additional path (the "DESTDIR") prepended to the > > > prefix. This is similar to providing an alternate root (e.g. prefix > > > is appended to DESTDIR instead of just "/"). > > > > Then what to do with p_group_cohomology? Its spkg-install script > > installs two sub-packages. Each of them, I think, follows the above > > scheme. However, Sage has no way of knowing that uninstalling > > p_group_cohomology means uninstalling two sub-packages. > > > > Perhaps one possibility is to split the current p_group_cohomology-3.3 > > into its two sub-packages, and make the first (which is autotoolized) > > a dependency for the second (which is a bunch of cython and python > > modules linked against the first sub-package. > > > > Say: > > - p_group_cohomology-3.4 shall only comprise what currently is the > > p/cython part of p_group_cohomology-3.3, and have modular_resolution > > as a dependency. > > - modular_resolution-1.0 shall only comprise what currently is the > > autotoolized c-library part of p_group_cohomology-3.3 > > > > Advantage: "make p_group_cohomology" would also install the dependency, > > hence, both packages get installed. And both packages can also be > > un-installed, I suppose. > > > > Disadvantage: The only purpose of modular_resolution-1.0 would be to > > serve as a dependency, it is (to my knowledge at least) of no > > independent use. > > > > Best regards, > > Simon > > > > -- > > You received this message because you are subscribed to the Google > Groups "sage-devel" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to sage-...@googlegroups.com . > > To view this discussion on the web visit > https://groups.google.com/d/msgid/sage-devel/qpuqe5%247598%241%40blaine.gmane.org. > > > -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/f30617cc-9e63-49b2-b097-b10682596c1b%40googlegroups.com.
Re: [sage-devel] Re: Package installation/uninstallation confusion
sdh_install installs a json file into SAGE_LOCAL/var/lib/sage/installed/ which are then used by unistallaller e.g. "make meataxe" produces SAGE_LOCAL/var/lib/sage/installed/meataxe-1.0.p0 { "package_name": "meataxe", "package_version": "1.0.p0", "install_date": "Wed Nov 6 17:24:57 GMT 2019", "system_uname": "Linux hilbert 5.1.12-gentoo #1 SMP Fri Jun 21 19:27:38 BST 2019 x86_64 Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz GenuineIntel GNU/Linux", "sage_version": "SageMath version 9.0.beta4, Release Date: 2019-11-02", "test_result": "", "files": [ "bin/cfcomp", "bin/chop", ... } I guess sdh_pip_install does not do this, I don't know. Anyhow, it would make perfect sense to split the package into two, as you propose. On Wed, Nov 6, 2019 at 3:55 PM Simon King wrote: > > Hi Eric, > > On 2019-11-06, E. Madison Bray wrote: > >> However, I heard rumours that in order to make a Sage optional package > >> uninstallable, one needs some new script analogous to spkg-install. > >> > >> Can someone give me a pointer on what should be done in that script and > >> what tools (sdh_*) are available for it? > > > > That's not quite accurate. In general you do *not* need any > > additional scripts, with some exception. > > These packaging guidelines still need better documentation, but > > essentially you need to make sure the package is built for what is > > referred in GNU standards as a "DESTDIR install" [1]. This means that > > the package is built for installation to some particular prefix (e.g. > > `./configure --prefix=$SAGE_LOCAL` in autoconf terms), but can then be > > *installed* with some additional path (the "DESTDIR") prepended to the > > prefix. This is similar to providing an alternate root (e.g. prefix > > is appended to DESTDIR instead of just "/"). > > Then what to do with p_group_cohomology? Its spkg-install script > installs two sub-packages. Each of them, I think, follows the above > scheme. However, Sage has no way of knowing that uninstalling > p_group_cohomology means uninstalling two sub-packages. > > Perhaps one possibility is to split the current p_group_cohomology-3.3 > into its two sub-packages, and make the first (which is autotoolized) > a dependency for the second (which is a bunch of cython and python > modules linked against the first sub-package. > > Say: > - p_group_cohomology-3.4 shall only comprise what currently is the > p/cython part of p_group_cohomology-3.3, and have modular_resolution > as a dependency. > - modular_resolution-1.0 shall only comprise what currently is the > autotoolized c-library part of p_group_cohomology-3.3 > > Advantage: "make p_group_cohomology" would also install the dependency, > hence, both packages get installed. And both packages can also be > un-installed, I suppose. > > Disadvantage: The only purpose of modular_resolution-1.0 would be to > serve as a dependency, it is (to my knowledge at least) of no > independent use. > > Best regards, > Simon > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sage-devel/qpuqe5%247598%241%40blaine.gmane.org. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq22%3DK%2Bss9jhCj2L57iBOe5i9_5iUE9Kdg-ZQO_SxwM50Q%40mail.gmail.com.
[sage-devel] Re: Package installation/uninstallation confusion
Hi Eric, On 2019-11-06, E. Madison Bray wrote: >> However, I heard rumours that in order to make a Sage optional package >> uninstallable, one needs some new script analogous to spkg-install. >> >> Can someone give me a pointer on what should be done in that script and >> what tools (sdh_*) are available for it? > > That's not quite accurate. In general you do *not* need any > additional scripts, with some exception. > These packaging guidelines still need better documentation, but > essentially you need to make sure the package is built for what is > referred in GNU standards as a "DESTDIR install" [1]. This means that > the package is built for installation to some particular prefix (e.g. > `./configure --prefix=$SAGE_LOCAL` in autoconf terms), but can then be > *installed* with some additional path (the "DESTDIR") prepended to the > prefix. This is similar to providing an alternate root (e.g. prefix > is appended to DESTDIR instead of just "/"). Then what to do with p_group_cohomology? Its spkg-install script installs two sub-packages. Each of them, I think, follows the above scheme. However, Sage has no way of knowing that uninstalling p_group_cohomology means uninstalling two sub-packages. Perhaps one possibility is to split the current p_group_cohomology-3.3 into its two sub-packages, and make the first (which is autotoolized) a dependency for the second (which is a bunch of cython and python modules linked against the first sub-package. Say: - p_group_cohomology-3.4 shall only comprise what currently is the p/cython part of p_group_cohomology-3.3, and have modular_resolution as a dependency. - modular_resolution-1.0 shall only comprise what currently is the autotoolized c-library part of p_group_cohomology-3.3 Advantage: "make p_group_cohomology" would also install the dependency, hence, both packages get installed. And both packages can also be un-installed, I suppose. Disadvantage: The only purpose of modular_resolution-1.0 would be to serve as a dependency, it is (to my knowledge at least) of no independent use. Best regards, Simon -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/qpuqe5%247598%241%40blaine.gmane.org.
Re: [sage-devel] Re: Package installation/uninstallation confusion
On Wed, Nov 6, 2019 at 7:58 AM Simon King wrote: > > On 2019-11-05, E. Madison Bray wrote: > > The generated file `build/make/Makefile` is output by the > > `./configure` script. In fact, that's its main purpose. When > > switching branches in this case the best thing to do is to re-run > > `./configure`. > > Concerning the original question how to uninstall stuff: In principle it > should be possible to cleanly uninstall p_group_cohomology, because it > consists of one autotoolized and one pip-installed sub-package. That's not the issue here; the issue is that when you uninstall the package files, some of the build system still needs to be reset. This is indeed a problem, and there should be a better way to handle what packages are or are not installed without necessarily having to re-run configure, but I haven't given much thought to it. > However, I heard rumours that in order to make a Sage optional package > uninstallable, one needs some new script analogous to spkg-install. > > Can someone give me a pointer on what should be done in that script and > what tools (sdh_*) are available for it? That's not quite accurate. In general you do *not* need any additional scripts, with some exception. These packaging guidelines still need better documentation, but essentially you need to make sure the package is built for what is referred in GNU standards as a "DESTDIR install" [1]. This means that the package is built for installation to some particular prefix (e.g. `./configure --prefix=$SAGE_LOCAL` in autoconf terms), but can then be *installed* with some additional path (the "DESTDIR") prepended to the prefix. This is similar to providing an alternate root (e.g. prefix is appended to DESTDIR instead of just "/"). This allows the package to be installed to a temporary "staging" area that makes it easy, in an otherwise agnostic way, to see exactly what files are going to be installed by the package so that we can generate a manifest from it. Any packaging using standard autoconf tools supports DESTDIR. Most CMake packages do as well I think but I forget if there's something special one has to do. Pip also supports this. Packages with weird custom build systems may need to be patched or worked around in other ways to support this, but fortunately there are not too many like that in Sage anymore. In some cases it is necessary to perform additional package installation steps more complicated than just copying files. In this case one can write an spkg-postinst script to perform such steps. This is mentioned here [2]. However, if an "spkg-postinst" installs or creates any files in $SAGE_LOCAL, although not necessarily required in all cases it might be good to uninstall those files as well, in which case one needs to write an "spkg-postrm" script. Normally this isn't necessary though, and there are only a few examples of this in use.* [1] https://www.gnu.org/prep/standards/html_node/DESTDIR.html [2] https://doc.sagemath.org/html/en/developer/packaging.html#build-and-install-scripts * One bug I am aware of in our packages is that there is a common file "share/info/dir" which some packages that support "info" modify/append to during installation. When using DESTDIR install this means a new share/info/dir file gets created, and then copied over any existing "share/info/dir". To handle this properly requires postinst/postrm scripts which handle this infodir for any packages that use it. I had a branch a long time ago that took care of this but it was part of a larger patchbomb and never got merged I think. However, it's not a high-priority issue; I don't think a lot of people are using `info` or someone would have complained by now, maybe... -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAOTD34ZmRQTJegm5f2zOVvFjOhhncQinRaTvj6SwFiVax4xQkg%40mail.gmail.com.
[sage-devel] Re: Package installation/uninstallation confusion
On 2019-11-05, E. Madison Bray wrote: > The generated file `build/make/Makefile` is output by the > `./configure` script. In fact, that's its main purpose. When > switching branches in this case the best thing to do is to re-run > `./configure`. Concerning the original question how to uninstall stuff: In principle it should be possible to cleanly uninstall p_group_cohomology, because it consists of one autotoolized and one pip-installed sub-package. However, I heard rumours that in order to make a Sage optional package uninstallable, one needs some new script analogous to spkg-install. Can someone give me a pointer on what should be done in that script and what tools (sdh_*) are available for it? Best regards, Simon -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/qptqu0%2410hh%241%40blaine.gmane.org.