[sage-devel] Linking libraries with spkg-install.in

['jonatha...@googlemail.com' via sage-devel]

Hi, I think I made a mistake in https://trac.sagemath.org/ticket/31482, 
which might cause latte_int to fail building, if you use your systems cdd 
(which almost no one does, because the headers in cdd/ and cddlib/ are 
rejected currently).

However, it appears that we make this mistake all over spkg-install.in's?
We configure libaries with e.g. `--with-zlib="$SAGE_LOCAL"` 
unconditionally. Why would this succeed, if we use the systems `zlib`?

It appears to me that this only works, because people either use all the 
system packages possible or no packages at all. The above example seems to 
not come up, because people usually have libgd installed if they have zlib 
installed.

Jonathan

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/40139c6b-d765-4e75-9739-30f8079ab599n%40googlegroups.com.

Re: [sage-devel] Linking libraries with spkg-install.in

[François Bissey]

> On 19/03/2021, at 23:05, 'jonatha...@googlemail.com' via sage-devel 
>  wrote:
> 
> We configure libaries with e.g. `--with-zlib="$SAGE_LOCAL"` unconditionally. 
> Why would this succeed, if we use the systems `zlib`?
> 

It depends a bit on whether you use a properly written configure script or not.
Autotools tests and finds things by trying to compile things. If there is no
lib in SAGE_LOCAL it will not prevent a compilation against a system zlib to 
fail.
gcc -o test -I$SAGE_LOCAL/include -L$SAGE_LOCAL/lib test.c -lz
will look first in the SAGE_LOCAL locations but will still use the standard 
location
if nothing is found in the custom ones.

Where it will fail, is with custom scripts that work by trying to find a file 
explicitly.
Like checking whether $SAGE_LOCAL/lib/libz.so exists. Those are bad tests in 
the first 
place.

François

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/2BAF2668-6CD0-4849-8293-F3D2523DDE8A%40gmail.com.

Re: [sage-devel] Linking libraries with spkg-install.in

['Jonathan Kliem' via sage-devel]

On 19.03.21 11:13, François Bissey wrote:




On 19/03/2021, at 23:05, 'jonatha...@googlemail.com' via sage-devel 
 wrote:

We configure libaries with e.g. `--with-zlib="$SAGE_LOCAL"` unconditionally. 
Why would this succeed, if we use the systems `zlib`?


It depends a bit on whether you use a properly written configure script or not.
Autotools tests and finds things by trying to compile things. If there is no
lib in SAGE_LOCAL it will not prevent a compilation against a system zlib to 
fail.
gcc -o test -I$SAGE_LOCAL/include -L$SAGE_LOCAL/lib test.c -lz
will look first in the SAGE_LOCAL locations but will still use the standard 
location
if nothing is found in the custom ones.

Where it will fail, is with custom scripts that work by trying to find a file 
explicitly.
Like checking whether $SAGE_LOCAL/lib/libz.so exists. Those are bad tests in 
the first
place.

François

I see. What we do is definitely against the instructions of libgd's 
configure ("where to find the zlib library"). But checking the actual 
code, $SAGE_LOCAL is completely ignored in this specific case.


So this really is a cdd specific problem (because I want to set the 
preference according to the path and not according to which of 
`setoper.h`, `cdd/setoper.h`, `cddlib/setoper.h` works).


And I guess we usually just wait until this leads to problems somewhere 
(e.g. for iml we specify `-with-gmp-include="$SAGE_GMP_INCLUDE").


--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/5ca25fcb-24c4-a3f0-4e48-616c61d480f2%40gmail.com.

Re: [sage-devel] Linking libraries with spkg-install.in

['jonatha...@googlemail.com' via sage-devel]

Actually, the code I wrote for latte is better than I remembered. At some 
point I wanted to hardcode the location of the libraries.

But I guess, I abandoned this (maybe because someone told me, this was a 
bad idea).
jonatha...@googlemail.com schrieb am Freitag, 19. März 2021 um 11:38:46 
UTC+1:

> On 19.03.21 11:13, François Bissey wrote:
>
> >
> >> On 19/03/2021, at 23:05, 'jonatha...@googlemail.com' via sage-devel <
> sage-...@googlegroups.com> wrote:
> >>
> >> We configure libaries with e.g. `--with-zlib="$SAGE_LOCAL"` 
> unconditionally. Why would this succeed, if we use the systems `zlib`?
> >>
> > It depends a bit on whether you use a properly written configure script 
> or not.
> > Autotools tests and finds things by trying to compile things. If there 
> is no
> > lib in SAGE_LOCAL it will not prevent a compilation against a system 
> zlib to fail.
> > gcc -o test -I$SAGE_LOCAL/include -L$SAGE_LOCAL/lib test.c -lz
> > will look first in the SAGE_LOCAL locations but will still use the 
> standard location
> > if nothing is found in the custom ones.
> >
> > Where it will fail, is with custom scripts that work by trying to find a 
> file explicitly.
> > Like checking whether $SAGE_LOCAL/lib/libz.so exists. Those are bad 
> tests in the first
> > place.
> >
> > François
> >
> I see. What we do is definitely against the instructions of libgd's 
> configure ("where to find the zlib library"). But checking the actual 
> code, $SAGE_LOCAL is completely ignored in this specific case.
>
> So this really is a cdd specific problem (because I want to set the 
> preference according to the path and not according to which of 
> `setoper.h`, `cdd/setoper.h`, `cddlib/setoper.h` works).
>
> And I guess we usually just wait until this leads to problems somewhere 
> (e.g. for iml we specify `-with-gmp-include="$SAGE_GMP_INCLUDE").
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/4eadab4f-e717-4b66-a23d-ddb21c6d595bn%40googlegroups.com.

Re: [sage-devel] Memory leak in integer multiplication in sage?

[Dima Pasechnik]

On Thu, Mar 18, 2021 at 8:29 PM Volker Braun  wrote:
>
> This is presumably the same memory leak as in 
> https://trac.sagemath.org/ticket/31340

yes, and it can be fixed by just removing custom memory allocators in ZZ.


>
> On Thursday, March 18, 2021 at 7:21:14 PM UTC+1 m.derick...@gmail.com wrote:
>>
>> Hi Vincent,
>>
>> Thanks for testing and good that you realized that indeed looked fishy 
>> indeed.
>>
>> I think your extension of my code with multiple loops actually confirms that 
>> there is a problem even after warming up:
>>
>> the prints:
>>
>> memory usage 30k: 1771.8046875
>> memory usage 40k: 1771.80859375
>>
>> mean that you are using 1.7 GB memory extra after the 3th and the 4th loop 
>> (notice you chance the baseline of memory usage every loop so the 1.7 GB is 
>> not the total memory usage of sage, it is the new memory usage relative to 
>> the previous loop). So that in the end sage was using around 885+3*1777 MB = 
>> 6.1 GB of memory in your session.
>>
>> Also,  the largest object ever created are a range object of size 1 
>> (which in python 3 is actually consuming almost no memory since under the 
>> hood it behaves very much like xrange from python 2) and an srange object of 
>> size 3000. Neither of these should occupy an amount of memory in the GB 
>> range.
>>
>> Also one thing that is maybe easy to miss, I doubled the size of the loop 
>> between the first loop and the second loop, which explains the 885 vs 1771.
>>
>>
>> On Thursday, 18 March 2021 at 17:52:19 UTC+1 vdelecroix wrote:
>>>
>>> Sorry, your example indeed looks fishy.
>>>
>>> Le 18/03/2021 à 17:36, Vincent Delecroix a écrit :
>>> > Maarten, in your example your can not fairly compare the first
>>> > and second run. For example R is not allocated at your first
>>> > call of "mem = get_memory_usage()". Also Integers have a pool
>>> > which is likely not being filled at startup. Ignoring the first
>>> > run, I do not notice any difference.
>>> >
>>> > (both on system sage 9.2 and compiled sage 9.3.beta8 on archlinux)
>>> >
>>> > sage: M = 3000
>>> > : import gc
>>> > : _ = gc.collect()
>>> > : mem = get_memory_usage()
>>> > : for i in range(1):
>>> > : R = srange(M)
>>> > : _ = gc.collect()
>>> > : print("memory usage 10k:", get_memory_usage(mem))
>>> > : mem = get_memory_usage()
>>> > : for i in range(2):
>>> > : R = srange(M)
>>> > : _ = gc.collect()
>>> > : print("memory usage 20k:", get_memory_usage(mem))
>>> > : mem = get_memory_usage()
>>> > : for i in range(2):
>>> > : R = srange(M)
>>> > : _ = gc.collect()
>>> > : print("memory usage 30k:", get_memory_usage(mem))
>>> > : mem = get_memory_usage()
>>> > : for i in range(2):
>>> > : R = srange(M)
>>> > : _ = gc.collect()
>>> > : print("memory usage 40k:", get_memory_usage(mem))
>>> >
>>> > memory usage 10k: 885.4453125
>>> > memory usage 20k: 1771.9375
>>> > memory usage 30k: 1771.8046875
>>> > memory usage 40k: 1771.80859375
>>> >
>>> > Le 18/03/2021 à 17:00, Maarten Derickx a écrit :
>>> >> Hi Guys,
>>> >>
>>> >> Thanks for the replies. I think this is enough info to know that it
>>> >> happens
>>> >> on multiple systems and that it's not just the cocalc enhanced version of
>>> >> sage. I have created:
>>> >> https://trac.sagemath.org/ticket/31511#ticket for this.
>>> >>
>>> >> In the meantime I found the problem is already with the srange
>>> >> command. So
>>> >> the integer multiplication seems to be a red herring.
>>> >>
>>> >> ~$ sage
>>> >> ┌┐
>>> >> │ SageMath version 9.2, Release Date: 2020-10-24 │
>>> >> │ Create a "Sage Worksheet" file for the notebook interface. │
>>> >> │ Enhanced for CoCalc.   │
>>> >> │ Using Python 3.8.5. Type "help()" for help.│
>>> >> └┘
>>> >> sage: M = 3000
>>> >> : import gc
>>> >> : gc.collect()
>>> >> : mem = get_memory_usage()
>>> >> : for i in range(1):
>>> >> : R = srange(M)
>>> >> : gc.collect()
>>> >> : print("memory usage 10k:", get_memory_usage(mem))
>>> >> : mem = get_memory_usage()
>>> >> : for i in range(2):
>>> >> : R = srange(M)
>>> >> : gc.collect()
>>> >> : print("memory usage 20k:", get_memory_usage(mem))
>>> >> 434
>>> >> 0
>>> >> memory usage 10k: 884.4296875
>>> >> 0
>>> >> memory usage 20k: 1770.71875
>>> >>
>>> >> On Thursday, 18 March 2021 at 16:01:56 UTC+1 David Joyner wrote:
>>> >>
>>> >>> On Thu, Mar 18, 2021 at 6:56 AM Maarten Derickx 
>>> >>> wrote:
>>> >>>
>>>  Hi All,
>>> 
>>>  tldr: the bottom of this post contains example code of which I would
>>>  like
>>>  the results on some other systems.
>>> 
>>>  I recently encountered a memory leak in the relati

Re: [sage-devel] Linking libraries with spkg-install.in

[Matthias Koeppe]

On Friday, March 19, 2021 at 3:13:46 AM UTC-7 François Bissey wrote:

>
> > On 19/03/2021, at 23:05, 'jonatha...@googlemail.com' via sage-devel <
> sage-...@googlegroups.com> wrote: 
> > 
> > We configure libaries with e.g. `--with-zlib="$SAGE_LOCAL"` 
> unconditionally. Why would this succeed, if we use the systems `zlib`? 
>
> It depends a bit on whether you use a properly written configure script or 
> not. 
> Autotools tests and finds things by trying to compile things. If there is 
> no 
> lib in SAGE_LOCAL it will not prevent a compilation against a system zlib 
> to fail. 
> gcc -o test -I$SAGE_LOCAL/include -L$SAGE_LOCAL/lib test.c -lz 
> will look first in the SAGE_LOCAL locations but will still use the 
> standard location 
> if nothing is found in the custom ones. 
>
> Where it will fail, is with custom scripts that work by trying to find a 
> file explicitly. 
> Like checking whether $SAGE_LOCAL/lib/libz.so exists. Those are bad tests 
> in the first 
> place. 
>

FWIW, latte_int 1.7.6 has eliminated these bad tests (for presence of 
header files) 
in 
https://github.com/latte-int/latte/commit/2011c2ec0bda70b6f7e02bdd3ed02176d427927b

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/5d3a8440-05fb-4bf1-9bfc-c03664b3e39en%40googlegroups.com.

[sage-devel] Currently advised way to build sage from a fresh git clone

[Maarten Derickx]

Hi All,

I haven't had a development version of sage for some time, and now am 
setting one up again. In setting it up I found out that both my 
understanding of how to do a build from a fresh git clone for some time 
ago, as well as the current developer documentation don't seem to matching 
the current situation anymore.

Here is how I got a working sage installation:

git clone git://github.com/sagemath/sage.git #I realize I need to 
add git.sagemath.org later
make #gives an error that I should run configure first
./configure 
make

The ./configure used and second make to be unnecessary according to my 
memory and the documentation 
on 
https://doc.sagemath.org/html/en/developer/walk_through.html#obtaining-the-sage-source-code
 

Additionally the ./configure command is not available until you run make at 
least once, so is there a step that I am missing (or an argument to the 
first make) so that I don't get an error after the first make?

Thanks,
Maarten

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/5d2c1492-0011-44db-a25c-8daf6ae90558n%40googlegroups.com.

[sage-devel] Re: Currently advised way to build sage from a fresh git clone

[Maarten Derickx]

On Friday, 19 March 2021 at 20:07:15 UTC+1 Maarten Derickx wrote:

> Hi All,
>
> I haven't had a development version of sage for some time, and now am 
> setting one up again. In setting it up I found out that both my 
> understanding of how to do a build from a fresh git clone for some time 
> ago, as well as the current developer documentation don't seem to matching 
> the current situation anymore.
>
> Here is how I got a working sage installation:
>
> git clone git://github.com/sagemath/sage.git #I realize I need to add 
> git.sagemath.org later
> make #gives an error that I should run configure first
> ./configure 
> make
>
> The ./configure used and second make to be unnecessary according to my 
> memory and the documentation on 
> https://doc.sagemath.org/html/en/developer/walk_through.html#obtaining-the-sage-source-code
>  
>
> *The ./configure  and second make used to be unnecessary 

> Additionally the ./configure command is not available until you run make 
> at least once, so is there a step that I am missing (or an argument to the 
> first make) so that I don't get an error after the first make?
>
> Thanks,
> Maarten
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/5066a55f-024f-4231-acbe-ce6fb4247f0cn%40googlegroups.com.

[sage-devel] Re: Currently advised way to build sage from a fresh git clone

[Samuel Lelievre]

Running `configure` before `make` became mandatory after

- Sage Trac ticket 29316: Require ./configure before make
  https://trac.sagemath.org/ticket/29316

merged in Sage 9.2.beta2.

With that in mind, the steps are now as follows.

Clone and change to the obtained folder:
```
$ git clone ...
$ cd ...
```
Then either
```
$ make configure
$ ./configure
$ make
```
or
```
$ ./bootstrap
$ ./configure
$ make
```
or if you like quiet:
```
$ git clone ... -q
$ cd ...
$ ./bootstrap -q
$ ./configure -q
$ ./make -s V=0
```
Happy building Sage!  --Samuel

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/92f933a1-2a81-4dcf-bcd4-b4412c821185n%40googlegroups.com.

[sage-devel] Re: Currently advised way to build sage from a fresh git clone

[Matthias Koeppe]

Our README.md is up to date. Unfortunately, the section "Sage Development 
Process" in Developer's Guide is not.
https://trac.sagemath.org/ticket/29784 describes the task of updating this 
section - which needs help.

On Friday, March 19, 2021 at 12:07:15 PM UTC-7 m.derick...@gmail.com wrote:

> Hi All,
>
> I haven't had a development version of sage for some time, and now am 
> setting one up again. In setting it up I found out that both my 
> understanding of how to do a build from a fresh git clone for some time 
> ago, as well as the current developer documentation don't seem to matching 
> the current situation anymore.
>
> Here is how I got a working sage installation:
>
> git clone git://github.com/sagemath/sage.git #I realize I need to add 
> git.sagemath.org later
> make #gives an error that I should run configure first
> ./configure 
> make
>
> The ./configure used and second make to be unnecessary according to my 
> memory and the documentation on 
> https://doc.sagemath.org/html/en/developer/walk_through.html#obtaining-the-sage-source-code
>  
>
> Additionally the ./configure command is not available until you run make 
> at least once, so is there a step that I am missing (or an argument to the 
> first make) so that I don't get an error after the first make?
>
> Thanks,
> Maarten
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/5c337cdc-75d7-4194-b968-7cc403a4cd53n%40googlegroups.com.

Re: [sage-devel] Linking libraries with spkg-install.in

[Michael Orlitzky]

On Fri, 2021-03-19 at 09:57 -0700, Matthias Koeppe wrote:
> 
> 
> FWIW, latte_int 1.7.6 has eliminated these bad tests (for presence of 
> header files) 
> in 
> https://github.com/latte-int/latte/commit/2011c2ec0bda70b6f7e02bdd3ed02176d427927b
> 

The "with" options that allow you to specify a path actually go against
the GNU coding standards,

  https://www.gnu.org/prep/standards/html_node/Configuration.html

Those standards aren't gospel, but the authors of autoconf can probably
believed when they say that using --with to specify a path is out-of-
scope. Instead, the correct locations should be appended to the
compiler/linker search paths with -I or -L before building the software
"normally."

Build systems are one place where you'll never figure out the right
thing to do by looking at lots of examples.



-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/bd0e8e16662e7d8e3aa06c15831ed1064c49ba40.camel%40orlitzky.com.

[sage-devel] Re: https://wiki.sagemath.org/ReleaseTours/sage-9.3

[Matthias Koeppe]

Another call for contributions to the Release Tour for the upcoming 9.3 
release.

So far we have:

Sage 9.3 Release Tour 

   
   1. Polyhedral geometry 
   
  1. New features 
  
  2. Implementation improvements 
  

   2. Graph theory 
   
   3. Algebra 
   4. Package upgrades, system packages, user packages 
   

  1. Python 
  2. FLINT, Arb 
  
  3. GAP 
  4. Other package upgrades 
  
  5. SageMath on repology.org 
  
  6. System package information for OpenSUSE 
  

  7. Chapter on packages in the Sage reference manual 
  

  8. For developers and packagers: Version constraints for Python 
  packages 
  

  9. For developers: Setting up Python packages from PyPI as Sage 
  packages 
  

   5. Cleaning of the Sage codebase to conform to best practices 
   

   6. Modularization and packaging of sagelib 
   

   7. Configuration changes 
   
  1. Editable ("in-place", "develop") installs of the Sage library 
  

  2. Use of system Jupyter notebook / JupyterLab 
  

   8. Tickets 
   9. Availability of Sage 9.3 and installation help 
   


(Some sections are stubs only)

On Monday, January 18, 2021 at 10:11:09 AM UTC-8 Matthias Koeppe wrote:

> On Saturday, December 26, 2020 at 10:54:42 PM UTC-8 Matthias Koeppe wrote:
>
>> It's time to start preparing
>> https://wiki.sagemath.org/ReleaseTours/sage-9.3
>>
>
> So far we have:
>
>1. Package upgrades, system packages 
>
> 
>   1. FLINT, Arb 
>   
>   2. GAP 
>   3. Other package upgrades 
>   
>   4. System package information for OpenSUSE 
>   
> 
>   5. Chapter on packages in the Sage reference manual 
>   
> 
>   6. For developers and packagers: Version constraints for Python 
>   packages 
>   
> 
>   7. For developers: Setting up Python packages from PyPI as Sage 
>   packages 
>   
> 
>2. Modularization and packaging of sagelib 
>
> 
>3. Use of system Jupyter notebook / JupyterLab 
>
> 
>4. Tickets 
>5. Availability of Sage 9.3 and installation help 
>
> 

Re: [sage-devel] Linking libraries with spkg-install.in

[Matthias Koeppe]

On Friday, March 19, 2021 at 3:38:26 PM UTC-7 Michael Orlitzky wrote:

> The "with" options that allow you to specify a path actually go against 
> the GNU coding standards, 
>
> https://www.gnu.org/prep/standards/html_node/Configuration.html 
>
> Those standards aren't gospel, but the authors of autoconf can probably 
> believed when they say that using --with to specify a path is out-of- 
> scope. 
>

Nice find. For additional amusement, I tracked down the version of the GNU 
Coding Standards in which this paragraph was added: Apparently between 1992 
and 1994. 
https://cvs.savannah.gnu.org/viewvc/gnustandards/gnustandards/standards.texi?r1=1.2&r2=1.3


-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/sage-devel/e98c6b97-12a5-4b2e-a050-3ca694c1c3b5n%40googlegroups.com.