[sage-devel] Re: Git (and "git trac") in the developer guide heads up

[Ralf Stephan]

On Sunday, April 6, 2014 11:17:12 PM UTC+2, Volker Braun wrote:
>
> Git cheat sheet: 
> http://boxen.math.washington.edu/home/vbraun/doc/git-cheat-sheet.pdf 
>

$ git trac checkout 13853
Loading ticket #13853...
Checking out Trac #13853 remote branch public/13853 -> local branch 
t/13853/public/13853...
Traceback (most recent call last):
...
git_trac.git_error.GitError: git returned with non-zero exit code (128) 
when executing "git branch --set-upstream-to remotes/trac/public/13853"
STDERR: fatal: Not a valid object name: 'remotes/trac/public/13853'.

 remotes/trac?

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Problem installing sagecell

[Andrey Novoseltsev]

On Sunday, 6 April 2014 15:00:13 UTC-6, Jesus Escribano wrote:
>
> Hello
> i'm trying a new installation, and I got exactly the same error. 
> Any idea?
>
> Jesús
>
>
I had  to do

./sage -sh -c "easy_install backports.ssl-match-hostname"

before installing the package - this way this thing is downloaded and 
installed, while during installation of the package extra downloads are 
prohibited.

Andrey



> El lunes, 3 de marzo de 2014 22:21:04 UTC+1, Andrey Novoseltsev escribió:
>>
>> On Saturday, 1 March 2014 19:39:13 UTC-7, jason wrote:
>>>
>>> I just updated the sagecell spkg: 
>>>
>>> http://boxen.math.washington.edu/home/jason/sagecell-spkg/sagecell-2014-03-01.spkg
>>>  
>>>
>>> I also updated my sage sagecell branch to 6.1.1. 
>>>
>>> Jason 
>>>
>>>
>>>
>> Doing exactly same things as before, I cannot use this package due to the 
>> blocked attempts to download extra stuff. Are there new dependencies? Below 
>> is the log of spkg installation.
>>
>> Thank you!
>> Andrey
>>
>>
>> Attempting to download package sagecell-2014-03-01
>> >>> Trying to download 
>> http://sage.math.washington.edu/home/jason/sagecell-spkg/sagecell-2014-03-01.spkg
>> []
>> sagecell-2014-03-01
>> 
>> Extracting package /home/sc_serv/sage/upstream/sagecell-2014-03-01.spkg
>> -rw-r--r-- 1 sc_serv sc_serv 202980026 Mar  3 14:02 
>> /home/sc_serv/sage/upstream/sagecell-2014-03-01.spkg
>> Finished extraction
>> 
>> Host system:
>> Linux sagecell_new 3.10-0.bpo.3-amd64 #1 SMP Debian 3.10.11-1~bpo70+1 
>> (2013-09-24) x86_64 GNU/Linux
>> 
>> C compiler: gcc
>> C compiler version:
>> Using built-in specs.
>> COLLECT_GCC=gcc
>> COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.7/lto-wrapper
>> Target: x86_64-linux-gnu
>> Configured with: ../src/configure -v --with-pkgversion='Debian 4.7.2-5' 
>> --with-bugurl=file:///usr/share/doc/gcc-4.7/README.Bugs 
>> --enable-languages=c,c++,go,fortran,objc,obj-c++ --prefix=/usr 
>> --program-suffix=-4.7 --enable-shared --enable-linker-build-id 
>> --with-system-zlib --libexecdir=/usr/lib --without-included-gettext 
>> --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.7 
>> --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu 
>> --enable-libstdcxx-debug --enable-libstdcxx-time=yes 
>> --enable-gnu-unique-object --enable-plugin --enable-objc-gc 
>> --with-arch-32=i586 --with-tune=generic --enable-checking=release 
>> --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
>> Thread model: posix
>> gcc version 4.7.2 (Debian 4.7.2-5) 
>> 
>> Installing python dependencies
>> Processing ecdsa-0.10.tar.gz
>> Writing /tmp/easy_install-YfqiA5/ecdsa-0.10/setup.cfg
>> Running ecdsa-0.10/setup.py -q bdist_egg --dist-dir 
>> /tmp/easy_install-YfqiA5/ecdsa-0.10/egg-dist-tmp-ATGBUl
>> zip_safe flag not set; analyzing archive contents...
>> ecdsa 0.10 is already the active version in easy-install.pth
>>
>> Installed 
>> /home/sc_serv/sage/local/lib/python2.7/site-packages/ecdsa-0.10-py2.7.egg
>> Processing dependencies for ecdsa==0.10
>> Finished processing dependencies for ecdsa==0.10
>> Processing paramiko-1.12.2.tar.gz
>> Writing /tmp/easy_install-CGnS5d/paramiko-1.12.2/setup.cfg
>> Running paramiko-1.12.2/setup.py -q bdist_egg --dist-dir 
>> /tmp/easy_install-CGnS5d/paramiko-1.12.2/egg-dist-tmp-E03k9t
>> zip_safe flag not set; analyzing archive contents...
>> paramiko 1.12.2 is already the active version in easy-install.pth
>>
>> Installed 
>> /home/sc_serv/sage/local/lib/python2.7/site-packages/paramiko-1.12.2-py2.7.egg
>> Processing dependencies for paramiko==1.12.2
>> Finished processing dependencies for paramiko==1.12.2
>> Processing tornado-3.2.tar.gz
>> Writing /tmp/easy_install-vZz7m3/tornado-3.2/setup.cfg
>> Running tornado-3.2/setup.py -q bdist_egg --dist-dir 
>> /tmp/easy_install-vZz7m3/tornado-3.2/egg-dist-tmp-qosdF5
>> zip_safe flag not set; analyzing archive contents...
>> tornado.autoreload: module references __file__
>> tornado.simple_httpclient: module references __file__
>> tornado.testing: module references __file__
>> tornado.test.httpserver_test: module references __file__
>> tornado.test.iostream_test: module references __file__
>> tornado.test.locale_test: module references __file__
>> tornado.test.options_test: module references __file__
>> tornado.test.template_test: module references __file__
>> tornado.test.web_test: module references __file__
>> tornado 3.2 is already the active version in easy-install.pth
>>
>> Installed 
>> /home/sc_serv/sage/local/lib/python2.7/site-packages/tornado-3.2-py2.7-linux-x86_64.egg
>> Processing dependencies for tornado==3.2
>> Searching for backports.ssl-match-hostname
>>
>> Link to 
>> http://pypi.python.org/simple/backports.ssl_match_

Re: [sage-devel] Git (and "git trac") in the developer guide heads up

[William Stein]

On Sun, Apr 6, 2014 at 2:17 PM, Volker Braun  wrote:
> I'm changing the developer guide to push people towards git + my "git trac"
> subcommand instead of the "sage -dev" scripts. Nothing actually changes as
> far as functionality goes, you can still use the dev scripts or just 100%
> plain git.

Awesome!   I was looking at the dev guide, since I'm going to teach
Sage+Git dev in my Sage class this quarter, and quickly wished that it
was more oriented toward your 'git + my "git trac"' instead of "sage
-dev" scripts.

William

>
> As Karl-Dieter pointed out, the updated developer guide not a fair and
> balanced review of all possible ways to interact with git. Rather, I want to
> clearly steer people in a particular direction. I did a lot of work on the
> dev scripts myself, so I'm not trying to diss anybody's work. But it hasn't
> really attracted a big following or any further development.
>
> Git cheat sheet:
> http://boxen.math.washington.edu/home/vbraun/doc/git-cheat-sheet.pdf
>
> Corresponding trac ticket is http://trac.sagemath.org/16030 (not finished)
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+unsubscr...@googlegroups.com.
> To post to this group, send email to sage-devel@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



-- 
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Building Sage with Cygwin 32 bit

[Jean-Pierre Flori]

or without the -c, don't remember.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Building Sage with Cygwin 32 bit

[Jean-Pierre Flori]

On Sunday, April 6, 2014 11:34:45 PM UTC+2, Evan Oman wrote:
>
> Alright so total noob question here but how do I make these changes to the 
> rpy2 files?
>
> I tried unpacking the tarball in the upstream folder(??), made the 
> appropriate changes, and then repackaged it but now I get a checksum error. 
> Is there some sort of touch or other command I need to use to update the 
> checksums? Or am I going about this in the entirely wrong way?
>

Go to the root of your sage install and run:
./sage -sh -c sage-fix-pkg-checksums

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Building Sage with Cygwin 32 bit

[Evan Oman]

Alright so total noob question here but how do I make these changes to the 
rpy2 files?

I tried unpacking the tarball in the upstream folder(??), made the 
appropriate changes, and then repackaged it but now I get a checksum error. 
Is there some sort of touch or other command I need to use to update the 
checksums? Or am I going about this in the entirely wrong way?

On Friday, April 4, 2014 3:09:16 PM UTC-5, Jean-Pierre Flori wrote:
>
>
>
> On Friday, April 4, 2014 10:03:44 PM UTC+2, Jean-Pierre Flori wrote:
>>
>> IIRC, you should modify lines like the following one:
>>
>> https://bitbucket.org/lgautier/rpy2/src/a66387e12e2419141dadb7464ea9205595ef89af/rpy/rinterface/na_values.c?at=default#cl-206
>> to include __CYGWIN__ in addition to Win32 and Win64.
>>
> Same in this file:
>
> https://bitbucket.org/lgautier/rpy2/src/a66387e12e2419141dadb7464ea9205595ef89af/rpy/rinterface/_rinterface.c?at=default#cl-3385
> (only for the tp_base stuff I'd say) 
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Git (and "git trac") in the developer guide heads up

[Volker Braun]

I'm changing the developer guide to push people towards git + my "git trac" 
subcommand instead of the "sage -dev" scripts. Nothing actually changes as 
far as functionality goes, you can still use the dev scripts or just 100% 
plain git. 

As Karl-Dieter pointed out, the updated developer guide not a fair and 
balanced review of all possible ways to interact with git. Rather, I want 
to clearly steer people in a particular direction. I did a lot of work on 
the dev scripts myself, so I'm not trying to diss anybody's work. But it 
hasn't really attracted a big following or any further development. 

Git cheat sheet: 
http://boxen.math.washington.edu/home/vbraun/doc/git-cheat-sheet.pdf

Corresponding trac ticket is http://trac.sagemath.org/16030 (not finished)

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: massive memory usage generating posets...

[Nathann Cohen]

Yo !

> It's not "self" that matters, it's the result returned by the cached method
> that shouldn't contain any UniqueRepresentation object that was constructed
> using self. That's expensive to check.

Oh. I see :-/

Nathann

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: massive memory usage generating posets...

[Nils Bruin]

On Sunday, April 6, 2014 1:35:17 PM UTC-7, Nathann Cohen wrote:
>
> No, I meant... Can't we add some parameter to cached_method, so that 
> it could not be applied to an object which inherits from 
> UniqueRepresentation (if this is the problem) ? 
>
> something like 
>
> def cached_method(self, function, 
> check_that_self_if_not_unique_representation=True): 
> if check_that_self_if_not_unique_representation and 
> isinstance(self,UniqueRepresentation): 
> raise RuntimeError("This will cause memory leaks ! If you know 
> what you are doing, disable this test") 
>

It's not "self" that matters, it's the result returned by the cached method 
that shouldn't contain any UniqueRepresentation object that was constructed 
using self. That's expensive to check.

As a debugging tool, one could rig cached_method to do a check like that, 
though. It would involve a reverse lookup of construction parameters for 
all UniqueRepresentation objects found in the return value.
 

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Problem installing sagecell

[Jesus Escribano]

Hello
i'm trying a new installation, and I got exactly the same error. 
Any idea?

Jesús


El lunes, 3 de marzo de 2014 22:21:04 UTC+1, Andrey Novoseltsev escribió:
>
> On Saturday, 1 March 2014 19:39:13 UTC-7, jason wrote:
>>
>> I just updated the sagecell spkg: 
>>
>> http://boxen.math.washington.edu/home/jason/sagecell-spkg/sagecell-2014-03-01.spkg
>>  
>>
>> I also updated my sage sagecell branch to 6.1.1. 
>>
>> Jason 
>>
>>
>>
> Doing exactly same things as before, I cannot use this package due to the 
> blocked attempts to download extra stuff. Are there new dependencies? Below 
> is the log of spkg installation.
>
> Thank you!
> Andrey
>
>
> Attempting to download package sagecell-2014-03-01
> >>> Trying to download 
> http://sage.math.washington.edu/home/jason/sagecell-spkg/sagecell-2014-03-01.spkg
> []
> sagecell-2014-03-01
> 
> Extracting package /home/sc_serv/sage/upstream/sagecell-2014-03-01.spkg
> -rw-r--r-- 1 sc_serv sc_serv 202980026 Mar  3 14:02 
> /home/sc_serv/sage/upstream/sagecell-2014-03-01.spkg
> Finished extraction
> 
> Host system:
> Linux sagecell_new 3.10-0.bpo.3-amd64 #1 SMP Debian 3.10.11-1~bpo70+1 
> (2013-09-24) x86_64 GNU/Linux
> 
> C compiler: gcc
> C compiler version:
> Using built-in specs.
> COLLECT_GCC=gcc
> COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.7/lto-wrapper
> Target: x86_64-linux-gnu
> Configured with: ../src/configure -v --with-pkgversion='Debian 4.7.2-5' 
> --with-bugurl=file:///usr/share/doc/gcc-4.7/README.Bugs 
> --enable-languages=c,c++,go,fortran,objc,obj-c++ --prefix=/usr 
> --program-suffix=-4.7 --enable-shared --enable-linker-build-id 
> --with-system-zlib --libexecdir=/usr/lib --without-included-gettext 
> --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.7 
> --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu 
> --enable-libstdcxx-debug --enable-libstdcxx-time=yes 
> --enable-gnu-unique-object --enable-plugin --enable-objc-gc 
> --with-arch-32=i586 --with-tune=generic --enable-checking=release 
> --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
> Thread model: posix
> gcc version 4.7.2 (Debian 4.7.2-5) 
> 
> Installing python dependencies
> Processing ecdsa-0.10.tar.gz
> Writing /tmp/easy_install-YfqiA5/ecdsa-0.10/setup.cfg
> Running ecdsa-0.10/setup.py -q bdist_egg --dist-dir 
> /tmp/easy_install-YfqiA5/ecdsa-0.10/egg-dist-tmp-ATGBUl
> zip_safe flag not set; analyzing archive contents...
> ecdsa 0.10 is already the active version in easy-install.pth
>
> Installed 
> /home/sc_serv/sage/local/lib/python2.7/site-packages/ecdsa-0.10-py2.7.egg
> Processing dependencies for ecdsa==0.10
> Finished processing dependencies for ecdsa==0.10
> Processing paramiko-1.12.2.tar.gz
> Writing /tmp/easy_install-CGnS5d/paramiko-1.12.2/setup.cfg
> Running paramiko-1.12.2/setup.py -q bdist_egg --dist-dir 
> /tmp/easy_install-CGnS5d/paramiko-1.12.2/egg-dist-tmp-E03k9t
> zip_safe flag not set; analyzing archive contents...
> paramiko 1.12.2 is already the active version in easy-install.pth
>
> Installed 
> /home/sc_serv/sage/local/lib/python2.7/site-packages/paramiko-1.12.2-py2.7.egg
> Processing dependencies for paramiko==1.12.2
> Finished processing dependencies for paramiko==1.12.2
> Processing tornado-3.2.tar.gz
> Writing /tmp/easy_install-vZz7m3/tornado-3.2/setup.cfg
> Running tornado-3.2/setup.py -q bdist_egg --dist-dir 
> /tmp/easy_install-vZz7m3/tornado-3.2/egg-dist-tmp-qosdF5
> zip_safe flag not set; analyzing archive contents...
> tornado.autoreload: module references __file__
> tornado.simple_httpclient: module references __file__
> tornado.testing: module references __file__
> tornado.test.httpserver_test: module references __file__
> tornado.test.iostream_test: module references __file__
> tornado.test.locale_test: module references __file__
> tornado.test.options_test: module references __file__
> tornado.test.template_test: module references __file__
> tornado.test.web_test: module references __file__
> tornado 3.2 is already the active version in easy-install.pth
>
> Installed 
> /home/sc_serv/sage/local/lib/python2.7/site-packages/tornado-3.2-py2.7-linux-x86_64.egg
> Processing dependencies for tornado==3.2
> Searching for backports.ssl-match-hostname
>
> Link to 
> http://pypi.python.org/simple/backports.ssl_match_hostname/***BLOCKED*** by 
> --allow-hosts
>
>
> Link to 
> http://pypi.python.org/simple/backports.ssl-match-hostname/***BLOCKED*** by 
> --allow-hosts
>
> Couldn't find index page for 'backports.ssl_match_hostname' (maybe 
> misspelled?)
> Scanning index of all packages (this may take a while)
>
> Link to http://pypi.python.org/simple/ ***BLOCKED*** by --allow-hosts
>
> No local packages or download links found for 

Re: [sage-devel] Re: massive memory usage generating posets...

[Nathann Cohen]

Helloo !!!

> If you mean by "automatic": write a script to create a lot of them and throw
> them away and see if it leaks then, yes. Otherwise: this really is a
> non-local program logic problem, so detecting it is hard.

No, I meant... Can't we add some parameter to cached_method, so that
it could not be applied to an object which inherits from
UniqueRepresentation (if this is the problem) ?

something like

def cached_method(self, function,
check_that_self_if_not_unique_representation=True):
if check_that_self_if_not_unique_representation and
isinstance(self,UniqueRepresentation):
raise RuntimeError("This will cause memory leaks ! If you know
what you are doing, disable this test")
...

This way it could only be used when developers DID think hard of what
they did, and not by mistake, which produces leaks like this one.

> Yep. The problem is:  UniqueRepresentation implements necessarily a GLOBAL
> cache and hence will hold strong references to the creation parameters. It's
> very unfortunate and I think if we had thought about the consequences of
> that a little more, we wouldn't have gone with this design choice at all.
> It's just too hard to work with. But now we're stuck with it.

I hated UniqueRepresentation for years already. I would be glad to see
it replaced by anything that makes sense. Please don't "stick with it"
if we encounter leaks like that as a result. We can change things and
improve them.

> UniqueRepresentation does try to alleviate the problems a little bit by
> using a WeakValueDictionary for its cache, but as you see: Construction
> parameter passed to a UniqueRepresentation constructor should NEVER hold a
> strong reference to the resulting object, because that creates a memory
> leak. Ideally, construction parameters should just not hold references at
> all, but that's unfortunately infeasible.
>
> Caching a routine that's simply a wrapper around a UniqueRepresentation
> constructor is nearly useless anyway: UniqueRepresentation is supposed to be
> fast and cached anyway! (it does have to do some argument processing,
> though, so it is a little slower that a straight cached routine). However, I
> can imagine that similar scenarios occur in less obvious ways and yes, they
> all leak.
>
> We can clean things up a little bit in this example by clearing the cache
> ourselves, i.e., run
>
> for p in P:
> dummy=p.linear_extensions()
> p.linear_extensions.clear_cache()
>
> but you'll find that you need to do multiple gc.collect() afterwards to get
> rid of all the crud: the garbage is so complicated that the collector can't
> clean it up in one go (the objects that get released from being
> WeakValueDictionary keys due to weakref callbacks are still part of
> reference cycles, and it takes a fresh gc run to recognize them as such)

Oh. That's what I advised Andrew to do... I hope it will work somehow :-/

Nathann

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: massive memory usage generating posets...

[Nils Bruin]

On Sunday, April 6, 2014 12:23:56 PM UTC-7, Nathann Cohen wrote:
>
> > As you can see for n=6, all the 318 
> sage.graphs.linearextensions.LinearExtensions instances are still in 
> memory.  I suspect it's something like this:
> >  - The "linear_extension" method is cached. This means that the poset 
> will keep the linear_extension alive
> >  - The unique_parent of linear_extension property means that its 
> construction arguments (which includes the poset!) are used as a strong key 
> in a weak_value_dictionary. This means that as long as the linear_extension 
> is alive, the key will remain and hence the category of linear_extension 
> will keep the poset alive.
> >
> > Voila, a classic memory leak. "cached_methods" and "unique_parent" 
> combined are basically designed to invite memory leaks like this. Don't use 
> them together unless you're prepared to think very hard about basically all 
> the objects that might have a glancing interaction with the objects you're 
> implementing.
>
> Are you serious ? I'm ready to bet that a LOT of classes use those two 
> things at the same time. But there is a way to identify them automatically, 
> isn't it ?
>

If you mean by "automatic": write a script to create a lot of them and 
throw them away and see if it leaks then, yes. Otherwise: this really is a 
non-local program logic problem, so detecting it is hard.

Yep. The problem is:  UniqueRepresentation implements necessarily a GLOBAL 
cache and hence will hold strong references to the creation parameters. 
It's very unfortunate and I think if we had thought about the consequences 
of that a little more, we wouldn't have gone with this design choice at 
all. It's just too hard to work with. But now we're stuck with it.

UniqueRepresentation does try to alleviate the problems a little bit by 
using a WeakValueDictionary for its cache, but as you see: Construction 
parameter passed to a UniqueRepresentation constructor should NEVER hold a 
strong reference to the resulting object, because that creates a memory 
leak. Ideally, construction parameters should just not hold references at 
all, but that's unfortunately infeasible.

Caching a routine that's simply a wrapper around a UniqueRepresentation 
constructor is nearly useless anyway: UniqueRepresentation is supposed to 
be fast and cached anyway! (it does have to do some argument processing, 
though, so it is a little slower that a straight cached routine). However, 
I can imagine that similar scenarios occur in less obvious ways and yes, 
they all leak.

We can clean things up a little bit in this example by clearing the cache 
ourselves, i.e., run

for p in P:
dummy=p.linear_extensions()
p.linear_extensions.clear_cache()

but you'll find that you need to do multiple gc.collect() afterwards to get 
rid of all the crud: the garbage is so complicated that the collector can't 
clean it up in one go (the objects that get released from being 
WeakValueDictionary keys due to weakref callbacks are still part of 
reference cycles, and it takes a fresh gc run to recognize them as such)

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Re: massive memory usage generating posets...

[Nathann Cohen]

> As you can see for n=6, all the 318
sage.graphs.linearextensions.LinearExtensions instances are still in
memory.  I suspect it's something like this:
>  - The "linear_extension" method is cached. This means that the poset
will keep the linear_extension alive
>  - The unique_parent of linear_extension property means that its
construction arguments (which includes the poset!) are used as a strong key
in a weak_value_dictionary. This means that as long as the linear_extension
is alive, the key will remain and hence the category of linear_extension
will keep the poset alive.
>
> Voila, a classic memory leak. "cached_methods" and "unique_parent"
combined are basically designed to invite memory leaks like this. Don't use
them together unless you're prepared to think very hard about basically all
the objects that might have a glancing interaction with the objects you're
implementing.

Are you serious ? I'm ready to bet that a LOT of classes use those two
things at the same time. But there is a way to identify them automatically,
isn't it ?

Nathann

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Building Sage with Cygwin 32 bit

[Jean-Pierre Flori]

On Sunday, April 6, 2014 4:40:35 PM UTC+2, Sebastien Gouezel wrote:
>
> Le 06/04/2014 13:45, Sebastien Gouezel a écrit : 
> > I am also trying to compile sage (6.2 beta 4) under Cygwin. I got a 
> > working copy of sage, but the doc does not build (so, I can not try 
> > "make test"). Here is the beginning of the log: 
> > 
> > Building reference manual, first pass. 
> > 
> > [combinat ] Configuration error: 
> > [combinat ] There is a syntax error in your configuration file: invalid 
> > syntax (conf.py, line 1) 
> > (the file combinat/conf.py is simply "../conf_sub.py"). 
>
> The problem is that git on windows (msysgit) converts symlinks to text 
> files containing the address of the symlinked file. So, it has nothing 
> to do with sage, it is a problem with msysgit. 
>
> Scripts to restore the symlinks are available at 
> http://stackoverflow.com/questions/5917249/git-symlinks-in-windows 
>
Good news.

If you're actually interested in Sage on Cygwin32/64, please don't be shy 
and review some of the easy tickets needing review.
See everyt link not green at http://trac.sagemath.org/wiki/Cygwin64Port 
(where I now report for goth Cygwin32/64).

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: [mpir-devel] MPIR-2.7.0-alpha1, testing needed

[Bill Hart]

We are now up to alpha4. New changes should only affect Mac. Thanks Leif!

http://mpir.org/mpir-2.7.0-alpha4.tar.bz2
http://mpir.org/mpir-2.7.0-alpha4.tar.lz
http://mpir.org/mpir-2.7.0-alpha4.zip

Bill.


On 6 April 2014 18:53, leif  wrote:

> Bill Hart wrote:
>
>> Can you tell me what the changes are so I can merge them?
>>
>
>
> diff -Naur mpir-2.7.0-alpha3/configure mpir-2.7.0-alpha3-patched/configure
> --- mpir-2.7.0-alpha3/configure
> +++ mpir-2.7.0-alpha3-patched/configure
> @@ -25408,7 +25408,7 @@
>
># Defined in mpn/x86_64/x86_64-defs.m4, but there currently
># hardcoded just for ELF, so redefine it here for Mach-O:
>
> -echo "define(\`JUMPTABSECT',RODATA)" >> $gmp_tmpconfigm4p
> +echo "define(\`JUMPTABSECT',\` .section__DATA,__const')" >>
> $gmp_tmpconfigm4p
>
>
>OBJECT_FORMAT="-f macho64" ;;
>  *-w64-mingw*|*-*-cygwin*)
> diff -Naur mpir-2.7.0-alpha3/configure.ac mpir-2.7.0-alpha3-patched/conf
> igure.ac
> --- mpir-2.7.0-alpha3/configure.ac
> +++ mpir-2.7.0-alpha3-patched/configure.ac
> @@ -2847,7 +2847,7 @@
>  *-*-darwin*)
>
># Defined in mpn/x86_64/x86_64-defs.m4, but there currently
># hardcoded just for ELF, so redefine it here for Mach-O:
> -  GMP_DEFINE_RAW(["define(\`JUMPTABSECT',RODATA)"],POST)
> +  GMP_DEFINE_RAW(["define(\`JUMPTABSECT',\`.section
> __DATA,__const')"],POST)
>
>OBJECT_FORMAT="-f macho64" ;;
>  *-w64-mingw*|*-*-cygwin*)
>OBJECT_FORMAT="-f x64"  ;;
>
>
> -leif
>
>
>  On 6 April 2014 18:03, leif > > wrote:
>>
>> leif wrote:
>>
>> Ok, here's minimalist's quick'n'dirty temporary solution;
>> minimally-invasive in that it touches only one file and affects
>> only
>> Darwin x86_64 :-) :
>>
>> --- mpir-2.7.0-alpha2/configure.ac 
>> +++ mpir-2.7.0-alpha2/configure.ac 
>> @@ -2843,15 +2843,18 @@
>>  ;;
>>
>>64)
>> +  GMP_INCLUDE_MPN(x86_64/x86_64-__defs.m4)
>>
>>  case $host in
>>*-*-darwin*)
>> +  # Defined in mpn/x86_64/x86_64-defs.m4, but there
>> currently
>> +  # hardcoded just for ELF, so redefine it here for
>> Mach-O:
>> +
>>   GMP_DEFINE_RAW(["define(\`__JUMPTABSECT',RODATA)"],POST)
>>
>>  OBJECT_FORMAT="-f macho64" ;;
>>*-w64-mingw*|*-*-cygwin*)
>>  OBJECT_FORMAT="-f x64"  ;;
>>*)
>>  OBJECT_FORMAT="-f elf64" ;;
>>  esac
>> -  GMP_INCLUDE_MPN(x86_64/x86_64-__defs.m4)
>>
>>  ;;
>>  esac
>>  AC_SUBST(OBJECT_FORMAT)
>>
>>
>> Patched tarball (of course also including a re-autoconfed
>> 'configure')
>> for the Apple fanboys to test is available here:
>>
>> http://boxen.math.washington.__edu/home/leif/tmp/mpir-2.7.0-
>> __alpha2-patched_for_darwin-x86___64.tar.bz2
>>
>> > alpha2-patched_for_darwin-x86_64.tar.bz2>
>>
>>
>>
>> Update, please try this one:
>>
>> http://boxen.math.washington.__edu/home/leif/tmp/mpir-2.7.0-
>> __alpha3-patched_for_darwin-x86___64.tar.bz2
>>
>> > alpha3-patched_for_darwin-x86_64.tar.bz2>
>>
>> (extracts to mpir-2.7.0-alpha3-patched/ )
>>
>>
>> -leif
>>
>>
>> Happy testing, we want to include it into Sage soon I think,
>>
>>
>> -leif
>>
>>
>> P.S.:  Haven't looked at Darwin PPC at all, but in case there
>> should be
>> any issues, they'll be different... :-)
>>
>
> --
> () The ASCII Ribbon Campaign
> /\   Help Cure HTML E-Mail
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] UniqueRepresentation and cached_method don't mix

[Nils Bruin]

The following paradigm happens for 
"sage.combinat.posets.posets.FinitePoset" and their "linear_extensions", 
but it's such an easy mistake to make that I'd almost consider it a flaw in 
our design:

class A(Parent,UniqueRepresentation):

@cached_method
def BfromA(self):
return B(A)

class B(Parent,UniqueRepresentation):
...

If one does

  a = A()
  b = a.BfromA()
  del a,b

one leaks memory. This is because the (global!) UniqueRepresentation cache 
for B will have a strong reference to a (as a key in a WeakValueDictionary) 
, since it's a construction parameter for b, and a will be holding a strong 
reference to b because BfromA was declared a cached method. Hence, a keeps 
b alive, and as long as b is alive, a will be kept alive due to the strong 
reference in the weakvaluedict.

The solution here probably is to NOT cache BfromA, since the lookup 
performed by B(A) will be very fast anyway (if the thing already exists), 
but you can see how difficult it now becomes to decide whether you can 
safely stick a @cached_method decorator somewhere: If the result contains a 
UniqueRepresentation object that is constructed using something linked to 
the object on which the cached_method lives, you probably can't. But this 
becomes a highly non-local thing to decide, because now you need to know 
the implementation details of lots of things you might be using.

The real problem here is UniqueRepresentation. Fundamentally, that breaks 
locality. We try to hide the problems a little bit by using a WeakValueDict 
to cache things, but as this example shows, it only takes one extra level 
to let the problems resurface. Any ideas on how we can mitigate these 
problems? Educating developers perhaps?

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: [mpir-devel] MPIR-2.7.0-alpha1, testing needed

[leif]

Bill Hart wrote:

Can you tell me what the changes are so I can merge them?



diff -Naur mpir-2.7.0-alpha3/configure mpir-2.7.0-alpha3-patched/configure
--- mpir-2.7.0-alpha3/configure
+++ mpir-2.7.0-alpha3-patched/configure
@@ -25408,7 +25408,7 @@
   # Defined in mpn/x86_64/x86_64-defs.m4, but there currently
   # hardcoded just for ELF, so redefine it here for Mach-O:

-echo "define(\`JUMPTABSECT',RODATA)" >> $gmp_tmpconfigm4p
+echo "define(\`JUMPTABSECT',\`	.section	__DATA,__const')" >> 
$gmp_tmpconfigm4p


   OBJECT_FORMAT="-f macho64" ;;
 *-w64-mingw*|*-*-cygwin*)
diff -Naur mpir-2.7.0-alpha3/configure.ac 
mpir-2.7.0-alpha3-patched/configure.ac

--- mpir-2.7.0-alpha3/configure.ac
+++ mpir-2.7.0-alpha3-patched/configure.ac
@@ -2847,7 +2847,7 @@
 *-*-darwin*)
   # Defined in mpn/x86_64/x86_64-defs.m4, but there currently
   # hardcoded just for ELF, so redefine it here for Mach-O:
-  GMP_DEFINE_RAW(["define(\`JUMPTABSECT',RODATA)"],POST)
+  GMP_DEFINE_RAW(["define(\`JUMPTABSECT',\`	.section 
__DATA,__const')"],POST)

   OBJECT_FORMAT="-f macho64" ;;
 *-w64-mingw*|*-*-cygwin*)
   OBJECT_FORMAT="-f x64"  ;;


-leif



On 6 April 2014 18:03, leif mailto:not.rea...@online.de>> wrote:

leif wrote:

Ok, here's minimalist's quick'n'dirty temporary solution;
minimally-invasive in that it touches only one file and affects only
Darwin x86_64 :-) :

--- mpir-2.7.0-alpha2/configure.ac 
+++ mpir-2.7.0-alpha2/configure.ac 
@@ -2843,15 +2843,18 @@
 ;;

   64)
+  GMP_INCLUDE_MPN(x86_64/x86_64-__defs.m4)
 case $host in
   *-*-darwin*)
+  # Defined in mpn/x86_64/x86_64-defs.m4, but there
currently
+  # hardcoded just for ELF, so redefine it here for
Mach-O:
+
  GMP_DEFINE_RAW(["define(\`__JUMPTABSECT',RODATA)"],POST)
 OBJECT_FORMAT="-f macho64" ;;
   *-w64-mingw*|*-*-cygwin*)
 OBJECT_FORMAT="-f x64"  ;;
   *)
 OBJECT_FORMAT="-f elf64" ;;
 esac
-  GMP_INCLUDE_MPN(x86_64/x86_64-__defs.m4)
 ;;
 esac
 AC_SUBST(OBJECT_FORMAT)


Patched tarball (of course also including a re-autoconfed
'configure')
for the Apple fanboys to test is available here:


http://boxen.math.washington.__edu/home/leif/tmp/mpir-2.7.0-__alpha2-patched_for_darwin-x86___64.tar.bz2





Update, please try this one:


http://boxen.math.washington.__edu/home/leif/tmp/mpir-2.7.0-__alpha3-patched_for_darwin-x86___64.tar.bz2



(extracts to mpir-2.7.0-alpha3-patched/ )


-leif


Happy testing, we want to include it into Sage soon I think,


-leif


P.S.:  Haven't looked at Darwin PPC at all, but in case there
should be
any issues, they'll be different... :-)


--
() The ASCII Ribbon Campaign
/\   Help Cure HTML E-Mail

--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: massive memory usage generating posets...

[Nils Bruin]

On Saturday, April 5, 2014 7:41:47 PM UTC-7, Andrew Juell wrote:
>
> Running sage locally under Ubuntu to avoid any esoteric problems in the 
> cloud, I discovered another issue...while I could iterate over Posets(7) 
> and (8) and count their linear extensions with little trouble, attempting 
> to do so with the 9-element posets overnight resulted in python eating up 
> 4+GB of memory before giving up and nearly freezing my machine outright. 

It's a memory leak:

sage: import gc
sage: from collections import Counter
sage: P=Posets(6)
sage: gc.collect()
1
sage: pre={id(c) for c in gc.get_objects()}
sage: for p in P: dummy=p.linear_extensions()
sage: gc.collect()
0
sage: post=Counter(type(o) for o in gc.get_objects() if id(o) not in pre)
sage: sorted(post.iteritems(),key=lambda t: t[1])
[(_ctypes.PyCFuncPtrType, 1),
 (_ast.Attribute, 1),
 (frame, 1),
 (sage.categories.category.JoinCategory_with_category, 1),
 (StgDict, 1),
 (sage.categories.facade_sets.FacadeSets_with_category, 1),
 (sage.structure.coerce_maps.DefaultConvertMap_unique, 1),
 (thread._local, 1),
 (listiterator, 1),
 (sage.categories.homset.Homset_with_category, 1),
 (sage.misc.cachefunc.CachedFunction, 1),
 (sage.misc.function_mangling.ArgumentFixer, 1),
 (collections.defaultdict, 1),
 (ctypes.CDLL, 1),
 (_ast.Module, 1),
 (_ast.Interactive, 1),
 (decimal._Log10Memoize, 1),
 (_ast.Compare, 1),
 (sage.categories.posets.Posets_with_category, 1),
 (_ast.Assign, 1),
 (sage.misc.cachefunc.CachedMethod, 1),
 (sage.categories.finite_posets.FinitePosets_with_category, 1),
 (sage.misc.constant_function.ConstantFunction, 2),
 (ctypes._FuncPtr, 2),
 (operator.itemgetter, 3),
 (decimal.Context, 3),
 (_ast.Call, 3),
 (instance, 3),
 (frozenset, 4),
 (abc.abstractproperty, 4),
 (uuid.UUID, 4),
 (_ast.Name, 4),
 (sage.misc.classcall_metaclass.ClasscallMetaclass, 4),
 (wrapper_descriptor, 5),
 (classmethod, 5),
 (abc.ABCMeta, 6),
 (classobj, 6),
 (sage.structure.dynamic_class.DynamicClasscallMetaclass, 6),
 (decimal.Decimal, 6),
 (sage.structure.dynamic_class.DynamicMetaclass, 8),
 (member_descriptor, 9),
 (method_descriptor, 9),
 (_weakrefset.WeakSet, 18),
 (cython_function_or_method, 20),
 (property, 27),
 (set, 38),
 (getset_descriptor, 55),
 (type, 67),
 (cell, 112),
 (staticmethod, 170),
 (module, 181),
 (sage.combinat.posets.hasse_diagram.HasseDiagram, 318),
 (sage.combinat.posets.linear_extensions.LinearExtensionsOfPoset_with_category, 
318),
 (sage.graphs.linearextensions.LinearExtensions, 318),
 (sage.misc.cachefunc.CachedMethodCaller, 318),
 (sage.graphs.base.static_sparse_backend.StaticSparseBackend, 318),
 (sage.combinat.posets.posets.FinitePoset_with_category, 318),
 (sage.misc.cachefunc.CachedMethodCallerNoArgs, 321),
 (builtin_function_or_method, 377),
 (sage.structure.coerce_dict.TripleDictEraser, 637),
 (sage.structure.coerce_dict.TripleDict, 637),
 (instancemethod, 638),
 (weakref.KeyedRef, 656),
 (sage.structure.coerce_dict.MonoDictEraser, 1274),
 (sage.structure.coerce_dict.MonoDict, 1274),
 (function, 1687),
 (weakref, 2054),
 (dict, 2684),
 (tuple, 3773),
 (list, 5675)]

As you can see for n=6, all the 318 
sage.graphs.linearextensions.LinearExtensions instances are still in 
memory.  I suspect it's something like this:
 - The "linear_extension" method is cached. This means that the poset will 
keep the linear_extension alive
 - The unique_parent of linear_extension property means that its 
construction arguments (which includes the poset!) are used as a strong key 
in a weak_value_dictionary. This means that as long as the linear_extension 
is alive, the key will remain and hence the category of linear_extension 
will keep the poset alive.

Voila, a classic memory leak. "cached_methods" and "unique_parent" combined 
are basically designed to invite memory leaks like this. Don't use them 
together unless you're prepared to think very hard about basically all the 
objects that might have a glancing interaction with the objects you're 
implementing.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: [mpir-devel] MPIR-2.7.0-alpha1, testing needed

[Bill Hart]

Can you tell me what the changes are so I can merge them?

Bill.


On 6 April 2014 18:03, leif  wrote:

> leif wrote:
>
>> Ok, here's minimalist's quick'n'dirty temporary solution;
>> minimally-invasive in that it touches only one file and affects only
>> Darwin x86_64 :-) :
>>
>> --- mpir-2.7.0-alpha2/configure.ac
>> +++ mpir-2.7.0-alpha2/configure.ac
>> @@ -2843,15 +2843,18 @@
>> ;;
>>
>>   64)
>> +  GMP_INCLUDE_MPN(x86_64/x86_64-defs.m4)
>> case $host in
>>   *-*-darwin*)
>> +  # Defined in mpn/x86_64/x86_64-defs.m4, but there currently
>> +  # hardcoded just for ELF, so redefine it here for Mach-O:
>> +  GMP_DEFINE_RAW(["define(\`JUMPTABSECT',RODATA)"],POST)
>> OBJECT_FORMAT="-f macho64" ;;
>>   *-w64-mingw*|*-*-cygwin*)
>> OBJECT_FORMAT="-f x64"  ;;
>>   *)
>> OBJECT_FORMAT="-f elf64" ;;
>> esac
>> -  GMP_INCLUDE_MPN(x86_64/x86_64-defs.m4)
>> ;;
>> esac
>> AC_SUBST(OBJECT_FORMAT)
>>
>>
>> Patched tarball (of course also including a re-autoconfed 'configure')
>> for the Apple fanboys to test is available here:
>>
>> http://boxen.math.washington.edu/home/leif/tmp/mpir-2.7.0-
>> alpha2-patched_for_darwin-x86_64.tar.bz2
>>
>
>
> Update, please try this one:
>
> http://boxen.math.washington.edu/home/leif/tmp/mpir-2.7.0-
> alpha3-patched_for_darwin-x86_64.tar.bz2
>
> (extracts to mpir-2.7.0-alpha3-patched/ )
>
>
> -leif
>
>
>  Happy testing, we want to include it into Sage soon I think,
>>
>>
>> -leif
>>
>>
>> P.S.:  Haven't looked at Darwin PPC at all, but in case there should be
>> any issues, they'll be different... :-)
>>
>
> --
> () The ASCII Ribbon Campaign
> /\   Help Cure HTML E-Mail
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: [mpir-devel] MPIR-2.7.0-alpha1, testing needed

[leif]

leif wrote:

Ok, here's minimalist's quick'n'dirty temporary solution;
minimally-invasive in that it touches only one file and affects only
Darwin x86_64 :-) :

--- mpir-2.7.0-alpha2/configure.ac
+++ mpir-2.7.0-alpha2/configure.ac
@@ -2843,15 +2843,18 @@
;;

  64)
+  GMP_INCLUDE_MPN(x86_64/x86_64-defs.m4)
case $host in
  *-*-darwin*)
+  # Defined in mpn/x86_64/x86_64-defs.m4, but there currently
+  # hardcoded just for ELF, so redefine it here for Mach-O:
+  GMP_DEFINE_RAW(["define(\`JUMPTABSECT',RODATA)"],POST)
OBJECT_FORMAT="-f macho64" ;;
  *-w64-mingw*|*-*-cygwin*)
OBJECT_FORMAT="-f x64"  ;;
  *)
OBJECT_FORMAT="-f elf64" ;;
esac
-  GMP_INCLUDE_MPN(x86_64/x86_64-defs.m4)
;;
esac
AC_SUBST(OBJECT_FORMAT)


Patched tarball (of course also including a re-autoconfed 'configure')
for the Apple fanboys to test is available here:

http://boxen.math.washington.edu/home/leif/tmp/mpir-2.7.0-alpha2-patched_for_darwin-x86_64.tar.bz2



Update, please try this one:

http://boxen.math.washington.edu/home/leif/tmp/mpir-2.7.0-alpha3-patched_for_darwin-x86_64.tar.bz2

(extracts to mpir-2.7.0-alpha3-patched/ )


-leif


Happy testing, we want to include it into Sage soon I think,


-leif


P.S.:  Haven't looked at Darwin PPC at all, but in case there should be
any issues, they'll be different... :-)


--
() The ASCII Ribbon Campaign
/\   Help Cure HTML E-Mail

--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Building Sage with Cygwin 32 bit

[Sebastien Gouezel]

Le 06/04/2014 13:45, Sebastien Gouezel a écrit :

I am also trying to compile sage (6.2 beta 4) under Cygwin. I got a
working copy of sage, but the doc does not build (so, I can not try
"make test"). Here is the beginning of the log:

Building reference manual, first pass.

[combinat ] Configuration error:
[combinat ] There is a syntax error in your configuration file: invalid
syntax (conf.py, line 1)
(the file combinat/conf.py is simply "../conf_sub.py").


The problem is that git on windows (msysgit) converts symlinks to text 
files containing the address of the symlinked file. So, it has nothing 
to do with sage, it is a problem with msysgit.


Scripts to restore the symlinks are available at 
http://stackoverflow.com/questions/5917249/git-symlinks-in-windows






--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: MPIR-2.7.0-alpha1, testing needed

[Bill Hart]

On 6 April 2014 15:44, Денис Крыськов  wrote:

>
>
> On Sunday, April 6, 2014 3:18:20 PM UTC+4, Bill Hart wrote:
>>
>> OK, according to this page:
>>
>> http://en.wikipedia.org/wiki/List_of_Intel_Core_i5_microprocessors
>>
>> your processor has an Intel Sandy Bridge core, with avx support.
>>
>> ... and if I install MPIR, asm code in mpn/x86_64w/sandybridge will be
> used, right? Or that code only runs under Windows?
>

That's the Windows assembly code for sandybridge. The linux assembly is in
mpn/x86_64/sandybridge (and possibly other directories too). It will have
been automatically installed.


>
> //gcc supporting Sandy Bridge since 4.6 (2011), Gentoo/GMP minimally
> supporting Sandy Bridge since 26 Mar 2014...That was stupid of me not to
> notice that
>
> Bill Hart, thanks for educating me
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: massive memory usage generating posets...

[Andrew Juell]

v=9
f = open('posetstats', 'w')
P=Posets(v)
Bean=0
for p in P:
  Bean+=1
  if Bean%1000==0: print Bean
  f.write(str(Bean)+ " "+ str(len(p.cover_relations()))+" 
"+str(p.linear_extensions().cardinality())+"\n")
print "done"
f.close()

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: Building Sage with Cygwin 32 bit

[Sebastien Gouezel]

I am also trying to compile sage (6.2 beta 4) under Cygwin. I got a 
working copy of sage, but the doc does not build (so, I can not try 
"make test"). Here is the beginning of the log:


Building reference manual, first pass.

[combinat ] Configuration error:
[combinat ] There is a syntax error in your configuration file: invalid 
syntax (conf.py, line 1)

[polynomia] Configuration error:
[polynomia] There is a syntax error in your configuration file: invalid 
syntax (conf.py, line 1)

[cmd  ] Configuration error:
[cmd  ] There is a syntax error in your configuration file: invalid 
syntax (conf.py, line 1)


And the list goes on like that for most directories, only [riemannia] 
succeeds.

(the file combinat/conf.py is simply "../conf_sub.py").
Any idea?


For the record, here are the steps I followed to compile sage:

- the patch trac.sagemath.org/ticket/15967 is necessary.

- there are problems in rpy2. As mentioned in the previous message, one 
should add __CYGWIN__ in addition  to Win32 and Win64 several times in 
rinterface/na_values.c. However, this is not enough: one still gets the 
following error


gcc -shared -Wl,--enable-auto-image-base 
-L/home/Sebastien/sage-git/sage/local/lib 
build/temp.cygwin-1.7.28-i686-2.7/./rpy/rinterface/_rinterface.o 
-L/home/Sebastien/sage-git/sage/local/lib/python2.7/config 
-L/home/Sebastien/sage-git/sage/local/lib 
-L/home/Sebastien/sage-git/sage/local/lib/R//lib 
-L/home/Sebastien/sage-git/sage/local/lib/R/modules -lR -lpython2.7 -lR 
-o build/lib.cygwin-1.7.28-i686-2.7/rpy2/rinterface/_rinterface.dll
build/temp.cygwin-1.7.28-i686-2.7/./rpy/rinterface/_rinterface.o: In 
function `EmbeddedR_init':
/home/Sebastien/sage-git/sage/local/var/tmp/sage/build/rpy2-2.3.8/src/./rpy/rinterface/_rinterface.c:1333: 
undefined reference to `rl_completer_word_break_characters'
/home/Sebastien/sage-git/sage/local/var/tmp/sage/build/rpy2-2.3.8/src/./rpy/rinterface/_rinterface.c:1334: 
undefined reference to `rl_completer_word_break_characters'
/home/Sebastien/sage-git/sage/local/var/tmp/sage/build/rpy2-2.3.8/src/./rpy/rinterface/_rinterface.c:1336: 
undefined reference to `rl_basic_word_break_characters'
/home/Sebastien/sage-git/sage/local/var/tmp/sage/build/rpy2-2.3.8/src/./rpy/rinterface/_rinterface.c:1337: 
undefined reference to `rl_basic_word_break_characters'

collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

(linking problem with readline, likely). I commented out the offending 
lines in _rinterface.c (I know this is not a proper fix, of course, but 
it is sufficient to get things to compile).


- I had to rebase once.

- Segmentation faults during zn_poly's tuning (in KS/FFT mul). Looks 
like http://trac.sagemath.org/ticket/13947. On the third attempt, 
everything went smoothly.




Le 04/04/2014 22:03, Jean-Pierre Flori a écrit :

IIRC, you should modify lines like the following one:
https://bitbucket.org/lgautier/rpy2/src/a66387e12e2419141dadb7464ea9205595ef89af/rpy/rinterface/na_values.c?at=default#cl-206
to include __CYGWIN__ in addition to Win32 and Win64.



--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] Re: MPIR-2.7.0-alpha1, testing needed

[Bill Hart]

OK, according to this page:

http://en.wikipedia.org/wiki/List_of_Intel_Core_i5_microprocessors

your processor has an Intel Sandy Bridge core, with avx support.

According to this page:

http://gcc.gnu.org/onlinedocs/gcc-4.8.1/gcc/i386-and-x86_002d64-Options.html

this is the right option (ok, it says corei7, but there is no corei5), if
it supports the following:

MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX, AES and PCLMUL

which the Wikipedia page confirms it does (it doesn't mention pclmul, but
your processor has the pclmulqdq flag listed in "flags") .

So I think it is the best compiler option for your processor.

Bill.


On 6 April 2014 12:59, Денис Крыськов  wrote:

>
> I wonder is this correct that -march=corei7-avx instead of corei5 (I got
>>> Intel(R) Core(TM) i5-2500K)?
>>>
>>
>> I think so. If you tell me the result of
>>
>> cat /proc/cpuinfo
>>
>> in particular the line that starts "model name", I will check and make
>> sure it is correct.
>>
>>
> processor : 0
> vendor_id : GenuineIntel
> cpu family : 6
> model : 42
> model name : Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
> stepping : 7
> microcode : 0x17
> cpu MHz : 1600.000
> cache size : 6144 KB
> physical id : 0
> siblings : 4
> core id : 0
> cpu cores : 4
> apicid : 0
> initial apicid : 0
> fpu : yes
> fpu_exception : yes
> cpuid level : 13
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
> pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
> constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc
> aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3
> cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx
> lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority
> ept vpid
> bogomips : 6584.79
> clflush size : 64
> cache_alignment : 64
> address sizes : 36 bits physical, 48 bits virtual
> power management:
>
> processor : 1
> vendor_id : GenuineIntel
> cpu family : 6
> model : 42
> model name : Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
> stepping : 7
> microcode : 0x17
> cpu MHz : 1600.000
> cache size : 6144 KB
> physical id : 0
> siblings : 4
> core id : 1
> cpu cores : 4
> apicid : 2
> initial apicid : 2
> fpu : yes
> fpu_exception : yes
> cpuid level : 13
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
> pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
> constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc
> aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3
> cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx
> lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority
> ept vpid
> bogomips : 6584.79
> clflush size : 64
> cache_alignment : 64
> address sizes : 36 bits physical, 48 bits virtual
> power management:
>
> processor : 2
> vendor_id : GenuineIntel
> cpu family : 6
> model : 42
> model name : Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
> stepping : 7
> microcode : 0x17
> cpu MHz : 1600.000
> cache size : 6144 KB
> physical id : 0
> siblings : 4
> core id : 2
> cpu cores : 4
> apicid : 4
> initial apicid : 4
> fpu : yes
> fpu_exception : yes
> cpuid level : 13
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
> pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
> constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc
> aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3
> cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx
> lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority
> ept vpid
> bogomips : 6584.79
> clflush size : 64
> cache_alignment : 64
> address sizes : 36 bits physical, 48 bits virtual
> power management:
>
> processor : 3
> vendor_id : GenuineIntel
> cpu family : 6
> model : 42
> model name : Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
> stepping : 7
> microcode : 0x17
> cpu MHz : 1600.000
> cache size : 6144 KB
> physical id : 0
> siblings : 4
> core id : 3
> cpu cores : 4
> apicid : 6
> initial apicid : 6
> fpu : yes
> fpu_exception : yes
> cpuid level : 13
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
> pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm
> constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc
> aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3
> cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx
> lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority
> ept vpid
> bogomips : 6584.79
> clflush size : 64
> cache_alignment : 64
> address sizes : 36 bits physical, 48 bits virtual
> power management:
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and s

[sage-devel] Re: MPIR-2.7.0-alpha1, testing needed

[Bill Hart]

On 6 April 2014 09:53, Денис Крыськов  wrote:

>
>
> On Sunday, April 6, 2014 10:12:36 AM UTC+4, 袁轶君 wrote:
>>
>> Well, I solved my problem. MPIR can only be built in a pure ASCII
>> path...No UNICODE characters is allowed. Besides that ,I have a question.
>> What's the difference between lib_mpir_core2
>>
>
> Can't confirm the statement about unicode. Looks like MPIR likes Russian
> more than Chinese (or maybe MPIR likes Linux better than Windoz).
> Successfully built in /tmp/ы where ы is a Russian letter.
>

This is likely a problem with Visual Studio. The linux build process is
completely different.  But thanks for checking.


>
> *g++ -DHAVE_CONFIG_H -I. -I../.. -I../.. -I../../tests-m64 -O2
> -march=corei7-avx -mtune=corei7-avx -c -o t-unary.o t-unary.cc*
> */bin/sh ../../libtool --tag=CXX   --mode=link g++  -m64 -O2
> -march=corei7-avx -mtune=corei7-avx   -o t-unary t-unary.o -L../../.libs
> ../../tests/libtests.la  ../../libmpirxx.la
>  ../../libmpir.la  *
> *libtool: link: g++ -m64 -O2 -march=corei7-avx -mtune=corei7-avx -o
> .libs/t-unary t-unary.o  -L../../.libs ../../tests/.libs/libtests.a
> /tmp/ы/mpir-2.7.0/.libs/libmpir.so -lm ../../.libs/libmpirxx.so
> ../../.libs/libmpir.so*
> *make[4]: Leaving directory `/tmp/ы/mpir-2.7.0/tests/cxx'*
> *make  check-TESTS*
> *make[4]: Entering directory `/tmp/ы/mpir-2.7.0/tests/cxx'*
> *make[5]: Entering directory `/tmp/ы/mpir-2.7.0/tests/cxx'*
> *PASS: t-assign*
> *PASS: t-binary*
> *PASS: t-cast*
> *PASS: t-constr*
> *PASS: t-headers*
> *PASS: t-istream*
> *PASS: t-locale*
> *PASS: t-misc*
> *PASS: t-ops*
> *PASS: t-ostream*
> *PASS: t-prec*
> *PASS: t-rand*
> *PASS: t-ternary*
> *PASS: t-unary*
> *===*
> *All 14 tests passed*
> *===*
>
> I wonder is this correct that -march=corei7-avx instead of corei5 (I got
> Intel(R) Core(TM) i5-2500K)?
>

I think so. If you tell me the result of

cat /proc/cpuinfo

in particular the line that starts "model name", I will check and make sure
it is correct.


> Has anybody benchmarked MPIR-2.6.0 against GMP in integer arithmetic under
> amd64/Linux? Is it possible to use MPIR instead of GMP for Sage/FLINT under
> Linux?
>

Sage uses MPIR by default, and flint 2.4 supports GMP and MPIR. We have
done some limited benchmarking. On a 2008 AMD Opteron, MPIR was faster than
GMP for many things. But we expect GMP to be faster for very recent chips
with avx (we don't have access to really recent chips, so we can't even
check this).

We should compare some more architectures. I think on Intel penryn
architecture some things are faster in GMP some faster in MPIR.


>
> Please forgive me if the questions sound silly. I can't sleep at night
> knowing that my program runs slower than it could.
>

Of course.

Bill.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[sage-devel] massive memory usage generating posets...

[Andrew Juell]

Running sage locally under Ubuntu to avoid any esoteric problems in the cloud, 
I discovered another issue...while I could iterate over Posets(7) and (8) and 
count their linear extensions with little trouble, attempting to do so with the 
9-element posets overnight resulted in python eating up 4+GB of memory before 
giving up and nearly freezing my machine outright.  Watching it run a second 
time even after inserting regular calls to gc.collect() I see the memory usage 
steadily climb even though each poset is only referenced once and then 
discarded...I recognize that there's a lot of calculation to do, but I don't 
understand why the intermediate results would accumulate in this fashion.  Is 
there any way to circumvent this?  Thanks in advance...

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.