Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Michael Orlitzky]

On 04/16/2016 11:06 AM, Jeroen Demeyer wrote:
> On 2016-04-16 16:19, Michael Orlitzky wrote:
>> It gets us one step closer to a Sage that will use what's already
>> installed on the system.
> 
> No, it does not do that. It's just a different way of installing packages.
> 

I had `pip install --user` in mind.

Suppose I use pip to install libfoo as part of the sage-8.0
installation. Then I wipe out my sage-8.0 directory and try to install
sage-9.0. Wouldn't it find the libfoo that I already installed?

Those pip-installed libraries should persist, and that means our "does
this library exist?" test should be something like "import foo" and not
"is the libfoo directory exactly where I think it should be". The
"import foo" test would also pick up libfoo if I've installed it with my
system's package manager.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Erik Bray]

On Sat, Apr 16, 2016 at 10:59 AM, Dima Pasechnik  wrote:
> Presently we distribute a number of tarballs that are copies of
> pip-installable Python packages, which are dependencies of something, and
> maintain placeholders for them in build/pkg/ (this will be completely true
> as soon as #14840 is merged).
> This is a lot of unnecessary stuff to carry around, and is quite different
> to what is done in pythonic (as well as ocaml, haskell, Macaulay2, etc)
> universe.
>
> My proposal is to get rid of these, and install pip-installable dependencies
> directly. Perhaps maintaining a catalog of such dependencies in some way;
> perhaps such facilities are provided by pip already, one needs to check this
> out.
> We may keep (some) placeholders in build/pkg, for the purpose of
> bootstrapping.
> One added advantage is  that we can provide a sane way to install packages
> which cannot be installed from distributed sources, such as ssl support.
>
> In case one wants all the sources needed for a complete Sage install
> packaged locally,
> this is easy to achieve by providing a custom pip repo; such a repo may
> become some kind of optional component.

Hi Dima,

I haven't read the rest of this thread yet, but I already started part
of this work back in http://trac.sagemath.org/ticket/20218 which I'm
nearly done testing a patch for.

The above isn't the whole store because it just switches from using
`setup.py install` (which has problems that I described in the ticket)
to using `pip install --no-deps .` for installing most Python
packages.

You're talking about going a step further and I'm inclined to agree,
but I haven't looked at what the others have stated on this issue.
Still, I think something like #20218 would be a good starting point
before proceeding further.

Thanks,
Erik

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Dima Pasechnik]

On Sunday, April 17, 2016 at 9:51:03 PM UTC+1, mmarco wrote:
>
> Would it be possible to ship such a pip repo (with only the packages 
> needed by sage) in the sage source tarball? Or maybe have two such 
> tarballs, one with it (so it would allow to compile Sage offline, and 
> source review before compiling), and another that would just rely on 
> downloading packages from pypi?
>

It looks like it would make more sense to provide a cache of pypi packages 
needed for Sage building.
It might either be a tarball, or just a list of files to download from a 
pypi mirror.
The less duplication, the better...


>
> El domingo, 17 de abril de 2016, 17:38:23 (UTC+2), William escribió:
>>
>> On Sun, Apr 17, 2016 at 8:27 AM, Dima Pasechnik  
>> wrote: 
>> > 
>> > 
>> > On Sunday, April 17, 2016 at 4:20:42 PM UTC+1, Dima Pasechnik wrote: 
>> >> 
>> >> 
>> >> 
>> >> On Sunday, April 17, 2016 at 4:15:12 PM UTC+1, William wrote: 
>> >>> 
>> >>> Dima: 
>> >>> > sorry, William, I don't see what you mean. 
>> >>> [...] 
>> >>> > 2) Putting data on your computer does not equal to installing crud 
>> on 
>> >>> > your 
>> >>> > computer, although people wearing tinfoil hats 
>> >>> > might disagree. 
>> >>> 
>> >>> They do disagree.  And they are right. 
>> >> 
>> >> 
>> >> Why would they trust somebody putting hundreds of megabytes of 3rd 
>> party 
>> >> sources into Sage? Is it because we mislead them into thinking that we 
>> did 
>> >> some kind of source review? 
>>
>> You are right.However, such a tinfoil hat person would rather 
>> (themselves) inspect the packages that are just needed for Sage, 
>> rather than all of Pypi. 
>>
>> > they actually would be less paranoid about having the 3rd party from 
>> its 
>> > original source rather than re-packaged, and this is precisely a part 
>> of my 
>> > proposal. And, by the way, 
>> > http://doc.devpi.net/latest/quickstart-pypimirror.html 
>> > is precisely a way to create a mini-repo containing only things needed 
>> by 
>> > Sage. 
>>
>> Perfect -- I agree that is the right approach. 
>>
>>
>> -- 
>> William (http://wstein.org) 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[mmarco]

Would it be possible to ship such a pip repo (with only the packages needed 
by sage) in the sage source tarball? Or maybe have two such tarballs, one 
with it (so it would allow to compile Sage offline, and source review 
before compiling), and another that would just rely on downloading packages 
from pypi?


El domingo, 17 de abril de 2016, 17:38:23 (UTC+2), William escribió:
>
> On Sun, Apr 17, 2016 at 8:27 AM, Dima Pasechnik  > wrote: 
> > 
> > 
> > On Sunday, April 17, 2016 at 4:20:42 PM UTC+1, Dima Pasechnik wrote: 
> >> 
> >> 
> >> 
> >> On Sunday, April 17, 2016 at 4:15:12 PM UTC+1, William wrote: 
> >>> 
> >>> Dima: 
> >>> > sorry, William, I don't see what you mean. 
> >>> [...] 
> >>> > 2) Putting data on your computer does not equal to installing crud 
> on 
> >>> > your 
> >>> > computer, although people wearing tinfoil hats 
> >>> > might disagree. 
> >>> 
> >>> They do disagree.  And they are right. 
> >> 
> >> 
> >> Why would they trust somebody putting hundreds of megabytes of 3rd 
> party 
> >> sources into Sage? Is it because we mislead them into thinking that we 
> did 
> >> some kind of source review? 
>
> You are right.However, such a tinfoil hat person would rather 
> (themselves) inspect the packages that are just needed for Sage, 
> rather than all of Pypi. 
>
> > they actually would be less paranoid about having the 3rd party from its 
> > original source rather than re-packaged, and this is precisely a part of 
> my 
> > proposal. And, by the way, 
> > http://doc.devpi.net/latest/quickstart-pypimirror.html 
> > is precisely a way to create a mini-repo containing only things needed 
> by 
> > Sage. 
>
> Perfect -- I agree that is the right approach. 
>
>
> -- 
> William (http://wstein.org) 
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[William Stein]

On Sun, Apr 17, 2016 at 8:27 AM, Dima Pasechnik  wrote:
>
>
> On Sunday, April 17, 2016 at 4:20:42 PM UTC+1, Dima Pasechnik wrote:
>>
>>
>>
>> On Sunday, April 17, 2016 at 4:15:12 PM UTC+1, William wrote:
>>>
>>> Dima:
>>> > sorry, William, I don't see what you mean.
>>> [...]
>>> > 2) Putting data on your computer does not equal to installing crud on
>>> > your
>>> > computer, although people wearing tinfoil hats
>>> > might disagree.
>>>
>>> They do disagree.  And they are right.
>>
>>
>> Why would they trust somebody putting hundreds of megabytes of 3rd party
>> sources into Sage? Is it because we mislead them into thinking that we did
>> some kind of source review?

You are right.However, such a tinfoil hat person would rather
(themselves) inspect the packages that are just needed for Sage,
rather than all of Pypi.

> they actually would be less paranoid about having the 3rd party from its
> original source rather than re-packaged, and this is precisely a part of my
> proposal. And, by the way,
> http://doc.devpi.net/latest/quickstart-pypimirror.html
> is precisely a way to create a mini-repo containing only things needed by
> Sage.

Perfect -- I agree that is the right approach.


-- 
William (http://wstein.org)

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Dima Pasechnik]

On Sunday, April 17, 2016 at 4:20:42 PM UTC+1, Dima Pasechnik wrote:
>
>
>
> On Sunday, April 17, 2016 at 4:15:12 PM UTC+1, William wrote:
>>
>> Dima: 
>> > sorry, William, I don't see what you mean. 
>> [...] 
>> > 2) Putting data on your computer does not equal to installing crud on 
>> your 
>> > computer, although people wearing tinfoil hats 
>> > might disagree. 
>>
>> They do disagree.  And they are right.  
>
>
> Why would they trust somebody putting hundreds of megabytes of 3rd party 
> sources into Sage? Is it because we mislead them into thinking that we did 
> some kind of source review?
>
 
they actually would be less paranoid about having the 3rd party from its 
original source rather than re-packaged, and this is precisely a part of my 
proposal. And, by the 
way, http://doc.devpi.net/latest/quickstart-pypimirror.html
is precisely a way to create a mini-repo containing only things needed by 
Sage.

Anyhow, it seems that what I propose is not evolutional enough to merit 
a normal discussion. :-(


>
>
>  
>
>>
>> -- 
>> William (http://wstein.org) 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Dima Pasechnik]

On Sunday, April 17, 2016 at 4:15:12 PM UTC+1, William wrote:
>
> Dima: 
> > sorry, William, I don't see what you mean. 
> [...] 
> > 2) Putting data on your computer does not equal to installing crud on 
> your 
> > computer, although people wearing tinfoil hats 
> > might disagree. 
>
> They do disagree.  And they are right.  


Why would they trust somebody putting hundreds of megabytes of 3rd party 
sources into Sage? Is it because we mislead them into thinking that we did 
some kind of source review?



 

>
> -- 
> William (http://wstein.org) 
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[William Stein]

Dima:
> sorry, William, I don't see what you mean.
[...]
> 2) Putting data on your computer does not equal to installing crud on your
> computer, although people wearing tinfoil hats
> might disagree.

They do disagree.  And they are right.

-- 
William (http://wstein.org)

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Dima Pasechnik]

On Sunday, April 17, 2016 at 3:43:15 PM UTC+1, William wrote:
>
>
>
> On Sunday, April 17, 2016, Dima Pasechnik  
> wrote:
>
>>
>>
>> On Sunday, April 17, 2016 at 2:14:09 AM UTC+1, kcrisman wrote:
>>>
>>>  
>>>

> (1) Providing a source tarball from which one can build Sage without 
> internet access is a basic requirement from a company that provides 
> some funding for Sage development over the years.   We don't 
> contractually have to do it, but I think it is a good to be able to do 
> so. (2) It is also helpful when we have Sage days workshops at places 
> with crappy internet (frequent), since at least everybody can build 
> Sage locally from a single tarball. 
>

 PS. in my eyes, my proposal would bring Sage closer to "normal 
 open-source development
 practices" that you were arguing for recently. No normal project I know 
 creates
 megamegatarball of everything; reducing these would be a step in the 
 right direction.

>
>
>>> Remind me again how this helps with (2)?  But more seriously, is there a 
>>> way to have the "normal" distribution be pip or whatever but have a 
>>> scriptable way to create a tarball for everything that is just not the 
>>> default one people are pointed to?  I assume yes. 
>>>
>> here is one I found
>> http://doc.devpi.net/latest/quickstart-pypimirror.html 
>>
>> it will create a pypi cache that can be served locally.
>>
>> Then, I saw a link saying that a complete pypi mirror is "just" 120GB.
>> Even if this is 5 times too optimistic, it would still fit on a portable 
>> 1TB or so disk you can take with you.
>>
>
> This fails to solve the problem of a security-paranoid person with an air 
> gapped computer, since such a person would be careful about what data they 
> put on their computer.  They would prefer to just put the 
> sage-related packages, not all pypi.
>
 
sorry, William, I don't see what you mean.

1) We talk about a workshop at a remote location situation, bad internet 
access etc.

2) Putting data on your computer does not equal to installing crud on your 
computer, although people wearing tinfoil hats
might disagree. And I never said: "copy the whole of pypi to evreyone's 
machine".
I said, basically, have a server on a local net with a mirror of pypi (or 
its part). 

3) if Sage must be approved for use in psychiatric wards, it's a different 
story all together. :-)


 
>
>>
>> Perhaps a single tarball was important to have 10 years ago, nowadays 
>> it's really not something so important...
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "sage-devel" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to sage-devel+unsubscr...@googlegroups.com.
>> To post to this group, send email to sage-devel@googlegroups.com.
>> Visit this group at https://groups.google.com/group/sage-devel.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> -- 
> Sent from my massive iPhone 6 plus.
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[William Stein]

On Sunday, April 17, 2016, Dima Pasechnik  wrote:

>
>
> On Sunday, April 17, 2016 at 2:14:09 AM UTC+1, kcrisman wrote:
>>
>>
>>
>>>
 (1) Providing a source tarball from which one can build Sage without
 internet access is a basic requirement from a company that provides
 some funding for Sage development over the years.   We don't
 contractually have to do it, but I think it is a good to be able to do
 so. (2) It is also helpful when we have Sage days workshops at places
 with crappy internet (frequent), since at least everybody can build
 Sage locally from a single tarball.

>>>
>>> PS. in my eyes, my proposal would bring Sage closer to "normal
>>> open-source development
>>> practices" that you were arguing for recently. No normal project I know
>>> creates
>>> megamegatarball of everything; reducing these would be a step in the
>>> right direction.
>>>


>> Remind me again how this helps with (2)?  But more seriously, is there a
>> way to have the "normal" distribution be pip or whatever but have a
>> scriptable way to create a tarball for everything that is just not the
>> default one people are pointed to?  I assume yes.
>>
> here is one I found
> http://doc.devpi.net/latest/quickstart-pypimirror.html
>
> it will create a pypi cache that can be served locally.
>
> Then, I saw a link saying that a complete pypi mirror is "just" 120GB.
> Even if this is 5 times too optimistic, it would still fit on a portable
> 1TB or so disk you can take with you.
>

This fails to solve the problem of a security-paranoid person with an air
gapped computer, since such a person would be careful about what data they
put on their computer.  They would prefer to just put the
sage-related packages, not all pypi.


>
> Perhaps a single tarball was important to have 10 years ago, nowadays it's
> really not something so important...
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+unsubscr...@googlegroups.com
> 
> .
> To post to this group, send email to sage-devel@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Sent from my massive iPhone 6 plus.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Isuru Fernando]

On Sun, Apr 17, 2016 at 7:15 PM, Dima Pasechnik  wrote:

>
>
> On Sunday, April 17, 2016 at 2:14:09 AM UTC+1, kcrisman wrote:
>>
>>
>>
>>>
 (1) Providing a source tarball from which one can build Sage without
 internet access is a basic requirement from a company that provides
 some funding for Sage development over the years.   We don't
 contractually have to do it, but I think it is a good to be able to do
 so. (2) It is also helpful when we have Sage days workshops at places
 with crappy internet (frequent), since at least everybody can build
 Sage locally from a single tarball.

>>>
>>> PS. in my eyes, my proposal would bring Sage closer to "normal
>>> open-source development
>>> practices" that you were arguing for recently. No normal project I know
>>> creates
>>> megamegatarball of everything; reducing these would be a step in the
>>> right direction.
>>>


>> Remind me again how this helps with (2)?  But more seriously, is there a
>> way to have the "normal" distribution be pip or whatever but have a
>> scriptable way to create a tarball for everything that is just not the
>> default one people are pointed to?  I assume yes.
>>
> here is one I found
> http://doc.devpi.net/latest/quickstart-pypimirror.html
>
> it will create a pypi cache that can be served locally.
>
> Then, I saw a link saying that a complete pypi mirror is "just" 120GB.
> Even if this is 5 times too optimistic, it would still fit on a portable
> 1TB or so disk you can take with you.
>

Or you can just download the packages to SAGE_ROOT/upstream from pip and
create a tarball for everything and use the downloaded packages to install
using pip without an internet connection

./sage -pip install --download $SAGE_ROOT/upstream sympy  # Download sympy
and dependencies to $SAGE_ROOT/upstream
./sage -pip install --no-index --find-links=$SAGE_ROOT/upstream --upgrade
sympy  # Use the downloaded packages


>
> Perhaps a single tarball was important to have 10 years ago, nowadays it's
> really not something so important...
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+unsubscr...@googlegroups.com.
> To post to this group, send email to sage-devel@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Dima Pasechnik]

On Sunday, April 17, 2016 at 2:14:09 AM UTC+1, kcrisman wrote:
>
>  
>
>>
>>> (1) Providing a source tarball from which one can build Sage without 
>>> internet access is a basic requirement from a company that provides 
>>> some funding for Sage development over the years.   We don't 
>>> contractually have to do it, but I think it is a good to be able to do 
>>> so. (2) It is also helpful when we have Sage days workshops at places 
>>> with crappy internet (frequent), since at least everybody can build 
>>> Sage locally from a single tarball. 
>>>
>>
>> PS. in my eyes, my proposal would bring Sage closer to "normal 
>> open-source development
>> practices" that you were arguing for recently. No normal project I know 
>> creates
>> megamegatarball of everything; reducing these would be a step in the 
>> right direction.
>>
>>>
>>>
> Remind me again how this helps with (2)?  But more seriously, is there a 
> way to have the "normal" distribution be pip or whatever but have a 
> scriptable way to create a tarball for everything that is just not the 
> default one people are pointed to?  I assume yes. 
>
here is one I found
http://doc.devpi.net/latest/quickstart-pypimirror.html 

it will create a pypi cache that can be served locally.

Then, I saw a link saying that a complete pypi mirror is "just" 120GB.
Even if this is 5 times too optimistic, it would still fit on a portable 
1TB or so disk you can take with you.

Perhaps a single tarball was important to have 10 years ago, nowadays it's 
really not something so important...


-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[kcrisman]

 

>
>> (1) Providing a source tarball from which one can build Sage without 
>> internet access is a basic requirement from a company that provides 
>> some funding for Sage development over the years.   We don't 
>> contractually have to do it, but I think it is a good to be able to do 
>> so. (2) It is also helpful when we have Sage days workshops at places 
>> with crappy internet (frequent), since at least everybody can build 
>> Sage locally from a single tarball. 
>>
>
> PS. in my eyes, my proposal would bring Sage closer to "normal open-source 
> development
> practices" that you were arguing for recently. No normal project I know 
> creates
> megamegatarball of everything; reducing these would be a step in the right 
> direction.
>
>>
>>
Remind me again how this helps with (2)?  But more seriously, is there a 
way to have the "normal" distribution be pip or whatever but have a 
scriptable way to create a tarball for everything that is just not the 
default one people are pointed to?  I assume yes. 

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Dima Pasechnik]

On Saturday, April 16, 2016 at 5:25:47 PM UTC+1, Jeroen Demeyer wrote:
>
> Let me also remind everyone that you don't have to use the full source 
> tarball. If you like to download at build-time from the internet, clone 
> the git repo (which does not contain tarballs) and build from that. 
>

Don't you see that making copies of perfectly good stuff that one can 
install using pip
raises few eyebrows?
See, some people are serious about security, and you are forcing them to 
check every bloody
tarfile for authenticity (and you rename them once in a while, making their 
job even harder)...
  

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Dima Pasechnik]

On Saturday, April 16, 2016 at 5:18:17 PM UTC+1, William wrote:
>
> On Sat, Apr 16, 2016 at 8:54 AM, Dima Pasechnik  > wrote: 
> > 
> > 
> > On Saturday, April 16, 2016 at 4:34:30 PM UTC+1, Jeroen Demeyer wrote: 
> >> 
> >> On 2016-04-16 17:29, Dima Pasechnik wrote: 
> >> > Any other largish Python-based  project that does these kind of 
> tarballs 
> >> > these days? 
> >> 
> >> The Sage source tarball is a distribution containing many packages, 
> it's 
> >> not a "Python project". 
> > 
> > 
> > whatever. 
>
> (1) Providing a source tarball from which one can build Sage without 
> internet access is a basic requirement from a company that provides 
> some funding for Sage development over the years.   We don't 
> contractually have to do it, but I think it is a good to be able to do 
> so. (2) It is also helpful when we have Sage days workshops at places 
> with crappy internet (frequent), since at least everybody can build 
> Sage locally from a single tarball. 
>

PS. in my eyes, my proposal would bring Sage closer to "normal open-source 
development
practices" that you were arguing for recently. No normal project I know 
creates
megamegatarball of everything; reducing these would be a step in the right 
direction.
 

>
>  - William 
>
>
> >  Distributions nowadays do not carry megamegatarballs with them, 
> > either 
> > (think e.g. Haskell Platform, or Macaulay2) 
> > If a Sage package (say, sagenb) needs to do 'pip install Blah' it should 
> > just do it. 
> > As opposed to to putting Blah.tar.gz into upstream, creating a 
> placeholder 
> > in build/pkgs/Blah, 
> > doing a vote on sage-devel on whether Blah might be included (oh, and 
> if, 
> > oops, Blah has a wrong 
> > sort of license, it's a no-starter...), hearing much cringing on how 
> this 
> > would bloat Sage even more, 
> > etc etc... 
> > 
> >> 
> >> > Of course it's hard to change this sort of attitude 
> >> 
> >> I don't think that there is a problem of attitude, we are most likely 
> >> just misunderstanding eachother. 
> > 
> > 
> > basically, I propose to reduce the over-packaging of dead-trivial things 
> > that 
> > can be pulled from the net with zero effort. 
> > You say it's no big deal, and I think you are plain wrong here. 
> > 
> > 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "sage-devel" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to sage-devel+...@googlegroups.com . 
> > To post to this group, send email to sage-...@googlegroups.com 
> . 
> > Visit this group at https://groups.google.com/group/sage-devel. 
> > For more options, visit https://groups.google.com/d/optout. 
>
>
>
> -- 
> William (http://wstein.org) 
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Dima Pasechnik]

On Saturday, April 16, 2016 at 5:18:17 PM UTC+1, William wrote:
>
> On Sat, Apr 16, 2016 at 8:54 AM, Dima Pasechnik  > wrote: 
> > 
> > 
> > On Saturday, April 16, 2016 at 4:34:30 PM UTC+1, Jeroen Demeyer wrote: 
> >> 
> >> On 2016-04-16 17:29, Dima Pasechnik wrote: 
> >> > Any other largish Python-based  project that does these kind of 
> tarballs 
> >> > these days? 
> >> 
> >> The Sage source tarball is a distribution containing many packages, 
> it's 
> >> not a "Python project". 
> > 
> > 
> > whatever. 
>
> (1) Providing a source tarball from which one can build Sage without 
> internet access is a basic requirement from a company that provides 
> some funding for Sage development over the years.   We don't 
> contractually have to do it, but I think it is a good to be able to do 
> so. (2) It is also helpful when we have Sage days workshops at places 
> with crappy internet (frequent), since at least everybody can build 
> Sage locally from a single tarball. 
>

one can easily make a local pip repo containing all the needed extra 
things, if 
required. I mentioned this in my original post.
 

>
>  - William 
>
>
> >  Distributions nowadays do not carry megamegatarballs with them, 
> > either 
> > (think e.g. Haskell Platform, or Macaulay2) 
> > If a Sage package (say, sagenb) needs to do 'pip install Blah' it should 
> > just do it. 
> > As opposed to to putting Blah.tar.gz into upstream, creating a 
> placeholder 
> > in build/pkgs/Blah, 
> > doing a vote on sage-devel on whether Blah might be included (oh, and 
> if, 
> > oops, Blah has a wrong 
> > sort of license, it's a no-starter...), hearing much cringing on how 
> this 
> > would bloat Sage even more, 
> > etc etc... 
> > 
> >> 
> >> > Of course it's hard to change this sort of attitude 
> >> 
> >> I don't think that there is a problem of attitude, we are most likely 
> >> just misunderstanding eachother. 
> > 
> > 
> > basically, I propose to reduce the over-packaging of dead-trivial things 
> > that 
> > can be pulled from the net with zero effort. 
> > You say it's no big deal, and I think you are plain wrong here. 
> > 
> > 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "sage-devel" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to sage-devel+...@googlegroups.com . 
> > To post to this group, send email to sage-...@googlegroups.com 
> . 
> > Visit this group at https://groups.google.com/group/sage-devel. 
> > For more options, visit https://groups.google.com/d/optout. 
>
>
>
> -- 
> William (http://wstein.org) 
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Jeroen Demeyer]

Let me also remind everyone that you don't have to use the full source 
tarball. If you like to download at build-time from the internet, clone 
the git repo (which does not contain tarballs) and build from that.


--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[William Stein]

On Sat, Apr 16, 2016 at 8:54 AM, Dima Pasechnik  wrote:
>
>
> On Saturday, April 16, 2016 at 4:34:30 PM UTC+1, Jeroen Demeyer wrote:
>>
>> On 2016-04-16 17:29, Dima Pasechnik wrote:
>> > Any other largish Python-based  project that does these kind of tarballs
>> > these days?
>>
>> The Sage source tarball is a distribution containing many packages, it's
>> not a "Python project".
>
>
> whatever.

(1) Providing a source tarball from which one can build Sage without
internet access is a basic requirement from a company that provides
some funding for Sage development over the years.   We don't
contractually have to do it, but I think it is a good to be able to do
so. (2) It is also helpful when we have Sage days workshops at places
with crappy internet (frequent), since at least everybody can build
Sage locally from a single tarball.

 - William


>  Distributions nowadays do not carry megamegatarballs with them,
> either
> (think e.g. Haskell Platform, or Macaulay2)
> If a Sage package (say, sagenb) needs to do 'pip install Blah' it should
> just do it.
> As opposed to to putting Blah.tar.gz into upstream, creating a placeholder
> in build/pkgs/Blah,
> doing a vote on sage-devel on whether Blah might be included (oh, and if,
> oops, Blah has a wrong
> sort of license, it's a no-starter...), hearing much cringing on how this
> would bloat Sage even more,
> etc etc...
>
>>
>> > Of course it's hard to change this sort of attitude
>>
>> I don't think that there is a problem of attitude, we are most likely
>> just misunderstanding eachother.
>
>
> basically, I propose to reduce the over-packaging of dead-trivial things
> that
> can be pulled from the net with zero effort.
> You say it's no big deal, and I think you are plain wrong here.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+unsubscr...@googlegroups.com.
> To post to this group, send email to sage-devel@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.



-- 
William (http://wstein.org)

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Dima Pasechnik]

On Saturday, April 16, 2016 at 4:34:30 PM UTC+1, Jeroen Demeyer wrote:
>
> On 2016-04-16 17:29, Dima Pasechnik wrote: 
> > Any other largish Python-based  project that does these kind of tarballs 
> > these days? 
>
> The Sage source tarball is a distribution containing many packages, it's 
> not a "Python project". 
>

whatever. Distributions nowadays do not carry megamegatarballs with them, 
either
(think e.g. Haskell Platform, or Macaulay2) 
If a Sage package (say, sagenb) needs to do 'pip install Blah' it should 
just do it.
As opposed to to putting Blah.tar.gz into upstream, creating a placeholder 
in build/pkgs/Blah,
doing a vote on sage-devel on whether Blah might be included (oh, and if, 
oops, Blah has a wrong
sort of license, it's a no-starter...), hearing much cringing on how this 
would bloat Sage even more,
etc etc...


> > Of course it's hard to change this sort of attitude 
>
> I don't think that there is a problem of attitude, we are most likely 
> just misunderstanding eachother. 
>

basically, I propose to reduce the over-packaging of dead-trivial things 
that
can be pulled from the net with zero effort.
You say it's no big deal, and I think you are plain wrong here.

 

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Jeroen Demeyer]

On 2016-04-16 17:29, Dima Pasechnik wrote:

Any other largish Python-based  project that does these kind of tarballs
these days?


The Sage source tarball is a distribution containing many packages, it's 
not a "Python project".



Of course it's hard to change this sort of attitude


I don't think that there is a problem of attitude, we are most likely 
just misunderstanding eachother.


--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Dima Pasechnik]

On Saturday, April 16, 2016 at 4:06:18 PM UTC+1, Jeroen Demeyer wrote:
>
> On 2016-04-16 16:19, Michael Orlitzky wrote: 
> > It gets us one step closer to a Sage that will use what's already 
> > installed on the system. 
>
> No, it does not do that. It's just a different way of installing packages. 
>
> > It will also make the Sage tarballs smaller 
>
> I don't think that "small tarballs" is a goal of Sage. On the other 
> hand, I do think that "tarballs containing all needed sources" is a goal 
> of Sage. 
>

Any other largish Python-based  project that does these kind of tarballs 
these days?
Of course it's hard to change this sort of attitude 

: 
(warning, the image on that link might not be very PC...)
 


> > make dependency upgrades easier 
>
> It's already pretty easy, but that might indeed be a minor advantage. 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Jeroen Demeyer]

On 2016-04-16 16:19, Michael Orlitzky wrote:

It gets us one step closer to a Sage that will use what's already
installed on the system.


No, it does not do that. It's just a different way of installing packages.


It will also make the Sage tarballs smaller


I don't think that "small tarballs" is a goal of Sage. On the other 
hand, I do think that "tarballs containing all needed sources" is a goal 
of Sage.



make dependency upgrades easier


It's already pretty easy, but that might indeed be a minor advantage.

--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Michael Orlitzky]

On 04/16/2016 08:06 AM, Jeroen Demeyer wrote:
> What is the problem that your proposal fixes?
> 

It gets us one step closer to a Sage that will use what's already
installed on the system. Users can upgrade pip-installed packages
themselves (pip --update, or whatever). And since Sage won't be looking
for its own copy of the library, using one installed by the system
package manager should become seamless.

It will also make the Sage tarballs smaller, simplify licensing (we
won't be distributing anything), and make dependency upgrades easier
(just edit the requirements file).

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Jeroen Demeyer]

What is the problem that your proposal fixes?

--
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

Re: [sage-devel] Proposal: install all pip-installable components of Sage via pip, directly

[Thierry]

Hi,

i am not sure to understand your proposal, but currently:
- you can install any pip-installable by 'sage -pip install '
- pip-installable optional packages can have their source directly
  downloaded and installed from pip. For this, just write 'pip' in the
  file "${SAGE_ROOT}/build/pkgs//type"
- for standard packages, the source must be provided since the tarball is
  supposed to be autonomous.

I do not get what is missing there ?

Ciao,
Thierry



On Sat, Apr 16, 2016 at 01:59:22AM -0700, Dima Pasechnik wrote:
> Presently we distribute a number of tarballs that are copies of 
> pip-installable Python packages, which are dependencies of something, and 
> maintain placeholders for them in build/pkg/ (this will be completely true 
> as soon as #14840 is merged).
> This is a lot of unnecessary stuff to carry around, and is quite different 
> to what is done in pythonic (as well as ocaml, haskell, Macaulay2, etc) 
> universe.
> 
> My proposal is to get rid of these, and install pip-installable 
> dependencies directly. Perhaps maintaining a catalog of such dependencies 
> in some way; perhaps such facilities are provided by pip already, one needs 
> to check this out.
> We may keep (some) placeholders in build/pkg, for the purpose of 
> bootstrapping.
> One added advantage is  that we can provide a sane way to install packages 
> which cannot be installed from distributed sources, such as ssl support.
> 
> In case one wants all the sources needed for a complete Sage install 
> packaged locally,
> this is easy to achieve by providing a custom pip repo; such a repo may 
> become some kind of optional component.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to sage-devel+unsubscr...@googlegroups.com.
> To post to this group, send email to sage-devel@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.