Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-28 Thread Nick Coghlan 
On Wed, Nov 28, 2012 at 2:46 PM, Eric Snow wrote:

> I knew there was one more: http://bugs.python.org/issue16499 ("CLI
> option for isolated mode").
>

Along with another PYIOENCODING related one that the Blender folks reported
(Christian Heimes pointed it out to me earlier today).

Anyway, I created a page on the wiki for the data gathering process:

http://wiki.python.org/moin/CPythonInterpreterInitialization

It's now a matter of going through and sorting out:

1. What gets set during startup?
2. Where does it get set (or modified)?
3. How is that configured?

Once we have a good view of that, *then* we can start looking for ways to
simplify the code, make the whole system more embedding friendly (i.e. by
giving the embedding app total control via a simple and clean API) and
still support the proposals for improvements.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-27 Thread Eric Snow 
On Tue, Nov 27, 2012 at 9:19 PM, Eric Snow  wrote:
> On Thu, Nov 22, 2012 at 5:09 AM, Nick Coghlan  wrote:
>> On Thu, Nov 22, 2012 at 9:44 PM, Kristján Valur Jónsson
>>  wrote:
>>>
>>> Where in the tracker?  I tried searching but didn’t find it.
>>
>>
>> This one: http://bugs.python.org/issue13475
>>
>> This and the issue about being able to configure coverage.py cleanly in
>> subprocesses (http://bugs.python.org/issue14803) are the two issues which
>> have got me thinking about the interpreter startup and configuration process
>> in general lately. Both would add *more* complexity to an already
>> complicated and hard to follow initialisation sequence, so I now think we
>> should be look at opportunities for rationalisation *first*, before we make
>> things even messier.
>
> There's also http://bugs.python.org/issue15716 ("Ability to specify
> the PYTHONPATH via a command line flag").
>
> -eric

I knew there was one more: http://bugs.python.org/issue16499 ("CLI
option for isolated mode").

-eric
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-27 Thread Eric Snow 
On Thu, Nov 22, 2012 at 5:09 AM, Nick Coghlan  wrote:
> On Thu, Nov 22, 2012 at 9:44 PM, Kristján Valur Jónsson
>  wrote:
>>
>> Where in the tracker?  I tried searching but didn’t find it.
>
>
> This one: http://bugs.python.org/issue13475
>
> This and the issue about being able to configure coverage.py cleanly in
> subprocesses (http://bugs.python.org/issue14803) are the two issues which
> have got me thinking about the interpreter startup and configuration process
> in general lately. Both would add *more* complexity to an already
> complicated and hard to follow initialisation sequence, so I now think we
> should be look at opportunities for rationalisation *first*, before we make
> things even messier.

There's also http://bugs.python.org/issue15716 ("Ability to specify
the PYTHONPATH via a command line flag").

-eric
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-22 Thread Nick Coghlan 
On Thu, Nov 22, 2012 at 9:44 PM, Kristján Valur Jónsson <
krist...@ccpgames.com> wrote:

>  Where in the tracker?  I tried searching but didn’t find it.
>

This one: http://bugs.python.org/issue13475

This and the issue about being able to configure coverage.py cleanly in
subprocesses (http://bugs.python.org/issue14803) are the two issues which
have got me thinking about the interpreter startup and configuration
process in general lately. Both would add *more* complexity to an already
complicated and hard to follow initialisation sequence, so I now think we
should be look at opportunities for rationalisation *first*, before we make
things even messier.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-22 Thread Kristján Valur Jónsson 
Where in the tracker?  I tried searching but didn't find it.

I contributed to the pep405 discussions with similar concerns back in march:
http://mail.python.org/pipermail/python-dev/2012-March/117894.html


From: Python-Dev [mailto:python-dev-bounces+kristjan=ccpgames@python.org] 
On Behalf Of Daniel Holth


There is some activity in the tracker about adding the missing "add this to 
PYTHONPATH" / "isolate self from the environment" command line arguments to 
Python.

Daniel Holth
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-20 Thread Daniel Holth 
On Tue, Nov 20, 2012 at 9:44 AM, Nick Coghlan  wrote:

> On Tue, Nov 20, 2012 at 11:45 PM, Christian Tismer 
> wrote:
>
>>  On 20.11.12 12:39, Nick Coghlan wrote:
>>
>> On Tue, Nov 20, 2012 at 7:06 PM, Kristján Valur Jónsson <
>> krist...@ccpgames.com> wrote:
>>
>>>  I’m intrigued.  I thought this was merely so that one could do
>>>
>>> python –m mypackage.mysubpackage
>>>
>>> Can you refer me to the rationale and discussion about this feature?
>>>
>>
>> It was part of a fairly long progression in changes to the command line
>> execution support :)
>>
>> Top level package execution with -m came first in 2.4, arbitrary package
>> execution for -m arrived in 2.5 (along with the runpy module), directory
>> and zipfile execution (by supplying a valid sys.path entry as the "script"
>> command line argument) was added in 2.6/3.0, and finally officially
>> supported package execution via -m only arrived in 2.7 and 3.1 (a broken
>> version of the last accidentally existed in 2.5, but that was just a
>> mistake that arose when merging the import emulations in runpy and pkgutil,
>> and had a side effect that violated at least one of the import system
>> invariants).
>>
>> In the specific case of directory and zipfile execution, discussion
>> happened on the tracker: http://bugs.python.org/issue1739468
>>
>> It was never brought up on python-dev because Guido was participating
>> directly in the tracker discussion. Unfortunately, we then also forgot to
>> mention it in the original version of the 2.6 What's New doc, so the vast
>> majority of people missed the addition :(
>>
>>
>> Hi Nick,
>>
>> thank you very much for this story and the link to the issue tracker!
>> A very good move for Python which is really not mentioned enough
>> to make people more aware of a great feature.
>>
>> I think part of this discussion should get a more prominent place
>> for gems that can be found without knowing what to search. ;-)
>>
>
> Technically, that's the "using" guide:
> http://docs.python.org/2/using/cmdline.html#interface-options (see the
> explanation of what's executable under the "

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-20 Thread Nick Coghlan 
On Tue, Nov 20, 2012 at 11:45 PM, Christian Tismer wrote:

>  On 20.11.12 12:39, Nick Coghlan wrote:
>
> On Tue, Nov 20, 2012 at 7:06 PM, Kristján Valur Jónsson <
> krist...@ccpgames.com> wrote:
>
>>  I’m intrigued.  I thought this was merely so that one could do
>>
>> python –m mypackage.mysubpackage
>>
>> Can you refer me to the rationale and discussion about this feature?
>>
>
> It was part of a fairly long progression in changes to the command line
> execution support :)
>
> Top level package execution with -m came first in 2.4, arbitrary package
> execution for -m arrived in 2.5 (along with the runpy module), directory
> and zipfile execution (by supplying a valid sys.path entry as the "script"
> command line argument) was added in 2.6/3.0, and finally officially
> supported package execution via -m only arrived in 2.7 and 3.1 (a broken
> version of the last accidentally existed in 2.5, but that was just a
> mistake that arose when merging the import emulations in runpy and pkgutil,
> and had a side effect that violated at least one of the import system
> invariants).
>
> In the specific case of directory and zipfile execution, discussion
> happened on the tracker: http://bugs.python.org/issue1739468
>
> It was never brought up on python-dev because Guido was participating
> directly in the tracker discussion. Unfortunately, we then also forgot to
> mention it in the original version of the 2.6 What's New doc, so the vast
> majority of people missed the addition :(
>
>
> Hi Nick,
>
> thank you very much for this story and the link to the issue tracker!
> A very good move for Python which is really not mentioned enough
> to make people more aware of a great feature.
>
> I think part of this discussion should get a more prominent place
> for gems that can be found without knowing what to search. ;-)
>

Technically, that's the "using" guide:
http://docs.python.org/2/using/cmdline.html#interface-options (see the
explanation of what's executable under the "

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-20 Thread Christian Tismer 

On 20.11.12 12:39, Nick Coghlan wrote:
On Tue, Nov 20, 2012 at 7:06 PM, Kristján Valur Jónsson 
mailto:krist...@ccpgames.com>> wrote:


I’m intrigued.  I thought this was merely so that one could do

python –m mypackage.mysubpackage

Can you refer me to the rationale and discussion about this feature?


It was part of a fairly long progression in changes to the command 
line execution support :)


Top level package execution with -m came first in 2.4, arbitrary 
package execution for -m arrived in 2.5 (along with the runpy module), 
directory and zipfile execution (by supplying a valid sys.path entry 
as the "script" command line argument) was added in 2.6/3.0, and 
finally officially supported package execution via -m only arrived in 
2.7 and 3.1 (a broken version of the last accidentally existed in 2.5, 
but that was just a mistake that arose when merging the import 
emulations in runpy and pkgutil, and had a side effect that violated 
at least one of the import system invariants).


In the specific case of directory and zipfile execution, discussion 
happened on the tracker: http://bugs.python.org/issue1739468


It was never brought up on python-dev because Guido was participating 
directly in the tracker discussion. Unfortunately, we then also forgot 
to mention it in the original version of the 2.6 What's New doc, so 
the vast majority of people missed the addition :(


Hi Nick,

thank you very much for this story and the link to the issue tracker!
A very good move for Python which is really not mentioned enough
to make people more aware of a great feature.

I think part of this discussion should get a more prominent place
for gems that can be found without knowing what to search. ;-)

Is the issue tracker permanent enough to reference it?
Maybe there could be some auxiliary info page with proper keywords
that collects links to relevant discussions like this.
Do we have such a thing already?

ciao - chris

--
Christian Tismer :^)   
Software Consulting  : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 :*Starship* http://starship.python.net/
14482 Potsdam: PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04   9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
  whom do you want to sponsor today?   http://www.stackless.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-20 Thread Nick Coghlan 
On Tue, Nov 20, 2012 at 7:06 PM, Kristján Valur Jónsson <
krist...@ccpgames.com> wrote:

>  I’m intrigued.  I thought this was merely so that one could do
>
> python –m mypackage.mysubpackage
>
> Can you refer me to the rationale and discussion about this feature?
>

It was part of a fairly long progression in changes to the command line
execution support :)

Top level package execution with -m came first in 2.4, arbitrary package
execution for -m arrived in 2.5 (along with the runpy module), directory
and zipfile execution (by supplying a valid sys.path entry as the "script"
command line argument) was added in 2.6/3.0, and finally officially
supported package execution via -m only arrived in 2.7 and 3.1 (a broken
version of the last accidentally existed in 2.5, but that was just a
mistake that arose when merging the import emulations in runpy and pkgutil,
and had a side effect that violated at least one of the import system
invariants).

In the specific case of directory and zipfile execution, discussion
happened on the tracker: http://bugs.python.org/issue1739468

It was never brought up on python-dev because Guido was participating
directly in the tracker discussion. Unfortunately, we then also forgot to
mention it in the original version of the 2.6 What's New doc, so the vast
majority of people missed the addition :(

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-20 Thread Kristján Valur Jónsson 
I'm intrigued.  I thought this was merely so that one could do
python -m mypackage.mysubpackage
Can you refer me to the rationale and discussion about this feature?

K

From: Nick Coghlan [mailto:ncogh...@gmail.com]
Sent: 18. nóvember 2012 11:25
To: Kristján Valur Jónsson
Cc: Christian Tismer; python-dev@python.org
Subject: Re: [Python-Dev] Generally boared by installation (Re: Setting project 
home path the best way)

Easily bundling dependencies is a key principle behind the ability to execute 
directories and zipfiles that contain a top level __main__.py file that was 
added back in 2.6 (although the zipfile version doesn't play nicely with 
extension modules).

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com<mailto:ncogh...@gmail.com>   |   
Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-18 Thread Nick Coghlan 
On Sun, Nov 18, 2012 at 8:54 PM, Kristján Valur Jónsson <
krist...@ccpgames.com> wrote:

> I don't have a well formed solution in mind, but I would see it desirable
> to have a way for someone to release his package with all its dependencies
> as a self-contained and isolated unit.  E.g. if package foo.py relies on
> functionality from version 1.7 of bar.py, then that functionality could be
> bottled up for foo´s exclusive usage.
> Another package, baz.py, could then also make use of bar, but version 1.8.
>  The two bar versions would be isolated.
>
> Perhaps this is just a pipedream.  Even unpossible.  But it doesn't harm
> to try to think about better ways to do things.
>
>
Easily bundling dependencies is a key principle behind the ability to
execute directories and zipfiles that contain a top level __main__.py file
that was added back in 2.6 (although the zipfile version doesn't play
nicely with extension modules).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-18 Thread Kristján Valur Jónsson 
Yes!
For many years I have been very frustrated by the install-centric nature of 
python.  I am biased, of course, by the fact that I am developing an 
application where python is embedded, an application that needs to run out of 
the box.  A developer may have many many versions (branches) of the application 
on his drive, and each needs to work separately.
We have managed to isolate things, by patching python (and contributing that 
patch) to override the default library seach path (and ignore environment 
paths) when python is started up thogh the api.  All well and good.
But recently we have started in increasing amount to use external libraries and 
packages and we have been introduced to the dependency hell that is public 
python packages.  In this install-centric world, developers reference huge 
external packages without a second thought, which cause large dependency trees. 
 Using a simple tool may require whole HTTP frameworks to be downloaded.
What is worse is when there are versioning conflicts between those dependencies.

I don't have a well formed solution in mind, but I would see it desirable to 
have a way for someone to release his package with all its dependencies as a 
self-contained and isolated unit.  E.g. if package foo.py relies on 
functionality from version 1.7 of bar.py, then that functionality could be 
bottled up for foo´s exclusive usage.
Another package, baz.py, could then also make use of bar, but version 1.8.  The 
two bar versions would be isolated.

Perhaps this is just a pipedream.  Even unpossible.  But it doesn't harm to try 
to think about better ways to do things.
K


-Original Message-
From: Christian Tismer [mailto:tis...@stackless.com] 
Sent: 15. nóvember 2012 23:10
To: Kristján Valur Jónsson
Cc: python-dev@python.org
Subject: Generally boared by installation (Re: [Python-Dev] Setting project 
home path the best way)

Hi guys,

I am bored of installing things. 
Bored of things that happen to not work for some minor reasons. 
Reasons that are not immediately obvious. 
Things that don't work because some special case was not handled. 
Things that compile for half an hour and then complain that something is not as 
expected. 
May it be a compiler, a library, a command like pip or easy-install, a system 
like macports or homebrew, virtualenv, whatsoever. 

These things are all great if they work. 

When they do not work, the user is in some real trouble. And he reads hundreds 
Of blogs and sites and emails, which all answer a bit of slightly related 
questions, but all in all - 

This is not how Python should work !!

I am really bored and exhausted and annoyed by those packages which Pretend to 
make my life eadier, but they don't really. 

Something is really missing. I want something that is easy to use in all cases, 
also if it fails. 

Son't get me wrong, I like things like pip or virtualenv or homebrew. 
I just think they have to be rewritten completely. They have the wrong 
assumption that things work!

The opposite should be the truth: by default, things go wrong. Correctness is 
very fragile. 

I am thinking of a better concept that is harder to break. I thin to design a 
setup tool that is much more checking itself and does not trust in any 
assumption. 

Scenario:
After hours and hours, I find how to modify setup.py to function almost 
correctly for PySide. 

This was ridiculously hard to do! Settings for certain directories, included 
and stuff are not checked when they could be, but after compiling a lot of 
things!

After a lot of tries and headaches, I find out that virtualenv barfs on a 
simple link like ./.Python, the executable, when switching from stock Python to 
a different (homebrew) version!!

This was obviously never tested well, so it frustrates me quite a lot.  

I could fill a huge list full of complaints like that if I had time. But I 
don't. 

Instead, I think installation scripts are generally still wrong by concept 
today and need to be written in a different way. 

I would like to start a task force and some sprints about improving this 
situation. 
My goal is some unbreakable system of building blocks that are self-contained 
with no dependencies, that have a defined interface to talk to, and that know 
themselves completely by introspection. 

They should not work because they happen to work around all known defects, but 
by design and control. 

Whoever is interested to work with me on this is hereby honestly welcomed!

Cheers - chris

Sent from my Ei4Steve

On Nov 15, 2012, at 10:17, Kristján Valur Jónsson  wrote:

> When python is being run from a compile environment, it detects this by 
> looking for "Lib" folders in directories above the one containing the 
> executable. 
> (I always thought that this "special" execution mode, hardwired in, 
> was a bit odd, and suggested that this could be made a function of pep405) 
> Anyway, keeping your executable as part of the tree is the trick I use, and 
> to make things nice I put

[Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

2012-11-15 Thread Christian Tismer 
Hi guys,

I am bored of installing things. 
Bored of things that happen to not work for some minor reasons. 
Reasons that are not immediately obvious. 
Things that don't work because some special case was not handled. 
Things that compile for half an hour and then complain that something is not as 
expected. 
May it be a compiler, a library, a command like pip or easy-install, a system 
like macports or homebrew,
virtualenv, whatsoever. 

These things are all great if they work. 

When they do not work, the user is in some real trouble. And he reads hundreds
Of blogs and sites and emails, which all answer a bit of slightly related 
questions, but all in all - 

This is not how Python should work !!

I am really bored and exhausted and annoyed by those packages which
Pretend to make my life eadier, but they don't really. 

Something is really missing. I want something that is easy to use in all
cases, also if it fails. 

Son't get me wrong, I like things like pip or virtualenv or homebrew. 
I just think they have to be rewritten completely. They have the wrong 
assumption that things work!

The opposite should be the truth: by default, things go wrong. Correctness is 
very fragile. 

I am thinking of a better concept that is harder to break. I thin to design a 
setup tool that is much more checking itself and does not trust in any 
assumption. 

Scenario:
After hours and hours, I find how to modify setup.py to function almost 
correctly for PySide. 

This was ridiculously hard to do! Settings for certain directories, included 
and stuff are not checked when they could be, but after compiling a lot of 
things!

After a lot of tries and headaches, I find out that virtualenv barfs on a simple
link like ./.Python, the executable, when switching from stock Python to a 
different (homebrew) version!!

This was obviously never tested well, so it frustrates me quite a lot.  

I could fill a huge list full of complaints like that if I had time. But I 
don't. 

Instead, I think installation scripts are generally still wrong by concept 
today and need to
be written in a different way. 

I would like to start a task force and some sprints about improving this
situation. 
My goal is some unbreakable system of building blocks that are self-contained 
with no dependencies, that have a defined interface to talk to, and that know 
themselves completely by introspection. 

They should not work because they happen to work around all known defects, but 
by design and control. 

Whoever is interested to work with me on this is hereby honestly welcomed!

Cheers - chris

Sent from my Ei4Steve

On Nov 15, 2012, at 10:17, Kristján Valur Jónsson  wrote:

> When python is being run from a compile environment, it detects this by 
> looking for "Lib" folders in directories above the one containing the 
> executable. 
> (I always thought that this "special" execution mode, hardwired in, was a bit 
> odd, and suggested that this could be made a function of pep405)
> Anyway, keeping your executable as part of the tree is the trick I use, and 
> to make things nice I put  right next to it:
> site.py
> sitecustomize.py
> 
> sitecustomize.py is where you would put the logic to set sys.path by walking 
> up the hierarchy and finding the proper root.
> site.py is there to merely import sitecustomize.py, in case a site.py is not 
> found in all the default places python looks.
> 
> K
> 
> 
>> -Original Message-
>> From: Python-Dev [mailto:python-dev-
>> bounces+kristjan=ccpgames@python.org] On Behalf Of Christian Tismer
>> Sent: 11. nóvember 2012 20:31
>> To: python-dev@python.org
>> Subject: [Python-Dev] Setting project home path the best way
>> 
>> Hi friends,
>> 
>> I have a project that has its root somewhere on my machine.
>> This project has many folders and contains quite some modules.
>> 
>> There is a common root of the module tree, and I want to use
>> - either absolute imports
>> - relative imports with '.'
>> 
>> Problem:
>> 
>> - I want to run any module inside the heirarchy from the command-line
>> 
>> - this should work, regardless what my 'cwd' is
>> 
>> - this should work with or without virtualenv.
>> 
>> So far, things work fine with virtualenv, because sys.executable is in the
>> project module tree.
>> 
>> Without virtualenv, this is not so. But I hate to make settings like
>> PYTHONPATH, because these are not permanent. .
>> 
>> Question:
>> 
>> How should I define my project root dir in a unique way, without setting an
>> environment variable?
>> What is the lest intrusive way to spell that?
>> 
>> Reason:
>> 
>> I'd like to make things work correctly and unambigously when I call a script
>> inside the module heirarchy. Things are not fixed: there exist many
>> checkouts In the file system, and each should know where to search its
>> home/root in the tree.
>> 
>> Is this elegantly possible to deduce from the actually executed script file?
>> 
>> Cheers - chris
>> 
>> Sent from my Ei4Steve
>> __