Re: urlopen returns forbidden

[Chris Rebert]

On Sun, Feb 27, 2011 at 9:38 PM, monkeys paw  wrote:
> I have a working urlopen routine which opens
> a url, parses it for  tags and prints out
> the links in the page. On some sites, wikipedia for
> instance, i get a
>
> HTTP error 403, forbidden.
>
> What is the difference in accessing the site through a web browser
> and opening/reading the URL with python urllib2.urlopen?

The User-Agent header (http://en.wikipedia.org/wiki/User_agent ).
"By default, the URLopener class sends a User-Agent header of
urllib/VVV, where VVV is the urllib version number."
– http://docs.python.org/library/urllib.html

Some sites block obvious non-search-engine bots based on their HTTP
User-Agent header value.

You can override the urllib default:
http://docs.python.org/library/urllib.html#urllib.URLopener.version

Sidenote: Wikipedia has a proper API for programmatic browsing, likely
hence why it's blocking your program.

Cheers,
Chris
-- 
http://mail.python.org/mailman/listinfo/python-list

urlopen returns forbidden

[monkeys paw]

I have a working urlopen routine which opens
a url, parses it for  tags and prints out
the links in the page. On some sites, wikipedia for
instance, i get a

HTTP error 403, forbidden.

What is the difference in accessing the site through a web browser
and opening/reading the URL with python urllib2.urlopen?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Executing js/ajax in a sandboxed environment

[Amit Sethi]

On Sun, Feb 27, 2011 at 10:26 AM, Rohan Malhotra
 wrote:

> BeautifulSoup library only fetches source of page. I need the access
> to DOM after js execution with url as input parameter.
>
> Any pointers?
>

I am not sure but in case you need to make some ajax requests
mechanize might help.

http://wwwsearch.sourceforge.net/mechanize/

Please share the experience of what finally worked for you.


-- 
A-M-I-T S|S
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [ANN]VTD-XML 2.10

[Dinh]

It is advertised as fastest tool. Have anyone done homework with it yet?

On Sun, Feb 27, 2011 at 7:52 PM,  wrote:

> Jimmy,
>
> How does VTD-XML compare to XML tools in the stdlib or to 3rd party
> alternatives like lxml?
>
> Thank you,
> Malcolm
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: nntplib encoding problem

[Thomas L. Shinnick]

At 08:12 PM 2/27/2011, you wrote:

On 28/02/2011 01:31, Laurent Duchesne wrote:

Hi,

I'm using python 3.2 and got the following error:


nntpClient = nntplib.NNTP_SSL(...)
nntpClient.group("alt.binaries.cd.lossless")
nntpClient.over((534157,534157))

... 'subject': 'Myl\udce8ne Farmer - Anamorphosee (Japan Edition) 1995
[02/41] "Back.jpg" yEnc (1/3)' ...

overview = nntpClient.over((534157,534157))
print(overview[1][0][1]['subject'])

Traceback (most recent call last):
File "", line 1, in 
UnicodeEncodeError: 'utf-8' codec can't encode character '\udce8' in
position 3: surrogates not allowed

I'm not sure if I should report this as a bug in nntplib or if I'm doing
something wrong.

Note that I get the same error if I try to write this data to a file:


h = open("output.txt", "a")
h.write(overview[1][0][1]['subject'])

Traceback (most recent call last):
File "", line 1, in 
UnicodeEncodeError: 'utf-8' codec can't encode character '\udce8' in
position 3: surrogates not allowed

It's looks like the subject was originally encoded as Latin-1 (or
similar) (b'Myl\xe8ne Farmer - Anamorphosee (Japan Edition) 1995
[02/41] "Back.jpg" yEnc (1/3)') but has been decoded as UTF-8 with
"surrogateescape" passed as the "errors" parameter.


3.2 Docs
  6.6. codecs — Codec registry and base classes
Possible values for errors are
  'surrogateescape': replace with surrogate U+DCxx, see PEP 383

Yes, it would have been 0xE8 -  Mylène

Googling on surrogateescape I can see lots of 
argument about unintended outcomes  yikes!



You can get the "correct" Unicode by encoding as UTF-8 with
"surrogateescape" and then decoding as Latin-1:


overview[1][0][1]['subject'].encode("utf-8", 
"surrogateescape").decode("latin-1")
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: nntplib encoding problem

[MRAB]

On 28/02/2011 01:31, Laurent Duchesne wrote:

Hi,

I'm using python 3.2 and got the following error:


nntpClient = nntplib.NNTP_SSL(...)
nntpClient.group("alt.binaries.cd.lossless")
nntpClient.over((534157,534157))

... 'subject': 'Myl\udce8ne Farmer - Anamorphosee (Japan Edition) 1995
[02/41] "Back.jpg" yEnc (1/3)' ...

overview = nntpClient.over((534157,534157))
print(overview[1][0][1]['subject'])

Traceback (most recent call last):
File "", line 1, in 
UnicodeEncodeError: 'utf-8' codec can't encode character '\udce8' in
position 3: surrogates not allowed

I'm not sure if I should report this as a bug in nntplib or if I'm doing
something wrong.

Note that I get the same error if I try to write this data to a file:


h = open("output.txt", "a")
h.write(overview[1][0][1]['subject'])

Traceback (most recent call last):
File "", line 1, in 
UnicodeEncodeError: 'utf-8' codec can't encode character '\udce8' in
position 3: surrogates not allowed


It's looks like the subject was originally encoded as Latin-1 (or
similar) (b'Myl\xe8ne Farmer - Anamorphosee (Japan Edition) 1995
[02/41] "Back.jpg" yEnc (1/3)') but has been decoded as UTF-8 with
"surrogateescape" passed as the "errors" parameter.

You can get the "correct" Unicode by encoding as UTF-8 with
"surrogateescape" and then decoding as Latin-1:

overview[1][0][1]['subject'].encode("utf-8", 
"surrogateescape").decode("latin-1")

--
http://mail.python.org/mailman/listinfo/python-list

Re: Various behaviors of doctest

[Gnarlodious]

Yeah, I just spent about 2 hours trying everything I could think of...
without success. Including your suggestions. Guess I'll have to skip
it. But thanks for the ideas.

-- Gnarlie
-- 
http://mail.python.org/mailman/listinfo/python-list

nntplib encoding problem

[Laurent Duchesne]

Hi,

I'm using python 3.2 and got the following error:


nntpClient = nntplib.NNTP_SSL(...)
nntpClient.group("alt.binaries.cd.lossless")
nntpClient.over((534157,534157))
... 'subject': 'Myl\udce8ne Farmer - Anamorphosee (Japan Edition) 1995 
[02/41] "Back.jpg" yEnc (1/3)' ...

overview = nntpClient.over((534157,534157))
print(overview[1][0][1]['subject'])

Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'utf-8' codec can't encode character '\udce8' in 
position 3: surrogates not allowed


I'm not sure if I should report this as a bug in nntplib or if I'm 
doing something wrong.


Note that I get the same error if I try to write this data to a file:


h = open("output.txt", "a")
h.write(overview[1][0][1]['subject'])

Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'utf-8' codec can't encode character '\udce8' in 
position 3: surrogates not allowed


Thanks,
Laurent
--
http://mail.python.org/mailman/listinfo/python-list

Re: Various behaviors of doctest

[Steven D'Aprano]

On Sun, 27 Feb 2011 08:55:10 -0800, Gnarlodious wrote:

> Using the doctest module, I get three different outputs:
> 
> 1) From the Terminal shell, I see a full report: 
> python ~/Sites/Sectrum/Filter.py -v

Can we assume that Filter.py, whatever that is, runs doctest.testmod()?


 
> 2) From the Terminal interactive session, I see an abbreviated report of
> only the failures:
> from doctest import testmod; testmod(Filter)

That's because you haven't specified the verbose option, either by 
injecting -v into sys.argv, or by verbose=True as an argument to testmod.

 
> 3) From a browser CGI, I see a named tuple: from doctest import testmod;
> doctest.testmod()
> 
> This returns output like:
> 
> TestResults(failed=1, attempted=5)
> 
> The browser is invoking the exact same code as the shell doctest, so why
> should it return a named tuple? 

Because testmod *always* returns a named tuple. Read the Fine Manual at 
the interactive interpreter with:

help(testmod)


> And how do I get it to return the full
> text so I can read it in the browser?

You don't. Consider a simplified version:

def testmod():
print "lots of stuff"
return (failures, total)


If you want to capture "lots of stuff", you have to either play nasty 
tricks with replacing standard out with a StringIO instance, or similar, 
or you have to look at the implementation of testmod() and re-implement 
it yourself.

Oh, here's a thought, if you're running Python 3, you could monkey-patch 
doctest:


# COMPLETELY UNTESTED!!!

from stringio import StringIO
from functools import partial
buffer = StringIO()
myprint = partial(print, file=buffer)
doctest.print = myprint

doctest.testmod()

Now somehow you have to pass buffer.as_string() to your CGI program, in 
whatever way you would normally do so.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python3.2m installed as (additional) binary

[andrew cooke]

[Sorry I clicked the wrong button so I think my prev reply went only to Tom]

Thanks.  Yes, they're hard linked.  And the bug report mentions PEP 3149 which 
says that "m" means --with-pymalloc was used 
http://www.python.org/dev/peps/pep-3149/

Cheers,
Andrew
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python3.2m installed as (additional) binary

[Tom Zych]

Tom Zych wrote:
> andrew cooke wrote:
>> -rwxr-xr-x 2 root root 7368810 2011-02-27 13:03 /usr/local/bin/python3.2
>> -rwxr-xr-x 2 root root 7368810 2011-02-27 13:03 /usr/local/bin/python3.2m

> I suspect the "m" name is what gets built and the "no m" is an alias for
> backwards-compatibility. Not sure why they did the "m", though.

Ah, this may be it:

http://bugs.python.org/issue9807

Search for "build flag". Apparently the "m" indicates that it was
compiled with a nonstandard compiler option, or some such; maybe
something to do with being on a 64-bit platform. The makefile or
configure log may shed some light.

-- 
Tom Zych / freethin...@pobox.com
Quidquid latine dictum sit, altum viditur.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python3.2m installed as (additional) binary

[Tom Zych]

andrew cooke wrote:
> I just downloaded, built and altinstalled Python3.2 on Linux x64.  I noticed 
> that in /usr/local/bin I have two identical (says diff) binaries called 
> Python3.2 and Python3.2m.  Is this expected?  I can find very little 
> reference to them apart from a short discussion in python-dev where someone 
> asks if users will be confused by this and Barry Warsaw replies saying that 
> they won't see them (well, I do!).
> 
> All I did was the usual ./configure; make; sudo make altinstall
> 
> So is this a bug?
> 
> pl6 Python-3.2: ls -l /usr/local/bin/python3.2*
> -rwxr-xr-x 2 root root 7368810 2011-02-27 13:03 /usr/local/bin/python3.2
> -rwxr-xr-x 2 root root 7368810 2011-02-27 13:03 /usr/local/bin/python3.2m
> -rwxr-xr-x 1 root root1826 2011-02-27 13:03 
> /usr/local/bin/python3.2m-config
> pl6 Python-3.2: diff  /usr/local/bin/python3.2 /usr/local/bin/python3.2m
> pl6 Python-3.2:

It looks like they're hard links of each other. Try:
  ls -li /usr/local/bin/python3.2*

Are the inode numbers the same?

I suspect the "m" name is what gets built and the "no m" is an alias for
backwards-compatibility. Not sure why they did the "m", though.

-- 
Tom Zych / freethin...@pobox.com
Quidquid latine dictum sit, altum viditur.
-- 
http://mail.python.org/mailman/listinfo/python-list

mortar_rdb 1.1.0 released!

[Chris Withers]

Hi All,

I'm pleased to announce a new release of mortar_rdb.

This package ties together SQLAlchemy, sqlalchemy-migrate and
the component architecture to make it easy to develop projects
using SQLAlchemy through their complete lifecycle.

This release allows you to register SessionExtensions with your sessions.

If you'd like to see what mortar_rdb can do for you, please have a read 
of the narrative usage docs, which give a quick run through of the 
lifespan of a project developers using mortar_rdb:


http://packages.python.org/mortar_rdb/use.html

Full package details including mailing list, irc and bug tracker details 
can be found here:


http://www.simplistix.co.uk/software/python/mortar_rdb

cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
   - http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list

python3.2m installed as (additional) binary

[andrew cooke]

Hi,

I just downloaded, built and altinstalled Python3.2 on Linux x64.  I noticed 
that in /usr/local/bin I have two identical (says diff) binaries called 
Python3.2 and Python3.2m.  Is this expected?  I can find very little reference 
to them apart from a short discussion in python-dev where someone asks if users 
will be confused by this and Barry Warsaw replies saying that they won't see 
them (well, I do!).

All I did was the usual ./configure; make; sudo make altinstall

So is this a bug?
Andrew

pl6 Python-3.2: ls -l /usr/local/bin/python3.2*
-rwxr-xr-x 2 root root 7368810 2011-02-27 13:03 /usr/local/bin/python3.2
-rwxr-xr-x 2 root root 7368810 2011-02-27 13:03 /usr/local/bin/python3.2m
-rwxr-xr-x 1 root root1826 2011-02-27 13:03 /usr/local/bin/python3.2m-config
pl6 Python-3.2: diff  /usr/local/bin/python3.2 /usr/local/bin/python3.2m
pl6 Python-3.2:

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Language Question?

[Paul Symonds]

cheers

--
http://mail.python.org/mailman/listinfo/python-list

Re: Python Language Question?

[Rotwang]

On 27/02/2011 16:45, Paul Symonds wrote:

Can someone give and explanation of what is happening with the following:


a,b = 0,1 # this assigns a = 0 and b = 1



while b < 10:

... print b
... a, b = b, a+b
...
1
1
2
3
5
8



a=0
b=1
while b < 1000:

... print b
... a = b
... b = a+b
...
1
2
4
8
16
32
64
128
256
512


Why is this statement .. a, b = b, a+b
different to ... a = b
... b = a+b


Because in the statement

   a, b = b, a + b

Python evaluates both expressions on the RHS before assigning their 
values to the variables on the LHS, whereas in the statements


   a = b
   b = a + b

Python assigns the RHS of the first statement to the variable a, /then/ 
evaluates the RHS of the second statement; the value of a has already 
been changed by the time the RHS of the second statement gets evaluated. 
So e.g. if a = 0 and b = 1, then the first statement says to let a = 1, 
b = 0 + 1, while the other two statements say to let a = 1, then b = 1 + 
1 (the first "1" being the new value of a).

--
http://mail.python.org/mailman/listinfo/python-list

Various behaviors of doctest

[Gnarlodious]

Using the doctest module, I get three different outputs:

1) From the Terminal shell, I see a full report:
python ~/Sites/Sectrum/Filter.py -v

2) From the Terminal interactive session, I see an abbreviated report
of only the failures:
from doctest import testmod; testmod(Filter)

3) From a browser CGI, I see a named tuple:
from doctest import testmod; doctest.testmod()

This returns output like:

TestResults(failed=1, attempted=5)

The browser is invoking the exact same code as the shell doctest, so
why should it return a named tuple? And how do I get it to return the
full text so I can read it in the browser?

-- Gnarlie
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Language Question?

[Benjamin Kaplan]

On Sun, Feb 27, 2011 at 11:45 AM, Paul Symonds  wrote:
> Can someone give and explanation of what is happening with the following:
>
 a,b = 0,1                       # this assigns a = 0 and b = 1
>
 while b < 10:
>
> ...     print b
> ...     a, b = b, a+b
> ...
> 1
> 1
> 2
> 3
> 5
> 8
>
>
 a=0
 b=1
 while b < 1000:
>
> ...     print b
> ...     a = b
> ...     b = a+b
> ...
> 1
> 2
> 4
> 8
> 16
> 32
> 64
> 128
> 256
> 512
>
>
> Why is this statement ..     a, b = b, a+b

Python evaluates the entire right side of the assignment, then assigns
it to the left side. So if a=1 and b=2, then
a, b = b, a+b
a,b = 2, 1+2
a,b = 2, 3
a = 2
b = 3


> different to ...     a = b
> ...                             b = a+b
>
Here, you're evaluating sequentially.
a = b
a = 2
b = a + b
b = 2 + 2
b = 4.

>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Python Language Question?

[Paul Symonds]

Can someone give and explanation of what is happening with the following:


a,b = 0,1   # this assigns a = 0 and b = 1



while b < 10:

... print b
... a, b = b, a+b
...
1
1
2
3
5
8



a=0
b=1
while b < 1000:

... print b
... a = b
... b = a+b
...
1
2
4
8
16
32
64
128
256
512


Why is this statement .. a, b = b, a+b
different to ... a = b
... b = a+b


--
http://mail.python.org/mailman/listinfo/python-list

Re: Python getting stuck

[Emile van Sebille]

On 2/26/2011 5:10 PM Dan Stromberg said...

Agreed, a more detailed description would greatly help us help you, but
if you're in the standard windows terminal emulator widget (command.com
, cmd.exe, powershell), then don't click in the
window without first turning on "quick edit" mode.  Otherwise the
emulator will just act silly.

It's kind of astonishing that any computer company would allow such an
annoying bug to live so long.




Well, people who are aware of command line terminal emulators are not 
the manufacturer's primary target audience.  Would be kinda like Henry 
Ford offering red cars...


Emile


--
http://mail.python.org/mailman/listinfo/python-list

Re: I'm happy with Python 2.5

[Bill Allen]

On Sun, Feb 27, 2011 at 07:34, n00m  wrote:

> Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
> (Intel)] on win32
>
> and Idon't move neither up nor down from it (the best & the fastest
> version)
> -- 


Trolls should also be happy...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I'm happy with Python 2.5

[Tom Zych]

n00m wrote:
> Am I turmoiling your wishful thinking?
> You may nourish it till the end of time.

Let us cease to nourish those fabled ones who dwell under bridges.

-- 
Tom Zych / freethin...@pobox.com
Quidquid latine dictum sit, altum viditur.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I'm happy with Python 2.5

[n00m]

http://www.spoj.pl/problems/TMUL/

Python's "print a * b" gets Time Limit Exceeded.

=
PHP's code
=
fscanf(STDIN, "%d\n", &$tcs);
while ($tcs--) {
fscanf(STDIN, "%s %s\n", &$n, &$m);
echo bcmul($n, $m, 0)."\n";
}
=
does it in 4.8s
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python getting stuck

[Tom Zych]

Colin J. Williams wrote:
> On 26-Feb-11 18:55 PM, Shanush Premathasarathan wrote:
>> When I use cut, copy, paste, and any keyboard shortcuts, Python
>> freezes and I am unable to use Python. Please Help as quick as
>> possible!!!

> What operating system are you using?

If it's some kind of Unix, are you using a text console, or X?
If X, what window manager?

-- 
Tom Zych / freethin...@pobox.com
Quidquid latine dictum sit, altum viditur.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with python 3.2 and circular imports

[Frank Millman]

"Steven D'Aprano"  wrote in message 
news:4d6a56aa$0$29972$c3e8da3$54964...@news.astraweb.com...

On Sun, 27 Feb 2011 12:08:12 +0200, Frank Millman wrote:


Assume the following structure -

main.py
/pkg
__init__.py
mod1.py
mod2.py

main.py
from pkg import mod1

mod1.py
import mod2

mod2.py
  import mod1



If you change the "import mod*" lines to "import pkg.mod*" it works for
me in Python 3.1 and 3.2.

According to my understand of PEP 328, "from . import mod*" should work,
but I agree with you that it doesn't.

If you get rid of the circular import, it does work. So I suspect a bug.




Thanks, Steven.

I confirm that 'import pkg.mod* works. Unfortunately I am using sub-packages 
as well, which means that to refer to an object in the sub-package I need to 
use w.x.y.z every time, which gets to be a lot of typing! I will stick to my 
hack of putting the package name in sys.path for now, unless someone comes 
up with a better idea.


Frank


--
http://mail.python.org/mailman/listinfo/python-list

Re: I'm happy with Python 2.5

[n00m]

On Feb 27, 3:58 pm, Grigory Javadyan 
wrote:
> what the hell does that have to do with anything
>
> On Sun, Feb 27, 2011 at 5:34 PM, n00m  wrote:
> > Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
> > (Intel)] on win32
>
> > and Idon't move neither up nor down from it (the best & the fastest
> > version)
> > --
> >http://mail.python.org/mailman/listinfo/python-list
>
>

Am I turmoiling your wishful thinking?
You may nourish it till the end of time.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: issue on internal import in a package

[Frank Millman]

"人言落日是天涯，望极天涯不见家"  wrote in message 
news:9529d52b-01b2-402c-a0a0-1e9240038...@l14g2000pre.googlegroups.com...

On Feb 27, 9:38 pm, "Frank Millman"  wrote:

"人言落日是天涯，望极天涯不见家"  wrote in message

news:fa94323b-d859-4599-b236-c78a22b3d...@t19g2000prd.googlegroups.com...

> On Feb 27, 9:22 pm, "Frank Millman"  wrote:

> This behavior is by design or just a bug for Python3.x ?

Definitely by design.

Have a look at PEP 328 -http://www.python.org/dev/peps/pep-0328/

"In Python 2.4 and earlier, if you're reading a module located inside a
package, it is not clear whether
import foo
refers to a top-level module or to another module inside the package. As
Python's library expands, more and more existing package internal modules
suddenly shadow standard library modules by accident. It's a particularly
difficult problem inside packages because there's no way to specify which
module is meant. To resolve the ambiguity, it is proposed that foo will
always be a module or package reachable from sys.path. This is called an
absolute import."

HTH

Frank


Yes, it's okay with the change in a.py with below line:
from . import b

But another issue occurred if I want to run the a.py separately.
$ cd module
$ python a.py
Traceback (most recent call last):
 File "a.py", line 1, in 
   from . import b
ValueError: Attempted relative import in non-package

Does that mean the relative import only allowed in the package.
And it cannot be run as __main__ program unless I change the relative
import back to absolute import?
I think this behavior is strange and difficult to use.



I think that PEP 366 addresses your question -

   http://www.python.org/dev/peps/pep-0366/

I came across this while investigating a separate problem. It looks a bit 
complicated, and I don't have such a requirement, so I did not look at it 
too closely.


Have a read and see if it solves your problem. If not post a follow-up 
message here and see if someone else can offer more information.


Frank


--
http://mail.python.org/mailman/listinfo/python-list

Re: I'm happy with Python 2.5

[n00m]

Steve, see a list of accepted langs there, in bottom dropdown:
http://www.spoj.pl/submit/ There *was* Python 2.6.
Then admins shifted back to 2.5. People vote by their legs.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I'm happy with Python 2.5

[Grigory Javadyan]

what the hell does that have to do with anything

On Sun, Feb 27, 2011 at 5:34 PM, n00m  wrote:
> Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
> (Intel)] on win32
>
> and Idon't move neither up nor down from it (the best & the fastest
> version)
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with python 3.2 and circular imports

[Steven D'Aprano]

On Sun, 27 Feb 2011 12:08:12 +0200, Frank Millman wrote:

> Assume the following structure -
> 
> main.py
> /pkg
> __init__.py
> mod1.py
> mod2.py
> 
> main.py
> from pkg import mod1
> 
> mod1.py
> import mod2
> 
> mod2.py
>   import mod1


If you change the "import mod*" lines to "import pkg.mod*" it works for 
me in Python 3.1 and 3.2.

According to my understand of PEP 328, "from . import mod*" should work, 
but I agree with you that it doesn't.

If you get rid of the circular import, it does work. So I suspect a bug.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: issue on internal import in a package

[人言落日是天涯，望极天涯不见家]

On Feb 27, 9:38 pm, "Frank Millman"  wrote:
> "人言落日是天涯，望极天涯不见家"  wrote in message
>
> news:fa94323b-d859-4599-b236-c78a22b3d...@t19g2000prd.googlegroups.com...
>
> > On Feb 27, 9:22 pm, "Frank Millman"  wrote:
>
> > This behavior is by design or just a bug for Python3.x ?
>
> Definitely by design.
>
> Have a look at PEP 328 -http://www.python.org/dev/peps/pep-0328/
>
> "In Python 2.4 and earlier, if you're reading a module located inside a
> package, it is not clear whether
> import foo
> refers to a top-level module or to another module inside the package. As
> Python's library expands, more and more existing package internal modules
> suddenly shadow standard library modules by accident. It's a particularly
> difficult problem inside packages because there's no way to specify which
> module is meant. To resolve the ambiguity, it is proposed that foo will
> always be a module or package reachable from sys.path. This is called an
> absolute import."
>
> HTH
>
> Frank

Yes, it's okay with the change in a.py with below line:
from . import b

But another issue occurred if I want to run the a.py separately.
$ cd module
$ python a.py
Traceback (most recent call last):
  File "a.py", line 1, in 
from . import b
ValueError: Attempted relative import in non-package

Does that mean the relative import only allowed in the package.
And it cannot be run as __main__ program unless I change the relative
import back to absolute import?
I think this behavior is strange and difficult to use.


Doesn't
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I'm happy with Python 2.5

[Steven D'Aprano]

On Sun, 27 Feb 2011 05:34:44 -0800, n00m wrote:

> Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
> (Intel)] on win32
> 
> and Idon't move neither up nor down from it (the best & the fastest
> version)

Congratulations.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: issue on internal import in a package

[Frank Millman]

"人言落日是天涯，望极天涯不见家"  wrote in message 
news:fa94323b-d859-4599-b236-c78a22b3d...@t19g2000prd.googlegroups.com...

On Feb 27, 9:22 pm, "Frank Millman"  wrote:

This behavior is by design or just a bug for Python3.x ?


Definitely by design.

Have a look at PEP 328 - http://www.python.org/dev/peps/pep-0328/

"In Python 2.4 and earlier, if you're reading a module located inside a 
package, it is not clear whether

import foo
refers to a top-level module or to another module inside the package. As 
Python's library expands, more and more existing package internal modules 
suddenly shadow standard library modules by accident. It's a particularly 
difficult problem inside packages because there's no way to specify which 
module is meant. To resolve the ambiguity, it is proposed that foo will 
always be a module or package reachable from sys.path. This is called an 
absolute import."


HTH

Frank


--
http://mail.python.org/mailman/listinfo/python-list

Re: I'm happy with Python 2.5

[n00m]

Python 3 is a tempor. lapse of reason.
Just my an intuitive sensation, nothing objective in it.
-- 
http://mail.python.org/mailman/listinfo/python-list

I'm happy with Python 2.5

[n00m]

Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit
(Intel)] on win32

and Idon't move neither up nor down from it (the best & the fastest
version)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with python 3.2 and circular imports

[Frank Millman]

"Ben Finney"  wrote in message 
news:87aahh6401@benfinney.id.au...

"Frank Millman"  writes:


Assume the following structure -

main.py
/pkg
   __init__.py
   mod1.py
   mod2.py

main.py
   from pkg import mod1

mod1.py
   import mod2

mod2.py
 import mod1


What are you expecting the result to be?



Simply that mod1 can refer to objects in mod2, and mod2 can refer to objects 
in mod1.



If it's about sharing objects between the modules, why not break the
circular dependency: factor out the common code to a module that both
the others import?


Any comments or suggestions will be appreciated.


Special cases aren't special enough to break the rules. If you think you
have a practical reason to do so, it would be best to make it explicit
when asking for help about this.



I am trying to understand what 'the rule' is. Your advice above suggests 
that you are one of those who recommend that circular imports are best 
avoided altogether. In an ideal world I would agree. However, the fact is 
that, no doubt due to a mental block I have, I do find myself in this 
situation from time to time, and I have not seen anything in the 
documentation or other literature that says it is absolutely wrong. 
Therefore, while I do try to avoid circular imports where possible, I would 
also like to know how to manage it in situations where I don't see a simple 
alternative.


From everything I have read about how the import mechanism works, I don't 
understand *why* the above construct fails. That is actually what I am 
asking for help with.


Thanks

Frank


--
http://mail.python.org/mailman/listinfo/python-list

Re: issue on internal import in a package

[人言落日是天涯，望极天涯不见家]

On Feb 27, 9:22 pm, "Frank Millman"  wrote:
> "Ben Finney"  wrote in message
>
> news:87ei6t646h@benfinney.id.au...
>
>
>
> > 人言落日是天涯，望极天涯不见家  writes:
>
> >> Here is a simple example:
> >> [app]
> >>       [module]
> >>             __init__.py   --> empty
> >>             a.py   --> import b
> >>             b.py  --> defined a function foo()
> >>       test.py
>
> >> In the test.py, contains the below statement:
> >> from module import a
> >> Execute the test.py will get error:
>
> > This works fine for me::
>
> >    $ mkdir --parents app/module/
> >    $ touch app/module/__init__.py
> >    $ printf "import b\n" > app/module/a.py
> >    $ printf "def foo(): pass\n" > app/module/b.py
> >    $ printf "from module import a\n" > app/test.py
> >    $ find .
> >    .
> >    ./app
> >    ./app/module
> >    ./app/module/__init__.py
> >    ./app/module/a.py
> >    ./app/module/b.py
> >    ./app/test.py
>
> >    $ python app/test.py
>
> >> Traceback (most recent call last):
> >>   File "", line 1, in 
> >>   File "module\a.py", line 1, in 
> >>     import b
> >> ImportError: No module named b
>
> >> Why the b.py can not be found by a.py?
>
> > I get no errors; the code appears to run fine. Perhaps the scenario is
> > not exactly as you describe?
>
> I get exactly the same result as the OP, using python 3.2 on both windows
> and linux. It works using python 2.6.
>
> I can fix it by changing a.py from 'import b' to 'from . import b'.
>
> As I understand it, the reason is that python 3.x will no longer look for an
> absolute import in the current package - it will only look in sys.path.
>
> Frank Millman

This behavior is by design or just a bug for Python3.x ?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: issue on internal import in a package

[Frank Millman]

"Ben Finney"  wrote in message 
news:87ei6t646h@benfinney.id.au...

人言落日是天涯，望极天涯不见家  writes:


Here is a simple example:
[app]
  [module]
__init__.py   --> empty
a.py   --> import b
b.py  --> defined a function foo()
  test.py

In the test.py, contains the below statement:
from module import a
Execute the test.py will get error:


This works fine for me::

   $ mkdir --parents app/module/
   $ touch app/module/__init__.py
   $ printf "import b\n" > app/module/a.py
   $ printf "def foo(): pass\n" > app/module/b.py
   $ printf "from module import a\n" > app/test.py
   $ find .
   .
   ./app
   ./app/module
   ./app/module/__init__.py
   ./app/module/a.py
   ./app/module/b.py
   ./app/test.py

   $ python app/test.py


Traceback (most recent call last):
  File "", line 1, in 
  File "module\a.py", line 1, in 
import b
ImportError: No module named b

Why the b.py can not be found by a.py?


I get no errors; the code appears to run fine. Perhaps the scenario is
not exactly as you describe?



I get exactly the same result as the OP, using python 3.2 on both windows 
and linux. It works using python 2.6.


I can fix it by changing a.py from 'import b' to 'from . import b'.

As I understand it, the reason is that python 3.x will no longer look for an 
absolute import in the current package - it will only look in sys.path.


Frank Millman


--
http://mail.python.org/mailman/listinfo/python-list

Re: Python getting stuck

[Colin J. Williams]

On 26-Feb-11 18:55 PM, Shanush Premathasarathan wrote:

Hi All,

When I use cut, copy, paste, and any keyboard shortcuts, Python freezes and I 
am unable to use Python. Please Help as quick as possible!!!

Thanks a lot.

Kind Regards
Big Python fan!
Shanush


What operating system are you using?

Colin W.
--
http://mail.python.org/mailman/listinfo/python-list

Re: issue on internal import in a package

[人言落日是天涯，望极天涯不见家]

On Feb 27, 8:40 pm, Ben Finney  wrote:
> 人言落日是天涯，望极天涯不见家  writes:
> > Here is a simple example:
> > [app]
> >       [module]
> >             __init__.py   --> empty
> >             a.py   --> import b
> >             b.py  --> defined a function foo()
> >       test.py
>
> > In the test.py, contains the below statement:
> > from module import a
> > Execute the test.py will get error:
>
> This works fine for me::
>
>     $ mkdir --parents app/module/
>     $ touch app/module/__init__.py
>     $ printf "import b\n" > app/module/a.py
>     $ printf "def foo(): pass\n" > app/module/b.py
>     $ printf "from module import a\n" > app/test.py
>     $ find .
>     .
>     ./app
>     ./app/module
>     ./app/module/__init__.py
>     ./app/module/a.py
>     ./app/module/b.py
>     ./app/test.py
>
>     $ python app/test.py
>
> > Traceback (most recent call last):
> >   File "", line 1, in 
> >   File "module\a.py", line 1, in 
> >     import b
> > ImportError: No module named b
>
> > Why the b.py can not be found by a.py?
>
> I get no errors; the code appears to run fine. Perhaps the scenario is
> not exactly as you describe?
>
> --
>  \       “If we listen only to those who are like us, we will squander |
>   `\   the great opportunity before us: To live together peacefully in |
> _o__)            a world of unresolved differences.” —David Weinberger |
> Ben Finney

Thanks for your reply.
What's the version of your Python?  My version is 3.2.
Python 2.5/2.6 doesn't have this issue.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [ANN]VTD-XML 2.10

[python]

Jimmy,

How does VTD-XML compare to XML tools in the stdlib or to 3rd party
alternatives like lxml?

Thank you,
Malcolm
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Problem with python 3.2 and circular imports

[Ben Finney]

"Frank Millman"  writes:

> Assume the following structure -
>
> main.py
> /pkg
>__init__.py
>mod1.py
>mod2.py
>
> main.py
>from pkg import mod1
>
> mod1.py
>import mod2
>
> mod2.py
>  import mod1

What are you expecting the result to be?

If it's about sharing objects between the modules, why not break the
circular dependency: factor out the common code to a module that both
the others import?

> Any comments or suggestions will be appreciated.

Special cases aren't special enough to break the rules. If you think you
have a practical reason to do so, it would be best to make it explicit
when asking for help about this.

-- 
 \   “The long-term solution to mountains of waste is not more |
  `\  landfill sites but fewer shopping centres.” —Clive Hamilton, |
_o__)_Affluenza_, 2005 |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: issue on internal import in a package

[Ben Finney]

人言落日是天涯，望极天涯不见家  writes:

> Here is a simple example:
> [app]
>   [module]
> __init__.py   --> empty
> a.py   --> import b
> b.py  --> defined a function foo()
>   test.py
>
> In the test.py, contains the below statement:
> from module import a
> Execute the test.py will get error:

This works fine for me::

$ mkdir --parents app/module/
$ touch app/module/__init__.py
$ printf "import b\n" > app/module/a.py
$ printf "def foo(): pass\n" > app/module/b.py
$ printf "from module import a\n" > app/test.py
$ find .
.
./app
./app/module
./app/module/__init__.py
./app/module/a.py
./app/module/b.py
./app/test.py

$ python app/test.py 

> Traceback (most recent call last):
>   File "", line 1, in 
>   File "module\a.py", line 1, in 
> import b
> ImportError: No module named b
>
> Why the b.py can not be found by a.py?

I get no errors; the code appears to run fine. Perhaps the scenario is
not exactly as you describe?

-- 
 \   “If we listen only to those who are like us, we will squander |
  `\   the great opportunity before us: To live together peacefully in |
_o__)a world of unresolved differences.” —David Weinberger |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: issue on internal import in a package

[人言落日是天涯，望极天涯不见家]

On Feb 27, 8:11 pm, 人言落日是天涯，望极天涯不见家  wrote:
> Here is a simple example:
> [app]
>       [module]
>             __init__.py   --> empty
>             a.py   --> import b
>             b.py  --> defined a function foo()
>       test.py
>
> In the test.py, contains the below statement:
> from module import a
> Execute the test.py will get error:
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "module\a.py", line 1, in 
>     import b
> ImportError: No module named b
>
> Why the b.py can not be found by a.py?

PS. This issue occurred on Python3.2
It's okay in the Python2.6 & Python2.5
-- 
http://mail.python.org/mailman/listinfo/python-list

issue on internal import in a package

[人言落日是天涯，望极天涯不见家]

Here is a simple example:
[app]
  [module]
__init__.py   --> empty
a.py   --> import b
b.py  --> defined a function foo()
  test.py

In the test.py, contains the below statement:
from module import a
Execute the test.py will get error:
Traceback (most recent call last):
  File "", line 1, in 
  File "module\a.py", line 1, in 
import b
ImportError: No module named b

Why the b.py can not be found by a.py?
-- 
http://mail.python.org/mailman/listinfo/python-list

Problem with python 3.2 and circular imports

[Frank Millman]

Hi all

I thought I was getting the hang of circular imports, but after upgrading to 
python 3.2 I am stumped again. I know some people think that circular 
imports are always bad, but others suggest that, provided you understand the 
potential problems, they can be acceptable.


Assume the following structure -

main.py
/pkg
   __init__.py
   mod1.py
   mod2.py

main.py
   from pkg import mod1

mod1.py
   import mod2

mod2.py
 import mod1

Using python 2.6, running main.py works.

After running 2to3.py on the above directory, mod1.py was changed to 'from . 
import mod2' and mod2.py was changed to 'from . import mod1'.


With python 3.2, it now fails with the following traceback -

Traceback (most recent call last):
 File "main.py", line 1, in 
   from pkg import mod1
 File "pkg\mod1.py", line 1, in 
   from . import mod2
 File "pkg\mod2.py", line 1, in 
   from . import mod1
ImportError: cannot import name mod1

I have read the relevant peps and various other material, understood them to 
a certain extent, tried several alternatives, but could not find a solution.


I have found a hack that works, but I don't like it very much. I added the 
following to '__init__.py' -

 import os
 import sys
 sys.path.insert(0, os.path.dirname(__file__))

This adds the package name into the search path.

Then I changed mod1.py and mod2.py back to the original 'import mod2' and 
'import mod1'.


It works, but it seems to be defeating the purpose of PEP 328, which I 
thought was an improvement.


Any comments or suggestions will be appreciated.

Frank Millman


--
http://mail.python.org/mailman/listinfo/python-list