Re: SOLVED! Re: Weird Stuff (Markdown, syntax highlighting and Python)

[dn via Python-list]

On 29/05/24 06:49, Gilmeh Serda via Python-list wrote:


Solved by using a different method.



Hedonist for hire... no job too easy!


This combination of sig-file and content seems sadly ironic.


How about CONTRIBUTING to the community by explaining 'the solution' to 
people who may find a similar problem - in the similar manner to the 
various members who have helped YOU, voluntarily (and despite the 
paucity of source-information and response)?


--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: SOLVED! Re: Weird Stuff (Markdown, syntax highlighting and Python)

[o1bigtenor via Python-list]

On Tue, May 28, 2024 at 9:48 PM Gilmeh Serda via Python-list <
python-list@python.org> wrote:

>
> Solved by using a different method.
>
>
- - - And that was how?

TIA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird Stuff (Markdown, syntax highlighting and Python)

[Thomas Passin via Python-list]

On 5/26/2024 2:28 AM, Gilmeh Serda via Python-list wrote:

The web claims (I think on all pages I've read about Markdown and Python)
that this code should work, with some very minor variants on the topic:

```python

import os

with open(os.path.join('/home/user/apath', 'somefile')) as f:
 print(f.read())
```


There are different flavors of Markdown, so that might be a factor so 
far as details of the block are concerned.


What do you mean by it not "working"?  What do you see and what did you 
expect to see?  What did you see different when you used the next example?


How did you generate the output HTML file?


However, that is not the case. At least not for me (using Python 3.12.3).
If instead I type it:

 #!python
 
 import os
 
 with open(os.path.join('/home/user/apath', 'somefile')) as f:

 print(f.read())

As an indented block (four spaces) and a shebang, THEN it works. You even
get line numbers by default.

N.b. if you don't know, you also need to generate a css file using
pygments to make this work.

Not until I started to read the markdown source code and its docs pages,
the coin dropped.

I'm posting this for other Markdown newbies that otherwise probably would
spend hours trying to make it work.


Speaking of Markdown. Does anybody out there have any idea how to turn on
table borders, adjust them (color/width/etc.) and such things? Currently I
have to add HTML to do so, which works, but isn't very nice. I'd hate to
spend an additional day or two, hunting for this info.

References:
https://pypi.org/project/Markdown/
https://python-markdown.github.io/



--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird Stuff (Markdown, syntax highlighting and Python)

[dn via Python-list]

With reference to another reply here, the "Weird stuff" came from 
reading the question, finding it unclear, and only later realising that 
whereas most people write Markdown-formatted documents for later 
processing, or perhaps docstrings in Markdown-format for collection by 
documentation systems; here, the objective appears to be using Python to 
generate Markdown.


How much have you used Markdown to any serious degree, before attempting 
this feat?



On 26/05/24 18:28, Gilmeh Serda via Python-list wrote:

The web claims (I think on all pages I've read about Markdown and Python)
that this code should work, with some very minor variants on the topic:


There are so many "variants", the problem is not "minor"!

Markdown users learn to use their tool (again, see @Grant's question) 
and work within the implementation of that "variant".


Like any other non-standardised tool, the users of some particular 
'version' often fail to realise that others using different versions may 
not enjoy the same experience. Plus-one for standardisation!



At the end of the message, the web.refs reveal use of a package which is 
based upon a variant of Markdown that is 20-years old(!), albeit with 
some updates to suit yet another variant. Neither variant-author famous 
for collaboration. The phrase YMMV springs to mind...



Some ten years ago, an effort was made to standardise Markup, and it 
ended-up being called CommonMark. Why is it not called "Standard 
Markdown" one might ask? Because the fellow who 'invented' Markdown 
objected. This very objection has likely led directly to your 
confusions, because the particular PyPi package is based upon that 
original definition...


Whereas, Markdown 3.6 is the most-recently updated Markdown search-hit 
on PyPi today, have you tried any of the others (which, ironically, may 
offer more recent and/or more standardised coverage)?



This has worked in all of the Markdown processors I have used or tried-out:

The (?reasonable) 'common-core', offers single back-ticks for code, 
triple back-ticks for a code-block, and the latter with or without a 
language specification which *usually* kicks-in syntax highlighting.




```python

import os

with open(os.path.join('/home/user/apath', 'somefile')) as f:
 print(f.read())
```

However, that is not the case. At least not for me (using Python 3.12.3).


It's not Python 3 that is the problem. It is the "Markdown 3.6" package!



If instead I type it:



I've not seen the hash-bang combination in-the-wild (but YMMV!)


 #!python
 
 import os
 
 with open(os.path.join('/home/user/apath', 'somefile')) as f:

 print(f.read())

As an indented block (four spaces) and a shebang, THEN it works. You even
get line numbers by default.


An indented-block is NOT necessarily the same as a code-block - just as 
"code" is not necessarily "Python".


Line numbers are great - although if a code snippet is extracted from 
the middle of some example code-file, the original line-numbers won't 
line-up with Markdown's...




N.b. if you don't know, you also need to generate a css file using
pygments to make this work.


That's not what the package's docs suggest: 
https://python-markdown.github.io/extensions/fenced_code_blocks/




Not until I started to read the markdown source code and its docs pages,
the coin dropped.

I'm posting this for other Markdown newbies that otherwise probably would
spend hours trying to make it work.


Speaking of Markdown. Does anybody out there have any idea how to turn on
table borders, adjust them (color/width/etc.) and such things? Currently I
have to add HTML to do so, which works, but isn't very nice. I'd hate to
spend an additional day or two, hunting for this info.


Again, heavily dependent upon the tool in-use. For example, most SSGs 
and doc-tools (which accept Markdown) have a .css or theming-system 
which enables 'decorations'.




References:
https://pypi.org/project/Markdown/
https://python-markdown.github.io/


Further reading:
https://en.wikipedia.org/wiki/Markdown
https://commonmark.org
https://pypi.org/search/?q=markdown

--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird Stuff (Markdown, syntax highlighting and Python)

[Grant Edwards via Python-list]

On 2024-05-26, Gilmeh Serda via Python-list  wrote:
> The web claims (I think on all pages I've read about Markdown and Python) 
> that this code should work, with some very minor variants on the topic:
>
> ```python
>
> import os
>
> with open(os.path.join('/home/user/apath', 'somefile')) as f:
> print(f.read())
> ```
>
> However, that is not the case.

For me, that block formats as expected using Python markdown.

What do you mean by "this code should work [...] that is not the case"?

What markdown rendering engine are you using?

--
Grant


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird strace of #! python script

[Barry]

> On 14 Mar 2022, at 21:29, Dan Stromberg  wrote:
> 
> I expected to see an exec of /usr/bin/python2, but I don't.  I just see an
> exec of /tmp/t.

I understand that the kernel handles the #! Line itself, which is why you do 
not see it in strace.

Barry



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird strace of #! python script

[Chris Angelico]

On Tue, 15 Mar 2022 at 08:28, Dan Stromberg  wrote:
>
> Hi folks.
>
> First off, I know, python2 is ancient history.  Porting to 3.x is on my
> list of things to do (though I'm afraid it's not yet at the top of the
> list), and the same thing happens with python3.
>
> So anyway, I'm strace'ing a #!/usr/bin/python2 script.
>
> I expected to see an exec of /usr/bin/python2, but I don't.  I just see an
> exec of /tmp/t.
>
> As follows:
> tact@celery_worker:/app$ strace -f -s 1024 -o /tmp/t.strace /tmp/t
> ^Z
> [1]+  Stopped strace -f -s 1024 -o /tmp/t.strace /tmp/t
> tact@celery_worker:/app$ bg
> [1]+ strace -f -s 1024 -o /tmp/t.strace /tmp/t &
> tact@celery_worker:/app$ ps axf
>   PID TTY  STAT   TIME COMMAND
>  1163 pts/0Ss 0:00 bash
>  1363 pts/0S  0:00  \_ strace -f -s 1024 -o /tmp/t.strace /tmp/t
>  1366 pts/0S  0:00  |   \_ /usr/bin/python2 /tmp/t
>  1367 pts/0R+ 0:00  \_ ps axf
> tact@celery_worker:/app$ fg
> bash: fg: job has terminated
> [1]+  Donestrace -f -s 1024 -o /tmp/t.strace /tmp/t
> tact@celery_worker:/app$ grep execve /tmp/t.strace
> 1366  execve("/tmp/t", ["/tmp/t"], 0x7ffd89f9c3b8 /* 49 vars */) = 0
> tact@celery_worker:/app$
>
> I've deleted some irrelevant processes from the 'ps axf'.
>
> /tmp/t is actually just:
> tact@celery_worker:/app$ cat /tmp/t
> #!/usr/bin/python2
>
> import time
>
> time.sleep(10)
>
>
> Was this some sort of security feature I never heard about?  I'm tracing a
> very simple time.sleep(10) here, but the same thing is (not) happening in a
> larger script that I need to track down a bug in.
>
> Is there a way I can coax Linux and/or strace to show all the exec's, like
> they used to?  Not having them makes me wonder what else is missing from
> the strace report.
>
> I'm on a Debian 11.2 system with strace 5.10 and Python 2.7.18.
>
> Thanks!

I'm not sure, but I suspect that strace is skipping the exec into the
process itself. When I try this myself, I can see the trace of the
sleep call itself (it's implemented on top of select()), so it does
seem to be doing the proper trace. My guess is that the shebang is
being handled inside the exec of /tmp/t itself, rather than being a
separate syscall.

In any case, it doesn't seem to be a Python thing; I can trigger the
same phenomenon with other interpreters.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird behavior for IDLE...

[Terry Reedy]

On 10/13/2020 4:51 AM, Steve wrote:

Why does IDLE always open with the lowest three lines of the window end up
hidden below the bottom of the screen behind the task bar? Every time I use
it,  I have to stop and grab the top of the window and drag it up to see the
line and row information.  I explored the Options/Configure IDLE but did not
see anything that would help.


The issue is lines on the screen, which depends on screen, font, and 
fontsize, versus lines in the editor window plus lines above the editor 
window.  On the General tab, you can ask for fewer lines in the editor 
window.


Use Options -> Zoom Height to get the max lines possible.

The positioning of the top of the window is the tk default.  This might 
be changed if it became the highest priority.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird Python Bug

[MRAB]

On 2019-09-13 20:17, CrazyVideoGamez wrote:

For some reason, if you put in the code

def odd_ones_out(numbers):
 for num in numbers:
 count = numbers.count(num)
 if not count % 2 == 0:
 for i in range(count):
 numbers.remove(num)
 return numbers

nums = [72, 4, 82, 67, 67]
print(odd_ones_out(nums))

For some reason, it outputs [4, 67, 67] when it should have deleted the 4 
because it was odd. Another interesting thing is that when you put print(num) 
in the for loop, the number 4 never shows up and neither does the last 67. Help!


Here's a simpler example to show what's going on:

>>> numbers = [1, 2, 3]
>>>
>>> for n in numbers:
... numbers.remove(n)
...
>>> print(numbers)
[2]

The 'for' loop steps through the list, one item at a time: first, 
second, third, etc.


In the first iteration, the first item is 1.

The body of the loop removes that item, and the following items are 
shifted down, leaving [2, 3].


In the second iteration, the second item is 3. (2 is now the first item, 
and as far as it's concerned it has already done the first item.)


The body of the loop removes that item, and the following items are 
shifted down, leaving [2].


It has now reached the end of the list.

And the moral is: don't change a list while you're iterating over it.

Iterate over the "input" list and build a new "output" list to hold the 
results.

--
https://mail.python.org/mailman/listinfo/python-list

RE: Weird Python Bug

[David Raymond]

2 comments:

First: Deleting from a list while you're iterating over it is a bad idea. Your 
first iteration gives nums[0] which is 72. But then you delete that and (in 
effect) everything moves up. So now the 4 is in the nums[0] slot. Your second 
iteration returns nums[1] which is now the 82 meaning you skipped over the 4... 
etc etc etc.,

Second: You are altering the original list that you're passing to the function, 
you're not working on a copy of it. Make sure that that is what you want, and 
that you didn't intend to leave the original thing alone.
>>> nums = [72, 4, 82, 67, 67]
>>> odd_ones_out(nums)
[4, 67, 67]
>>> nums
[4, 67, 67]
>>>



-Original Message-
From: Python-list On Behalf Of CrazyVideoGamez
Sent: Friday, September 13, 2019 3:18 PM
To: python-list@python.org
Subject: Weird Python Bug

For some reason, if you put in the code

def odd_ones_out(numbers):
for num in numbers:
count = numbers.count(num)
if not count % 2 == 0:
for i in range(count):
numbers.remove(num)
return numbers

nums = [72, 4, 82, 67, 67]
print(odd_ones_out(nums))

For some reason, it outputs [4, 67, 67] when it should have deleted the 4 
because it was odd. Another interesting thing is that when you put print(num) 
in the for loop, the number 4 never shows up and neither does the last 67. Help!
-- 
https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird side effect of default parameter

[Robert Latest via Python-list]

Steven D'Aprano wrote:
> Python function default values use *early binding*: the default parameter 
> is evaluated, ONCE, when the function is defined, and that value is used 
> each time it is needed.

Thanks, "early binding" was the clue I was missing.

robert
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird side effect of default parameter

[Steven D'Aprano]

On Thu, 03 May 2018 19:47:37 +, Robert Latest via Python-list wrote:

> Hello,
> 
> I don't understand the behavior of the code below. Why does the dict
> property "a" of both objects contain the same keys? This is only if
> "a=dict" is in the initializer. If I put self.a = dict() into the init
> function, I get two separate dicts


Python function default values use *early binding*: the default parameter 
is evaluated, ONCE, when the function is defined, and that value is used 
each time it is needed.

The opposite is *late binding*: the default parameter is not evaluated 
until it is actually needed, and then re-evaluated each time it is needed.

Both approaches have their pros and cons, but early binding is better as 
the language supported feature, since it is easy to build late binding 
for yourself:

def __init__(self, x, a=None):
if a is None:
a = dict()
self.x = x
self.a = a
self.a[x] = x


whereas if the language used late binding, it would be really difficult 
to build early binding on top of it.

The interaction between early-binding defaults and mutable values like 
lists and dicts is a well-known "gotcha" (or at least, it is well-known 
to those of us who have come across it before). It is even a FAQ:

https://docs.python.org/dev/faq/programming.html#why-are-default-values-
shared-between-objects


See also: http://www.effbot.org/zone/default-values.htm



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird side effect of default parameter

[Gary Herron]

This is a well known feature of Python.   It's a very common "gotcha" to 
new Python programmers.


Google "Mutable default parameters in Python" for long list of 
explanations and fixes.


In short, don't use a mutable object as a default parameter.


Gary Herron



On 05/03/2018 12:47 PM, python-list@python.org wrote:

Hello,

I don't understand the behavior of the code below. Why does the dict property
"a" of both objects contain the same keys? This is only if "a=dict" is in
the initializer. If I put self.a = dict() into the init function, I get two
separate dicts



class Foo(object):
 def __init__(self, x, a=dict()):
 self.x = x
 self.a = a
 self.a[x] = x


c = Foo(1)
d = Foo(2)

print(c.__dict__)
print(d.__dict__)


robert


--
Dr. Gary Herron
Professor of Computer Science
DigiPen Institute of Technology
(425) 895-4418

--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird python 2.7 import thing

[Dan Stromberg]

On Fri, Feb 19, 2016 at 10:33 AM, Dan Stromberg  wrote:
> On Thu, Feb 18, 2016 at 5:40 PM, Dan Stromberg  wrote:
>
>>
>> BTW, this is not a package I pip installed from pypi - it's an
>> internal-only project.
>
> I need to correct this: It's not installed from pypi.python.org, but
> it probably is installed from our internal pypi server.
>
> Thanks!

I've taken this to Stackoverflow:
http://stackoverflow.com/questions/35512989/weird-cpython-2-7-import-traceback-not-an-importerror

Thanks again.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird python 2.7 import thing

[Dan Stromberg]

On Thu, Feb 18, 2016 at 5:40 PM, Dan Stromberg  wrote:

>
> BTW, this is not a package I pip installed from pypi - it's an
> internal-only project.

I need to correct this: It's not installed from pypi.python.org, but
it probably is installed from our internal pypi server.

Thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird and sparese cgi:error

[Φώντας Λαδοπρακόπουλος]

[root@nexus cgi-bin]# head -10 ./metrites.py 
#!/usr/bin/python3
# coding=utf-8

import cgitb; cgitb.enable()
import cgi, re, os, sys, socket, time, datetime, locale, codecs, random, 
smtplib, subprocess, pymysql, geoip2.database
from datetime import datetime, timedelta
from http import cookies


#needed line, script does *not* work without it
[root@nexus cgi-bin]# ^C
[root@nexus cgi-bin]# /usr/bin/python3 
Python 3.3.2 (default, Aug 14 2014, 14:25:52) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
KeyboardInterrupt
>>> 
KeyboardInterrupt
>>> quit()
[root@nexus cgi-bin]# 
[root@nexus cgi-bin]# /usr/bin/python3 ./metrites.py 
Error in sys.excepthook:
ValueError: underlying buffer has been detached

Original exception was:
Traceback (most recent call last):
  File "/opt/rh/python33/root/usr/lib64/python3.3/os.py", line 673, in 
__getitem__
value = self._data[self.encodekey(key)]
KeyError: b'HTTP_USER_AGENT'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./metrites.py", line 22, in 
userinfo = os.environ['HTTP_USER_AGENT']
  File "/opt/rh/python33/root/usr/lib64/python3.3/os.py", line 676, in 
__getitem__
raise KeyError(key)
KeyError: 'HTTP_USER_AGENT'
[root@nexus cgi-bin]# ^C
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird and sparese cgi:error

[Φώντας Λαδοπρακόπουλος]

#!/usr/bin/python3
# coding=utf-8

import cgitb; cgitb.enable()
import cgi, re, os, sys, socket, time, datetime, locale, codecs, random, 
smtplib, subprocess, pymysql, geoip2.database
from datetime import datetime, timedelta
from http import cookies
===

[root@nexus cgi-bin]# /usr/bin/python3 
Python 3.3.2 (default, Aug 14 2014, 14:25:52) 
[GCC 4.8.2 20140120 (Red Hat 4.8.2-16)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
=

[root@nexus cgi-bin]# /usr/bin/python3 ./metrites.py 
Error in sys.excepthook:
ValueError: underlying buffer has been detached

Original exception was:
Traceback (most recent call last):
  File "/opt/rh/python33/root/usr/lib64/python3.3/os.py", line 673, in 
__getitem__
value = self._data[self.encodekey(key)]
KeyError: b'HTTP_USER_AGENT'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./metrites.py", line 22, in 
userinfo = os.environ['HTTP_USER_AGENT']
  File "/opt/rh/python33/root/usr/lib64/python3.3/os.py", line 676, in 
__getitem__
raise KeyError(key)
KeyError: 'HTTP_USER_AGENT'
[root@nexus cgi-bin]# 
===

everything is they should but still iam gettign that weird cgi:err error
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird and sparese cgi:error

[Chris Angelico]

On Thu, Feb 18, 2016 at 12:47 AM, Jeremy Leonard  wrote:
> On Tuesday, February 16, 2016 at 3:02:40 PM UTC-5, Φώντας Λαδοπρακόπουλος 
> wrote:
>> Hello,
>>
>> I recentely changed VPS server and when i try to load my webiste iam 
>> receiving 500 error which i wasnt receiving in my old VPS server with the 
>> same exact cgi scripts.
>>
>> After looking at Apacher's error_log iam seeing the following output when i 
>> try to load scripts from cgi-bin directory.
>>
>>
>> [Tue Feb 16 21:28:55.970302 2016] [cgi:error] [pid 17478] [client 
>> 180.76.15.28:21311] End of script output before headers: metrites.py
>> [Tue Feb 16 21:29:31.193281 2016] [cgi:error] [pid 18802] [client 
>> 180.76.15.29:34155] End of script output before headers: metrites.py
>> [Tue Feb 16 21:32:21.306627 2016] [cgi:error] [pid 17478] [client 
>> 188.138.124.110:40212] End of script output before headers: metrites.py
>> [Tue Feb 16 21:37:21.128472 2016] [cgi:error] [pid 19731] [client 
>> 204.152.200.42:54376] End of script output before headers: metrites.py
>> [Tue Feb 16 21:42:21.198207 2016] [cgi:error] [pid 17478] [client 
>> 184.75.210.226:57252] End of script output before headers: metrites.py
>> [Tue Feb 16 21:47:21.228361 2016] [cgi:error] [pid 19731] [client 
>> 76.72.172.208:34730] End of script output before headers: metrites.py
>> [Tue Feb 16 21:52:21.422066 2016] [cgi:error] [pid 18802] [client 
>> 85.17.156.11:52071] End of script output before headers: metrites.py
>>
>>
>> Please i need your assistance to this.
>> thank you.Prob
>
> If you import the cgitb module and then put cgitb.enable() in the first line 
> of your code this can help give you more descriptive error messages on the 
> page that is producing the errors
>

And if that doesn't work, I'd look at the very beginning of code
execution - can the Python interpreter even be found? What happens if
you run one of the scripts from a shell? Maybe there's a shebang
problem.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird and sparese cgi:error

[Jeremy Leonard]

On Tuesday, February 16, 2016 at 3:02:40 PM UTC-5, Φώντας Λαδοπρακόπουλος wrote:
> Hello,
> 
> I recentely changed VPS server and when i try to load my webiste iam 
> receiving 500 error which i wasnt receiving in my old VPS server with the 
> same exact cgi scripts.
> 
> After looking at Apacher's error_log iam seeing the following output when i 
> try to load scripts from cgi-bin directory.
> 
> 
> [Tue Feb 16 21:28:55.970302 2016] [cgi:error] [pid 17478] [client 
> 180.76.15.28:21311] End of script output before headers: metrites.py
> [Tue Feb 16 21:29:31.193281 2016] [cgi:error] [pid 18802] [client 
> 180.76.15.29:34155] End of script output before headers: metrites.py
> [Tue Feb 16 21:32:21.306627 2016] [cgi:error] [pid 17478] [client 
> 188.138.124.110:40212] End of script output before headers: metrites.py
> [Tue Feb 16 21:37:21.128472 2016] [cgi:error] [pid 19731] [client 
> 204.152.200.42:54376] End of script output before headers: metrites.py
> [Tue Feb 16 21:42:21.198207 2016] [cgi:error] [pid 17478] [client 
> 184.75.210.226:57252] End of script output before headers: metrites.py
> [Tue Feb 16 21:47:21.228361 2016] [cgi:error] [pid 19731] [client 
> 76.72.172.208:34730] End of script output before headers: metrites.py
> [Tue Feb 16 21:52:21.422066 2016] [cgi:error] [pid 18802] [client 
> 85.17.156.11:52071] End of script output before headers: metrites.py
> 
> 
> Please i need your assistance to this.
> thank you.Prob

If you import the cgitb module and then put cgitb.enable() in the first line of 
your code this can help give you more descriptive error messages on the page 
that is producing the errors
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird list conversion

[Chris Angelico]

On Mon, Dec 14, 2015 at 7:31 AM, Erik  wrote:
> On 13/12/15 20:28, Erik wrote:
>>
>> When you call "print", then the list class's __repr__() method is called
>> which in turn calls the contained objects' __repr__() methods in turn
>
>
> I mean the __str__() method, not __repr__() in this case - however, the
> answer is otherwise the same.

It actually makes no difference; lists don't have __str__, and fall
back on object.__str__, which returns self.__repr__(). Inside
list.__repr__, the contained objects' reprs are combined. With lists,
you always get the repr. :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird list conversion

[Larry Hudson via Python-list]

On 12/13/2015 12:05 PM, KP wrote:

On Sunday, 13 December 2015 11:57:57 UTC-8, Laura Creighton  wrote:

In a message of Sun, 13 Dec 2015 11:45:19 -0800, KP writes:

Hi all,

  f = open("stairs.bin", "rb")
  data = list(f.read(16))
  print data

returns

['=', '\x04', '\x00', '\x05', '\x00', '\x01', '\x00', '\x00', '\x00', '\x00', 
'\x00', '\x00', '\x00', '\x00', '\x00', '\x00']

The first byte of the file is 0x3D according to my hex editor, so why does 
Python return '=' and not '\x3D'?

As always, thanks for any help!


0x3d is the ascii code for '='


I am aware of that - so is the rule that non-printables are returned in hex 
notation whereas printables come in their ASCII representation?


No.  The data is returned as raw bytes.
It's the print that is responsible for the way these bytes are _displayed_.

 -=- Larry -=-

--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird list conversion

[Erik]

On 13/12/15 20:28, Erik wrote:

When you call "print", then the list class's __repr__() method is called
which in turn calls the contained objects' __repr__() methods in turn


I mean the __str__() method, not __repr__() in this case - however, the 
answer is otherwise the same.


E.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird list conversion

[Ian Kelly]

On Sun, Dec 13, 2015 at 1:05 PM, KP  wrote:
> On Sunday, 13 December 2015 11:57:57 UTC-8, Laura Creighton  wrote:
>> In a message of Sun, 13 Dec 2015 11:45:19 -0800, KP writes:
>> >Hi all,
>> >
>> >  f = open("stairs.bin", "rb")
>> >  data = list(f.read(16))
>> >  print data
>> >
>> >returns
>> >
>> >['=', '\x04', '\x00', '\x05', '\x00', '\x01', '\x00', '\x00', '\x00', 
>> >'\x00', '\x00', '\x00', '\x00', '\x00', '\x00', '\x00']
>> >
>> >The first byte of the file is 0x3D according to my hex editor, so why does 
>> >Python return '=' and not '\x3D'?
>> >
>> >As always, thanks for any help!
>>
>> 0x3d is the ascii code for '='
>
> I am aware of that - so is the rule that non-printables are returned in hex 
> notation whereas printables come in their ASCII representation?

What you're seeing there is the repr() of the string. The rule is that
it should be a printable representation that can be passed to eval to
reconstruct the string. The printable requirement means that NUL
should always come out escaped as '\x00' or perhaps '\0', but for
printable bytes escaping isn't necessary. I don't know if there's a
requirement that the repr of '\x3D' be '=', but it's probably the most
generally useful and I doubt that any implementation would depart from
that.

If you specifically want hex representations, then you could use hex(ord(foo)).

py> hex(ord('='))
'0x3d'
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird list conversion

[Erik]

On 13/12/15 20:05, KP wrote:

On Sunday, 13 December 2015 11:57:57 UTC-8, Laura Creighton  wrote:

In a message of Sun, 13 Dec 2015 11:45:19 -0800, KP writes:

Hi all,

f = open("stairs.bin", "rb") data = list(f.read(16)) print data

returns

['=', '\x04', '\x00', '\x05', '\x00', '\x01', '\x00', '\x00',
'\x00', '\x00', '\x00', '\x00', '\x00', '\x00', '\x00', '\x00']

The first byte of the file is 0x3D according to my hex editor, so
why does Python return '=' and not '\x3D'?

As always, thanks for any help!


0x3d is the ascii code for '='


I am aware of that - so is the rule that non-printables are returned
in hex notation whereas printables come in their ASCII
representation?


No, what is _returned_ is a list of one-byte strings.

When you call "print", then the list class's __repr__() method is called 
which in turn calls the contained objects' __repr__() methods in turn - 
it is those methods (in this case, they are all the string class's) 
which is deciding to render ASCII printable characters as just their 
value and ASCII non-printable characters using their hex notation.


E.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird list conversion

[KP]

On Sunday, 13 December 2015 11:57:57 UTC-8, Laura Creighton  wrote:
> In a message of Sun, 13 Dec 2015 11:45:19 -0800, KP writes:
> >Hi all,
> >
> >  f = open("stairs.bin", "rb") 
> >  data = list(f.read(16))
> >  print data
> >
> >returns
> >
> >['=', '\x04', '\x00', '\x05', '\x00', '\x01', '\x00', '\x00', '\x00', 
> >'\x00', '\x00', '\x00', '\x00', '\x00', '\x00', '\x00']
> >
> >The first byte of the file is 0x3D according to my hex editor, so why does 
> >Python return '=' and not '\x3D'?
> >
> >As always, thanks for any help!
> 
> 0x3d is the ascii code for '='

I am aware of that - so is the rule that non-printables are returned in hex 
notation whereas printables come in their ASCII representation?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird list conversion

[Ian Kelly]

On Sun, Dec 13, 2015 at 12:45 PM,   wrote:
> Hi all,
>
>   f = open("stairs.bin", "rb")
>   data = list(f.read(16))
>   print data
>
> returns
>
> ['=', '\x04', '\x00', '\x05', '\x00', '\x01', '\x00', '\x00', '\x00', '\x00', 
> '\x00', '\x00', '\x00', '\x00', '\x00', '\x00']
>
> The first byte of the file is 0x3D according to my hex editor, so why does 
> Python return '=' and not '\x3D'?

They're equivalent representations of the same thing.

py> '\x3D'
'='
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird list conversion

[Laura Creighton]

In a message of Sun, 13 Dec 2015 11:45:19 -0800, high5stor...@gmail.com writes:
>Hi all,
>
>  f = open("stairs.bin", "rb") 
>  data = list(f.read(16))
>  print data
>
>returns
>
>['=', '\x04', '\x00', '\x05', '\x00', '\x01', '\x00', '\x00', '\x00', '\x00', 
>'\x00', '\x00', '\x00', '\x00', '\x00', '\x00']
>
>The first byte of the file is 0x3D according to my hex editor, so why does 
>Python return '=' and not '\x3D'?
>
>As always, thanks for any help!

0x3d is the ascii code for '='
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird behavior on __dict__ attr

[Steven D'Aprano]

Shiyao Ma wrote:

> Hi.
> 
> My context is a little hard to reproduce.
> 
> NS3 is a network simulation tool written in C++. I am using its Python
> binding.
> 
> So the class I am dealing with is from a .so file.

So it is written in (probably) C and you don't have source code for it.

> Say, I do the following:
> 
> %
> 
> import ns.network.Node as Node
> 
> # Node is a class
> # it has a __dict__ attr
> 
> # Now I instantiate an instance of Node
> n = Node()
> 
> # I checked, there is no __dict__ on 'n'
> # but the following succeeds.
> 
> n.foobar = 3

It is hard to say exactly what is happening without source code. If Node is 
an extension class written in C, it could do almost anything. It certainly 
doesn't need to follow Python conventions for Python classes.


If this were a pure-Python class, I can think of at least three ways to get 
this behaviour:

- use __slots__ so that the class has a fixed set of members 
  with no __dict__ used or needed;

- use a custom descriptor

- use a custom metaclass that changes the behaviour of lookups;

I presume that extension classes written in C can do similar things.


> My understanding is the foobar is stored in n.__dict__, but seemingly n
> has no __dict__.
> 
> So where does the foobar go?

Here are some diagnostic tools you can use to try to determine what is going 
on:

print(vars(n))
print(dir(n))
print(n.__slots__)
o = Node.__dict__['foobar']
print(type(o), repr(o))




-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird behavior on __dict__ attr

[Dave Angel]

On 02/09/2015 09:52 PM, Shiyao Ma wrote:

Hi.

My context is a little hard to reproduce.


WHY don't you try?  Telling us about a class without showing how it's 
defined leaves us all guessing.


Start by telling us Python version.  And if it's 2.x, tell us whether 
this class is an old style or new style class.




NS3 is a network simulation tool written in C++. I am using its Python binding.

So the class I am dealing with is from a .so file.

Say, I do the following:

%

import ns.network.Node as Node

# Node is a class
# it has a __dict__ attr

# Now I instantiate an instance of Node
n = Node()

# I checked, there is no __dict__ on 'n'
# but the following succeeds.

n.foobar = 3



My understanding is the foobar is stored in n.__dict__, but seemingly n has no 
__dict__.

So where does the foobar go?



Lots of possibilities.  Simplest is slots.  if you define __slots__, 
then there's no dictionary in each instance.


https://docs.python.org/3.4/reference/datamodel.html#slots

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird connection problem

[Roland Hedberg]

It’s a special HTTPS url and searching further it seems to be a SNI problem 
talked about here:

http://stackoverflow.com/questions/18578439/using-requests-with-tls-doesnt-give-sni-support

> 25 okt 2014 kl. 08:48 skrev Joel Goldstick :
> 
> On Sat, Oct 25, 2014 at 9:40 AM, Roland Hedberg  wrote:
>> When I try to access a URL using requests I always get:
>> socket.error: [Errno 104] Connection reset by peer
>> 
>> If I try to access the same URL using curl I get no error message instead I 
>> get the page.
>> The same result if I use a web browser like Safari.
>> 
>> But, if I use python httplib I also get Errno 104.
>> 
>> What gives ?
>> 
>> Oh, by the way it’s a HTTPS url.
>> 
>> — Roland
> 
> Is this for any url or a specific one?
>> --
>> https://mail.python.org/mailman/listinfo/python-list
> 
> 
> 
> -- 
> Joel Goldstick
> http://joelgoldstick.com
> -- 
> https://mail.python.org/mailman/listinfo/python-list

— Roland

”Being able to think like a child is an important attribute of being an adult” 
- Eddie Izzard

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird connection problem

[Joel Goldstick]

On Sat, Oct 25, 2014 at 9:40 AM, Roland Hedberg  wrote:
> When I try to access a URL using requests I always get:
> socket.error: [Errno 104] Connection reset by peer
>
> If I try to access the same URL using curl I get no error message instead I 
> get the page.
> The same result if I use a web browser like Safari.
>
> But, if I use python httplib I also get Errno 104.
>
> What gives ?
>
> Oh, by the way it’s a HTTPS url.
>
> — Roland

Is this for any url or a specific one?
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Joel Goldstick
http://joelgoldstick.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird connection problem

[Roland Hedberg]

Oh, by the way!
To make this more interesting :-/

I saw this behavior on a Linux machine (Ubuntu 14.04 LTS) using Python 2.7.6 if 
I do the same exercise 
on a Mac OS X machine also with Python 2.7.6 - no problem what so ever.

> 25 okt 2014 kl. 08:40 skrev Roland Hedberg :
> 
> When I try to access a URL using requests I always get: 
> socket.error: [Errno 104] Connection reset by peer
> 
> If I try to access the same URL using curl I get no error message instead I 
> get the page.
> The same result if I use a web browser like Safari.
> 
> But, if I use python httplib I also get Errno 104.
> 
> What gives ?
> 
> Oh, by the way it’s a HTTPS url.
> 
> — Roland
> 
> ”Being able to think like a child is an important attribute of being an 
> adult” - Eddie Izzard
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

— Roland

”Being able to think like a child is an important attribute of being an adult” 
- Eddie Izzard

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird SSL problem

[Roland Hedberg]

30 sep 2014 kl. 00:55 skrev Ned Deily :

> In article ,
> Roland Hedberg  wrote:
> 
>> Hi!
>> 
>> I¹m trying to access 
>> https://stsadweb.one.microsoft.com/adfs/.well-known/openid-configuration
>> 
>> Doing it the simplest way I get the following:
>> 
> import urllib
> f = 
> urllib.urlopen("https://stsadweb.one.microsoft.com/adfs/.well-known/openid
> -configuration")
>> Traceback (most recent call last):
>>  File "", line 1, in 
>>  File 
>>  "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py",
>>   line 87, in urlopen
>>return opener.open(url)
>>  File 
>>  "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py",
>>   line 208, in open
>>return getattr(self, name)(url)
>>  File 
>>  "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py",
>>   line 437, in open_https
>>h.endheaders(data)
>>  File 
>>  "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py"
>>  , line 969, in endheaders
>>self._send_output(message_body)
>>  File 
>>  "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py"
>>  , line 829, in _send_output
>>self.send(msg)
>>  File 
>>  "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py"
>>  , line 791, in send
>>self.connect()
>>  File 
>>  "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py"
>>  , line 1176, in connect
>>self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
>>  File 
>>  "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", 
>>  line 387, in wrap_socket
>>ciphers=ciphers)
>>  File 
>>  "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", 
>>  line 143, in __init__
>>self.do_handshake()
>>  File 
>>  "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", 
>>  line 305, in do_handshake
>>self._sslobj.do_handshake()
>> IOError: [Errno socket error] [Errno 54] Connection reset by peer
> import ssl
> ssl.OPENSSL_VERSION
>> ¹OpenSSL 0.9.8za 5 Jun 2014'
>> 
>> Now, using Safari, or curl for that matter, from the same machine works 
>> without a hitch.
>> 
>> The URL above is also the only URL I¹ve encountered this problem with.
>> 
>> Anyone got an idea ?
> 
> I believe the problem is that the connection is protected by a 
> multi-hostname server certificate and Python 2's urllib (and underlying 
> httplib and ssl modules) do not support SNI extensions to TLS.  The 
> request above works fine with Python 3 (which has supported client-side 
> SNI since Python 3.2).  See http://bugs.python.org/issue5639 for more 
> discussion of the matter.  If Python 3 is not an option for you, the 
> requests package available via PyPI should help.

You’re absolutely correct in that it’s a SNI problem.
Python 3 is not an option and I was already using requests obviously missing 
something.
Ah, had to install some extra libraries.

— Roland

”Being able to think like a child is an important attribute of being an adult” 
- Eddie Izzard

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird SSL problem

[Ned Deily]

In article ,
 Roland Hedberg  wrote:

> Hi!
> 
> I¹m trying to access 
> https://stsadweb.one.microsoft.com/adfs/.well-known/openid-configuration
> 
> Doing it the simplest way I get the following:
> 
> >>> import urllib
> >>> f = 
> >>> urllib.urlopen("https://stsadweb.one.microsoft.com/adfs/.well-known/openid
> >>> -configuration")
> Traceback (most recent call last):
>   File "", line 1, in 
>   File 
>   "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py",
>line 87, in urlopen
> return opener.open(url)
>   File 
>   "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py",
>line 208, in open
> return getattr(self, name)(url)
>   File 
>   "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py",
>line 437, in open_https
> h.endheaders(data)
>   File 
>   "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py"
>   , line 969, in endheaders
> self._send_output(message_body)
>   File 
>   "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py"
>   , line 829, in _send_output
> self.send(msg)
>   File 
>   "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py"
>   , line 791, in send
> self.connect()
>   File 
>   "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py"
>   , line 1176, in connect
> self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
>   File 
>   "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", 
>   line 387, in wrap_socket
> ciphers=ciphers)
>   File 
>   "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", 
>   line 143, in __init__
> self.do_handshake()
>   File 
>   "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", 
>   line 305, in do_handshake
> self._sslobj.do_handshake()
> IOError: [Errno socket error] [Errno 54] Connection reset by peer
> >>> import ssl
> >>> ssl.OPENSSL_VERSION
> ¹OpenSSL 0.9.8za 5 Jun 2014'
> 
> Now, using Safari, or curl for that matter, from the same machine works 
> without a hitch.
> 
> The URL above is also the only URL I¹ve encountered this problem with.
> 
> Anyone got an idea ?

I believe the problem is that the connection is protected by a 
multi-hostname server certificate and Python 2's urllib (and underlying 
httplib and ssl modules) do not support SNI extensions to TLS.  The 
request above works fine with Python 3 (which has supported client-side 
SNI since Python 3.2).  See http://bugs.python.org/issue5639 for more 
discussion of the matter.  If Python 3 is not an option for you, the 
requests package available via PyPI should help.

-- 
 Ned Deily,
 n...@acm.org

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird problem with UDP and gevent

[Grant Edwards]

On 2013-10-18, James Harris  wrote:
> "Roy Smith"  wrote in message 
> news:l3riea$82$1...@panix2.panix.com...
>> I'm running:
>>
>> Ubuntu Precise
>> Python 2.7.3
>> django 1.4.5
>> gunicorn 0.17.4
>> gevent 1.0dev (rc3)
>>
>> I haven't been able to pin this down exactly, but it looks like if I
>> do (inside of a custom logging.Handler subclass):
>>
>>   # Paraphrased from the actual code
>> remote_addr = ("localhost", 9700)
>>  self.socket = socket.socket(type=socket.SOCK_DGRAM)
>>payload = "..."
>> self.socket.connect(remote_addr)
>>self.socket.send(payload)
>>
>> I get intermittant hangs in the connect() call.  If I rewrite this as:
>>
>> remote_addr = ("localhost", 9700)
>>self.socket = socket.socket(type=socket.SOCK_DGRAM)
>>payload = "..."
>>self.socket.sendto(payload, remote_addr)
>>
>> everything works fine.  Has anybody seen anything like this?  I'm
>> guessing this is some kind of gevent bug.
>
> Those are two different things.

Yes.  They differ in the addresses _from_which_ packets can be
received. In the "connect" example, you will only be able to receive
packets from ("localhost",9700).  In the second example you can
receive packets from any address.  Since the snippets only send
packets not receive packets, they should both behave identically.

> You would normally use connect() on a SOCK_STREAM socket. It requires
> that the remote endpoint, in this case localhost:9700, has an open
> socket listening for connections. sendto() is the right thing to use
> with SOCK_DGRAM.

Either will work.  You can use connect() with a SOCK_DGRAM.  What it
does is 

 1) save the destination address and use it when you call send()

 2) limit the addresses from which packets will be received. 

>From the Linux connect(2) man page:

   If the socket sockfd is of type SOCK_DGRAM then addr is the address
   to which datagrams are sent by default, and the only address from
   which datagrams are received.

-- 
Grant
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird problem with UDP and gevent

[Roy Smith]

On Friday, October 18, 2013 1:04:38 PM UTC-4, James Harris wrote:
> Those are two different things. You would normally use connect() on a 
> SOCK_STREAM socket. It requires that the remote endpoint, in this case 
> localhost:9700, has an open socket listening for connections. sendto() is 
> the right thing to use with SOCK_DGRAM.

You are supposed to be able to call connect() on a UDP socket.  All it does is 
store the remote address in the kernel socket structure.  Doing connect() 
followed by a series of send()s is nominally more efficient than doing a series 
of sendto()s.

I forgot to mention that the connect()/send() code works just fine if I switch 
my gunicorn config from gevent to sync workers.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird problem with UDP and gevent

[James Harris]

"Roy Smith"  wrote in message 
news:l3riea$82$1...@panix2.panix.com...
> I'm running:
>
> Ubuntu Precise
> Python 2.7.3
> django 1.4.5
> gunicorn 0.17.4
> gevent 1.0dev (rc3)
>
> I haven't been able to pin this down exactly, but it looks like if I
> do (inside of a custom logging.Handler subclass):
>
>   # Paraphrased from the actual code
> remote_addr = ("localhost", 9700)
>  self.socket = socket.socket(type=socket.SOCK_DGRAM)
>payload = "..."
> self.socket.connect(remote_addr)
>self.socket.send(payload)
>
> I get intermittant hangs in the connect() call.  If I rewrite this as:
>
> remote_addr = ("localhost", 9700)
>self.socket = socket.socket(type=socket.SOCK_DGRAM)
>payload = "..."
>self.socket.sendto(payload, remote_addr)
>
> everything works fine.  Has anybody seen anything like this?  I'm
> guessing this is some kind of gevent bug.

Those are two different things. You would normally use connect() on a 
SOCK_STREAM socket. It requires that the remote endpoint, in this case 
localhost:9700, has an open socket listening for connections. sendto() is 
the right thing to use with SOCK_DGRAM.

James


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird bahaviour from shlex - line no

[Piet van Oostrum]

Peter Otten <__pete...@web.de> writes:

> Dave Angel wrote:
>
>> On 28/9/2013 02:26, Daniel Stojanov wrote:
>> 
>>> Can somebody explain this. The line number reported by shlex depends
>>> on the previous token. I want to be able to tell if I have just popped
>>> the last token on a line.
[...]
> The explanation seems obvious: a word may be continued by the next character 
> if that is in wordchars, so the parser has to look at that character. If it 
> happens to be '\n' the lineno is immediately incremented. Non-wordchars are 
> returned as single characters, so there is no need to peek ahead and the 
> lineno is not altered.
>
> In short: this looks like an implementation accident. 

I think shlex should be changed to give the line number of the start of
the token in self.lineno. It isn't hard.
-- 
Piet van Oostrum 
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird bahaviour from shlex - line no

[Peter Otten]

Dave Angel wrote:

> On 28/9/2013 02:26, Daniel Stojanov wrote:
> 
>> Can somebody explain this. The line number reported by shlex depends
>> on the previous token. I want to be able to tell if I have just popped
>> the last token on a line.
>>
> 
> I agree that it seems weird.  However, I don't think you have made
> clear why it's not what you (and I) expect.
> 
> import shlex
> 
> def parseit(string):
> print
> print "Parsing -", string
> first = shlex.shlex(string)
> token = "dummy"
> while token:
> token = first.get_token()
> print token, " -- line", first.lineno
> 
> parseit("word1 word2\nword3") #first
> parseit("word1 word2,\nword3")#second
> parseit("word1 word2,word3\nword4")
> parseit("word1 word2+,?\nword3")
> 
> This will display the lineno attribute for every token.
> 
> shlex is documented at:
> 
> http://docs.python.org/2/library/shlex.html
> 
> And lineno is documented on that page as:
> 
> """shlex.lineno
> Source line number (count of newlines seen so far plus one).
> """
> 
> It's not at all clear what "seen so far" is intended to mean, but in
> practice, the line number is incremented for the last token on the
> line. Thus your first example
> 
> Parsing - word1 word2
> word3
> word1  -- line 1
> word2  -- line 2
> word3  -- line 2
>   -- line 2
> 
> word2 has the incremented line number.
> 
> But when the token is neither whitespace nor ASCII letters, then it
> doesn't increment lineno.  Thus second example:
> 
> Parsing - word1 word2,
> word3
> word1  -- line 1
> word2  -- line 1
> ,  -- line 1  #we would expect this to be "line 2"
> word3 -- line 2 -- line 2
> 
> Anybody else have some explanation 

The explanation seems obvious: a word may be continued by the next character 
if that is in wordchars, so the parser has to look at that character. If it 
happens to be '\n' the lineno is immediately incremented. Non-wordchars are 
returned as single characters, so there is no need to peek ahead and the 
lineno is not altered.

In short: this looks like an implementation accident. 

OP: I don't see a usecase for the current behaviour -- I suggest that you 
file a bug report.

> or advice for Daniel, other than
> preprocessing the string by stripping any non letters off the end of the
> line?

The following gives the tokens' starting line for your examples

def shlexiter(s):
p = shlex.shlex(s)
p.whitespace = p.whitespace.replace("\n", "")
while True:
lineno = p.lineno
token = p.get_token()
if not token:
break
if token == "\n":
continue
yield lineno, token

def parseit(string):
print("Parsing - {!r}".format(string))
for lineno, token in shlexiter(string):
print("{:3} {!r}".format(lineno, token))
print("")

but I have no idea about the implications for more complex input.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird bahaviour from shlex - line no

[Dave Angel]

On 28/9/2013 02:26, Daniel Stojanov wrote:

> Can somebody explain this. The line number reported by shlex depends
> on the previous token. I want to be able to tell if I have just popped
> the last token on a line.
>

I agree that it seems weird.  However, I don't think you have made
clear why it's not what you (and I) expect.

import shlex

def parseit(string):
print
print "Parsing -", string
first = shlex.shlex(string)
token = "dummy"
while token:
token = first.get_token()
print token, " -- line", first.lineno

parseit("word1 word2\nword3") #first
parseit("word1 word2,\nword3")#second
parseit("word1 word2,word3\nword4")
parseit("word1 word2+,?\nword3")

This will display the lineno attribute for every token.

shlex is documented at:

http://docs.python.org/2/library/shlex.html

And lineno is documented on that page as:

"""shlex.lineno
Source line number (count of newlines seen so far plus one).
"""

It's not at all clear what "seen so far" is intended to mean, but in
practice, the line number is incremented for the last token on the
line. Thus your first example

Parsing - word1 word2
word3
word1  -- line 1
word2  -- line 2
word3  -- line 2
  -- line 2

word2 has the incremented line number.

But when the token is neither whitespace nor ASCII letters, then it
doesn't increment lineno.  Thus second example:

Parsing - word1 word2,
word3
word1  -- line 1
word2  -- line 1
,  -- line 1  #we would expect this to be "line 2"
word3 -- line 2 -- line 2

Anybody else have some explanation or advice for Daniel, other than
preprocessing the string by stripping any non letters off the end of the
line?

-- 
DaveA


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird bahaviour from shlex - line no

[Andreas Perstinger]

On 28.09.2013 08:26, Daniel Stojanov wrote:

Can somebody explain this. The line number reported by shlex depends
on the previous token. I want to be able to tell if I have just popped
the last token on a line.


[SNIP]


second = shlex.shlex("word1 word2,\nword3")


Punctuation characters like the comma are not considered as word 
characters by default and thus are seen as different tokens (consisting 
of only a single character):


>>> lexer = shlex.shlex("foo, bar, ...")
>>> token = lexer.get_token()
>>> while token != lexer.eof:
...   print token
...   token = lexer.get_token()
...
foo
,
bar
,
.
.
.

If you want to treat them as "word" characters you need to add them to 
the string "wordchars" (a public attribute of the "shlex" instance):


>>> lexer = shlex.shlex("foo.bar, baz")
>>> lexer.wordchar += '.,'
>>> print lexer.get_token()
foo.bar,
>>> print lexer.get_token()
baz

There is also a "debug" attribute (with three different levels: 1, 2, 3; 
default value 0 means no debug output):


>>> lexer = shlex.shlex("foo, bar, ...")
>>> lexer.debug = 1
>>> print lexer.get_token()
shlex: token='foo'
foo
>>> print lexer.get_token()
shlex: popping token ','
,

Bye, Andreas
--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird ttk behaviour

[Chris Angelico]

On Wed, Sep 18, 2013 at 2:11 AM, Rotwang  wrote:
> I don't know tkinter well enough either, but the fact that it behaves
> differently on Linux and Windows suggests to me that at least one version is
> bugging out. Do you think this is worth raising on bugs.python.org?

Possibly, but I'd keep it here on python-list for the moment; it may
be you're actually using undefined behaviour somewhere, in which case
the solution is to change your code. Once someone else (someone who
actually knows what he's talking about, for preference) can confirm
what's going on, it'll be bug report time, I think.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird ttk behaviour

[Rotwang]

On 17/09/2013 15:35, Chris Angelico wrote:

On Wed, Sep 18, 2013 at 12:25 AM, Rotwang  wrote:

In fact, if I replace tkderp with this:


# begin tkderp.py

import tkinter as tk

_root = tk.Tk()
_root.withdraw()

# end tkderp.py


then simply importing tkderp before tkderp2 is enough to make the latter
work properly


Nice piece of detective work! Alas, I don't know tkinter well enough
to help with the details, but this is exactly what I'd like to hear if
I were trying to pick this up and debug it :)


I don't know tkinter well enough either, but the fact that it behaves 
differently on Linux and Windows suggests to me that at least one 
version is bugging out. Do you think this is worth raising on 
bugs.python.org?




Tkinter experts, anywhere? Where's Ranting Rick when you need him...


Last time I saw him was in this thread:

https://mail.python.org/pipermail/python-list/2013-June/650257.html
--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird ttk behaviour

[Chris Angelico]

On Wed, Sep 18, 2013 at 12:25 AM, Rotwang  wrote:
> In fact, if I replace tkderp with this:
>
>
> # begin tkderp.py
>
> import tkinter as tk
>
> _root = tk.Tk()
> _root.withdraw()
>
> # end tkderp.py
>
>
> then simply importing tkderp before tkderp2 is enough to make the latter
> work properly

Nice piece of detective work! Alas, I don't know tkinter well enough
to help with the details, but this is exactly what I'd like to hear if
I were trying to pick this up and debug it :)

Tkinter experts, anywhere? Where's Ranting Rick when you need him...

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird ttk behaviour

[Rotwang]

On 17/09/2013 12:32, Chris Angelico wrote:

[...]

If reloading and doing it again makes things different, what happens
if you simply trigger your code twice without reloading?

I've no idea if it'll help, it just seems like an attack vector on the
problem, so to speak.


Thanks for the suggestion, here's what I've found with some more 
testing. If I rewrite the function f() like this:


def f(fail):
style = ttk.Style(_root)
style.theme_create('newtheme', parent = 'default')
tk.messagebox.showwarning('test', 'test')
if fail:
style.theme_use('newtheme')
tk.messagebox.showwarning('test', 'test')


then I import the module and call f(False) followed by f(True), the 
second call raises an exception just like the original function. I've 
tried variations of the above, such as defining a module-level global 
style instead of having one created during the function call, and the 
end result is always the same. However, suppose instead I define two 
modules, tkderp and tkderp2; both have a function f as defined in my OP, 
but the first has the last two lines of f commented out, and the second 
doesn't (i.e. tkderp is the modified tkderp from before, and tkderp2 is 
the original). Then I do this:


>>> import tkderp
>>> tkderp.f()
>>> import tkderp2
>>> tkderp2.f()


In that case the second call to f() works fine - two warnings, no 
exception. In fact, if I replace tkderp with this:


# begin tkderp.py

import tkinter as tk

_root = tk.Tk()
_root.withdraw()

# end tkderp.py


then simply importing tkderp before tkderp2 is enough to make the latter 
work properly - this


>>> import tkderp2
>>> tkderp2.f()


raises an exception, but this

>>> import tkderp
>>> import tkderp2
>>> tkderp2.f()


doesn't. Any ideas what may be going on?
--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird ttk behaviour

[Chris Angelico]

On Tue, Sep 17, 2013 at 9:27 PM, Rotwang  wrote:
> On 16/09/2013 23:34, Chris Angelico wrote:
>>
>> On Tue, Sep 17, 2013 at 2:28 AM, Rotwang  wrote:
>>>
>>> If I then uncomment those two lines, reload the module and call f() again
>>> (by entering tkderp.reload(tkderp).f()), the function works like it was
>>> supposed to in the first place: two warnings, no exceptions. I can reload
>>> the module as many times as I like and f() will continue to work without
>>> any
>>> problems.
>>
>>
>> Reloading modules in Python is a bit messy. Are you able to tinker
>> with it and make it work in some way without reloading? It'd be easier
>> to figure out what's going on that way.
>
>
> I can't think what else I could try, do you have any suggestions? The
> problem first appeared in a much larger module (I was trying to replace some
> tkinter widgets that looked bad on Linux with their ttk equivalents); the
> only reason I noticed the thing about reloading is that I was trying to
> reproduce the error in a short module by repeatedly making changes and
> reloading.

If reloading and doing it again makes things different, what happens
if you simply trigger your code twice without reloading?

I've no idea if it'll help, it just seems like an attack vector on the
problem, so to speak.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird ttk behaviour

[Rotwang]

On 16/09/2013 23:34, Chris Angelico wrote:

On Tue, Sep 17, 2013 at 2:28 AM, Rotwang  wrote:

If I then uncomment those two lines, reload the module and call f() again
(by entering tkderp.reload(tkderp).f()), the function works like it was
supposed to in the first place: two warnings, no exceptions. I can reload
the module as many times as I like and f() will continue to work without any
problems.


Reloading modules in Python is a bit messy. Are you able to tinker
with it and make it work in some way without reloading? It'd be easier
to figure out what's going on that way.


I can't think what else I could try, do you have any suggestions? The 
problem first appeared in a much larger module (I was trying to replace 
some tkinter widgets that looked bad on Linux with their ttk 
equivalents); the only reason I noticed the thing about reloading is 
that I was trying to reproduce the error in a short module by repeatedly 
making changes and reloading.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird ttk behaviour

[Rotwang]

On 16/09/2013 19:43, Serhiy Storchaka wrote:

16.09.13 19:28, Rotwang написав(ла):

On Windows 7 (sys.version is '3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012,
10:57:17) [MSC v.1600 64 bit (AMD64)]') there's no problem; f() works
fine in the first place. Does anybody know what's going on?


What _root.wantobjects() returns?


It returns True, both before and after the call to 
style.theme_use('newtheme'). Why do you ask? I've no idea what 
Tk.wantobjects is supposed to do (I tried help() and some web searches 
with no luck).


--
https://mail.python.org/mailman/listinfo/python-list

Re: Weird ttk behaviour

[Chris Angelico]

On Tue, Sep 17, 2013 at 2:28 AM, Rotwang  wrote:
> If I then uncomment those two lines, reload the module and call f() again
> (by entering tkderp.reload(tkderp).f()), the function works like it was
> supposed to in the first place: two warnings, no exceptions. I can reload
> the module as many times as I like and f() will continue to work without any
> problems.

Reloading modules in Python is a bit messy. Are you able to tinker
with it and make it work in some way without reloading? It'd be easier
to figure out what's going on that way.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Weird ttk behaviour

[Serhiy Storchaka]

16.09.13 19:28, Rotwang написав(ла):

On Windows 7 (sys.version is '3.3.0 (v3.3.0:bd8afb90ebf2, Sep 29 2012,
10:57:17) [MSC v.1600 64 bit (AMD64)]') there's no problem; f() works
fine in the first place. Does anybody know what's going on?


What _root.wantobjects() returns?


--
https://mail.python.org/mailman/listinfo/python-list

Re: weird behavior. bug perhaps?

[rusi]

On Jun 18, 8:31 pm, zoom  wrote:
>
> yes, that's the hing.
>
> thanks a lot
>
> FYI this happens because
>  >>> shape(mean(m,1))
> (4, 1)
>  >>> shape(mean(array(m),1))
> (4,)
>
> thanks again

And thank you for the 'Thank you'  !!
Given the noob-questions the list is currently dealing with, your
question was a relief and a pleasure.
You have (unwittingly maybe??) followed http://sscce.org/ Thats great.
Also the 'Thanks' at end helps not just to tickle the ego of the
responder but also to confirm the answer which is often not sure.

The only small suggestion I would make is next time please spend a
minute on choosing a pointed subject line.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: weird behavior. bug perhaps?

[zoom]

On 06/18/2013 05:25 PM, Robert Kern wrote:

On 2013-06-18 15:23, zoom wrote:

Hi, I have a strange problem here. Perhaps someone would care to help me.

In the file test.py I have the following code:

from scipy import matrix, tile, mean, shape
import unittest

class TestSequenceFunctions(unittest.TestCase):

def setUp(self):
self.m = [[1,2],[3,4],[3,4],[3,4]]

def test_simplify(self):
m = matrix(self.m)
print shape(m)
print [shape(m)[1],1]
print shape(tile(mean(m,1),[shape(m)[1],1]).T)

if __name__ == '__main__':
unittest.main()

(Note that test.py, is just a simplification of my testing file,
sufficient to
reproduce the weird behavior that I'm about to describe.)

If i run it in terminal via "python test.py" command I get the
following output:

(4, 2)
[2, 1]
(1, 8)
.
--
Ran 1 test in 0.000s

OK


Now comes the funny part.
Let's try to run the following code in python interpreter:

>>> m = [[1,2],[3,4],[3,4],[3,4]]
>>>
>>> from scipy import matrix, tile, mean, shape
>>> print shape(m)
(4, 2)
>>> print [shape(m)[1],1]
[2, 1]
>>> print shape(tile(mean(m,1),[shape(m)[1],1]).T)
(4, 2)

Note the difference between outputs of:
print shape(tile(mean(m,1),[shape(m)[1],1]).T)


I mean, WTF?
This is definitely not the expected behavior.
Anybody knows what just happened here?


As rusi noted, the difference between your two snippets is that in one,
you converted the list of lists to a matrix object in your test suite
but not in your interactive session. Most numpy functions like mean()
will convert their arguments to regular numpy.ndarray objects rather
than matrix objects. matrix is a subclass of ndarray that adds special
behavior: in particular, operations on matrix objects retain their
2D-ness even when an ndarray would flatten down to a 1D array.

[~]
|1> import numpy as np

[~]
|2> m = [[1,2],[3,4],[3,4],[3,4]]

[~]
|3> a = np.array(m)

[~]
|4> b = np.matrix(m)

[~]
|5> np.mean(a, axis=1)
array([ 1.5, 3.5, 3.5, 3.5])

[~]
|6> np.mean(b, axis=1)
matrix([[ 1.5],
[ 3.5],
[ 3.5],
[ 3.5]])

[~]
|7> np.mean(a, axis=1).shape
(4,)

[~]
|8> np.mean(b, axis=1).shape
(4, 1)


This will propagate through the rest of your computation.

Personally, I recommend avoiding the matrix type. It causes too many
problems. Stick to plain ndarrays.

You will probably want to ask further numpy questions on the
numpy-discussion mailing list:

http://www.scipy.org/scipylib/mailing-lists.html


didn't see it earlier.

thanks.

so ndarrays, you say...


--
http://mail.python.org/mailman/listinfo/python-list

Re: weird behavior. bug perhaps?

[zoom]

On 06/18/2013 04:27 PM, rusi wrote:

On Jun 18, 7:23 pm, zoom  wrote:

Hi, I have a strange problem here. Perhaps someone would care to help me.

In the file test.py I have the following code:

from scipy import matrix, tile, mean, shape
import unittest

class TestSequenceFunctions(unittest.TestCase):

  def setUp(self):
  self.m = [[1,2],[3,4],[3,4],[3,4]]

  def test_simplify(self):
  m = matrix(self.m)
  print shape(m)
  print [shape(m)[1],1]
  print shape(tile(mean(m,1),[shape(m)[1],1]).T)

if __name__ == '__main__':
  unittest.main()

(Note that test.py, is just a simplification of my testing file,
sufficient to reproduce the weird behavior that I'm  about to describe.)

If i run it in terminal via "python test.py" command I get the following
output:

(4, 2)
[2, 1]
(1, 8)
.
--
Ran 1 test in 0.000s

OK

Now comes the funny part.
Let's try to run the following code in python interpreter:

  >>>  m = [[1,2],[3,4],[3,4],[3,4]]
  >>>
  >>>  from scipy import matrix, tile, mean, shape
  >>>  print shape(m)
(4, 2)
  >>>  print [shape(m)[1],1]
[2, 1]
  >>>  print shape(tile(mean(m,1),[shape(m)[1],1]).T)
(4, 2)

Note the difference between outputs of:
print shape(tile(mean(m,1),[shape(m)[1],1]).T)

I mean, WTF?
This is definitely not the expected behavior.
Anybody knows what just happened here?


[Never used scipy so pls excuse if I am off...]

Given list m, in the class you are doing m ->  matrix(m)
which you dont seem to be doing in the interpreter.


yes, that's the thing.

thanks a lot

FYI this happens because

>>> shape(mean(m,1))
(4, 1)
>>> shape(mean(array(m),1))
(4,)

thanks again
--
http://mail.python.org/mailman/listinfo/python-list

Re: weird behavior. bug perhaps?

[Robert Kern]

On 2013-06-18 15:23, zoom wrote:

Hi, I have a strange problem here. Perhaps someone would care to help me.

In the file test.py I have the following code:

from scipy import matrix, tile, mean, shape
import unittest

class TestSequenceFunctions(unittest.TestCase):

 def setUp(self):
 self.m = [[1,2],[3,4],[3,4],[3,4]]

 def test_simplify(self):
 m = matrix(self.m)
 print shape(m)
 print [shape(m)[1],1]
 print shape(tile(mean(m,1),[shape(m)[1],1]).T)

if __name__ == '__main__':
 unittest.main()

(Note that test.py, is just a simplification of my testing file, sufficient to
reproduce the weird behavior that I'm  about to describe.)

If i run it in terminal via "python test.py" command I get the following output:

(4, 2)
[2, 1]
(1, 8)
.
--
Ran 1 test in 0.000s

OK


Now comes the funny part.
Let's try to run the following code in python interpreter:

 >>> m = [[1,2],[3,4],[3,4],[3,4]]
 >>>
 >>> from scipy import matrix, tile, mean, shape
 >>> print shape(m)
(4, 2)
 >>> print [shape(m)[1],1]
[2, 1]
 >>> print shape(tile(mean(m,1),[shape(m)[1],1]).T)
(4, 2)

Note the difference between outputs of:
print shape(tile(mean(m,1),[shape(m)[1],1]).T)


I mean, WTF?
This is definitely not the expected behavior.
Anybody knows what just happened here?


As rusi noted, the difference between your two snippets is that in one, you 
converted the list of lists to a matrix object in your test suite but not in 
your interactive session. Most numpy functions like mean() will convert their 
arguments to regular numpy.ndarray objects rather than matrix objects. matrix is 
a subclass of ndarray that adds special behavior: in particular, operations on 
matrix objects retain their 2D-ness even when an ndarray would flatten down to a 
1D array.


[~]
|1> import numpy as np

[~]
|2> m = [[1,2],[3,4],[3,4],[3,4]]

[~]
|3> a = np.array(m)

[~]
|4> b = np.matrix(m)

[~]
|5> np.mean(a, axis=1)
array([ 1.5,  3.5,  3.5,  3.5])

[~]
|6> np.mean(b, axis=1)
matrix([[ 1.5],
[ 3.5],
[ 3.5],
[ 3.5]])

[~]
|7> np.mean(a, axis=1).shape
(4,)

[~]
|8> np.mean(b, axis=1).shape
(4, 1)


This will propagate through the rest of your computation.

Personally, I recommend avoiding the matrix type. It causes too many problems. 
Stick to plain ndarrays.


You will probably want to ask further numpy questions on the numpy-discussion 
mailing list:


  http://www.scipy.org/scipylib/mailing-lists.html

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco

--
http://mail.python.org/mailman/listinfo/python-list

Re: weird behavior. bug perhaps?

[Marcel Rodrigues]

Note that

print [shape(m)[1],1]

just prints a list with two elements where the first element is shape(m)[1]
and the second is the number 1 (regardless of the value of m). I'm pretty
sure that's not what you want.


2013/6/18 zoom 

> Hi, I have a strange problem here. Perhaps someone would care to help me.
>
> In the file test.py I have the following code:
>
> from scipy import matrix, tile, mean, shape
> import unittest
>
> class TestSequenceFunctions(**unittest.TestCase):
>
> def setUp(self):
> self.m = [[1,2],[3,4],[3,4],[3,4]]
>
> def test_simplify(self):
> m = matrix(self.m)
> print shape(m)
> print [shape(m)[1],1]
> print shape(tile(mean(m,1),[shape(m)**[1],1]).T)
>
> if __name__ == '__main__':
> unittest.main()
>
> (Note that test.py, is just a simplification of my testing file,
> sufficient to reproduce the weird behavior that I'm  about to describe.)
>
> If i run it in terminal via "python test.py" command I get the following
> output:
>
> (4, 2)
> [2, 1]
> (1, 8)
> .
> --**--**--
> Ran 1 test in 0.000s
>
> OK
>
>
> Now comes the funny part.
> Let's try to run the following code in python interpreter:
>
> >>> m = [[1,2],[3,4],[3,4],[3,4]]
> >>>
> >>> from scipy import matrix, tile, mean, shape
> >>> print shape(m)
> (4, 2)
> >>> print [shape(m)[1],1]
> [2, 1]
> >>> print shape(tile(mean(m,1),[shape(m)**[1],1]).T)
> (4, 2)
>
> Note the difference between outputs of:
> print shape(tile(mean(m,1),[shape(m)**[1],1]).T)
>
>
> I mean, WTF?
> This is definitely not the expected behavior.
> Anybody knows what just happened here?
>
> --
> http://mail.python.org/**mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: weird behavior. bug perhaps?

[rusi]

On Jun 18, 7:23 pm, zoom  wrote:
> Hi, I have a strange problem here. Perhaps someone would care to help me.
>
> In the file test.py I have the following code:
>
> from scipy import matrix, tile, mean, shape
> import unittest
>
> class TestSequenceFunctions(unittest.TestCase):
>
>      def setUp(self):
>          self.m = [[1,2],[3,4],[3,4],[3,4]]
>
>      def test_simplify(self):
>          m = matrix(self.m)
>          print shape(m)
>          print [shape(m)[1],1]
>          print shape(tile(mean(m,1),[shape(m)[1],1]).T)
>
> if __name__ == '__main__':
>      unittest.main()
>
> (Note that test.py, is just a simplification of my testing file,
> sufficient to reproduce the weird behavior that I'm  about to describe.)
>
> If i run it in terminal via "python test.py" command I get the following
> output:
>
> (4, 2)
> [2, 1]
> (1, 8)
> .
> --
> Ran 1 test in 0.000s
>
> OK
>
> Now comes the funny part.
> Let's try to run the following code in python interpreter:
>
>  >>> m = [[1,2],[3,4],[3,4],[3,4]]
>  >>>
>  >>> from scipy import matrix, tile, mean, shape
>  >>> print shape(m)
> (4, 2)
>  >>> print [shape(m)[1],1]
> [2, 1]
>  >>> print shape(tile(mean(m,1),[shape(m)[1],1]).T)
> (4, 2)
>
> Note the difference between outputs of:
> print shape(tile(mean(m,1),[shape(m)[1],1]).T)
>
> I mean, WTF?
> This is definitely not the expected behavior.
> Anybody knows what just happened here?

[Never used scipy so pls excuse if I am off...]

Given list m, in the class you are doing m -> matrix(m)
which you dont seem to be doing in the interpreter.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird python behavior

[rusi]

On Apr 26, 11:25 am, rusi  wrote:
> To present these kind of errors, Erlang has a concept of sticky
> modules -- those that come from the system…

??present?? should have been 'prevent'
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird python behavior

[rusi]

On Apr 26, 11:04 am, Tim Roberts  wrote:
> Forafo San  wrote:
>
> >OK, lesson learned: Take care not to have module names that conflict with 
> >python's built ins. Sorry for being so obtuse.
>
> You don't have to apologize.  We've all been bitten by this at least once.

Yes…
When it comes to keywords it is reasonable to expect the programmer to
know all the keywords and if he uses one for a function/variable the
error will be immediate.

When it comes to modules it is less reasonable to expect the
programmer to know all available modules.
To present these kind of errors, Erlang has a concept of sticky
modules -- those that come from the system -- for which special
efforts need to be made to 'unstick' them if one wants them
overridden.  This is helpful because the default which is to raise an
error, is in most cases a more sound option than to silently override
a builtin.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird python behavior

[Tim Roberts]

Forafo San  wrote:
>
>OK, lesson learned: Take care not to have module names that conflict with 
>python's built ins. Sorry for being so obtuse.

You don't have to apologize.  We've all been bitten by this at least once.
-- 
Tim Roberts, t...@probo.com
Providenza & Boekelheide, Inc.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird python behavior

[Forafo San]

On Wednesday, April 24, 2013 3:08:27 PM UTC-4, Neil Cerutti wrote:
> On 2013-04-24, Forafo San  wrote:
> 
> > Hello All,
> 
> > I'm running Python version 2.7.3_6 on a FreeBSD system. The following 
> > session in a Python interpreter throws a mysterious TypeError:
> 
> >
> 
> > --
> 
> > [ppvora@snowfall ~/xbrl]$ python 
> 
> > Python 2.7.3 (default, Apr 22 2013, 18:42:18) 
> 
> > [GCC 4.2.1 20070719  [FreeBSD]] on freebsd8
> 
> > Type "help", "copyright", "credits" or "license" for more information.
> 
>  import glob
> 
> > Traceback (most recent call last):
> 
> >   File "", line 1, in 
> 
> >   File "glob.py", line 14, in 
> 
> > myl = glob.glob('data/*.xml')
> 
> > TypeError: 'module' object is not callable
> 
> > --
> 
> > The file glob.py that the error refers to was run under another
> 
> > screen session. It's a mystery why even that program threw an
> 
> > error. Regardless, why should that session throw an import
> 
> > error in this session? Weird. Any help is appreciated. Thanks,
> 
> 
> 
> 'Cause Python's import statement looks in the current directory
> 
> first for files to import. So you're importing your own
> 
> error-riddled and mortal glob.py, rather than Python's pristine
> 
> and Olympian glob.py.
> 
> 
> 
> -- 
> 
> Neil Cerutti
> 
>   "This room is an illusion and is a trap devisut by Satan.  Go
> 
> ahead and dauntlessly!  Make rapid progres!"
> 
>   --Ghosts 'n Goblins

OK, lesson learned: Take care not to have module names that conflict with 
python's built ins. Sorry for being so obtuse.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird python behavior

[Neil Cerutti]

On 2013-04-24, Forafo San  wrote:
> Hello All,
> I'm running Python version 2.7.3_6 on a FreeBSD system. The following session 
> in a Python interpreter throws a mysterious TypeError:
>
> --
> [ppvora@snowfall ~/xbrl]$ python 
> Python 2.7.3 (default, Apr 22 2013, 18:42:18) 
> [GCC 4.2.1 20070719  [FreeBSD]] on freebsd8
> Type "help", "copyright", "credits" or "license" for more information.
 import glob
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "glob.py", line 14, in 
> myl = glob.glob('data/*.xml')
> TypeError: 'module' object is not callable
> --
> The file glob.py that the error refers to was run under another
> screen session. It's a mystery why even that program threw an
> error. Regardless, why should that session throw an import
> error in this session? Weird. Any help is appreciated. Thanks,

'Cause Python's import statement looks in the current directory
first for files to import. So you're importing your own
error-riddled and mortal glob.py, rather than Python's pristine
and Olympian glob.py.

-- 
Neil Cerutti
  "This room is an illusion and is a trap devisut by Satan.  Go
ahead and dauntlessly!  Make rapid progres!"
  --Ghosts 'n Goblins
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird behaviour?

[Steven D'Aprano]

On Mon, 22 Apr 2013 07:29:57 -0700, nn wrote:

> On Apr 21, 9:19 pm, Steven D'Aprano  +comp.lang.pyt...@pearwood.info> wrote:
>> On Mon, 22 Apr 2013 10:56:11 +1000, Chris Angelico wrote:
>> > You're running this under Windows. The convention on Windows is for
>> > end-of-line to be signalled with \r\n, but the convention inside
>> > Python is to use just \n. With the normal use of buffered and parsed
>> > input, this is all handled for you; with unbuffered input, that
>> > translation also seems to be disabled, so your string actually
>> > contains '120\r', as will be revealed by its repr().
>>
>> If that's actually the case, then I would call that a bug in raw_input.
>>
>> Actually, raw_input doesn't seem to cope well with embedded newlines
>> even without the -u option. On Linux, I can embed a control character
>> by typing Ctrl-V followed by Ctrl-. E.g. Ctrl-V Ctrl-M to embed a
>> carriage return, Ctrl-V Ctrl-J to embed a newline. So watch:
>>
>> [steve@ando ~]$ python2.7 -c "x = raw_input('Hello? '); print repr(x)"
>> Hello? 120^M^Jabc
>> '120\r'
>>
>> Everything after the newline is lost.
>>
>> --
>> Steven
> 
> Maybe it is related to this bug?
> 
> http://bugs.python.org/issue11272



I doubt it, I'm not using Windows and that bug is specific to Windows.


Here's the behaviour on Python 3.3:

py> result = input("Type something with control chars: ")
Type something with control chars: something ^T^M else
and a second line
py> print(repr(result))
'something \x14\r else \nand a second line'


Much better!



-- 
Steven

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird behaviour?

[Chris Angelico]

On Tue, Apr 23, 2013 at 9:06 AM,   wrote:
> On Tuesday, April 23, 2013 12:29:57 AM UTC+10, nn wrote:
>
>> Maybe it is related to this bug?
>>
>> http://bugs.python.org/issue11272
>
> I'm running Python 2.7.2 (on Windows) and that version doesn't appear to have 
> that bug:
>
>   Python 2.7.2 (default, Apr 23 2013, 11:49:52) [MSC v.1500 32 bit (Intel)] 
> on win32
>   Type "help", "copyright", "credits" or "license" for more information.
>   >>> print(repr(input()))
>   "testing"
>   'testing'
>   >>>

Careful there; go with raw_input() on Py2. And then it does happen.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird behaviour?

[jussij]

On Tuesday, April 23, 2013 12:29:57 AM UTC+10, nn wrote:

> Maybe it is related to this bug?
> 
> http://bugs.python.org/issue11272

I'm running Python 2.7.2 (on Windows) and that version doesn't appear to have 
that bug:

  Python 2.7.2 (default, Apr 23 2013, 11:49:52) [MSC v.1500 32 bit (Intel)] on 
win32
  Type "help", "copyright", "credits" or "license" for more information.
  >>> print(repr(input()))
  "testing"
  'testing'
  >>>

Cheers Jussi

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird behaviour?

[nn]

On Apr 21, 9:19 pm, Steven D'Aprano  wrote:
> On Mon, 22 Apr 2013 10:56:11 +1000, Chris Angelico wrote:
> > You're running this under Windows. The convention on Windows is for
> > end-of-line to be signalled with \r\n, but the convention inside Python
> > is to use just \n. With the normal use of buffered and parsed input,
> > this is all handled for you; with unbuffered input, that translation
> > also seems to be disabled, so your string actually contains '120\r', as
> > will be revealed by its repr().
>
> If that's actually the case, then I would call that a bug in raw_input.
>
> Actually, raw_input doesn't seem to cope well with embedded newlines even
> without the -u option. On Linux, I can embed a control character by
> typing Ctrl-V followed by Ctrl-. E.g. Ctrl-V Ctrl-M to embed a
> carriage return, Ctrl-V Ctrl-J to embed a newline. So watch:
>
> [steve@ando ~]$ python2.7 -c "x = raw_input('Hello? '); print repr(x)"
> Hello? 120^M^Jabc
> '120\r'
>
> Everything after the newline is lost.
>
> --
> Steven

Maybe it is related to this bug?

http://bugs.python.org/issue11272


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird behaviour?

[jussij]

On Monday, April 22, 2013 11:05:11 AM UTC+10, Steven D'Aprano wrote:

> I cannot confirm that behaviour. It works fine for me.

As Chris pointed out there is a \r character at the end of the string and that 
is causing the if to fail. 

I can now see the \r :)

So this is *Windows only* behaviour. 

Cheers Jussi
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird behaviour?

[Steven D'Aprano]

On Mon, 22 Apr 2013 10:56:11 +1000, Chris Angelico wrote:

> You're running this under Windows. The convention on Windows is for
> end-of-line to be signalled with \r\n, but the convention inside Python
> is to use just \n. With the normal use of buffered and parsed input,
> this is all handled for you; with unbuffered input, that translation
> also seems to be disabled, so your string actually contains '120\r', as
> will be revealed by its repr().


If that's actually the case, then I would call that a bug in raw_input.

Actually, raw_input doesn't seem to cope well with embedded newlines even 
without the -u option. On Linux, I can embed a control character by 
typing Ctrl-V followed by Ctrl-. E.g. Ctrl-V Ctrl-M to embed a 
carriage return, Ctrl-V Ctrl-J to embed a newline. So watch:


[steve@ando ~]$ python2.7 -c "x = raw_input('Hello? '); print repr(x)"
Hello? 120^M^Jabc
'120\r'

Everything after the newline is lost.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird behaviour?

[Chris Angelico]

On Mon, Apr 22, 2013 at 11:05 AM, Steven D'Aprano
 wrote:
> I cannot confirm that behaviour. It works fine for me.

I should mention: Under Linux, there's no \r, so -u or no -u, the
program will work fine.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird behaviour?

[jussij]

On Monday, April 22, 2013 10:56:11 AM UTC+10, Chris Angelico wrote:

> so your string actually contains '120\r', as will be revealed 
> by its repr().

Thanks Chris. That makes sense.

Cheers Jussi

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird behaviour?

[Steven D'Aprano]

On Sun, 21 Apr 2013 17:37:18 -0700, jussij wrote:

> Can someone please explain the following behaviour?
> 
> I downloaded and compiled the Python 2.7.2 code base.
> 
> I then created this simple c:\temp\test.py macro:
> 
> import sys
> 
> def main():
> print("Please Input 120: ")
> input = raw_input()
> 
> print("Value Inputed: " + input)
> 
> if str(input) == "120":
> print("Yes")
> else:
> print("No")
> 
> main()
> 
> If I run the macro using the -u (flush buffers) option the if statement
> always fails:
> 
> C:\Temp>python.exe -u c:\temp\test.py Please Input 120:
> 120
> Value Inputed: 120
> No


I cannot confirm that behaviour. It works fine for me.

Are you sure that the code you show above is *exactly* the code you're 
using? Actually, it cannot possibly be the exact code you are using, 
since it has been indented. What other changes did you make when retyping 
it in your message? Please copy and paste the actual code used, without 
retyping or making any modifications.

If the problem still persists, try printing repr(input) and see what it 
shows.

Also, a comment on your code: there is no need to call str(input), since 
input is already a string. If you replace the test:

str(input) == '120'

with just 

input == '120'

does the problem go away? If so, then there's something funny going on 
with the call to str(). Please print(str), and see what it shows.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird behaviour?

[Chris Angelico]

On Mon, Apr 22, 2013 at 10:37 AM,   wrote:
> Can someone please explain the following behaviour?
>
> If I run the macro using the -u (flush buffers) option the if statement 
> always fails:
>
> C:\Temp>python.exe -u c:\temp\test.py
> Please Input 120:
> 120
> Value Inputed: 120
> No

Here's the essence of your program:

print(repr(raw_input()))

You can use that to verify what's going on. Try running that with and
without the -u option; note, by the way, that -u actually means
"unbuffered", not "flush buffers".

You're running this under Windows. The convention on Windows is for
end-of-line to be signalled with \r\n, but the convention inside
Python is to use just \n. With the normal use of buffered and parsed
input, this is all handled for you; with unbuffered input, that
translation also seems to be disabled, so your string actually
contains '120\r', as will be revealed by its repr().

By the way, raw_input() already returns a string. There's no need to
str() it. :)

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird exception handling behavior -- late evaluation in except clause

[Chris Angelico]

On Mon, Dec 3, 2012 at 6:30 PM, Steven D'Aprano
 wrote:
> Yeah, in hindsight it was a pretty crappy example. But this sort of
> dynamism really is useful:
>
> def testRaises(exc, func, *args):
> try:
> result = func(*args)
> except exc:
> return
> raise AssertionError("expected exception but didn't get one")
>
>
> def wrap(func, exc, default=None):
> @functools.wraps(func)
> def inner(*args):
> try:
> return func(*args)
> except exc:
> return default
> return inner

Ah, that makes good sense. The 'except' clause takes a parameter, so
it follows logically that you could pass a parameter to something that
wraps an except clause.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird exception handling behavior -- late evaluation in except clause

[Steven D'Aprano]

On Mon, 03 Dec 2012 16:24:50 +1100, Chris Angelico wrote:

> On Mon, Dec 3, 2012 at 8:31 AM, Steven D'Aprano
>  wrote:
>> Consider this piece of legal Python code:
>>
>> Err = None
>> if condition(x) > 100:
>> Err = OneException
>> elif another_condition(x):
>> Err = AnotherException
>> try:
>> spam(a, b, c)
>> except Err:
>> recover()
> 
> Legal it may be, but are there times when you actually _need_ this level
> of dynamism? It strikes me as a pretty weird way of going about things.
> 
> I agree with the point you're making, but this feels like a contrived
> example, and I'm curious as to whether it can be uncontrived.


Yeah, in hindsight it was a pretty crappy example. But this sort of 
dynamism really is useful:

def testRaises(exc, func, *args):
try:
result = func(*args)
except exc:
return
raise AssertionError("expected exception but didn't get one")


def wrap(func, exc, default=None):
@functools.wraps(func)
def inner(*args):
try:
return func(*args)
except exc:
return default
return inner


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird exception handling behavior -- late evaluation in except clause

[Chris Angelico]

On Mon, Dec 3, 2012 at 8:31 AM, Steven D'Aprano
 wrote:
> Consider this piece of legal Python code:
>
> Err = None
> if condition(x) > 100:
> Err = OneException
> elif another_condition(x):
> Err = AnotherException
> try:
> spam(a, b, c)
> except Err:
> recover()

Legal it may be, but are there times when you actually _need_ this
level of dynamism? It strikes me as a pretty weird way of going about
things.

I agree with the point you're making, but this feels like a contrived
example, and I'm curious as to whether it can be uncontrived.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird exception handling behavior -- late evaluation in except clause

[Steven D'Aprano]

On Sun, 02 Dec 2012 12:25:22 -0500, Roy Smith wrote:

> This is kind of weird (Python 2.7.3):
> 
> try:
> print "hello"
> except foo:
> print "foo"
> 
> prints "hello".  The problem (IMHO) is that apparently the except clause
> doesn't get evaluated until after some exception is caught.  Which means
> it never notices that foo is not defined until it's too late.

This is exactly the same as, well, everything in Python. Nothing is 
evaluated until needed.

Consider this piece of legal Python code:

Err = None
if condition(x) > 100:
Err = OneException
elif another_condition(x):
Err = AnotherException
try:
spam(a, b, c)
except Err:
recover()


And consider that spam() may very well set the global Err to yet another 
value, or delete it altogether, before raising. How should Python check 
that Err is defined to an actual exception ahead of time?

The names of exceptions are no different from any other names in Python. 
They're merely names, and they're looked up at runtime. If you want to 
boggle/confuse/annoy your friends, shadowing builtins can help:


py> def spam(x):
... global UnicodeEncodeError
... UnicodeEncodeError = ZeroDivisionError
... return 1/x
...
py> try:
... spam(0)
... except UnicodeEncodeError:
... print("Divided by zero")
...
Divided by zero


(By the way, if you ever do this by accident, you can recover with a 
simple "del UnicodeEncodeError" to restore access to the builtin.)

As Terry has said, those sort of pre-runtime checks are the 
responsibility of pylint or pychecker or equivalent. Python is too 
dynamic to allow the sort of compile-time checks you can get from a 
Haskell, C or Pascal, so the Python compiler delegates responsibility to 
third-party applications. There's no point in making an exceptions for 
exceptions.


> This just came up in some code, where I was trying to catch a very rare
> exception.  When the exception finally happened, I discovered that I had
> a typo in the except clause (I had mis-spelled the name of the
> exception).  So, instead of getting some useful information, I got an
> AttributeError :-(

One of the nice features of Python 3:

py> try:
... 1/0
... except ZeroDividingError:
... pass
...
Traceback (most recent call last):
  File "", line 2, in 
ZeroDivisionError: division by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "", line 3, in 
NameError: name 'ZeroDividingError' is not defined


Very useful for debugging unexpected errors in except clauses. The 
downside is that until Python 3.3, there's no way to turn it off when you 
intentionally catch one exception and raise another in it's place.


> Is this a bug, or intended behavior?  It seems to me it would be much
> more useful (if slightly more expensive) to evaluate the names of the
> exceptions in the expect clause before running the try block.

"Slightly" more expensive? Methinks you are underestimating the level of 
dynamism of Python.

py> def pick_an_exception():
... global x
... if 'x' in globals():
... return TypeError
... x = None
... return NameError
...
py> for i in range(3):
... try:
... print(x + 1)
... except pick_an_exception() as err:
... print("Caught", err)
...
Caught name 'x' is not defined
Caught unsupported operand type(s) for +: 'NoneType' and 'int'
Caught unsupported operand type(s) for +: 'NoneType' and 'int'


The except expression can be arbitrarily expensive, with arbitrary side-
effects.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird exception handling behavior -- late evaluation in except clause

[Terry Reedy]

On 12/2/2012 12:25 PM, Roy Smith wrote:

This is kind of weird (Python 2.7.3):

try:
 print "hello"
except foo:
 print "foo"

prints "hello".  The problem (IMHO) is that apparently the except clause
doesn't get evaluated until after some exception is caught.  Which means
it never notices that foo is not defined until it's too late.

This just came up in some code, where I was trying to catch a very rare
exception.  When the exception finally happened, I discovered that I had
a typo in the except clause (I had mis-spelled the name of the
exception).  So, instead of getting some useful information, I got an
AttributeError :-(


You would have the same problem if you spelled the exception name right 
but misspelled a name in the corresponding block. This is PyLint, 
PyChecker, xxx territory.


Note that proper checking requires execution of imports.

import mymod

try: pass
except mymod.MymogException: pass  # whoops


Is this a bug, or intended behavior?  It seems to me it would be much
more useful (if slightly more expensive) to evaluate the names of the
exceptions in the expect clause before running the try block.


'try:' is very cheap. Searching ahead any checking names in all the 
except clauses would be much more expensive -- and useless after the 
first time the code is run after being revised.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: Weird exception handling behavior -- late evaluation in except clause

[Hans Mulder]

On 2/12/12 18:25:22, Roy Smith wrote:
> This is kind of weird (Python 2.7.3):
> 
> try:
> print "hello"
> except foo:
> print "foo"
> 
> prints "hello".  The problem (IMHO) is that apparently the except clause 
> doesn't get evaluated until after some exception is caught.  Which means 
> it never notices that foo is not defined until it's too late.
> 
> This just came up in some code, where I was trying to catch a very rare 
> exception.  When the exception finally happened, I discovered that I had 
> a typo in the except clause (I had mis-spelled the name of the 
> exception).  So, instead of getting some useful information, I got an 
> AttributeError :-(
> 
> Is this a bug, or intended behavior?  It seems to me it would be much 
> more useful (if slightly more expensive) to evaluate the names of the 
> exceptions in the expect clause before running the try block.

It's intended behaviour: Python runs your script from top to bottom.
It doesn't peek ahead at except: clauses.  Even it an exception is
raised, the exception names are not eveluated all at once.  Python
begins by evaluating the first exception name.  It the exception at
hand is an instance of that, Python has found a match and the rest
of the names are left unevaluated:

>>> try:
...  1/0
... except ZeroDivisionError:
...  print "hello"
... except 1/0:
...  pass
...
hello

There's probably some lint-like tool that can find this kind of issue.


Hope this helps,

-- HansM
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird import failure with "nosetests --processes=1"

[Roy Smith]

In article <50b78e26$0$6945$e4fe5...@news2.news.xs4all.nl>,
 Hans Mulder  wrote:

> That is baffling indeed.  It looks like nose is adding some
> directory to sys.path, which contains a module pyza.py instead
> of a package.

We finally figured it out.  As it turns out, that's pretty close.

> Another idea might be to delete all *.pyc files below
> /home/roy/deploy/current/ and /home/roy/songza/

I've got

sys.path.insert(0, '/home/roy/deploy/current')

in my virtualenv's usercustomize.py.  Also, though historical accident, 
I had (but don't any longer)

PYTHONPATH=/home/roy/songza

in my .profile.  And /home/roy/deploy/current is a symlink to 
/home/roy/songza!  So, I had two different paths to the same directory.  
Why this only showed up as a problem when running under nose in 
multiprocess mode, I have no clue.  And how it ended up thinking pyza 
was a module instead of a package, I also have no idea.

Crazy.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: weird isinstance/issubclass behavior?

[Terry Reedy]

On 11/29/2012 9:59 AM, lars van gemerden wrote:

Hi,

I have encountered some strange behavior of isinstance(/issubclass): depending 
on the import path used for classes i get different output, while the classes i 
compare are in the same file.

Basically if i import a class as:

 from mod1.mod2 import A
or:
 from mod0.mod1.mod2 import A

which both result in importing the same class,


As other said, both import the same abstract class but create two 
different concrete class objects.


> a call to isinstance(inst, A) in another module can have a different 
output.

In this module
 print type(inst), A, isinstance(inst, A), issubclass(type(inst), A)
gives:
   False False


Add print id(type(inst)), id(A) and you will see that they are different 
objects. The isinstance and issubclass calls compare the classes by 
identity (the default meaning of ==) and so False, False are correct.


--
Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

Re: weird isinstance/issubclass behavior?

[Ian Kelly]

On Thu, Nov 29, 2012 at 9:07 AM, lars van gemerden wrote:

> > PS: this is somewhat simpler than the actual case i've encountered, and
> i haven't tested this exact case, but for now i hope this is enough to get
> some of your insight.
>
> I know for sure that the imports both import the same file, though if i
> understand you correctly, it creates 2 different module objects? Are module
> object only created on an import statement?
>
>
Yes, Python sees imports of two different absolute module paths and does
not recognize them as being the same file, so it imports each separately.
This scenario typically arises when your sys.path includes a folder that is
actually a module itself.  Recommended usage: don't do that.  Your sys.path
entries should contain modules and packages, not point inside them.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird import failure with "nosetests --processes=1"

[Hans Mulder]

On 29/11/12 04:13:57, Roy Smith wrote:
> I've got a minimal test script:
> 
> -
> $ cat test_foo.py
> import pyza.models
> print pyza.models
> 
> def test_foo():
> pass
> -
> 
> pyza.models is a package.  Under normal conditions, I can import it fine:
> 
> $ python
> Python 2.7.3 (default, Aug  1 2012, 05:14:39) 
> [GCC 4.6.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
 import pyza.models
 print pyza.models
>  '/home/roy/deploy/current/pyza/models/__init__.pyc'>

> 
> But when I run nosetests in parallel mode, the import fails in a way 
> which has me baffled.  What's going on here?
> 
> $ nosetests --processes=1 -s test_foo:test_foo
>  '/home/roy/deploy/current/pyza/models/__init__.pyc'>
> E
> ==
> ERROR: Failure: AttributeError ('module' object has no attribute 
> 'models')
> --
> Traceback (most recent call last):
>   File 
> "/home/roy/production/python/local/lib/python2.7/site-packages/nose/loade
> r.py", line 390, in loadTestsFromName
> addr.filename, addr.module)
>   File 
> "/home/roy/production/python/local/lib/python2.7/site-packages/nose/impor
> ter.py", line 39, in importFromPath
> return self.importFromDir(dir_path, fqname)
>   File 
> "/home/roy/production/python/local/lib/python2.7/site-packages/nose/impor
> ter.py", line 86, in importFromDir
> mod = load_module(part_fqname, fh, filename, desc)
>   File "/home/roy/songza/pyza/djapi/test_foo.py", line 2, in 
> print pyza.models
> AttributeError: 'module' object has no attribute 'models'
> 
> --
> Ran 1 test in 0.107s
> 
> FAILED (errors=1)


That is baffling indeed.  It looks like nose is adding some
directory to sys.path, which contains a module pyza.py instead
of a package.

One thing you could try, is changing line 2 of you script to just

print pyza

, to see if pyza is being loaded from
/home/roy/deploy/current/pyza/__init__.pyc, as you'd expect.  If it
isn't, you'll be
one step closer to a solution.

Another idea might be to delete all *.pyc files below
/home/roy/deploy/current/ and /home/roy/songza/pyza/djapi/.
That would help if a .pyc file is somehow out of sync with
the corresponding .py file.  That's not supposed to happen,
but if it does, you can get weird results.

One last idea: put these lines at the top of test_foo.py

import pdb
pdb.set_trace()

Running under nosetest should then drop you in the debugger.
Single step into the next statement ("import pyza.models") to
see where this gets you.  It should import pyza, and then
pyza.models.  Add some print statements to the __init__.py
files of those packages, so that there's somewhere for pdb
to stop and prompt.


Hope this helps,

-- HansM








-- 
http://mail.python.org/mailman/listinfo/python-list

Re: weird isinstance/issubclass behavior?

[lars van gemerden]

On Thursday, November 29, 2012 3:59:37 PM UTC+1, lars van gemerden wrote:
> Hi,
> 
> 
> 
> I have encountered some strange behavior of isinstance(/issubclass): 
> depending on the import path used for classes i get different output, while 
> the classes i compare are in the same file. 
> 
> 
> 
> Basically if i import a class as:
> 
> 
> 
> from mod1.mod2 import A
> 
> 
> 
> or:
> 
> 
> 
> from mod0.mod1.mod2 import A
> 
> 
> 
> which both result in importing the same class, a call to isinstance(inst, A) 
> in another module can have a different output. In this module
> 
> 
> 
> print type(inst), A, isinstance(inst, A), issubclass(type(inst), A)
> 
> 
> 
> gives:
> 
> 
> 
>   False False
> 
> 
> 
> resp.
> 
> 
> 
>   True True
> 
> 
> 
> which seems somewhat logical, but in my case a strange gotcha. My question is:
> 
> Is this intended, inevitable or a bug?
> 
> 
> 
> Cheers, Lars
> 
> 
> 
> PS: this is somewhat simpler than the actual case i've encountered, and i 
> haven't tested this exact case, but for now i hope this is enough to get some 
> of your insight.

I know for sure that the imports both import the same file, though if i 
understand you correctly, it creates 2 different module objects? Are module 
object only created on an import statement?  

The 2 parameters of isinstance() follow a very different import path. 

Anyway, to not spend too much time us this, i'll just file it under 'dangerous' 
and use the second import statement. It does solve my problem. 

Thanks, Lars
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: weird isinstance/issubclass behavior?

[Chris Angelico]

On Fri, Nov 30, 2012 at 1:59 AM, lars van gemerden  wrote:
> Basically if i import a class as:
>
> from mod1.mod2 import A
>
> or:
>
> from mod0.mod1.mod2 import A
>
> which both result in importing the same class, a call to isinstance(inst, A) 
> in another module can have a different output.

What you may be seeing there is that you've imported the module twice.
There are two entirely separate module objects, taken from the same
file. Something instantiated from one class isn't an instance of the
other class even if they're indistinguishable classes; a little
monkeypatching might show you what's going on (fiddle with one class,
check the other, fiddle hasn't happened).

As a general rule, importing a module in different ways is considered
dangerous. Be consistent, and then this sort of oddity won't occur -
and you also won't have other oddities, eg with other global state.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: weird behaviour: pygame plays in shell but not in script

[Jean-Michel Pichavant]

Mik wrote:

Oh thanks alex!
that's kind!

PS: It looks like a party indeed: plenty of interesting
discussions :-)

On Mar 30, 4:33 am, alex23  wrote:
  

On Mar 29, 10:41 pm, Mik  wrote:



What a nice way to introduce myself to the group!!! :-)
  

Hey, don't beat yourself up, you:

 - summarised the problem in the subject heading
 - included actual code showing the problem
 - reported back on the problem you found

That puts you ahead of most new posters.



sorry for bothering you guys :-)
  

No bother at all, welcome to the party :)



  

Welcome Mik !

JM

PS : please don't top post in this list
--
http://mail.python.org/mailman/listinfo/python-list

Re: weird behaviour: pygame plays in shell but not in script

[Chris Angelico]

On Thu, Mar 29, 2012 at 11:41 PM, Mik  wrote:
>
> I can't believe I am so dumb!
>
> after sound.play() the script was terminating
> I didn't notice that 'play()' actually returns...
>
> What a nice way to introduce myself to the group!!! :-)
>
> sorry for bothering you guys :-)

You've just proven one of the great benefits of asking a question: you
suddenly discover the answer for yourself. It's arguable whether it's
primarily caused by the action of coalescing your question into text,
or some outworking of Murphy's Law, but both are involved.

Welcome to the party. There's some trolls over there *gestures to the
corner* and some smart guys over there *gestures to the other corner*
and a few snakes and comedians around *gestures to the random pythons
and Pythons and Pythons and pythons* and don't forget the lurkers
lurking around. Enjoy yourself, learn something, educate someone!

Chris Angelico
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: weird behaviour: pygame plays in shell but not in script

[rusi]

On Mar 30, 8:33 am, alex23  wrote:
> On Mar 29, 10:41 pm, Mik  wrote:
>
> > What a nice way to introduce myself to the group!!! :-)
>
> Hey, don't beat yourself up, you:
>
>  - summarised the problem in the subject heading
>  - included actual code showing the problem
>  - reported back on the problem you found
>
> That puts you ahead of most new posters.

Well said.
I only disagree with the 'new' 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: weird behaviour: pygame plays in shell but not in script

[Mik]

Oh thanks alex!
that's kind!

PS: It looks like a party indeed: plenty of interesting
discussions :-)

On Mar 30, 4:33 am, alex23  wrote:
> On Mar 29, 10:41 pm, Mik  wrote:
>
> > What a nice way to introduce myself to the group!!! :-)
>
> Hey, don't beat yourself up, you:
>
>  - summarised the problem in the subject heading
>  - included actual code showing the problem
>  - reported back on the problem you found
>
> That puts you ahead of most new posters.
>
> > sorry for bothering you guys :-)
>
> No bother at all, welcome to the party :)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: weird behaviour: pygame plays in shell but not in script

[alex23]

On Mar 29, 10:41 pm, Mik  wrote:
> What a nice way to introduce myself to the group!!! :-)

Hey, don't beat yourself up, you:

 - summarised the problem in the subject heading
 - included actual code showing the problem
 - reported back on the problem you found

That puts you ahead of most new posters.

> sorry for bothering you guys :-)

No bother at all, welcome to the party :)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: weird behaviour: pygame plays in shell but not in script

[Mik]

I can't believe I am so dumb!

after sound.play() the script was terminating
I didn't notice that 'play()' actually returns...

What a nice way to introduce myself to the group!!! :-)

sorry for bothering you guys :-)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird newbie question

[Matty Sarro]

The issue was a version mismatch. When installing python on windows,
it installs a shortcut so when you right click, you can edit things in
IDLE. I had installed python 3 after installing 2.7, so it apparently
changed the extension.

As for not showing tracebacks, I couldn't. When I ran the program from
the command line, it ran the version 2.7 python because that's the one
I have set in my $PATH. However when I was editing it in IDLE, it
would highlight the second set of quotation marks in the statement and
simply spit out an error saying "Invalid Syntax." There was no
traceback because IDLE wouldn't run the program in the interpreter
that opens along side the editor.

Re-opening the file in the 2.7 version of IDLE removed any issues.

Thanks for your help everyone.

On Thu, Jan 26, 2012 at 5:53 PM, Rick Johnson
 wrote:
> On Jan 26, 4:05 pm, Matty Sarro  wrote:
>
>> Here's my code:
>>
>> from sys import argv
>> script,filename=argv
>> txt=open(filename)
>> print "Here is your file %r:" % filename
>> print txt.read()
>> print "I'll also ask you to type it again:"
>> file_again=raw_input("> ")
>> txt_again=open(file_again)
>> print txt_again.read()
>>
>> IDLE is saying that my error is on line 4, at the second set of
>> quotation marks. [...]
>
> Forget about line 4 because line 2 is a problem waiting to happen
> (PEP8 formating added for clarity)
>
> Line2: script, filename = argv
>
> Is this a case of the blind leading the blind?
>
> --
> http://mail.python.org/mailman/listinfo/python-list
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird newbie question

[Rick Johnson]

On Jan 26, 4:05 pm, Matty Sarro  wrote:

> Here's my code:
>
> from sys import argv
> script,filename=argv
> txt=open(filename)
> print "Here is your file %r:" % filename
> print txt.read()
> print "I'll also ask you to type it again:"
> file_again=raw_input("> ")
> txt_again=open(file_again)
> print txt_again.read()
>
> IDLE is saying that my error is on line 4, at the second set of
> quotation marks. [...]

Forget about line 4 because line 2 is a problem waiting to happen
(PEP8 formating added for clarity)

Line2: script, filename = argv

Is this a case of the blind leading the blind?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird newbie question

[John Gordon]

In  Matty Sarro 
 writes:

> Hey everyone. I'm running into a funky error as I work through "Learn
> Python the Hard Way." What's weird is both idle and the python
> interpreter in idle spit out an error about syntax, but when I run the
> same script from the command line it works just fine, with no issue.
> I'm not sure if its an issue with IDLE or if I'm doing something
> wrong.

> Here's the book's code:

> from sys import argv
> script, filename = argv
> txt = open(filename)
> print "Here's your file %r:" % filename
> print txt.read()
> print "Type the filename again:"
> file_again = raw_input("> ")
> txt_again = open(file_again)
> print txt_again.read()

> Here's my code:

> from sys import argv
> script,filename=argv
> txt=open(filename)
> print "Here is your file %r:" % filename
> print txt.read()
> print "I'll also ask you to type it again:"
> file_again=raw_input("> ")
> txt_again=open(file_again)
> print txt_again.read()


> IDLE is saying that my error is on line 4, at the second set of
> quotation marks. Since I can't get the error from the command line, I
> can't actually see what the syntax error is that it's complaining
> about. Any advice would really be appreciated, thank you!!

This may be a Python version mismatch error.

In python version 2.x (and 1.x for that matter), "print" is a statement,
but it was changed to be a function in python version 3.x.

In other words, in python 2.x you can say this:

  print x
  print "hello there"

But in python 3.x you have to do it a little differently:

  print(x)
  print("hello there")

It sounds like your IDLE is running python 3, but your command line is
running python 2.

-- 
John Gordon   A is for Amy, who fell down the stairs
gor...@panix.com  B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird newbie question

[Ethan Furman]

Matty Sarro wrote:

from sys import argv
script,filename=argv
txt=open(filename)
print "Here is your file %r:" % filename
print txt.read()
print "I'll also ask you to type it again:"
file_again=raw_input("> ")
txt_again=open(file_again)
print txt_again.read()

IDLE is saying that my error is on line 4, at the second set of
quotation marks.


The code is for Python 2 -- are you using Python 3?

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Weird newbie question

[Steven D'Aprano]

On Thu, 26 Jan 2012 17:05:57 -0500, Matty Sarro wrote:

> Hey everyone. I'm running into a funky error as I work through "Learn
> Python the Hard Way." What's weird is both idle and the python
> interpreter in idle spit out an error about syntax, but when I run the
> same script from the command line it works just fine, with no issue.

Either you are mistaken about it being the same script, or you have two 
versions of Python installed, Python 2.x and Python 3.x.

My guess is that you have two versions installed, and when you run IDLE 
you are running Python 3, and when you run the script from the command 
line you are running Python 2.


> I'm
> not sure if its an issue with IDLE or if I'm doing something wrong.
> 
> Here's the book's code:

Irrelevant. Unless you think that somehow your computer is running the 
code in the book instead of the code in your computer?


> Here's my code:
> 
> from sys import argv
> script,filename=argv
> txt=open(filename)
> print "Here is your file %r:" % filename
> print txt.read()
> print "I'll also ask you to type it again:" 
> file_again=raw_input("> ")
> txt_again=open(file_again)
> print txt_again.read()
> 
> 
> IDLE is saying that my error is on line 4, at the second set of
> quotation marks.

Line four is not the second set of quotation marks. It is the first set 
of quotation marks.

Please show the ENTIRE error displayed, including the full traceback, and 
not just the message. That is, everything starting from "Traceback (most 
recent call last)" to the end. Please copy and paste the full error, do 
not summarise or paraphrase it.


> Since I can't get the error from the command line,

Earlier you said that the script works correctly when you run it from the 
command line.


> I
> can't actually see what the syntax error is that it's complaining about.
> Any advice would really be appreciated, thank you!!

(1) Inside IDLE, type this:

import sys
sys.version

What does it show?

(2) Show us the exact command you give on the command line that 
successfully runs the script.


There may be more questions later, but this will do to start.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird newbie question

[Andrea Crotti]

On 01/26/2012 10:05 PM, Matty Sarro wrote:

Hey everyone. I'm running into a funky error as I work through "Learn
Python the Hard Way." What's weird is both idle and the python
interpreter in idle spit out an error about syntax, but when I run the
same script from the command line it works just fine, with no issue.
I'm not sure if its an issue with IDLE or if I'm doing something
wrong.

Here's the book's code:

from sys import argv
script, filename = argv
txt = open(filename)
print "Here's your file %r:" % filename
print txt.read()
print "Type the filename again:"
file_again = raw_input(">  ")
txt_again = open(file_again)
print txt_again.read()

Here's my code:

from sys import argv
script,filename=argv
txt=open(filename)
print "Here is your file %r:" % filename
print txt.read()
print "I'll also ask you to type it again:"
file_again=raw_input(">  ")
txt_again=open(file_again)
print txt_again.read()


IDLE is saying that my error is on line 4, at the second set of
quotation marks. Since I can't get the error from the command line, I
can't actually see what the syntax error is that it's complaining
about. Any advice would really be appreciated, thank you!!


Let me try, maybe you're running the IDLE for Python 3 and the shell for 
python 2?

--
http://mail.python.org/mailman/listinfo/python-list

Re: Weird Loop Behaviour

[Arnaud Delobelle]

On 20 January 2012 20:47, Yigit Turgut  wrote:
> Hi,
>
> In the following code, I am trying to run "black" screen for 3 seconds
> and respectively 2 seconds "white" screen. Black routine takes 3
> seconds and white 2 seconds, 2 x black + white = 8 seconds which
> should be the expected value but when I run it I get black-white-black-
> white   instead of black-white-black. Couldn't figure out what is
> wrong thus sharing the code as well ;
>
> white = False
>     while(end<8.00):
>      end = time.time() - start
>      if white:
>       screen.fill((255, 255, 255))
>        time.sleep(2)
>      else:
>        screen.fill((0, 0, 0))
>        time.sleep(3)
>      white = not white
>      pygame.display.update()
>     pygame.quit()

This is cryptic.  You'd be better off with something like

black = 0, 0, 0
white = 255, 255, 255
for color, wait in (black, 3), (white, 2), (black, 3):
screen.fill(color)
pygame.display.update()
time.sleep(wait)
pygame.quit()

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird Loop Behaviour

[Emile van Sebille]

On 1/20/2012 12:47 PM Yigit Turgut said...

Hi,

In the following code, I am trying to run "black" screen for 3 seconds
and respectively 2 seconds "white" screen. Black routine takes 3
seconds and white 2 seconds, 2 x black + white = 8 seconds which
should be the expected value but when I run it I get black-white-black-
white   instead of black-white-black. Couldn't figure out what is
wrong thus sharing the code as well ;

white = False
  while(end<8.00):
   end = time.time() - start
you're setting end's value before the display happens, so while tests 
the values 0,3,5 before getting after the fourth pass 8.  Move this 
after to white = note white and I suspect you'll be OK.


HTH,

Emile





   if white:
screen.fill((255, 255, 255))
time.sleep(2)
   else:
screen.fill((0, 0, 0))
time.sleep(3)
   white = not white
   pygame.display.update()
  pygame.quit()



--
http://mail.python.org/mailman/listinfo/python-list

Re: Weird Loop Behaviour

[MRAB]

On 20/01/2012 20:47, Yigit Turgut wrote:

Hi,

In the following code, I am trying to run "black" screen for 3 seconds
and respectively 2 seconds "white" screen. Black routine takes 3
seconds and white 2 seconds, 2 x black + white = 8 seconds which
should be the expected value but when I run it I get black-white-black-
white   instead of black-white-black. Couldn't figure out what is
wrong thus sharing the code as well ;

white = False
while(end<8.00):
end = time.time() - start
if white:
screen.fill((255, 255, 255))
time.sleep(2)
else:
screen.fill((0, 0, 0))
time.sleep(3)
white = not white
pygame.display.update()
pygame.quit()


Could it be because you're setting 'end' after testing it?

It might be simpler as:

  while time.time() - start < 8:

Also, should it really be sleeping before updating the display? I
would've thought that it should be sleeping _after_ updating the
display.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Weird interaction with nested functions inside a decorator-producing function and closuring of outer data...

[Steven D'Aprano]

Adam Jorgensen wrote:

> Thanks :-) Sorry about the size, I wasn't sure what was relevant...

We prefer that you don't top-post here, because it makes it hard to see
context when people reply.

In general, people asking questions should always try to reduce the problem
to the simplest code that will demonstrate the problem. This has two huge
advantages:

(1) We're all volunteers here, none of us are paid to solve your problems.
The more work needed to answer a question that we might not care that much
about, the less likely it is that you will get good answers. The easier you
make it for us, the more likely you will get many good, prompt answers.

(2) Even more important... the process of cutting out all the irrelevant
details and reducing the code to the smallest possible example may reveal
to you what the problem is.

As the old proverb goes, "solve a coder's problem, and you've solved one
problem... teach a coder how to solve his own problems, and you've solved
them all" *wink*

I can't begin to count how many times I've started writing up a post to ask
a question, and in the process of reducing the example code to the smallest
it can be, I've solved it myself.


-- 
Steven

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Weird interaction with nested functions inside a decorator-producing function and closuring of outer data...

[Adam Jorgensen]

Thanks :-) Sorry about the size, I wasn't sure what was relevant...

On 24 August 2011 15:29, Peter Otten <__pete...@web.de> wrote:
> Adam Jorgensen wrote:
>
>> Hi all, I'm experiencing a weird issue with closuring of parameters
>> and some nested functions I have inside two functions that
>> return decorators. I think it's best illustrated with the actual code:
>
> You should have made an effort to reduce its size
>> # This decorator doesn't work. For some reason python refuses to
>> closure the *decode_args parameter into the scope of the nested
>> decorate and decorate_with_rest_wrapper functions
>> # Renaming *decode_args has no effect
>> def rest_wrapper(*decode_args, **deco_kwargs):
>>     def decorate(func):
>>         argspec = getfullargspec(func)
>>         decode_args = [argspec.args.index(decode_arg) for decode_arg
>> in decode_args]
>
> I didn't read the whole thing, but:
>
 def f(a):
> ...     def g():
> ...             a = a + 42
> ...             return a
> ...     return g
> ...
 f(1)()
> Traceback (most recent call last):
>  File "", line 1, in 
>  File "", line 3, in g
> UnboundLocalError: local variable 'a' referenced before assignment
>
> Python treats variables as local if you assign a value to them anywhere in
> the function. The fix is easy, just use another name:
>
 def f(a):
> ...     def g():
> ...             b = a + 42
> ...             return b
> ...     return g
> ...
 f(1)()
> 43
>
> In Python 3 you can also use the nonlocal statement.
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list