Re: Interpreter Bug

[Barry Scott]

> On 2 Jun 2021, at 10:34, Alice Braim  wrote:
> 
>   Good morning-
> 
> 
> 
>   I am having some very serious issues with Python, and I was hoping you
>   could help?
> 
>   I downloaded both Python and PyCharm, and the 2 do not seem to be working.
>   Every time I select Python as an interpreter, the whole thing crashes.
>   Obviously, I went onto repairs, but even then it told me there was a
>   `fatal error'. I tried uninstalling and restarting multiple times, and
>   nothing improved. I've tried redownloading loads of times, no improvement.
>   I have the latest version (3.9.5) and there definitely seems to be
>   something going on at your end.

Details please: Which OS which version of Python etc.

What does crashes mean exactly?

Was it PyCharm that crashes or python that crashes?

Barry


> 
>   Any ideas?
> 
> 
> 
>   ~Alexa
> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: bug in download

[Terry Reedy]

On 12/16/2020 12:15 PM, Erick Willum wrote:




Begin forwarded message:


From: Erick Willum 
Date: 16 December 2020 at 15:53:40 GMT
To: python-list@python.org
Subject: bug in download

Hallo and good afternoon,

Having installed python (big thank you) and sublime text, i get the next 
message when trying to download numpy or matplotlib in sublime text:

fails to pass a sanity check due to a bug in the windows runtime. See this 
issue for more information:
https://tinyurl.com/y3dm3h86I


The answer on stackoverflow.com, easily accessed by their local search 
or Google web search: use 1.9.3 on Windows 2004, 1.9.4 on *nix.  1.9.3 
did not work on *nix, and the 1.9.4 hotfix does not work on on some 
Windows.  I have not seen anything either way about the 2009 Windows update


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: [BUG] missing ')' causes syntax error on next line

[Greg Ewing]

On 23/07/20 12:23 pm, dn wrote:

You be sure to have a nice day, now!


How do you feel about you be sure to have a nice day, now?

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: [BUG] missing ')' causes syntax error on next line

[Jeff Linahan]

Interesting PEG thing.  C++ compilers are getting better at suggesting
fixes for minor syntax errors (in 2011 on MSVC I remember seeing pages of
errors for forgetting a semicolon after a struct.) but python seems to be
lagging behind in this regard.. will check out superhelp, maybe it'll help
me patiently wait for the day where the python compiler does machine
learning on my specific style of mistakes to suggest fixes lol.

On Thu, Jul 23, 2020, 2:09 PM Chris Angelico  wrote:

> On Fri, Jul 24, 2020 at 4:00 AM Dennis Lee Bieber 
> wrote:
> >
> > On Thu, 23 Jul 2020 09:51:42 +1000, Chris Angelico 
> > declaimed the following:
> >
> > >On Thu, Jul 23, 2020 at 9:17 AM dn via Python-list
> > > wrote:
> > >> However, questions remain:-
> > >>
> > >> Robot: any machine or mechanical device that operates automatically
> with
> > >> humanlike skill
> > >>
> > >
> > >What about a human that operates mechanically with merely robot-like
> > >skill? I'm pretty sure I've spoken to them on the phone many times.
> >
> > Would that be... a computer?
> >
> >
> >
> > Using the original meaning of "one who computes"
> >
>
> I suspect that even that is too generous for some of them. They merely
> follow instructions like good little automatons, until the caller
> gives up...
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-l
> 
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: [BUG] missing ')' causes syntax error on next line

[Chris Angelico]

On Fri, Jul 24, 2020 at 4:00 AM Dennis Lee Bieber  wrote:
>
> On Thu, 23 Jul 2020 09:51:42 +1000, Chris Angelico 
> declaimed the following:
>
> >On Thu, Jul 23, 2020 at 9:17 AM dn via Python-list
> > wrote:
> >> However, questions remain:-
> >>
> >> Robot: any machine or mechanical device that operates automatically with
> >> humanlike skill
> >>
> >
> >What about a human that operates mechanically with merely robot-like
> >skill? I'm pretty sure I've spoken to them on the phone many times.
>
> Would that be... a computer?
>
>
>
> Using the original meaning of "one who computes"
>

I suspect that even that is too generous for some of them. They merely
follow instructions like good little automatons, until the caller
gives up...

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: [BUG] missing ')' causes syntax error on next line

[Chris Angelico]

On Thu, Jul 23, 2020 at 10:30 AM Gene Heskett  wrote:
>
> On Wednesday 22 July 2020 19:51:42 Chris Angelico wrote:
>
> > On Thu, Jul 23, 2020 at 9:17 AM dn via Python-list
> >
> >  wrote:
> > > However, questions remain:-
> > >
> > > Robot: any machine or mechanical device that operates automatically
> > > with humanlike skill
> >
> > What about a human that operates mechanically with merely robot-like
> > skill? I'm pretty sure I've spoken to them on the phone many times.
> >
> > ChrisA
>
> Thats more common than you might think Chris.  Whats nice is that you can
> often take over that robot and use his or her hands to troubleshoot an
> off the air tv transmitter from a thousand miles away.  That was my
> field of expertise for 60 years.
>

Ah yes. I was more thinking of the ones you get when you call for
support. "Have you rebooted your computer and router?" "THERE IS NO
ADSL SYNC. Rebooting my computer is not going to change that."

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: [BUG] missing ')' causes syntax error on next line

[Gene Heskett]

On Wednesday 22 July 2020 19:51:42 Chris Angelico wrote:

> On Thu, Jul 23, 2020 at 9:17 AM dn via Python-list
>
>  wrote:
> > However, questions remain:-
> >
> > Robot: any machine or mechanical device that operates automatically
> > with humanlike skill
>
> What about a human that operates mechanically with merely robot-like
> skill? I'm pretty sure I've spoken to them on the phone many times.
>
> ChrisA

Thats more common than you might think Chris.  Whats nice is that you can 
often take over that robot and use his or her hands to troubleshoot an 
off the air tv transmitter from a thousand miles away.  That was my 
field of expertise for 60 years.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: [BUG] missing ')' causes syntax error on next line

[dn via Python-list]

On 23/07/2020 11:51, Chris Angelico wrote:

On Thu, Jul 23, 2020 at 9:17 AM dn via Python-list
 wrote:

However, questions remain:-

Robot: any machine or mechanical device that operates automatically with
humanlike skill



What about a human that operates mechanically with merely robot-like
skill? I'm pretty sure I've spoken to them on the phone many times.

Can't say - I've never tried to speak with @Ethan over the phone.

...and what do you call the robot which can't/won't operate by phone, 
and takes a week to respond to your email - and then often by asking for 
information previously provided, eg a certain book-seller's "Mr Neil 
would you please confirm that your voucher number is 12345?"



However, on the Turing machine side, I recall creating an abbreviated 
version for a University Open Day - way back in the seventies and when 
VDU-terminals were a novelty and thus curiosities.


The program worked in similar manner to the famous "Eliza" 
(https://en.wikipedia.org/wiki/ELIZA) - attempting to give an illusion 
of understanding and conversation. It stored a mere top-twenty phrases. 
When the user entered some text, it would randomly select one of these 
and 'reply'. The user was invited to give feedback, ie whether it was a 
good reply or not. These were used to score/grade the phrases. Then the 
user's data-entry was added to the 'database of phrases' in place of the 
existing phrase with the lowest score. Thus the conversation continued.


'On the day', many of our visitors were convinced that either the 
computer was capable of holding a conversation (à la Turing), or that 
there was a person hidden-away somewhere, providing the 'answers'.


Most telling, reviewing the 'database' at the end, showed twenty of the 
most innocuous phrases in English conversation - those which were so 
multi-purpose that they could be used in almost any situation (phatic 
phrases: https://en.wikipedia.org/wiki/Phatic_expression).


You be sure to have a nice day, now!
--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: [BUG] missing ')' causes syntax error on next line

[Chris Angelico]

On Thu, Jul 23, 2020 at 9:17 AM dn via Python-list
 wrote:
> However, questions remain:-
>
> Robot: any machine or mechanical device that operates automatically with
> humanlike skill
>

What about a human that operates mechanically with merely robot-like
skill? I'm pretty sure I've spoken to them on the phone many times.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: [BUG] missing ')' causes syntax error on next line

[dn via Python-list]

On 23/07/2020 10:46, Ethan Furman wrote:

On 7/22/20 2:57 PM, Jeff Linahan wrote:


Subscribing to the mailing list as per the bot's request and resending.


*beep* *whir*

WE ARE NOT

*click* *whi*

A BOT.

*bzzzt*

WE ARE

*bzzzt* *click*

ADVANCED LIFE

*whi*

FORMS

*click*

*beep*

.

--
~eTHAN~
pYTHON lIST mODERATOR
NoT a BoT  (i ThInK...)



Enjoyed the joke!
(and thanks for all the efforts you exert on our behalf)


However, questions remain:-

Robot: any machine or mechanical device that operates automatically with 
humanlike skill


See also, Turing Machine 
(https://www.cl.cam.ac.uk/projects/raspberrypi/tutorials/turing-machine/one.html)

--
Regards =dn
--
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: [BUG] missing ')' causes syntax error on next line

[Kyle Stanley]

>
> *beep* *whir*
>
> WE ARE NOT
>
> *click* *whi*
>
> A BOT.


*dial up noises*

SPEAK FOR YOURSELF


On Wed, Jul 22, 2020 at 6:46 PM Ethan Furman  wrote:

> On 7/22/20 2:57 PM, Jeff Linahan wrote:
>
> > Subscribing to the mailing list as per the bot's request and resending.
>
> *beep* *whir*
>
> WE ARE NOT
>
> *click* *whi*
>
> A BOT.
>
> *bzzzt*
>
> WE ARE
>
> *bzzzt* *click*
>
> ADVANCED LIFE
>
> *whi*
>
> FORMS
>
> *click*
>
> *beep*
>
> .
>
> --
> ~eTHAN~
> pYTHON lIST mODERATOR
> NoT a BoT  (i ThInK...)
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: [BUG] missing ')' causes syntax error on next line

[Joe Pfeiffer]

Jeff Linahan  writes:
>
> See attached image.  Would be nice if it printed "SyntaxError: unbalanced
> parens" as it can difficult to see the problem if code like this is run in
> an environment that only prints the problematic line, which in this case
> the compiler is confused and one line off.

It would be indeed be nice, but it's reporting the error when it finds
it.  Suppose you were typing the program in at the console -- how could
it possibly go back and report the error on the previous line?  Given
python's syntax, does the error even really exist yet on that
previous line?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fwd: [BUG] missing ')' causes syntax error on next line

[Ethan Furman]

On 7/22/20 2:57 PM, Jeff Linahan wrote:


Subscribing to the mailing list as per the bot's request and resending.


*beep* *whir*

WE ARE NOT

*click* *whi*

A BOT.

*bzzzt*

WE ARE

*bzzzt* *click*

ADVANCED LIFE

*whi*

FORMS

*click*

*beep*

.

--
~eTHAN~
pYTHON lIST mODERATOR
NoT a BoT  (i ThInK...)
--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Barry Scott]

> On 25 Mar 2020, at 06:12, Frank Millman  wrote:
> 
> On 2020-03-24 8:39 PM, Barry Scott wrote:
>>> On 24 Mar 2020, at 11:54, Frank Millman  wrote:
>>> 
>>> 
>>> I decided to concentrate on using Wireshark to detect the difference 
>>> between a Python3.7 session and a Python3.8 session. Already I can see some 
>>> differences.
>>> 
>>> There is only one version of my program. I am simply running it with either 
>>> 'py -3.7 ' or 'py -3.8'. And I am ignoring Chrome at this stage, as it is 
>>> only that Edge shows the problem.
>>> 
>>> First point - Python3.7 also shows a lot of [RST, ACK] lines. My guess is 
>>> that this is caused by my 'protocol violation' of sending a 'Keep-Alive' 
>>> header and then closing the connection. Python3.7 does not suffer from 
>>> dropping files, so I now think this is a sidetrack. I will fix my program 
>>> when this is all over, but for now I don't want to touch it.
>> Yes your protocol violation is why you see [RST, ACK].
>> I'm confused you know that the code has a critical bug in it and you have 
>> not fixed it?
>> Just send "Connection: close" and I'd assume all will work.
> 
> Well, the reason is simply that I wanted to understand why my code that 
> worked all the way from 3.4 through 3.7 stopped working in 3.8. I realise 
> that my code is faulty, but I still wanted to know what the trigger was that 
> caused the bug to appear.

Got it, I'd not picked up on you wishes to find the details of why.

> 
> From my testing with Wireshark, I can see that both Edge and Chrome create 20 
> connections to GET 20 files. The difference seems to be that Chrome does not 
> attempt to re-use a connection, even though both client and server have sent 
> Keep-Alive headers. Edge does attempt to re-use the connection.

Chrome will reuse the connections if you load enough files from the same server.
All browser do this to get a page displayed as fast as possible.
Because HTTP is half-duplex its the round trip time rather then bandwidth that 
controls
the time taken get all the asserts needed to render a page most of the time.

> 
> The difference between 3.7 and 3.8 is that 3.7 sends the data in separate 
> packets for the status, each header, and then each chunk, whereas 3.8 sends 
> the whole lot in a single packet.

That will change the timing enough to expose the problem.

I often find with network code bugs that its changes that makes the
timing different that expos bugs in supposed tested and working code.

In this case you could track down what was the cause, but it's often the case 
that it's not practical to do that.
You may not know what changed between when the code worked and when it broke in 
many cases.

> My guess is that 3.7 is slower to send the files, so Edge starts up all 20 
> connections before it has finished receiving the first one, whereas with 3.8, 
> by the time it has opened a few connections the first file has been received, 
> so it tries to re-use the same connection to receive the next one. By then I 
> have closed the connection. If I am right, it is surprising that my program 
> worked *some* of the time.

I have lost count of the number of times when we have found a bug in some code 
that we say "how did it ever work?".

> 
> The same reasoning would explain why it worked when connecting from a remote 
> host. There would be enough delay to force it into the same behaviour as 3.7.

Yep and if there are any middle boxes, routers, man-in-middle-proxies, etc the 
timing/packets get pushed around.

> 
> It has been an interesting ride, and I have learned a lot. I will now look 
> into fixing my program. The easy fix is to just send 'Connection: Close', but 
> I will do it properly and implement 'Keep-Alive'.

Barry

> 
> Thanks all
> 
> Frank
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Chris Angelico]

On Wed, Mar 25, 2020 at 5:14 PM Frank Millman  wrote:
> My guess is that 3.7 is slower to send the files, so Edge starts up all
> 20 connections before it has finished receiving the first one, whereas
> with 3.8, by the time it has opened a few connections the first file has
> been received, so it tries to re-use the same connection to receive the
> next one. By then I have closed the connection. If I am right, it is
> surprising that my program worked *some* of the time.

That is definitely plausible. It would also mean that, depending on
exact browser settings, you might see the same problem crop up in
other situations. Definitely worth the time to investigate this one,
since you now know the ACTUAL reason and can solve the ACTUAL bug.

> It has been an interesting ride, and I have learned a lot. I will now
> look into fixing my program. The easy fix is to just send 'Connection:
> Close', but I will do it properly and implement 'Keep-Alive'.

Yep, and honestly, popping a quick "Connection: Close" isn't a
terrible solution. I have one app where I had a similar weird problem
with keep-alive, and made the opposite decision to you: rather than
sink in the hours to figure out the actual cause, I just slapped in a
"Connection: Close" and moved on. Part of the problem is that my issue
only shows up on HTTPS connections and not on plain HTTP ones, which
makes it extremely fiddly to wireshark (since all the traffic I care
about is encrypted, unless through some miracle the bug is visible
when it has no symptoms). But hey, that's what TODO files are for

Thank you for including us on your debugging journey. To anyone else
out there with a bizarre problem, I recommend reading over this thread
to get an idea of how to get help - Frank demonstrated a willingness
to sink his own time into this, a comprehensive writeup of all useful
symptoms, and a truly interesting problem report.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-24 8:39 PM, Barry Scott wrote:




On 24 Mar 2020, at 11:54, Frank Millman  wrote:


I decided to concentrate on using Wireshark to detect the difference between a 
Python3.7 session and a Python3.8 session. Already I can see some differences.

There is only one version of my program. I am simply running it with either 'py 
-3.7 ' or 'py -3.8'. And I am ignoring Chrome at this stage, as it is only that 
Edge shows the problem.

First point - Python3.7 also shows a lot of [RST, ACK] lines. My guess is that 
this is caused by my 'protocol violation' of sending a 'Keep-Alive' header and 
then closing the connection. Python3.7 does not suffer from dropping files, so 
I now think this is a sidetrack. I will fix my program when this is all over, 
but for now I don't want to touch it.


Yes your protocol violation is why you see [RST, ACK].

I'm confused you know that the code has a critical bug in it and you have not 
fixed it?
Just send "Connection: close" and I'd assume all will work.



Well, the reason is simply that I wanted to understand why my code that 
worked all the way from 3.4 through 3.7 stopped working in 3.8. I 
realise that my code is faulty, but I still wanted to know what the 
trigger was that caused the bug to appear.


From my testing with Wireshark, I can see that both Edge and Chrome 
create 20 connections to GET 20 files. The difference seems to be that 
Chrome does not attempt to re-use a connection, even though both client 
and server have sent Keep-Alive headers. Edge does attempt to re-use the 
connection.


The difference between 3.7 and 3.8 is that 3.7 sends the data in 
separate packets for the status, each header, and then each chunk, 
whereas 3.8 sends the whole lot in a single packet.


My guess is that 3.7 is slower to send the files, so Edge starts up all 
20 connections before it has finished receiving the first one, whereas 
with 3.8, by the time it has opened a few connections the first file has 
been received, so it tries to re-use the same connection to receive the 
next one. By then I have closed the connection. If I am right, it is 
surprising that my program worked *some* of the time.


The same reasoning would explain why it worked when connecting from a 
remote host. There would be enough delay to force it into the same 
behaviour as 3.7.


It has been an interesting ride, and I have learned a lot. I will now 
look into fixing my program. The easy fix is to just send 'Connection: 
Close', but I will do it properly and implement 'Keep-Alive'.


Thanks all

Frank
--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Barry Scott]

> On 24 Mar 2020, at 11:54, Frank Millman  wrote:
> 
> On 2020-03-23 1:56 PM, Frank Millman wrote:
>> On 2020-03-23 12:57 PM, Chris Angelico wrote:
>>> On Mon, Mar 23, 2020 at 8:03 PM Frank Millman  wrote:
 
 On 2020-03-22 12:11 PM, Chris Angelico wrote:
> On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:
>> 
>> On 2020-03-22 10:45 AM, Chris Angelico wrote:
> 
> If you can recreate the problem with a single socket and multiple
> requests, that would be extremely helpful. I also think it's highly
> likely that this is the case.
> 
 
 I am working on a stripped-down version, but I realise there are a few
 things I have not grasped.
 
 Hope you don't mind, but can you scan through what follows and tell me
 if I am on the right lines?
>>> 
>>> No probs!
>>> 
>> [...]
>> Really appreciate the one-on-one tuition. I am learning a lot!
>>> 
 If this all makes sense, I should write two versions of the client
 program, one using a single connection, and one using a pool of 
 connections.
 
>>> 
>>> Possibly! I think you'll most likely see that one of those behaves
>>> perfectly normally, and you only trigger the issue in the other. So
>>> you could move forward with just one test program.
>>> 
>> Well, I have got the first one working - single connection - and so far it 
>> has not gone wrong.
>> However, it is difficult to be sure that I am comparing apples with apples. 
>> I have written my test server to handle 'Keep-Alive' correctly, but as I 
>> mentioned earlier, my live program closes the connection after each 
>> transfer. So now I have to make my test server do the same, and change my 
>> test client to react to that and re-open the connection each time. I will 
>> make the changes and see how that behaves.
>> Of course now I am in the murky waters of trying to second-guess how Edge 
>> reacts to that. Presumably that is where Wireshark will be useful. I will 
>> keep you posted.
> 
> Here is a progress report.
> 
> I decided to concentrate on using Wireshark to detect the difference between 
> a Python3.7 session and a Python3.8 session. Already I can see some 
> differences.
> 
> There is only one version of my program. I am simply running it with either 
> 'py -3.7 ' or 'py -3.8'. And I am ignoring Chrome at this stage, as it is 
> only that Edge shows the problem.
> 
> First point - Python3.7 also shows a lot of [RST, ACK] lines. My guess is 
> that this is caused by my 'protocol violation' of sending a 'Keep-Alive' 
> header and then closing the connection. Python3.7 does not suffer from 
> dropping files, so I now think this is a sidetrack. I will fix my program 
> when this is all over, but for now I don't want to touch it.

Yes your protocol violation is why you see [RST, ACK].

I'm confused you know that the code has a critical bug in it and you have not 
fixed it?
Just send "Connection: close" and I'd assume all will work.

If my last message was not clear let me know what is unclear.

It not interesting that you get away with it in 3.7 and not 3.8 because the 
code is wrong.

> When I send the response to the initial connection, I write a status line, 
> then multiple header lines, then an html file. I then close the connection. 
> Python3.7 sends a packet with the status, then a separate packet for each 
> header, then a packet with the file. Python3.8 sends a packet with the 
> status, then merges everything else into a single packet and sends it in one 
> go. I just mention this as an indication that quite a lot has changed between 
> my two versions of Python.
> 
> I have one frustration with Wireshark. I will mention it in case anyone has a 
> solution.
> 
> I can see that Edge opens multiple connections. I am trying to track the 
> activity on each connection separately. I can export the data to csv, which 
> makes it easier to work on. But while the TCP lines include the source and 
> destination ports, the HTTP lines do not, so I don't know which connection 
> they belong to. If I view the data in Wireshark's gui it does show the ports, 
> so the data is there somewhere. Does anyone know how to include it in the csv 
> output?

There is a option to follow a single TCP or HTTP connection in Wireshark.
Right click on one pf the packets then choose Follow/HTTP stream.

The way to share the data is as a PCAP file that allows someone else to look at 
the capture
with tools like wireshark and others.

> 
> That's all for now. I will keep you posted.

Barry

> 
> Frank
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-24 1:54 PM, Frank Millman wrote:

On 2020-03-23 1:56 PM, Frank Millman wrote:

I have one frustration with Wireshark. I will mention it in case anyone 
has a solution.


I can see that Edge opens multiple connections. I am trying to track the 
activity on each connection separately. I can export the data to csv, 
which makes it easier to work on. But while the TCP lines include the 
source and destination ports, the HTTP lines do not, so I don't know 
which connection they belong to. If I view the data in Wireshark's gui 
it does show the ports, so the data is there somewhere. Does anyone know 
how to include it in the csv output?




I solved my Wireshark problem by exporting the data as text. A much more 
wordy format, and fiddly to parse, but the info I am looking for is there.


I am getting some interesting results now, such as a second GET 
interrupting an existing session. However, my brain is now frazzled, so 
I will continue tomorrow.


Frank
--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-23 1:56 PM, Frank Millman wrote:

On 2020-03-23 12:57 PM, Chris Angelico wrote:

On Mon, Mar 23, 2020 at 8:03 PM Frank Millman  wrote:


On 2020-03-22 12:11 PM, Chris Angelico wrote:
On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  
wrote:


On 2020-03-22 10:45 AM, Chris Angelico wrote:


If you can recreate the problem with a single socket and multiple
requests, that would be extremely helpful. I also think it's highly
likely that this is the case.



I am working on a stripped-down version, but I realise there are a few
things I have not grasped.

Hope you don't mind, but can you scan through what follows and tell me
if I am on the right lines?


No probs!



[...]

Really appreciate the one-on-one tuition. I am learning a lot!




If this all makes sense, I should write two versions of the client
program, one using a single connection, and one using a pool of 
connections.




Possibly! I think you'll most likely see that one of those behaves
perfectly normally, and you only trigger the issue in the other. So
you could move forward with just one test program.



Well, I have got the first one working - single connection - and so far 
it has not gone wrong.


However, it is difficult to be sure that I am comparing apples with 
apples. I have written my test server to handle 'Keep-Alive' correctly, 
but as I mentioned earlier, my live program closes the connection after 
each transfer. So now I have to make my test server do the same, and 
change my test client to react to that and re-open the connection each 
time. I will make the changes and see how that behaves.


Of course now I am in the murky waters of trying to second-guess how 
Edge reacts to that. Presumably that is where Wireshark will be useful. 
I will keep you posted.


Here is a progress report.

I decided to concentrate on using Wireshark to detect the difference 
between a Python3.7 session and a Python3.8 session. Already I can see 
some differences.


There is only one version of my program. I am simply running it with 
either 'py -3.7 ' or 'py -3.8'. And I am ignoring Chrome at this stage, 
as it is only that Edge shows the problem.


First point - Python3.7 also shows a lot of [RST, ACK] lines. My guess 
is that this is caused by my 'protocol violation' of sending a 
'Keep-Alive' header and then closing the connection. Python3.7 does not 
suffer from dropping files, so I now think this is a sidetrack. I will 
fix my program when this is all over, but for now I don't want to touch it.


When I send the response to the initial connection, I write a status 
line, then multiple header lines, then an html file. I then close the 
connection. Python3.7 sends a packet with the status, then a separate 
packet for each header, then a packet with the file. Python3.8 sends a 
packet with the status, then merges everything else into a single packet 
and sends it in one go. I just mention this as an indication that quite 
a lot has changed between my two versions of Python.


I have one frustration with Wireshark. I will mention it in case anyone 
has a solution.


I can see that Edge opens multiple connections. I am trying to track the 
activity on each connection separately. I can export the data to csv, 
which makes it easier to work on. But while the TCP lines include the 
source and destination ports, the HTTP lines do not, so I don't know 
which connection they belong to. If I view the data in Wireshark's gui 
it does show the ports, so the data is there somewhere. Does anyone know 
how to include it in the csv output?


That's all for now. I will keep you posted.

Frank

--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Chris Angelico]

On Mon, Mar 23, 2020 at 10:58 PM Frank Millman  wrote:
>
> On 2020-03-23 12:57 PM, Chris Angelico wrote:
> > On Mon, Mar 23, 2020 at 8:03 PM Frank Millman  wrote:
> >>
> >> On 2020-03-22 12:11 PM, Chris Angelico wrote:
> >>> On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:
> 
>  On 2020-03-22 10:45 AM, Chris Angelico wrote:
> >>>
> >>> If you can recreate the problem with a single socket and multiple
> >>> requests, that would be extremely helpful. I also think it's highly
> >>> likely that this is the case.
> >>>
> >>
> >> I am working on a stripped-down version, but I realise there are a few
> >> things I have not grasped.
> >>
> >> Hope you don't mind, but can you scan through what follows and tell me
> >> if I am on the right lines?
> >
> > No probs!
> >
>
> [...]
>
> Really appreciate the one-on-one tuition. I am learning a lot!

Heh, I'm always happy to help people who are willing to learn. Also,
you've brought an interesting problem to the table. :)

> Of course now I am in the murky waters of trying to second-guess how
> Edge reacts to that. Presumably that is where Wireshark will be useful.
> I will keep you posted.
>

Yep, it's going to be a matter of monitoring the exact headers sent
each direction.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Barry Scott]

> On 23 Mar 2020, at 09:02, Frank Millman  wrote:
> 
> On 2020-03-22 12:11 PM, Chris Angelico wrote:
>> On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:
>>> 
>>> On 2020-03-22 10:45 AM, Chris Angelico wrote:
>> If you can recreate the problem with a single socket and multiple
>> requests, that would be extremely helpful. I also think it's highly
>> likely that this is the case.
> 
> I am working on a stripped-down version, but I realise there are a few things 
> I have not grasped.
> 
> Hope you don't mind, but can you scan through what follows and tell me if I 
> am on the right lines?
> 
> Both the client and the server can send a header with 'Keep-alive', but what 
> does it actually mean?
> 
> If the client sends it, does that mean that it wants the server to keep the 
> connection open, and only close it when the client closes it from the other 
> end?
> 
> Conversely, if the server sends it, does it mean that it wants the client to 
> keep the connection open? If so, under what condition would the server close 
> the connection (other than a timeout). Should the server only send 
> 'Keep-alive' if it receives 'Keep-alive'?

See https://tools.ietf.org/html/rfc2616  
secton 14.10 for the details.

When the browser sends "Connection: keep-alive" it mean that the browser wishes 
to have
a persistent connection. The server is free to choose.

If the server replies with "Connection: close" the connection is not persistent 
and will
be closed after the response.

If the server replies with "Connection: keep-aiive" then connect is persistent 
that means that
after this response the server will allow another transaction, GET, POST etc.

Note: You need to know if the client is using HTTP/1.0 or HTTP/1.1 protocol the 
rules are different in
each version.

> 
> In my program, when I send a file in response to a GET, I send a header with 
> 'Keep-alive', then I send the file, then I close the connection. This now 
> seems wrong. It could even be the cause of my bug, but then why has it only 
> appeared now? Both Edge and Chrome send 'Keep-alive' headers.

Its a protocol violation. In your case you must send "Connection: close". Fix 
that and I think your code is likely to work.

If you care about performance add the extra code to handle persistent 
connections. Its speeds up the browsers
and should reduce the impact on the server as well.


> If I am thinking along the right lines, then the exchange should go like this 
> -
> 
> Client sends request, with 'Keep-alive' header.
> 
> Server sends response, but does not close the connection. The server should 
> reply with a 'Keep-alive' header. If it does not, the client will close the 
> connection, in which case the server will also close.

You must include a "Connection:" head for HTTP/1.1 I believe.

> 
> Assuming that they both send 'Keep-alive', the onus is on the client to close 
> the connection when it has no more requests.

Yes.

> The server should have a timeout in case the client goes away.

Yes to stop resources running out.

> 
> Assuming that the above is correct, the client will rely on 'Content-length' 
> to determine when it has received the entire request. If the client has more 
> than one request, it will send the first, wait for the response, when fully 
> received as per the 'Content-length' it will send the next one, until all 
> requests have been sent and all responses received, at which point it will 
> close the connection.

You can use "Content-Length" header or chunked encoding.

> 
> All this assumes only one connection. Alternatively the client could open 
> multiple connections for the requests. In that case it would make sense to 
> use 'Connection: Close', so that the server can close the connection straight 
> away, making it available for reuse.

Modern browsers open many persistent connection. I recall its 8/site in the 
case of chrome.


> 
> If this all makes sense, I should write two versions of the client program, 
> one using a single connection, and one using a pool of connections.

Is this for a test program?

You can use curl for the single connection case.


> 
> All comments appreciated!
> 
> Frank

Barry

> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-23 12:57 PM, Chris Angelico wrote:

On Mon, Mar 23, 2020 at 8:03 PM Frank Millman  wrote:


On 2020-03-22 12:11 PM, Chris Angelico wrote:

On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:


On 2020-03-22 10:45 AM, Chris Angelico wrote:


If you can recreate the problem with a single socket and multiple
requests, that would be extremely helpful. I also think it's highly
likely that this is the case.



I am working on a stripped-down version, but I realise there are a few
things I have not grasped.

Hope you don't mind, but can you scan through what follows and tell me
if I am on the right lines?


No probs!



[...]

Really appreciate the one-on-one tuition. I am learning a lot!




If this all makes sense, I should write two versions of the client
program, one using a single connection, and one using a pool of connections.



Possibly! I think you'll most likely see that one of those behaves
perfectly normally, and you only trigger the issue in the other. So
you could move forward with just one test program.



Well, I have got the first one working - single connection - and so far 
it has not gone wrong.


However, it is difficult to be sure that I am comparing apples with 
apples. I have written my test server to handle 'Keep-Alive' correctly, 
but as I mentioned earlier, my live program closes the connection after 
each transfer. So now I have to make my test server do the same, and 
change my test client to react to that and re-open the connection each 
time. I will make the changes and see how that behaves.


Of course now I am in the murky waters of trying to second-guess how 
Edge reacts to that. Presumably that is where Wireshark will be useful. 
I will keep you posted.


Frank

--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Chris Angelico]

On Mon, Mar 23, 2020 at 8:03 PM Frank Millman  wrote:
>
> On 2020-03-22 12:11 PM, Chris Angelico wrote:
> > On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:
> >>
> >> On 2020-03-22 10:45 AM, Chris Angelico wrote:
> >
> > If you can recreate the problem with a single socket and multiple
> > requests, that would be extremely helpful. I also think it's highly
> > likely that this is the case.
> >
>
> I am working on a stripped-down version, but I realise there are a few
> things I have not grasped.
>
> Hope you don't mind, but can you scan through what follows and tell me
> if I am on the right lines?

No probs!

> Both the client and the server can send a header with 'Keep-alive', but
> what does it actually mean?
>
> If the client sends it, does that mean that it wants the server to keep
> the connection open, and only close it when the client closes it from
> the other end?
>
> Conversely, if the server sends it, does it mean that it wants the
> client to keep the connection open? If so, under what condition would
> the server close the connection (other than a timeout). Should the
> server only send 'Keep-alive' if it receives 'Keep-alive'?

I'm pretty sure the server should never say "keep-alive" if the client
said "close". (There's a bit of an oddity in that the default changed
between, I think, HTTP 1.0 and 1.1. But that's a minor discrepancy.)
Generally, you'll see one of these options:

1) Client says "Connection: close". This means "I'm only gonna be
doing this one request, then we're done". Good for non-browser usage,
automated scripts etc.

2) Client says "Keep-Alive", server says "Close". This means the
client would be okay with connection reuse, but the server's saying no
(maybe it doesn't support it, or maybe it's in a soft shutdown state
where it wants to end connections, or whatever). After the current
request, the connection will be closed.

3) Both ends say "Connection: Keep-Alive". After the request, the
socket isn't closed, and can be reused for some other request.

I'm not sure how a client would (or should) handle it if the server
says "Connection: Keep-Alive" after the client said "Connection:
Close". Probably the client would just close its end of the socket and
that'd be that.

> In my program, when I send a file in response to a GET, I send a header
> with 'Keep-alive', then I send the file, then I close the connection.
> This now seems wrong. It could even be the cause of my bug, but then why
> has it only appeared now? Both Edge and Chrome send 'Keep-alive' headers.
>

It mightn't be fundamentally WRONG (there needs to be *some* way to
signal that you're now done), but it would probably be suboptimal. If
you're always going to close the connection after one request, just
send Connection: Close.

> If I am thinking along the right lines, then the exchange should go like
> this -
>
> Client sends request, with 'Keep-alive' header.
>
> Server sends response, but does not close the connection. The server
> should reply with a 'Keep-alive' header. If it does not, the client will
> close the connection, in which case the server will also close.
>
> Assuming that they both send 'Keep-alive', the onus is on the client to
> close the connection when it has no more requests. The server should
> have a timeout in case the client goes away.

I think either end is allowed to close the connection.

> Assuming that the above is correct, the client will rely on
> 'Content-length' to determine when it has received the entire request.
> If the client has more than one request, it will send the first, wait
> for the response, when fully received as per the 'Content-length' it
> will send the next one, until all requests have been sent and all
> responses received, at which point it will close the connection.

Yep. Which makes things very interesting (for serene values of
"interesting", such as "we're all gonna die") when you want to use
connection pooling with something that fundamentally doesn't have a
length, such as a chunked transfer. Which you are doing. That may or
may not actually be significant.

> All this assumes only one connection. Alternatively the client could
> open multiple connections for the requests. In that case it would make
> sense to use 'Connection: Close', so that the server can close the
> connection straight away, making it available for reuse.

What you'll often see is that the client has some sort of connection
limit, and reuses those same connections. For instance, it might have
a maximum of four connections per server (where "server" has to
include the name as well as its IP and port, due to TLS SNI), so it
requests an HTML page on one connection (with keep-alive), parses
that, and then fires off one request on the reused connection and
three others on new connections. As those requests complete, it sends
new requests on the same sockets.

> If this all makes sense, I should write two versions of the client
> program, one using a single connection, and one using 

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-22 12:11 PM, Chris Angelico wrote:

On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:


On 2020-03-22 10:45 AM, Chris Angelico wrote:


If you can recreate the problem with a single socket and multiple
requests, that would be extremely helpful. I also think it's highly
likely that this is the case.



I am working on a stripped-down version, but I realise there are a few 
things I have not grasped.


Hope you don't mind, but can you scan through what follows and tell me 
if I am on the right lines?


Both the client and the server can send a header with 'Keep-alive', but 
what does it actually mean?


If the client sends it, does that mean that it wants the server to keep 
the connection open, and only close it when the client closes it from 
the other end?


Conversely, if the server sends it, does it mean that it wants the 
client to keep the connection open? If so, under what condition would 
the server close the connection (other than a timeout). Should the 
server only send 'Keep-alive' if it receives 'Keep-alive'?


In my program, when I send a file in response to a GET, I send a header 
with 'Keep-alive', then I send the file, then I close the connection. 
This now seems wrong. It could even be the cause of my bug, but then why 
has it only appeared now? Both Edge and Chrome send 'Keep-alive' headers.


If I am thinking along the right lines, then the exchange should go like 
this -


Client sends request, with 'Keep-alive' header.

Server sends response, but does not close the connection. The server 
should reply with a 'Keep-alive' header. If it does not, the client will 
close the connection, in which case the server will also close.


Assuming that they both send 'Keep-alive', the onus is on the client to 
close the connection when it has no more requests. The server should 
have a timeout in case the client goes away.


Assuming that the above is correct, the client will rely on 
'Content-length' to determine when it has received the entire request. 
If the client has more than one request, it will send the first, wait 
for the response, when fully received as per the 'Content-length' it 
will send the next one, until all requests have been sent and all 
responses received, at which point it will close the connection.


All this assumes only one connection. Alternatively the client could 
open multiple connections for the requests. In that case it would make 
sense to use 'Connection: Close', so that the server can close the 
connection straight away, making it available for reuse.


If this all makes sense, I should write two versions of the client 
program, one using a single connection, and one using a pool of connections.


All comments appreciated!

Frank

--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Barry Scott]

> On 22 Mar 2020, at 09:41, Frank Millman  wrote:
> 
> On 2020-03-22 11:00 AM, Barry Scott wrote:
>>> On 22 Mar 2020, at 07:56, Frank Millman  wrote:
>>> 
>>> On 2020-03-21 8:04 PM, Barry Scott wrote:
 I'd look at the network traffic with wireshark to see if there is anything 
 different between edge and the other browsers.
>>> 
>>> You are leading me into deep waters here :-)  I have never used Wireshark 
>>> before. I have now downloaded it and am running it - it generates a *lot* 
>>> of data, most of which I do not understand yet!
>> You can tell wireshark to only capture on one interface and to only capture 
>> packets for port 80.
>> (Captureing HTTPS means you cannot decode the packets without going deeper I 
>> recall)
>> Then you can tell wireshark to decode the captured data for http to drop a 
>> lot of the lower level details.
> 
> Thanks. I am more or less doing that. Interestingly the [RST,ACK] messages 
> appear on the tcp packets, so if I filter on http I do not see them.
> 
>>> 
>>> One thing immediately stands out. When I run it with MS Edge and Python3.8, 
>>> it shows a lot of lines highlighted in red, with the symbols [RST,ACK]. 
>>> They do not appear when running Chrome, and they do not appear when running 
>>> Python3.7.
>> As Chris said that should not happen.
> 
> As I replied to Chris, they appear in packets sent *from* Python *to* Edge.
> 
>>> 
>>> I have another data point. I tried putting an asyncio.sleep() after sending 
>>> each file. A value of 0.01 made no difference, but a value of 0.1 makes the 
>>> problem go away.
>> What is the async wait to wait for the transmit buffers to drain?
> 
> Not sure what you are asking. I am just doing what it says in the docs -
> 
> =
> 
> write(data)
> The method attempts to write the data to the underlying socket immediately. 
> If that fails, the data is queued in an internal write buffer until it can be 
> sent.
> 
> The method should be used along with the drain() method:
> 
> stream.write(data)
> await stream.drain()
> 
> =
> 
> coroutine drain()
> Wait until it is appropriate to resume writing to the stream. Example:
> 
> writer.write(data)
> await writer.drain()
> This is a flow control method that interacts with the underlying IO write 
> buffer. When the size of the buffer reaches the high watermark, drain() 
> blocks until the size of the buffer is drained down to the low watermark and 
> writing can be resumed. When there is nothing to wait for, the drain() 
> returns immediately.
> 
> =

That you called drain() is what I was asking. I'm a Twisted user and have not 
used asyncio yet.

Barry

> 
> Frank
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Barry Scott]

> On 22 Mar 2020, at 11:59, Frank Millman  wrote:
> 
> On 2020-03-22 1:01 PM, Chris Angelico wrote:
>> On Sun, Mar 22, 2020 at 12:45 AM Frank Millman  wrote:
>>> 
>>> Hi all
>>> 
>>> I have a strange intermittent bug.
>>> 
>>> The role-players -
>>>  asyncio on Python 3.8 running on Windows 10
>>>  Microsoft Edge running as a browser on the same machine
>>> 
>>> The bug does not occur with Python 3.7.
>>> It does not occur with Chrome or Firefox.
>>> It does not occur when MS Edge connects to another host on the network,
>>> running the same Python program (Python 3.8 on Fedora 31).
>> What exact version of Python 3.7 did you test? I'm looking through the
>> changes to asyncio and came across this one, which may have some
>> impact.
> 
> Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 
> bit (AMD64)] on win32

Can you confirm that you have implemented Connection: keep-alive?
This means that the browser can send a 2nd GET on the same connection.

Barry


> 
> Frank
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-22 1:01 PM, Chris Angelico wrote:

On Sun, Mar 22, 2020 at 12:45 AM Frank Millman  wrote:


Hi all

I have a strange intermittent bug.

The role-players -
  asyncio on Python 3.8 running on Windows 10
  Microsoft Edge running as a browser on the same machine

The bug does not occur with Python 3.7.
It does not occur with Chrome or Firefox.
It does not occur when MS Edge connects to another host on the network,
running the same Python program (Python 3.8 on Fedora 31).


What exact version of Python 3.7 did you test? I'm looking through the
changes to asyncio and came across this one, which may have some
impact.


Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 
64 bit (AMD64)] on win32


Frank

--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Kouli]

The RST from Python is probably caused here by HTTP 1.1 server closing TCP
connection without signalling "Connection: Close" in response headers: a
fast HTTP client will send another HTTP request before its TCP stack
detects the connection is being closed - and packets containing this new
requests will be replied with RST.

When you delay your response (as you mentioned), the Edge browser probably
opens more HTTP connections and will not send more HTTP requests in a
single connection as described above.

You should search for the cause Python closes the TCP connection (instead
of waiting for another HTTP request).

Kouli

On Sun, Mar 22, 2020 at 12:04 PM Chris Angelico  wrote:

> On Sun, Mar 22, 2020 at 12:45 AM Frank Millman  wrote:
> >
> > Hi all
> >
> > I have a strange intermittent bug.
> >
> > The role-players -
> >  asyncio on Python 3.8 running on Windows 10
> >  Microsoft Edge running as a browser on the same machine
> >
> > The bug does not occur with Python 3.7.
> > It does not occur with Chrome or Firefox.
> > It does not occur when MS Edge connects to another host on the network,
> > running the same Python program (Python 3.8 on Fedora 31).
>
> What exact version of Python 3.7 did you test? I'm looking through the
> changes to asyncio and came across this one, which may have some
> impact.
>
> https://bugs.python.org/issue36801
>
> Also this one made a change that introduced a regression that was
> subsequently fixed. Could be interesting.
>
> https://bugs.python.org/issue36802
>
> What happens if you try awaiting your writes? I think it probably
> won't make any difference, though - from my reading of the source, I
> believe that "await writer.write(...)" is the same as
> "writer.write(...); await writer.drain()", so it's going to be exactly
> the same as you're already doing.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Chris Angelico]

On Sun, Mar 22, 2020 at 12:45 AM Frank Millman  wrote:
>
> Hi all
>
> I have a strange intermittent bug.
>
> The role-players -
>  asyncio on Python 3.8 running on Windows 10
>  Microsoft Edge running as a browser on the same machine
>
> The bug does not occur with Python 3.7.
> It does not occur with Chrome or Firefox.
> It does not occur when MS Edge connects to another host on the network,
> running the same Python program (Python 3.8 on Fedora 31).

What exact version of Python 3.7 did you test? I'm looking through the
changes to asyncio and came across this one, which may have some
impact.

https://bugs.python.org/issue36801

Also this one made a change that introduced a regression that was
subsequently fixed. Could be interesting.

https://bugs.python.org/issue36802

What happens if you try awaiting your writes? I think it probably
won't make any difference, though - from my reading of the source, I
believe that "await writer.write(...)" is the same as
"writer.write(...); await writer.drain()", so it's going to be exactly
the same as you're already doing.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Chris Angelico]

On Sun, Mar 22, 2020 at 8:42 PM Frank Millman  wrote:
>
> On 2020-03-22 11:00 AM, Barry Scott wrote:
> >
> >
> >> On 22 Mar 2020, at 07:56, Frank Millman  wrote:
> >>
> >> On 2020-03-21 8:04 PM, Barry Scott wrote:
> >>> I'd look at the network traffic with wireshark to see if there is 
> >>> anything different between edge and the other browsers.
> >>
> >> You are leading me into deep waters here :-)  I have never used Wireshark 
> >> before. I have now downloaded it and am running it - it generates a *lot* 
> >> of data, most of which I do not understand yet!
> >
> > You can tell wireshark to only capture on one interface and to only capture 
> > packets for port 80.
> > (Captureing HTTPS means you cannot decode the packets without going deeper 
> > I recall)
> >
> > Then you can tell wireshark to decode the captured data for http to drop a 
> > lot of the lower level details.
> >
>
> Thanks. I am more or less doing that. Interestingly the [RST,ACK]
> messages appear on the tcp packets, so if I filter on http I do not see
> them.
>

I'm not 100% sure what "filter on HTTP" actually means, and it might
show only the data packets. Instead, filter on "from port 80 or to
port 80", which should show you the entire connection including the
SYN - SYN/ACK - ACK handshake, every data and acknowledgement packet,
and then whichever closing sequence gets used.

> >> I have another data point. I tried putting an asyncio.sleep() after 
> >> sending each file. A value of 0.01 made no difference, but a value of 0.1 
> >> makes the problem go away.
> >
> > What is the async wait to wait for the transmit buffers to drain?
> >
>
> Not sure what you are asking. I am just doing what it says in the docs -
>

I think his question is "what would asyncio be sleeping for, since you
already made it wait till it was drained". And my guess is that
there's a bug somewhere; the question is where.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Chris Angelico]

On Sun, Mar 22, 2020 at 8:30 PM Frank Millman  wrote:
>
> On 2020-03-22 10:45 AM, Chris Angelico wrote:
> > On Sun, Mar 22, 2020 at 6:58 PM Frank Millman  wrote:
> >>> I'd look at the network traffic with wireshark to see if there is 
> >>> anything different between edge and the other browsers.
> >>>
> >>
> >> You are leading me into deep waters here :-)  I have never used
> >> Wireshark before. I have now downloaded it and am running it - it
> >> generates a *lot* of data, most of which I do not understand yet!
> >>
> >> One thing immediately stands out. When I run it with MS Edge and
> >> Python3.8, it shows a lot of lines highlighted in red, with the symbols
> >> [RST,ACK]. They do not appear when running Chrome, and they do not
> >> appear when running Python3.7.
> >
> > Interesting. RST means "Reset" and is sent when the connection is
> > closed. Which direction were these packets sent (Edge to Python or
> > Python to Edge)? You can tell by the source and destination ports -
> > one of them is going to be the port Python is listening on (eg 80 or
> > 443), so if the destination port is 80, it's being sent *to* Python,
> > and if the source port is 80, it's being sent *from* Python.
> >
>
> They are all being sent *from* Python *to* Edge.

Very interesting indeed. What that *might* mean is that Python is
misinterpreting something and then believing that the connection has
been closed, so it responds "Okay, I'm closing the connection". Or
possibly it sees some sort of error condition.

> >> I have another data point. I tried putting an asyncio.sleep() after
> >> sending each file. A value of 0.01 made no difference, but a value of
> >> 0.1 makes the problem go away.
> >
> > Interesting also.
> >
> > Can you recreate the problem without Edge? It sounds like something's
> > going on with concurrent transfers, so it'd be awesome if you can
> > replace Edge with another Python program, and then post both programs.
> >
>
> Do you mean write a program that emulates a browser - make a connection,
> receive the HTML page, send a GET request for each file, and receive the
> results?
>
> I will give it a go!

Yes - although the HTML page is most likely irrelevant. You could just
make a connection and then spam requests. Or make multiple
connections.

Actually, that's another thing worth checking. Is Edge using a single
connection for all requests, or separate connections for each request,
or something in between (eg a pool of four connections and spreading
requests between them)? You'll be able to recognize different
connections by the port numbers Edge uses, which are guaranteed unique
among concurrent connections, and are most likely to not be reused for
a while even if the other is closed.

> > Also of interest: Does the problem go away if you change "Connection:
> > Keep-Alive" to "Connection: Close" in your headers?
> >
>
> Yes, the problem does go away.
>

This makes me think that the answer to the previous question is going
to involve some connection reuse.

If you can recreate the problem with a single socket and multiple
requests, that would be extremely helpful. I also think it's highly
likely that this is the case.

My theory: Your program is sending a large amount of data to the lower
level API functions, which attempt to send a large amount of data to
the socket. At some point, something gets told "sorry, can't handle
all that data, hold some of it back" at a time when it's not prepared
to do so, and it misinterprets it as an error. This error results in
the connection being closed.

But that's just a theory.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-22 11:00 AM, Barry Scott wrote:




On 22 Mar 2020, at 07:56, Frank Millman  wrote:

On 2020-03-21 8:04 PM, Barry Scott wrote:

I'd look at the network traffic with wireshark to see if there is anything 
different between edge and the other browsers.


You are leading me into deep waters here :-)  I have never used Wireshark 
before. I have now downloaded it and am running it - it generates a *lot* of 
data, most of which I do not understand yet!


You can tell wireshark to only capture on one interface and to only capture 
packets for port 80.
(Captureing HTTPS means you cannot decode the packets without going deeper I 
recall)

Then you can tell wireshark to decode the captured data for http to drop a lot 
of the lower level details.



Thanks. I am more or less doing that. Interestingly the [RST,ACK] 
messages appear on the tcp packets, so if I filter on http I do not see 
them.






One thing immediately stands out. When I run it with MS Edge and Python3.8, it 
shows a lot of lines highlighted in red, with the symbols [RST,ACK]. They do 
not appear when running Chrome, and they do not appear when running Python3.7.


As Chris said that should not happen.



As I replied to Chris, they appear in packets sent *from* Python *to* Edge.



I have another data point. I tried putting an asyncio.sleep() after sending 
each file. A value of 0.01 made no difference, but a value of 0.1 makes the 
problem go away.


What is the async wait to wait for the transmit buffers to drain?



Not sure what you are asking. I am just doing what it says in the docs -

=

write(data)
The method attempts to write the data to the underlying socket 
immediately. If that fails, the data is queued in an internal write 
buffer until it can be sent.


The method should be used along with the drain() method:

stream.write(data)
await stream.drain()

=

coroutine drain()
Wait until it is appropriate to resume writing to the stream. Example:

writer.write(data)
await writer.drain()
This is a flow control method that interacts with the underlying IO 
write buffer. When the size of the buffer reaches the high watermark, 
drain() blocks until the size of the buffer is drained down to the low 
watermark and writing can be resumed. When there is nothing to wait for, 
the drain() returns immediately.


=

Frank

--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-22 10:45 AM, Chris Angelico wrote:

On Sun, Mar 22, 2020 at 6:58 PM Frank Millman  wrote:

I'd look at the network traffic with wireshark to see if there is anything 
different between edge and the other browsers.



You are leading me into deep waters here :-)  I have never used
Wireshark before. I have now downloaded it and am running it - it
generates a *lot* of data, most of which I do not understand yet!

One thing immediately stands out. When I run it with MS Edge and
Python3.8, it shows a lot of lines highlighted in red, with the symbols
[RST,ACK]. They do not appear when running Chrome, and they do not
appear when running Python3.7.


Interesting. RST means "Reset" and is sent when the connection is
closed. Which direction were these packets sent (Edge to Python or
Python to Edge)? You can tell by the source and destination ports -
one of them is going to be the port Python is listening on (eg 80 or
443), so if the destination port is 80, it's being sent *to* Python,
and if the source port is 80, it's being sent *from* Python.



They are all being sent *from* Python *to* Edge.


I have another data point. I tried putting an asyncio.sleep() after
sending each file. A value of 0.01 made no difference, but a value of
0.1 makes the problem go away.


Interesting also.

Can you recreate the problem without Edge? It sounds like something's
going on with concurrent transfers, so it'd be awesome if you can
replace Edge with another Python program, and then post both programs.



Do you mean write a program that emulates a browser - make a connection, 
receive the HTML page, send a GET request for each file, and receive the 
results?


I will give it a go!


Also of interest: Does the problem go away if you change "Connection:
Keep-Alive" to "Connection: Close" in your headers?



Yes, the problem does go away.

Frank

--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Barry Scott]

> On 22 Mar 2020, at 07:56, Frank Millman  wrote:
> 
> On 2020-03-21 8:04 PM, Barry Scott wrote:
>>> On 21 Mar 2020, at 13:43, Frank Millman  wrote:
>>> 
>>> Hi all
>>> 
>>> I have a strange intermittent bug.
>>> 
>>> The role-players -
>>>asyncio on Python 3.8 running on Windows 10
>>>Microsoft Edge running as a browser on the same machine
>>> 
>>> The bug does not occur with Python 3.7.
>>> It does not occur with Chrome or Firefox.
>>> It does not occur when MS Edge connects to another host on the network, 
>>> running the same Python program (Python 3.8 on Fedora 31).
>>> 
>>> The symptoms -
>>>On receiving a connection, I send an HTML page to the browser,
>>>which has 20 lines like this -
>>> 
>>>
>>>
>>>...
>>> 
>>> Intermittently, one or other of the script files is not received by MS Edge.
>>> 
> [...]
>>> >> I don't know whether the problem lies with Python or MS Edge, but as 
> it does not happen with Python 3.7, I am suspecting that something changed in 
> 3.8 which does not match MS Edge's expectations.
>> I'd look at the network traffic with wireshark to see if there is anything 
>> different between edge and the other browsers.
> 
> You are leading me into deep waters here :-)  I have never used Wireshark 
> before. I have now downloaded it and am running it - it generates a *lot* of 
> data, most of which I do not understand yet!

You can tell wireshark to only capture on one interface and to only capture 
packets for port 80.
(Captureing HTTPS means you cannot decode the packets without going deeper I 
recall)

Then you can tell wireshark to decode the captured data for http to drop a lot 
of the lower level details.


> 
> One thing immediately stands out. When I run it with MS Edge and Python3.8, 
> it shows a lot of lines highlighted in red, with the symbols [RST,ACK]. They 
> do not appear when running Chrome, and they do not appear when running 
> Python3.7.

As Chris said that should not happen.

> 
> I have another data point. I tried putting an asyncio.sleep() after sending 
> each file. A value of 0.01 made no difference, but a value of 0.1 makes the 
> problem go away.

What is the async wait to wait for the transmit buffers to drain?

> 
> I will keep digging, but I thought I would post this information now in case 
> it helps with diagnosis.
> 
> Frank
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
Barry
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Chris Angelico]

On Sun, Mar 22, 2020 at 6:58 PM Frank Millman  wrote:
> > I'd look at the network traffic with wireshark to see if there is anything 
> > different between edge and the other browsers.
> >
>
> You are leading me into deep waters here :-)  I have never used
> Wireshark before. I have now downloaded it and am running it - it
> generates a *lot* of data, most of which I do not understand yet!
>
> One thing immediately stands out. When I run it with MS Edge and
> Python3.8, it shows a lot of lines highlighted in red, with the symbols
> [RST,ACK]. They do not appear when running Chrome, and they do not
> appear when running Python3.7.

Interesting. RST means "Reset" and is sent when the connection is
closed. Which direction were these packets sent (Edge to Python or
Python to Edge)? You can tell by the source and destination ports -
one of them is going to be the port Python is listening on (eg 80 or
443), so if the destination port is 80, it's being sent *to* Python,
and if the source port is 80, it's being sent *from* Python.

> I have another data point. I tried putting an asyncio.sleep() after
> sending each file. A value of 0.01 made no difference, but a value of
> 0.1 makes the problem go away.

Interesting also.

Can you recreate the problem without Edge? It sounds like something's
going on with concurrent transfers, so it'd be awesome if you can
replace Edge with another Python program, and then post both programs.

Also of interest: Does the problem go away if you change "Connection:
Keep-Alive" to "Connection: Close" in your headers?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Frank Millman]

On 2020-03-21 8:04 PM, Barry Scott wrote:




On 21 Mar 2020, at 13:43, Frank Millman  wrote:

Hi all

I have a strange intermittent bug.

The role-players -
asyncio on Python 3.8 running on Windows 10
Microsoft Edge running as a browser on the same machine

The bug does not occur with Python 3.7.
It does not occur with Chrome or Firefox.
It does not occur when MS Edge connects to another host on the network, running 
the same Python program (Python 3.8 on Fedora 31).

The symptoms -
On receiving a connection, I send an HTML page to the browser,
which has 20 lines like this -



...

Intermittently, one or other of the script files is not received by MS Edge.


[...]
>> I don't know whether the problem lies with Python or MS Edge, but as 
it does not happen with Python 3.7, I am suspecting that something 
changed in 3.8 which does not match MS Edge's expectations.


I'd look at the network traffic with wireshark to see if there is anything 
different between edge and the other browsers.



You are leading me into deep waters here :-)  I have never used 
Wireshark before. I have now downloaded it and am running it - it 
generates a *lot* of data, most of which I do not understand yet!


One thing immediately stands out. When I run it with MS Edge and 
Python3.8, it shows a lot of lines highlighted in red, with the symbols 
[RST,ACK]. They do not appear when running Chrome, and they do not 
appear when running Python3.7.


I have another data point. I tried putting an asyncio.sleep() after 
sending each file. A value of 0.01 made no difference, but a value of 
0.1 makes the problem go away.


I will keep digging, but I thought I would post this information now in 
case it helps with diagnosis.


Frank

--
https://mail.python.org/mailman/listinfo/python-list

Re: Intermittent bug with asyncio and MS Edge

[Barry Scott]

> On 21 Mar 2020, at 13:43, Frank Millman  wrote:
> 
> Hi all
> 
> I have a strange intermittent bug.
> 
> The role-players -
>asyncio on Python 3.8 running on Windows 10
>Microsoft Edge running as a browser on the same machine
> 
> The bug does not occur with Python 3.7.
> It does not occur with Chrome or Firefox.
> It does not occur when MS Edge connects to another host on the network, 
> running the same Python program (Python 3.8 on Fedora 31).
> 
> The symptoms -
>On receiving a connection, I send an HTML page to the browser,
>which has 20 lines like this -
> 
>
>
>...
> 
> Intermittently, one or other of the script files is not received by MS Edge.
> 
> I have checked the Network tab in Developer Tools in MS Edge. It shows the 
> first few requests getting a Status 200 OK, then some are shown as 'Pending'. 
> This seems to be where the problem occurs.
> 
> I am hoping that someone can give me some hints about how to debug this.
> 
> My function to send the script file looks like this -
> 
>async def send_file(writer, fname):
> 
>status = 'HTTP/1.1 200 OK\r\n'
>writer.write(status.encode())
> 
>headers = []
>headers.append(('CONNECTION', 'keep-alive'))
>headers.append(('DATE',
>email.utils.formatdate(usegmt=True)))
>headers.append(('SERVER',
>   f'Python {sys.version.split()[0]} asyncio'))
>headers.append(('Content-type', 'text/javascript'))
>headers('Transfer-Encoding', 'chunked'))
>for key, val in headers:
>writer.write(f'{key}: {val}\r\n'.encode())
>writer.write('\r\n'.encode())
>await writer.drain()
> 
>with open(fname 'rb') as fd:
>chunk = fd.read(8192)
>while chunk:
>writer.write(hex(len(chunk))[2:].encode() + b'\r\n')
>writer.write(chunk + b'\r\n')
>await writer.drain()
>chunk = fd.read(8192)
>writer.write(b'0\r\n\r\n')
>await writer.drain()
> 
>writer.close()
>await writer.wait_closed()
> 
> I have asked the same question on StackOverflow, from an MS Edge perspective -
> https://stackoverflow.com/questions/60785767/ms-edge-randomly-does-not-load-script
> 
> I don't know whether the problem lies with Python or MS Edge, but as it does 
> not happen with Python 3.7, I am suspecting that something changed in 3.8 
> which does not match MS Edge's expectations.

I'd look at the network  traffic with wireshark to see if there is anything 
different between edge and the other browsers.

Aside: headers are case blind, but usually written capitalised. Some buggy 
software expects that its capitalised.

I wonder if it will start working if you do not do chunked (your chunked 
encoding looks correct).
You can use os.stat to find the size of the file for the Content-Length header.

Barry



> 
> Any hints much appreciated.
> 
> Frank Millman
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python bug in ArcGIS - Urban Network analysis tool

[Станимира Николова]

понеделник, 30 юли 2018 г., 15:19:43 UTC+3, Andrew MacIntyre написа:
> On 30/07/2018 4:02 PM, Станимира Николова wrote:
> 
> > I run Urban network analysis but It shows similar mistake several 
> > times. The UNA tool is free plugin that i downloaded, it's not from the 
> > default intalled in ArcGIS packed. It ask for integer data.
> > I checked the type of the attributes, it's all integer. PLus it's all 
> > in geo data based file.
> >
> > Unfortunately I don't understand Python, I'm one of those who use 
> > ArcGIS as sample customer.
> >
> > This is the mistake:
> >
> > Start Time: Fri Jul 27 14:48:32 2018
> > Running script Centrality...
> > [started] Copying input buildings
> > [finished]
> > [1 started] Computing adjacency list
> > [1 failed]
> > Not successful
> > Completed script Centrality...
> > Succeeded at Fri Jul 27 14:48:36 2018 (Elapsed Time: 4,56 seconds)
> >
> > Any suggestions? How it's calling these adjaency list? What could be 
> > wrong? I even don't know how to get debugger, so it could give me more 
> > information.
> 
> If it's a third party plugin, contact the author or source.  This 
> mailing list sees very little traffic about ArcGIS usage as it is a 
> highly specialised commercial product.
> 
> > I add in a project the .py file for the adjacency list.
> > 
> > That's the main from the debuger:
> > 
> > pydev debugger: process 8904 is connecting
> > 
> > Connected to pydev debugger (build 182.3684.100)
> > Traceback (most recent call last):
> >File "C:\Program Files\JetBrains\PyCharm 
> > 2018.2\helpers\pydev\pydevd.py", line 1664, in 
> >  main()
> >File "C:\Program Files\JetBrains\PyCharm 
> > 2018.2\helpers\pydev\pydevd.py", line 1658, in main
> >  globals = debugger.run(setup['file'], None, None, is_module)
> >File "C:\Program Files\JetBrains\PyCharm 
> > 2018.2\helpers\pydev\pydevd.py", line 1068, in run
> >  pydev_imports.execfile(file, globals, locals)  # execute the script
> >File "C:\Program Files\JetBrains\PyCharm 
> > 2018.2\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
> >  exec(compile(contents+"\n", file, 'exec'), glob, loc)
> >File "D:/INSTALL/Urban Network Analysis Toolbox 
> > 1.01/src/Centrality/Adjacency_List_Computation.py", line 14, in 
> >  from arcpy import AddField_management
> > ModuleNotFoundError: No module named 'arcpy'
> > 
> > May I found the following lines from the code?
> 
> Is it documented that the file you've attempted to run via PyCharm 
> actually supports being run from outside ArcGIS?  Most plugins don't 
> seem to be...  This failure is because PyCharm isn't starting the script 
> with the necessary environment (including the arcpy ArcGIS Python 
> module) accessible.
> 
> The fact that it starts running from within ArcGIS and produces some 
> status messages before bailing out with the failure message strongly 
> suggests to me that the plugin is working properly but the input given 
> is not sufficient to produce the expected output.  In the absence of 
> sufficient documentation for you to figure out the required input, my 
> advice above stands: contact the author or the download source.
> 
> If you or your organisation has a current ArcGIS maintenance agreement, 
> you might also be able to access the community forums that ESRI run to 
> ask for more info about this plugin.
> 
> -- 
> -
> Andrew I MacIntyre "These thoughts are mine alone..."
> E-mail: andy...@bullseye.apana.org.au  (pref) | Snail: PO Box 370
>  andy...@pcug.org.au (alt) |Belconnen ACT 2616
> Web:http://www.andymac.org/   |Australia

Hi Andrew,
Thanks for your reply.

I managed to resolve the problem.

It seems that just the plugin doesn't found the python paths of it's 
environments and libraries. As I understood the adj. list is "calling" for 
functions in the other python files of the plugin (plus functions from ArcGIS).

What I've done (me and one IT specialist, actually):
 - Found the pythonpath of the module arcpy in the folder of default installed 
ArcGIS
 - Copied the pythonpath file, for the connection with module arcpy, in the 
folder with scripts of the plugin (UNA tool)
 - opened Python 3.7 and called for arcpy of the plugin. There we saw the paths 
of all enviroments that might use the plugin. The path to the arcpy appeared.

After that I got few mistakes of the input data, because I had to make unique 
ID of all the data points in the folder.

And tadamm … it works. 

Thanks for all the suggestions.
Mira.
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Python bug in ArcGIS - Urban Network analysis tool

[David Raymond]

A note that Arc may have installed its own version of Python, which it is using 
from within its own tools. For example, I've got a full Python installation in 
C:\Python27\ArcGIS10.2 which Arc installed on top of a preexisting installation 
in C:\Python27. So you may need to explicitly run it with that version to get 
it to work and to import arcpy ok.

ie instead of just running...

pydevd.py

or...

python pydevd.py

You might need to explicitly do...

C:\Python27\ArcGIS10.2\python.exe pydevd.py

...or the equivalent of wherever it installed it on your system.


-Original Message-
From: Python-list 
[mailto:python-list-bounces+david.raymond=tomtom@python.org] On Behalf Of 
? 
Sent: Monday, July 30, 2018 2:03 AM
To: python-list@python.org
Subject: Re: Python bug in ArcGIS - Urban Network analysis tool

понеделник, 30 юли 2018 г., 3:29:44 UTC+3, MRAB написа:
> On 2018-07-29 22:00, Станимира Николова wrote:
> > неделя, 29 юли 2018 г., 23:41:01 UTC+3, MRAB написа:
> >> On 2018-07-29 18:56, stanimira.s...@gmail.com wrote:
> >> > Hi there,
> >> > 
> >> > I'm trying make reach analyze with Urban network analysis (UNA) tool in 
> >> > ArcGIS. It's based on points for buildings and pedestrian network of 
> >> > acceptability steets. The aim is to show how many people from each 
> >> > building can reach different building in radius of 150 meters (so i can 
> >> > find the density of the pedestrian routes).
> >> > 
> >> > I run Urban network analysis but It shows similar mistake several times. 
> >> > The UNA tool is free plugin that i downloaded, it's not from the default 
> >> > intalled in ArcGIS packed. It ask for integer data.
> >> > I checked the type of the attributes, it's all integer. PLus it's all in 
> >> > geo data based file.
> >> > 
> >> > Unfortunately I don't understand Python, I'm one of those who use ArcGIS 
> >> > as sample customer.
> >> > 
> >> > This is the mistake:
> >> > 
> >> > Start Time: Fri Jul 27 14:48:32 2018
> >> > Running script Centrality...
> >> > [started] Copying input buildings
> >> > [finished]
> >> > [1 started] Computing adjacency list
> >> > [1 failed]
> >> > Not successful
> >> > Completed script Centrality...
> >> > Succeeded at Fri Jul 27 14:48:36 2018 (Elapsed Time: 4,56 seconds)
> >> > 
> >> > Any suggestions? How it's calling these adjaency list? What could be 
> >> > wrong? I even don't know how to get debugger, so it could give me more 
> >> > information.
> >> > 
> >> > Thank you previously,
> >> > Mira
> >> > 
> >> All you've shown is that says that it was unsuccessful, and it doesn't 
> >> say why.
> >> 
> >> We'd need a lot more info than that!
> >> 
> >> The best way to debug it is to try it with the minimal amount of test 
> >> data that should give a result.
> > 
> > I'm sorry.
> > Could someone help me by telling me how to make these test data? (so i 
> > could send it to You)
> > 
> > I realize the review of the bug is just the customer view. Any chance to 
> > help me out with the test of the process?
> > 
> > Thank you previously. If there is another forum that i have to write to, i 
> > will switch. The fact is that in the Themes for UNA tool or GIS software, 
> > almost no one respond.
> > 
> I'm unfamiliar with ArcGIS, so I doubt I could help.
> 
> It might have generated a log file somewhere that gives more details.

Ok, thank you for the patiance.
I installed PyCharm for trying make some tests. 

I add in a project the .py file for the adjacency list.

That's the main from the debuger:

pydev debugger: process 8904 is connecting

Connected to pydev debugger (build 182.3684.100)
Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm 2018.2\helpers\pydev\pydevd.py", 
line 1664, in 
main()
  File "C:\Program Files\JetBrains\PyCharm 2018.2\helpers\pydev\pydevd.py", 
line 1658, in main
globals = debugger.run(setup['file'], None, None, is_module)
  File "C:\Program Files\JetBrains\PyCharm 2018.2\helpers\pydev\pydevd.py", 
line 1068, in run
pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 
2018.2\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "D:/INSTALL/Urban Network Analysis Toolbox 
1.01/src/Centrality/Adjacency_List_Computation.py", line 14, in 
from arcpy import AddField_management
ModuleNotFoundError: No module named 'arcpy'

May I found the following lines from the code?
-- 
https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python bug in ArcGIS - Urban Network analysis tool

[Andrew MacIntyre]

On 30/07/2018 4:02 PM, Станимира Николова wrote:


I run Urban network analysis but It shows similar mistake several times. The 
UNA tool is free plugin that i downloaded, it's not from the default intalled 
in ArcGIS packed. It ask for integer data.
I checked the type of the attributes, it's all integer. PLus it's all in geo 
data based file.

Unfortunately I don't understand Python, I'm one of those who use ArcGIS as 
sample customer.

This is the mistake:

Start Time: Fri Jul 27 14:48:32 2018
Running script Centrality...
[started] Copying input buildings
[finished]
[1 started] Computing adjacency list
[1 failed]
Not successful
Completed script Centrality...
Succeeded at Fri Jul 27 14:48:36 2018 (Elapsed Time: 4,56 seconds)

Any suggestions? How it's calling these adjaency list? What could be wrong? I 
even don't know how to get debugger, so it could give me more information.


If it's a third party plugin, contact the author or source.  This 
mailing list sees very little traffic about ArcGIS usage as it is a 
highly specialised commercial product.



I add in a project the .py file for the adjacency list.

That's the main from the debuger:

pydev debugger: process 8904 is connecting

Connected to pydev debugger (build 182.3684.100)
Traceback (most recent call last):
   File "C:\Program Files\JetBrains\PyCharm 2018.2\helpers\pydev\pydevd.py", line 
1664, in 
 main()
   File "C:\Program Files\JetBrains\PyCharm 2018.2\helpers\pydev\pydevd.py", 
line 1658, in main
 globals = debugger.run(setup['file'], None, None, is_module)
   File "C:\Program Files\JetBrains\PyCharm 2018.2\helpers\pydev\pydevd.py", 
line 1068, in run
 pydev_imports.execfile(file, globals, locals)  # execute the script
   File "C:\Program Files\JetBrains\PyCharm 
2018.2\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
 exec(compile(contents+"\n", file, 'exec'), glob, loc)
   File "D:/INSTALL/Urban Network Analysis Toolbox 
1.01/src/Centrality/Adjacency_List_Computation.py", line 14, in 
 from arcpy import AddField_management
ModuleNotFoundError: No module named 'arcpy'

May I found the following lines from the code?


Is it documented that the file you've attempted to run via PyCharm 
actually supports being run from outside ArcGIS?  Most plugins don't 
seem to be...  This failure is because PyCharm isn't starting the script 
with the necessary environment (including the arcpy ArcGIS Python 
module) accessible.


The fact that it starts running from within ArcGIS and produces some 
status messages before bailing out with the failure message strongly 
suggests to me that the plugin is working properly but the input given 
is not sufficient to produce the expected output.  In the absence of 
sufficient documentation for you to figure out the required input, my 
advice above stands: contact the author or the download source.


If you or your organisation has a current ArcGIS maintenance agreement, 
you might also be able to access the community forums that ESRI run to 
ask for more info about this plugin.


--
-
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: andy...@bullseye.apana.org.au  (pref) | Snail: PO Box 370
andy...@pcug.org.au (alt) |Belconnen ACT 2616
Web:http://www.andymac.org/   |Australia
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python bug in ArcGIS - Urban Network analysis tool

[Станимира Николова]

понеделник, 30 юли 2018 г., 3:29:44 UTC+3, MRAB написа:
> On 2018-07-29 22:00, Станимира Николова wrote:
> > неделя, 29 юли 2018 г., 23:41:01 UTC+3, MRAB написа:
> >> On 2018-07-29 18:56, stanimira.s...@gmail.com wrote:
> >> > Hi there,
> >> > 
> >> > I'm trying make reach analyze with Urban network analysis (UNA) tool in 
> >> > ArcGIS. It's based on points for buildings and pedestrian network of 
> >> > acceptability steets. The aim is to show how many people from each 
> >> > building can reach different building in radius of 150 meters (so i can 
> >> > find the density of the pedestrian routes).
> >> > 
> >> > I run Urban network analysis but It shows similar mistake several times. 
> >> > The UNA tool is free plugin that i downloaded, it's not from the default 
> >> > intalled in ArcGIS packed. It ask for integer data.
> >> > I checked the type of the attributes, it's all integer. PLus it's all in 
> >> > geo data based file.
> >> > 
> >> > Unfortunately I don't understand Python, I'm one of those who use ArcGIS 
> >> > as sample customer.
> >> > 
> >> > This is the mistake:
> >> > 
> >> > Start Time: Fri Jul 27 14:48:32 2018
> >> > Running script Centrality...
> >> > [started] Copying input buildings
> >> > [finished]
> >> > [1 started] Computing adjacency list
> >> > [1 failed]
> >> > Not successful
> >> > Completed script Centrality...
> >> > Succeeded at Fri Jul 27 14:48:36 2018 (Elapsed Time: 4,56 seconds)
> >> > 
> >> > Any suggestions? How it's calling these adjaency list? What could be 
> >> > wrong? I even don't know how to get debugger, so it could give me more 
> >> > information.
> >> > 
> >> > Thank you previously,
> >> > Mira
> >> > 
> >> All you've shown is that says that it was unsuccessful, and it doesn't 
> >> say why.
> >> 
> >> We'd need a lot more info than that!
> >> 
> >> The best way to debug it is to try it with the minimal amount of test 
> >> data that should give a result.
> > 
> > I'm sorry.
> > Could someone help me by telling me how to make these test data? (so i 
> > could send it to You)
> > 
> > I realize the review of the bug is just the customer view. Any chance to 
> > help me out with the test of the process?
> > 
> > Thank you previously. If there is another forum that i have to write to, i 
> > will switch. The fact is that in the Themes for UNA tool or GIS software, 
> > almost no one respond.
> > 
> I'm unfamiliar with ArcGIS, so I doubt I could help.
> 
> It might have generated a log file somewhere that gives more details.

Ok, thank you for the patiance.
I installed PyCharm for trying make some tests. 

I add in a project the .py file for the adjacency list.

That's the main from the debuger:

pydev debugger: process 8904 is connecting

Connected to pydev debugger (build 182.3684.100)
Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm 2018.2\helpers\pydev\pydevd.py", 
line 1664, in 
main()
  File "C:\Program Files\JetBrains\PyCharm 2018.2\helpers\pydev\pydevd.py", 
line 1658, in main
globals = debugger.run(setup['file'], None, None, is_module)
  File "C:\Program Files\JetBrains\PyCharm 2018.2\helpers\pydev\pydevd.py", 
line 1068, in run
pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 
2018.2\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "D:/INSTALL/Urban Network Analysis Toolbox 
1.01/src/Centrality/Adjacency_List_Computation.py", line 14, in 
from arcpy import AddField_management
ModuleNotFoundError: No module named 'arcpy'

May I found the following lines from the code?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python bug in ArcGIS - Urban Network analysis tool

[MRAB]

On 2018-07-29 22:00, Станимира Николова wrote:

неделя, 29 юли 2018 г., 23:41:01 UTC+3, MRAB написа:

On 2018-07-29 18:56, stanimira.s...@gmail.com wrote:
> Hi there,
> 
> I'm trying make reach analyze with Urban network analysis (UNA) tool in ArcGIS. It's based on points for buildings and pedestrian network of acceptability steets. The aim is to show how many people from each building can reach different building in radius of 150 meters (so i can find the density of the pedestrian routes).
> 
> I run Urban network analysis but It shows similar mistake several times. The UNA tool is free plugin that i downloaded, it's not from the default intalled in ArcGIS packed. It ask for integer data.

> I checked the type of the attributes, it's all integer. PLus it's all in geo 
data based file.
> 
> Unfortunately I don't understand Python, I'm one of those who use ArcGIS as sample customer.
> 
> This is the mistake:
> 
> Start Time: Fri Jul 27 14:48:32 2018

> Running script Centrality...
> [started] Copying input buildings
> [finished]
> [1 started] Computing adjacency list
> [1 failed]
> Not successful
> Completed script Centrality...
> Succeeded at Fri Jul 27 14:48:36 2018 (Elapsed Time: 4,56 seconds)
> 
> Any suggestions? How it's calling these adjaency list? What could be wrong? I even don't know how to get debugger, so it could give me more information.
> 
> Thank you previously,

> Mira
> 
All you've shown is that says that it was unsuccessful, and it doesn't 
say why.


We'd need a lot more info than that!

The best way to debug it is to try it with the minimal amount of test 
data that should give a result.


I'm sorry.
Could someone help me by telling me how to make these test data? (so i could 
send it to You)

I realize the review of the bug is just the customer view. Any chance to help 
me out with the test of the process?

Thank you previously. If there is another forum that i have to write to, i will 
switch. The fact is that in the Themes for UNA tool or GIS software, almost no 
one respond.


I'm unfamiliar with ArcGIS, so I doubt I could help.

It might have generated a log file somewhere that gives more details.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python bug in ArcGIS - Urban Network analysis tool

[Станимира Николова]

неделя, 29 юли 2018 г., 23:41:01 UTC+3, MRAB написа:
> On 2018-07-29 18:56, stanimira.s...@gmail.com wrote:
> > Hi there,
> > 
> > I'm trying make reach analyze with Urban network analysis (UNA) tool in 
> > ArcGIS. It's based on points for buildings and pedestrian network of 
> > acceptability steets. The aim is to show how many people from each building 
> > can reach different building in radius of 150 meters (so i can find the 
> > density of the pedestrian routes).
> > 
> > I run Urban network analysis but It shows similar mistake several times. 
> > The UNA tool is free plugin that i downloaded, it's not from the default 
> > intalled in ArcGIS packed. It ask for integer data.
> > I checked the type of the attributes, it's all integer. PLus it's all in 
> > geo data based file.
> > 
> > Unfortunately I don't understand Python, I'm one of those who use ArcGIS as 
> > sample customer.
> > 
> > This is the mistake:
> > 
> > Start Time: Fri Jul 27 14:48:32 2018
> > Running script Centrality...
> > [started] Copying input buildings
> > [finished]
> > [1 started] Computing adjacency list
> > [1 failed]
> > Not successful
> > Completed script Centrality...
> > Succeeded at Fri Jul 27 14:48:36 2018 (Elapsed Time: 4,56 seconds)
> > 
> > Any suggestions? How it's calling these adjaency list? What could be wrong? 
> > I even don't know how to get debugger, so it could give me more information.
> > 
> > Thank you previously,
> > Mira
> > 
> All you've shown is that says that it was unsuccessful, and it doesn't 
> say why.
> 
> We'd need a lot more info than that!
> 
> The best way to debug it is to try it with the minimal amount of test 
> data that should give a result.

I'm sorry. 
Could someone help me by telling me how to make these test data? (so i could 
send it to You)

I realize the review of the bug is just the customer view. Any chance to help 
me out with the test of the process? 

Thank you previously. If there is another forum that i have to write to, i will 
switch. The fact is that in the Themes for UNA tool or GIS software, almost no 
one respond.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python bug in ArcGIS - Urban Network analysis tool

[MRAB]

On 2018-07-29 18:56, stanimira.s...@gmail.com wrote:

Hi there,

I'm trying make reach analyze with Urban network analysis (UNA) tool in ArcGIS. 
It's based on points for buildings and pedestrian network of acceptability 
steets. The aim is to show how many people from each building can reach 
different building in radius of 150 meters (so i can find the density of the 
pedestrian routes).

I run Urban network analysis but It shows similar mistake several times. The 
UNA tool is free plugin that i downloaded, it's not from the default intalled 
in ArcGIS packed. It ask for integer data.
I checked the type of the attributes, it's all integer. PLus it's all in geo 
data based file.

Unfortunately I don't understand Python, I'm one of those who use ArcGIS as 
sample customer.

This is the mistake:

Start Time: Fri Jul 27 14:48:32 2018
Running script Centrality...
[started] Copying input buildings
[finished]
[1 started] Computing adjacency list
[1 failed]
Not successful
Completed script Centrality...
Succeeded at Fri Jul 27 14:48:36 2018 (Elapsed Time: 4,56 seconds)

Any suggestions? How it's calling these adjaency list? What could be wrong? I 
even don't know how to get debugger, so it could give me more information.

Thank you previously,
Mira

All you've shown is that says that it was unsuccessful, and it doesn't 
say why.


We'd need a lot more info than that!

The best way to debug it is to try it with the minimal amount of test 
data that should give a result.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Possible bug in ThreadPoolExecutor, or just misinterpretation

[INADA Naoki]

On Mon, May 28, 2018 at 1:12 AM Santiago Basulto

wrote:

> Hey list! I might have encountered a "bug", or maybe it's just a "design
> decision" :)

> Here's some example code:
> https://gist.github.com/santiagobasulto/3513a50ec0dc939e8f7bb2ecfa8d4ae2

> The problem is `ThreadPoolExecutor.map()`. It's not returning tasks "as
> completed" but sequentially.  I understand that might be the desired
> behavior since the function is `map()` and you *might* be expecting the
> results "in order".

> So, the questions are: Is this expected behavior?

Yes, you can use `future.as_completed()` instead.

>   Is there any chance to
> add an `map_unordered` as multiprocessing has
> <
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.imap_unordered
> ?

I don't it's reasonable.
You can use multiprocessing.ThreadPool instead.

> Another option might be making `as_completed` work with map results too
> (which was my original intention).

I don't like this idea.

Regards,

-- 
INADA Naoki  
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Problem/bug with class definition inside function definition

[Gregory Ewing]

Alexey Muranov wrote:

   x = 42

   class C:
   x = x  # Works


I'd say it kind of works by accident, and is not really an
intended feature.

if Python does not allow to refer "simultaneously" to 
variables from different scopes if they have the same name.


It seems perfectly reasonable to me to have to use different
names to refer to different things in the same context. :-)

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Problem/bug with class definition inside function definition

[Ned Batchelder]

On 5/8/18 3:55 AM, Alexey Muranov wrote:
Sorry, i was confused.  I would say that this mostly works as 
expected, though the difference between


   x = 42

   class C:
   x = x  # Works

and

   def f2(a):
   class D:
   a = a  # Does not work <
   return D

is still surprising to me.

Otherwise, probably the solution with

   def f(a):
   _a = a
   class D:
   a = _a
   return D

is good enough, if Python does not allow to refer "simultaneously" to 
variables from different scopes if they have the same name.


Alexey.


I'm curious to see the real code you're writing that uses this style of 
class creation.  It feels like there might be an easier way to achieve 
your goal.


--Ned.



On Tue, 8 May, 2018 at 12:21 AM, Alexey Muranov 
 wrote:

To be more exact, i do see a few workarounds, for example:


   def f4(a):
   b = a
   class D:
   a = b  # Works
   return D

But this is not what i was hoping for.

Alexey.

On Tue, 8 May, 2018 at 12:02 AM, Alexey Muranov 
 wrote:
I have discovered the following bug or problem: it looks like i am 
forced to choose different names for class attributes and function 
arguments, and i see no workaround.  Am i missing some special 
syntax feature ?


Alexey.

---
x = 42

class C1:
   y = x  # Works

class C2:
   x = x  # Works

# ---
def f1(a):
   class D:
   b = a  # Works
   return D

def f2(a):
   class D:
   a = a  # Does not work <
   return D

def f3(a):
   class D:
   nonlocal a
   a = a  # Does not work either <
   return D

# ---
def g1(a):
   def h():
   b = a  # Works
   return b
   return h

def g2(a):
   def h():
   a = a  # Does not work (as expected)
   return a
   return h

def g3(a):
   def h():
   nonlocal a
   a = a  # Works
   return a
   return h

# ---
if __name__ == "__main__":
   assert C1.y == 42
   assert C2.x == 42

   assert f1(13).b == 13

   try:
   f2(13)  # NameError
   except NameError:
   pass
   except Exception as e:
   raise Exception( 'Unexpected exception raised: '
    '{}'.format(type(e).__name__) )
   else:
   raise Exception('No exception')

   try:
   f3(13).a  # AttributeError
   except AttributeError:
   pass
   except Exception as e:
   raise Exception( 'Unexpected exception raised: '
    '{}'.format(type(e).__name__) )
   else:
   raise Exception('No exception')

   assert g1(13)() == 13

   try:
   g2(13)()  # UnboundLocalError
   except UnboundLocalError:
   pass
   except Exception as e:
   raise Exception( 'Unexpected exception raised: '
    '{}'.format(type(e).__name__) )
   else:
   raise Exception('No exception')

   assert g3(13)() == 13







--
https://mail.python.org/mailman/listinfo/python-list

Re: Problem/bug with class definition inside function definition

[Alexey Muranov]

Sorry, i was confused.  I would say that this mostly works as expected, 
though the difference between


   x = 42

   class C:
   x = x  # Works

and

   def f2(a):
   class D:
   a = a  # Does not work <
   return D

is still surprising to me.

Otherwise, probably the solution with

   def f(a):
   _a = a
   class D:
   a = _a
   return D

is good enough, if Python does not allow to refer "simultaneously" to 
variables from different scopes if they have the same name.


Alexey.


On Tue, 8 May, 2018 at 12:21 AM, Alexey Muranov 
 wrote:

To be more exact, i do see a few workarounds, for example:


   def f4(a):
   b = a
   class D:
   a = b  # Works
   return D

But this is not what i was hoping for.

Alexey.

On Tue, 8 May, 2018 at 12:02 AM, Alexey Muranov 
 wrote:
I have discovered the following bug or problem: it looks like i am 
forced to choose different names for class attributes and function 
arguments, and i see no workaround.  Am i missing some special 
syntax feature ?


Alexey.

---
x = 42

class C1:
   y = x  # Works

class C2:
   x = x  # Works

# ---
def f1(a):
   class D:
   b = a  # Works
   return D

def f2(a):
   class D:
   a = a  # Does not work <
   return D

def f3(a):
   class D:
   nonlocal a
   a = a  # Does not work either <
   return D

# ---
def g1(a):
   def h():
   b = a  # Works
   return b
   return h

def g2(a):
   def h():
   a = a  # Does not work (as expected)
   return a
   return h

def g3(a):
   def h():
   nonlocal a
   a = a  # Works
   return a
   return h

# ---
if __name__ == "__main__":
   assert C1.y == 42
   assert C2.x == 42

   assert f1(13).b == 13

   try:
   f2(13)  # NameError
   except NameError:
   pass
   except Exception as e:
   raise Exception( 'Unexpected exception raised: '
'{}'.format(type(e).__name__) )
   else:
   raise Exception('No exception')

   try:
   f3(13).a  # AttributeError
   except AttributeError:
   pass
   except Exception as e:
   raise Exception( 'Unexpected exception raised: '
'{}'.format(type(e).__name__) )
   else:
   raise Exception('No exception')

   assert g1(13)() == 13

   try:
   g2(13)()  # UnboundLocalError
   except UnboundLocalError:
   pass
   except Exception as e:
   raise Exception( 'Unexpected exception raised: '
'{}'.format(type(e).__name__) )
   else:
   raise Exception('No exception')

   assert g3(13)() == 13





--
https://mail.python.org/mailman/listinfo/python-list

Re: Problem/bug with class definition inside function definition

[Gregory Ewing]

Python 3.5.1 (default, Jun  1 2016, 13:15:26)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(a):
...  class D:
...   pass
...  D.a = a
...  return D
...
>>> c = f(42)
>>> c
.D'>
>>> c.a
42

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Problem/bug with class definition inside function definition

[Alexey Muranov]

To be more exact, i do see a few workarounds, for example:


   def f4(a):
   b = a
   class D:
   a = b  # Works
   return D

But this is not what i was hoping for.

Alexey.

On Tue, 8 May, 2018 at 12:02 AM, Alexey Muranov 
 wrote:
I have discovered the following bug or problem: it looks like i am 
forced to choose different names for class attributes and function 
arguments, and i see no workaround.  Am i missing some special syntax 
feature ?


Alexey.

---
x = 42

class C1:
   y = x  # Works

class C2:
   x = x  # Works

# ---
def f1(a):
   class D:
   b = a  # Works
   return D

def f2(a):
   class D:
   a = a  # Does not work <
   return D

def f3(a):
   class D:
   nonlocal a
   a = a  # Does not work either <
   return D

# ---
def g1(a):
   def h():
   b = a  # Works
   return b
   return h

def g2(a):
   def h():
   a = a  # Does not work (as expected)
   return a
   return h

def g3(a):
   def h():
   nonlocal a
   a = a  # Works
   return a
   return h

# ---
if __name__ == "__main__":
   assert C1.y == 42
   assert C2.x == 42

   assert f1(13).b == 13

   try:
   f2(13)  # NameError
   except NameError:
   pass
   except Exception as e:
   raise Exception( 'Unexpected exception raised: '
'{}'.format(type(e).__name__) )
   else:
   raise Exception('No exception')

   try:
   f3(13).a  # AttributeError
   except AttributeError:
   pass
   except Exception as e:
   raise Exception( 'Unexpected exception raised: '
'{}'.format(type(e).__name__) )
   else:
   raise Exception('No exception')

   assert g1(13)() == 13

   try:
   g2(13)()  # UnboundLocalError
   except UnboundLocalError:
   pass
   except Exception as e:
   raise Exception( 'Unexpected exception raised: '
'{}'.format(type(e).__name__) )
   else:
   raise Exception('No exception')

   assert g3(13)() == 13



--
https://mail.python.org/mailman/listinfo/python-list

Re: Python bug report

[breamoreboy]

On Friday, December 22, 2017 at 1:28:17 PM UTC, Ranya wrote:
> Hi,
> Am trying to use clr.AddReference and clr.AddReferenceToFile, but
> python(2.7) keeps making this error:
> 
> Traceback (most recent call last):
>   File "", line 1, in 
> clr.AddReference("UnityEngine")AttributeError: 'module' object has
> no attribute 'AddReference'
> 
> How can I fix this?
> Thanks in advance.

Are you actually using the IronPython clr module, have you downloaded by 
mistake the module of the same name from pypi, or do you have a module of the 
same name on your path?

--
Kindest regards.

Mark Lawrence.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python bug report

[Peter Pearson]

On Thu, 21 Dec 2017 23:54:17 +0100, Ranya  wrote:
> Hi,
> Am trying to use clr.AddReference and clr.AddReferenceToFile, but
> python(2.7) keeps making this error:
>
> Traceback (most recent call last):
>   File "", line 1, in 
> clr.AddReference("UnityEngine")AttributeError: 'module' object has
> no attribute 'AddReference'
>
> How can I fix this?
> Thanks in advance.

What is clr?  Whatever it is, it doesn't have the AddReference
attribute that you seem to be expecting.

-- 
To email me, substitute nowhere->runbox, invalid->com.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: possible bug in while loop test

[MRAB]

On 2017-09-02 18:53, Charles Hixson wrote:

python3 --version
Python 3.5.3

Running on Debian stretch

In this code s is a string parameter

while (j < k   and \
   (s[j].isalnum())  or   \
(s[j] in seps and s[j+1].isalnum()) ):
 j   =   j + 1
 print ("i = {0}, j = {1}, k = {2}, len[s] = {3}".   \
   format(i, j, k, len(s) ) )

Yields the result:

i = 25, j = 27, k = 31, len[s] = 32
i = 25, j = 28, k = 31, len[s] = 32
i = 25, j = 29, k = 31, len[s] = 32
i = 25, j = 30, k = 31, len[s] = 32
i = 25, j = 31, k = 31, len[s] = 32
Traceback (most recent call last):
File "parse1.py", line 40, in 
  print (parse1("The gostack distimms the doshes."))
File "parse1.py", line 21, in parse1
  (s[j] in seps and s[j+1].isalnum()) ):
IndexError: string index out of range

I hesitate to report this, because I've been wrong so often, but it
looks to me like the last iteration should not have happened since j is
not less than k.

The last iteration happened because j < k when you tested the while 
loop's condition.


You then added 1 to j before printing out j and k. At this print, j >= k.
--
https://mail.python.org/mailman/listinfo/python-list

Re: possible bug in while loop test

[Peter Otten]

Charles Hixson wrote:

> python3 --version
> Python 3.5.3
> 
> Running on Debian stretch
> 
> In this code s is a string parameter
> 
> while (j < k   and \
>   (s[j].isalnum())  or   \
>(s[j] in seps and s[j+1].isalnum()) ):
> j   =   j + 1
> print ("i = {0}, j = {1}, k = {2}, len[s] = {3}".   \
>   format(i, j, k, len(s) ) )
> 
> Yields the result:
> 
> i = 25, j = 27, k = 31, len[s] = 32
> i = 25, j = 28, k = 31, len[s] = 32
> i = 25, j = 29, k = 31, len[s] = 32
> i = 25, j = 30, k = 31, len[s] = 32
> i = 25, j = 31, k = 31, len[s] = 32
> Traceback (most recent call last):
>File "parse1.py", line 40, in 
>  print (parse1("The gostack distimms the doshes."))
>File "parse1.py", line 21, in parse1
>  (s[j] in seps and s[j+1].isalnum()) ):
> IndexError: string index out of range
> 
> I hesitate to report this, because I've been wrong so often, but it
> looks to me like the last iteration should not have happened since j is
> not less than k.

You have made a bit of a mess of the while condition. Removing random space 
and newlines:

(j < k and (s[j].isalnum()) or (s[j] in seps and s[j+1].isalnum()))

The structure is

a and b or (c and d)  # a: j False as you suspect, but then (c and d) is evalued, and if c 
succeeds also the expression d aka s[j+1].isalnum() which fails on accessing 
s[j+1].


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: configparser bug

[Christos Malliopoulos]

Thanx Peter for your prompt response.
I replaced tabs with spaces and had no problem reading the file.
I think however it is sth to correct in next versions

BR
Chris


On Wed, Jan 25, 2017 at 1:32 PM Peter Otten <__pete...@web.de> wrote:

> Christos Malliopoulos wrote:
>
> > Hi,
> >
> > I run Ubuntu 16.04 LTS in a VM using VMWare Workstation on a Windows 10
> > host.
> > apt show python-configparser shows 3.3.0r2-2
> > On python 2.7.12 I use the following code:
> >
> > import configparser as cfg
> > root =
> > u'/'.join(os.path.split(os.path.abspath('cfg.py'))[0].split('/')[:-2])
> > cp = cfg.ConfigParser(interpolation = cfg.ExtendedInterpolation())
> > cp.read(os.path.abspath(os.path.join(root, u'config/sampling.cfg')))
>
> This looks confusing. Are you sure you are reading the right sampling.cfg?
>
> > cp.items('Sampling')
> >
> > sampling.cfg contains the following lines:
> > [Sampling]
> > nobs = 10
> > nzin = 4
> > nzout = 3
> > ndrops = 1
> >
> > inh = day,loc,z0,value
> > outh = day,loc,sku,value
> >
> > invalran = 1,1e3
> > outvalran = 1,1000
> >
> > cp.items(u'Sampling') prints the following:
> > [(u'nobs', u'10'),
> >  (u'nzin', u'4'),
> >  (u'nzout', u'3\nndrops = 1'),
> >  (u'inh', u'day,loc,z0,value'),
> >  (u'outh', u'day,loc,sku,value'),
> >  (u'invalran', u'1,1e3'),
> >  (u'outvalran', u'1,1000')]
> >
> > ndrops = 1 is not parsed correctly
>
> It looks like the parser does not expand tabs, and your problem may be
> mixing tabs and spaces:
>
> $ cat demo.py
> import configparser
>
> data = """\
> [Sampling]
> \talpha = 1
> \tbeta = 2
>  \tgamma = 3
> \tdelta = 4
> """
> print "what you see:"
> print data.expandtabs(4)
>
> print "what the parser sees:"
> print data.replace("\t", " ")
>
> with open("sampling.cfg", "w") as f:
> f.write(data)
>
> cp = configparser.ConfigParser(
> interpolation=configparser.ExtendedInterpolation()
> )
> cp.read("sampling.cfg")
> for item in cp.items('Sampling'):
> print item
> $ python demo.py
> what you see:
> [Sampling]
> alpha = 1
> beta = 2
> gamma = 3
> delta = 4
>
> what the parser sees:
> [Sampling]
>  alpha = 1
>  beta = 2
>   gamma = 3
>  delta = 4
>
> (u'alpha', u'1')
> (u'beta', u'2\ngamma = 3')
> (u'delta', u'4')
> $
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: configparser bug

[Peter Otten]

Christos Malliopoulos wrote:

> Hi,
> 
> I run Ubuntu 16.04 LTS in a VM using VMWare Workstation on a Windows 10
> host.
> apt show python-configparser shows 3.3.0r2-2
> On python 2.7.12 I use the following code:
> 
> import configparser as cfg
> root =
> u'/'.join(os.path.split(os.path.abspath('cfg.py'))[0].split('/')[:-2])
> cp = cfg.ConfigParser(interpolation = cfg.ExtendedInterpolation())
> cp.read(os.path.abspath(os.path.join(root, u'config/sampling.cfg')))

This looks confusing. Are you sure you are reading the right sampling.cfg?

> cp.items('Sampling')
> 
> sampling.cfg contains the following lines:
> [Sampling]
> nobs = 10
> nzin = 4
> nzout = 3
> ndrops = 1
> 
> inh = day,loc,z0,value
> outh = day,loc,sku,value
> 
> invalran = 1,1e3
> outvalran = 1,1000
> 
> cp.items(u'Sampling') prints the following:
> [(u'nobs', u'10'),
>  (u'nzin', u'4'),
>  (u'nzout', u'3\nndrops = 1'),
>  (u'inh', u'day,loc,z0,value'),
>  (u'outh', u'day,loc,sku,value'),
>  (u'invalran', u'1,1e3'),
>  (u'outvalran', u'1,1000')]
> 
> ndrops = 1 is not parsed correctly

It looks like the parser does not expand tabs, and your problem may be 
mixing tabs and spaces:

$ cat demo.py
import configparser

data = """\
[Sampling]
\talpha = 1
\tbeta = 2
 \tgamma = 3
\tdelta = 4
"""
print "what you see:"
print data.expandtabs(4)

print "what the parser sees:"
print data.replace("\t", " ")

with open("sampling.cfg", "w") as f:
f.write(data)

cp = configparser.ConfigParser(
interpolation=configparser.ExtendedInterpolation()
)
cp.read("sampling.cfg")
for item in cp.items('Sampling'):
print item
$ python demo.py 
what you see:
[Sampling]
alpha = 1
beta = 2
gamma = 3
delta = 4

what the parser sees:
[Sampling]
 alpha = 1
 beta = 2
  gamma = 3
 delta = 4

(u'alpha', u'1')
(u'beta', u'2\ngamma = 3')
(u'delta', u'4')
$ 


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[Xristos Xristoou]

Τη Σάββατο, 14 Ιανουαρίου 2017 - 8:09:53 μ.μ. UTC+2, ο χρήστης duncan smith 
έγραψε:
> On 14/01/17 14:59, Xristos Xristoou wrote:
> > Τη Σάββατο, 14 Ιανουαρίου 2017 - 4:38:39 μ.μ. UTC+2, ο χρήστης duncan smith 
> > έγραψε:
> >> On 14/01/17 11:18, Xristos Xristoou wrote:
> >>> i want  to create a simple spatial joing using geopandas but i thing so 
> >>> geopandas has bug ?
> >>>
> >>>
> >>>
> >>> geopandas code :
> >>>
> >>> from geopandas import gpd
> >>> import geopandas
> >>> points = geopandas.GeoDataFrame.from_file('points.shp') # or geojson etc
> >>> polys = geopandas.GeoDataFrame.from_file('polygons.shp')
> >>> pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> >>>
> >>> error :
> >>>
> >>> Traceback (most recent call last):
> >>>   File "/home/sarantis/testshapely/sumpointsinsidepolygon/testgeo.py", 
> >>> line 7, in 
> >>> pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> >>>   File "/usr/local/lib/python2.7/dist-packages/geopandas/tools/sjoin.py", 
> >>> line 57, in sjoin
> >>> r_idx = np.concatenate(idxmatch.values)
> >>> ValueError: need at least one array to concatenate
> >>>
> >>> and if i change the imports with the some code :
> >>>
> >>> import geopandas
> >>> import pandas as pd
> >>> import geopandas as gpd
> >>> from geopandas import GeoDataFrame, read_file
> >>> from geopandas.tools import sjoin
> >>> from shapely.geometry import Point, mapping,shape
> >>> import pandas as gpd
> >>>
> >>> i take that error :
> >>>
> >>> pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> >>> AttributeError: 'module' object has no attribute 'sjoin'
> >>>
> >>>
> >>> any idea why ?
> >>>
> >>
> >>
> >> import geopandas as gpd
> >> import pandas as gpd
> >>
> >> Does pandas have an attribute 'sjoin'?
> >>
> >> Duncan
> > 
> > i dont know i follow detais i am newbie
> > 
> 
> You import geopandas as gpd, then import pandas as gpd. So maybe you
> think gpd refers to geopandas while it actually refers to pandas. I
> don't know geopandas or pandas, but you should check your imports.
> 
> Duncan

@Duncan i see that and i remove line with pandas ans no change
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[duncan smith]

On 14/01/17 14:59, Xristos Xristoou wrote:
> Τη Σάββατο, 14 Ιανουαρίου 2017 - 4:38:39 μ.μ. UTC+2, ο χρήστης duncan smith 
> έγραψε:
>> On 14/01/17 11:18, Xristos Xristoou wrote:
>>> i want  to create a simple spatial joing using geopandas but i thing so 
>>> geopandas has bug ?
>>>
>>>
>>>
>>> geopandas code :
>>>
>>> from geopandas import gpd
>>> import geopandas
>>> points = geopandas.GeoDataFrame.from_file('points.shp') # or geojson etc
>>> polys = geopandas.GeoDataFrame.from_file('polygons.shp')
>>> pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
>>>
>>> error :
>>>
>>> Traceback (most recent call last):
>>>   File "/home/sarantis/testshapely/sumpointsinsidepolygon/testgeo.py", line 
>>> 7, in 
>>> pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
>>>   File "/usr/local/lib/python2.7/dist-packages/geopandas/tools/sjoin.py", 
>>> line 57, in sjoin
>>> r_idx = np.concatenate(idxmatch.values)
>>> ValueError: need at least one array to concatenate
>>>
>>> and if i change the imports with the some code :
>>>
>>> import geopandas
>>> import pandas as pd
>>> import geopandas as gpd
>>> from geopandas import GeoDataFrame, read_file
>>> from geopandas.tools import sjoin
>>> from shapely.geometry import Point, mapping,shape
>>> import pandas as gpd
>>>
>>> i take that error :
>>>
>>> pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
>>> AttributeError: 'module' object has no attribute 'sjoin'
>>>
>>>
>>> any idea why ?
>>>
>>
>>
>> import geopandas as gpd
>> import pandas as gpd
>>
>> Does pandas have an attribute 'sjoin'?
>>
>> Duncan
> 
> i dont know i follow detais i am newbie
> 

You import geopandas as gpd, then import pandas as gpd. So maybe you
think gpd refers to geopandas while it actually refers to pandas. I
don't know geopandas or pandas, but you should check your imports.

Duncan
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[Peter Otten]

Xristos Xristoou wrote:

> Τη Σάββατο, 14 Ιανουαρίου 2017 - 6:33:54 μ.μ. UTC+2, ο χρήστης Peter Otten
> έγραψε:
>> Xristos Xristoou wrote:
>> 
>> >> I suggest that you file a bug report.
>> > 
>> > Mr.Peter Otten do you see my shapefiles ?have instersection 100 to 100
>> > i use instersection on QGIS ad work fine
>> 
>> Yes, I downloaded the zipfile at
>> 
>> > https://www.dropbox.com/s/2693nfi248z0y9q/files.zip?dl=0 with the
>> 
>> and when I ran your code I got the very error that you saw. There are
>> many NaN values in your data, so if it works elsewhere perhaps the data
>> is corrupted in some way. I'm sorry I cannot help you any further.
>> 
>> Good luck!
> 
> one more question,i have a idea what is wrong,but if my code work how to
> export spatial join "pointInPoly" to new shapefile ?

You can find an object's methods in the interactive interpreter with dir()

>>> dir(pointInPoly)
['T', '_AXIS_ALIASES', '_AXIS_IALIASES', '_AXIS_LEN', '_AXIS_NAMES', 
'_AXIS_NUMBERS', '_AXIS_ORDERS', '_AXIS_REVERSED', '_AXIS_SLICEMAP', 



'rmod', 'rmul', 'rotate', 'rpow', 'rsub', 'rtruediv', 'save', 'scale', 
'select', 'set_geometry', 'set_index', 'set_value', 'shape', 'shift', 
'simplify', 'sindex', 'skew', 'sort', 'sort_index', 'sortlevel', 'squeeze', 
'stack', 'std', 'sub', 'subtract', 'sum', 'swapaxes', 'swaplevel', 
'symmetric_difference', 'tail', 'take', 'to_clipboard', 'to_crs', 'to_csv', 
'to_dense', 'to_dict', 'to_excel', 'to_file', 'to_gbq', 'to_hdf', 'to_html', 
'to_json', 'to_latex', 'to_msgpack', 'to_panel', 'to_period', 'to_pickle', 
'to_records', 'to_sparse', 'to_sql', 'to_stata', 'to_string', 
'to_timestamp', 'to_wide', 'total_bounds', 'touches', 'translate', 
'transpose', 'truediv', 'truncate', 'tshift', 'type', 'tz_convert', 
'tz_localize', 'unary_union', 'union', 'unstack', 'update', 'values', 'var', 
'where', 'within', 'xs']

OK, that's quite a lot, but to_file() seems to be a good candidate. Let's 
see what it does:

>>> help(pointInPoly.to_file)
Help on method to_file in module geopandas.geodataframe:

to_file(filename, driver='ESRI Shapefile', schema=None, **kwargs) metod of 
geopandas.geodataframe.GeoDataFrame instance
Write this GeoDataFrame to an OGR data source

A dictionary of supported OGR providers is available via:
>>> import fiona
>>> fiona.supported_drivers

Parameters
--
filename : string
File path or file handle to write to.
driver : string, default 'ESRI Shapefile'
The OGR format driver used to write the vector file.
schema : dict, default None
If specified, the schema dictionary is passed to Fiona to
better control how the file is written.

The *kwargs* are passed to fiona.open and can be used to write
to multi-layer data, store data within archives (zip files), etc.

Looks good, run it:

>>> pointInPoly.to_file("point_in_poly")

No error. Does it round-trip?

>>> pointInPoly == gpd.GeoDataFrame.from_file("point_in_poly")
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3/dist-packages/pandas/core/ops.py", line 875, in f
return self._compare_frame(other, func, str_rep)
  File "/usr/lib/python3/dist-packages/pandas/core/frame.py", line 2860, in 
_compare_frame
raise ValueError('Can only compare identically-labeled '
ValueError: Can only compare identically-labeled DataFrame objects

Ouch, unfortunately not. Upon further inspection:

>>> pip.columns
Index(['geometry', 'index_righ', 'name_left', 'name_right'], dtype='object')
>>> pointInPoly.columns
Index(['geometry', 'name_left', 'index_right', 'name_right'], 
dtype='object')

Looks like column names are either corrupted or limited to 10 characters by 
default. Again I don't know how to overcome this, but as a special service 
here's the first hit for 'shp file column name limit' on a popular search 
engine:

http://gis.stackexchange.com/questions/15784/how-to-bypass-10-character-limit-of-field-name-in-shapefiles

I'm out of this now.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[Xristos Xristoou]

Τη Σάββατο, 14 Ιανουαρίου 2017 - 6:33:54 μ.μ. UTC+2, ο χρήστης Peter Otten 
έγραψε:
> Xristos Xristoou wrote:
> 
> >> I suggest that you file a bug report.
> > 
> > Mr.Peter Otten do you see my shapefiles ?have instersection 100 to 100 i
> > use instersection on QGIS ad work fine
> 
> Yes, I downloaded the zipfile at
> 
> > https://www.dropbox.com/s/2693nfi248z0y9q/files.zip?dl=0 with the
> 
> and when I ran your code I got the very error that you saw. There are many 
> NaN values in your data, so if it works elsewhere perhaps the data is 
> corrupted in some way. I'm sorry I cannot help you any further. 
> 
> Good luck!

one more question,i have a idea what is wrong,but if my code work how to export 
spatial join "pointInPoly" to new shapefile ?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[Peter Otten]

Xristos Xristoou wrote:

>> I suggest that you file a bug report.
> 
> Mr.Peter Otten do you see my shapefiles ?have instersection 100 to 100 i
> use instersection on QGIS ad work fine

Yes, I downloaded the zipfile at

> https://www.dropbox.com/s/2693nfi248z0y9q/files.zip?dl=0 with the

and when I ran your code I got the very error that you saw. There are many 
NaN values in your data, so if it works elsewhere perhaps the data is 
corrupted in some way. I'm sorry I cannot help you any further. 

Good luck!

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[Xristos Xristoou]

Τη Σάββατο, 14 Ιανουαρίου 2017 - 6:01:39 μ.μ. UTC+2, ο χρήστης Peter Otten 
έγραψε:
> Xristos Xristoou wrote:
> 
> > Τη Σάββατο, 14 Ιανουαρίου 2017 - 4:30:48 μ.μ. UTC+2, ο χρήστης Peter Otten
> > έγραψε:
> >> Xristos Xristoou wrote:
> >> 
> >> > Τη Σάββατο, 14 Ιανουαρίου 2017 - 3:43:10 μ.μ. UTC+2, ο χρήστης Peter
> >> > Otten έγραψε:
> >> >> Xristos Xristoou wrote:
> >> >> 
> >> >> > i want  to create a simple spatial joing using geopandas but i thing
> >> >> > so geopandas has bug ?
> >> >> 
> >> >> Have you tried the examples on
> >> >> ? Do they work? If yes, inspect
> >> >> your data, does it have the same format?
> >> 
> >> Looks like you chose to ignore the hard part.
> >>  
> >> >> > geopandas code :
> >> >> > 
> >> >> > from geopandas import gpd
> >> >> > import geopandas
> >> >> > points = geopandas.GeoDataFrame.from_file('points.shp') # or geojson
> >> >> > etc polys = geopandas.GeoDataFrame.from_file('polygons.shp')
> >> >> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> >> >> > 
> >> >> > error :
> >> >> > 
> >> >> > Traceback (most recent call last):
> >> >> >   File
> >> >> >   "/home/sarantis/testshapely/sumpointsinsidepolygon/testgeo.py",
> >> >> >   line 7, in 
> >> >> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> >> >> >   File
> >> >> >   "/usr/local/lib/python2.7/dist-packages/geopandas/tools/sjoin.py",
> >> >> >   line 57, in sjoin
> >> >> > r_idx = np.concatenate(idxmatch.values)
> >> >> > ValueError: need at least one array to concatenate
> >> 
> >> >> > any idea why ?
> >> >> 
> >> >> My crystal ball says that either points or polys is empty ;)
> >> > 
> >> > is not empty and yes i have two shapefiles from qgis.
> >> 
> >> Can I download those files somewhere?
> >> 
> >> > what is the error ?
> >> 
> >> No idea. Without those files I cannot run your code.
> > 
> > https://www.dropbox.com/s/2693nfi248z0y9q/files.zip?dl=0 with the
> > shapefiles
> 
> It looks like there are no intersections in your data.
> With the proviso that I've learned about the library only today I think you 
> should get an empty result set rather than the ValueError.
> Here's a way to reproduce the error (?) with the data provided in the 
> project:
> 
> import geopandas
> from geopandas import gpd
> 
> world = gpd.read_file(gpd.datasets.get_path('naturalearth_lowres'))
> cities = gpd.read_file(gpd.datasets.get_path('naturalearth_cities'))
> countries = world[['geometry', 'name']]
> 
> def find(items, name):
> # didn't find the idomatic way quickly, so
> for index, n in enumerate(items["name"]):
> if n == name:
> return items[index:index+1]
> raise ValueError
> 
> berlin = find(cities, "Berlin")
> paris = find(cities, "Paris")
> germany = find(countries, "Germany")
> 
> print(gpd.sjoin(berlin, germany))
> print(gpd.sjoin(paris, germany)) # ValueError
>  
> $ python demo.py
> geometry name_left  index_right  \
> 175  POINT (13.39960276470055 52.52376452225116)Berlin   41   
> 
> name_right  
> 175Germany  
> 
> [1 rows x 4 columns]
> Traceback (most recent call last):
>   File "demo.py", line 20, in 
> print(gpd.sjoin(paris, germany)) # ValueError
>   File "/home/peter/virt/geopandas/lib/python3.4/site-
> packages/geopandas/tools/sjoin.py", line 57, in sjoin
> r_idx = np.concatenate(idxmatch.values)
> ValueError: need at least one array to concatenate
> $
> 
> I suggest that you file a bug report.

Mr.Peter Otten do you see my shapefiles ?have instersection 100 to 100 i use 
instersection on QGIS ad work fine
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[Peter Otten]

Xristos Xristoou wrote:

> Τη Σάββατο, 14 Ιανουαρίου 2017 - 4:30:48 μ.μ. UTC+2, ο χρήστης Peter Otten
> έγραψε:
>> Xristos Xristoou wrote:
>> 
>> > Τη Σάββατο, 14 Ιανουαρίου 2017 - 3:43:10 μ.μ. UTC+2, ο χρήστης Peter
>> > Otten έγραψε:
>> >> Xristos Xristoou wrote:
>> >> 
>> >> > i want  to create a simple spatial joing using geopandas but i thing
>> >> > so geopandas has bug ?
>> >> 
>> >> Have you tried the examples on
>> >> ? Do they work? If yes, inspect
>> >> your data, does it have the same format?
>> 
>> Looks like you chose to ignore the hard part.
>>  
>> >> > geopandas code :
>> >> > 
>> >> > from geopandas import gpd
>> >> > import geopandas
>> >> > points = geopandas.GeoDataFrame.from_file('points.shp') # or geojson
>> >> > etc polys = geopandas.GeoDataFrame.from_file('polygons.shp')
>> >> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
>> >> > 
>> >> > error :
>> >> > 
>> >> > Traceback (most recent call last):
>> >> >   File
>> >> >   "/home/sarantis/testshapely/sumpointsinsidepolygon/testgeo.py",
>> >> >   line 7, in 
>> >> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
>> >> >   File
>> >> >   "/usr/local/lib/python2.7/dist-packages/geopandas/tools/sjoin.py",
>> >> >   line 57, in sjoin
>> >> > r_idx = np.concatenate(idxmatch.values)
>> >> > ValueError: need at least one array to concatenate
>> 
>> >> > any idea why ?
>> >> 
>> >> My crystal ball says that either points or polys is empty ;)
>> > 
>> > is not empty and yes i have two shapefiles from qgis.
>> 
>> Can I download those files somewhere?
>> 
>> > what is the error ?
>> 
>> No idea. Without those files I cannot run your code.
> 
> https://www.dropbox.com/s/2693nfi248z0y9q/files.zip?dl=0 with the
> shapefiles

It looks like there are no intersections in your data.
With the proviso that I've learned about the library only today I think you 
should get an empty result set rather than the ValueError.
Here's a way to reproduce the error (?) with the data provided in the 
project:

import geopandas
from geopandas import gpd

world = gpd.read_file(gpd.datasets.get_path('naturalearth_lowres'))
cities = gpd.read_file(gpd.datasets.get_path('naturalearth_cities'))
countries = world[['geometry', 'name']]

def find(items, name):
# didn't find the idomatic way quickly, so
for index, n in enumerate(items["name"]):
if n == name:
return items[index:index+1]
raise ValueError

berlin = find(cities, "Berlin")
paris = find(cities, "Paris")
germany = find(countries, "Germany")

print(gpd.sjoin(berlin, germany))
print(gpd.sjoin(paris, germany)) # ValueError
 
$ python demo.py
geometry name_left  index_right  \
175  POINT (13.39960276470055 52.52376452225116)Berlin   41   

name_right  
175Germany  

[1 rows x 4 columns]
Traceback (most recent call last):
  File "demo.py", line 20, in 
print(gpd.sjoin(paris, germany)) # ValueError
  File "/home/peter/virt/geopandas/lib/python3.4/site-
packages/geopandas/tools/sjoin.py", line 57, in sjoin
r_idx = np.concatenate(idxmatch.values)
ValueError: need at least one array to concatenate
$

I suggest that you file a bug report.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[Xristos Xristoou]

Τη Σάββατο, 14 Ιανουαρίου 2017 - 4:38:39 μ.μ. UTC+2, ο χρήστης duncan smith 
έγραψε:
> On 14/01/17 11:18, Xristos Xristoou wrote:
> > i want  to create a simple spatial joing using geopandas but i thing so 
> > geopandas has bug ?
> > 
> > 
> > 
> > geopandas code :
> > 
> > from geopandas import gpd
> > import geopandas
> > points = geopandas.GeoDataFrame.from_file('points.shp') # or geojson etc
> > polys = geopandas.GeoDataFrame.from_file('polygons.shp')
> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> > 
> > error :
> > 
> > Traceback (most recent call last):
> >   File "/home/sarantis/testshapely/sumpointsinsidepolygon/testgeo.py", line 
> > 7, in 
> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> >   File "/usr/local/lib/python2.7/dist-packages/geopandas/tools/sjoin.py", 
> > line 57, in sjoin
> > r_idx = np.concatenate(idxmatch.values)
> > ValueError: need at least one array to concatenate
> > 
> > and if i change the imports with the some code :
> > 
> > import geopandas
> > import pandas as pd
> > import geopandas as gpd
> > from geopandas import GeoDataFrame, read_file
> > from geopandas.tools import sjoin
> > from shapely.geometry import Point, mapping,shape
> > import pandas as gpd
> > 
> > i take that error :
> > 
> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> > AttributeError: 'module' object has no attribute 'sjoin'
> > 
> > 
> > any idea why ?
> > 
> 
> 
> import geopandas as gpd
> import pandas as gpd
> 
> Does pandas have an attribute 'sjoin'?
> 
> Duncan

i dont know i follow detais i am newbie
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[Xristos Xristoou]

Τη Σάββατο, 14 Ιανουαρίου 2017 - 4:30:48 μ.μ. UTC+2, ο χρήστης Peter Otten 
έγραψε:
> Xristos Xristoou wrote:
> 
> > Τη Σάββατο, 14 Ιανουαρίου 2017 - 3:43:10 μ.μ. UTC+2, ο χρήστης Peter Otten
> > έγραψε:
> >> Xristos Xristoou wrote:
> >> 
> >> > i want  to create a simple spatial joing using geopandas but i thing so
> >> > geopandas has bug ?
> >> 
> >> Have you tried the examples on ?
> >> Do they work? If yes, inspect your data, does it have the same format?
> 
> Looks like you chose to ignore the hard part.
>  
> >> > geopandas code :
> >> > 
> >> > from geopandas import gpd
> >> > import geopandas
> >> > points = geopandas.GeoDataFrame.from_file('points.shp') # or geojson
> >> > etc polys = geopandas.GeoDataFrame.from_file('polygons.shp')
> >> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> >> > 
> >> > error :
> >> > 
> >> > Traceback (most recent call last):
> >> >   File "/home/sarantis/testshapely/sumpointsinsidepolygon/testgeo.py",
> >> >   line 7, in 
> >> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> >> >   File
> >> >   "/usr/local/lib/python2.7/dist-packages/geopandas/tools/sjoin.py",
> >> >   line 57, in sjoin
> >> > r_idx = np.concatenate(idxmatch.values)
> >> > ValueError: need at least one array to concatenate
> 
> >> > any idea why ?
> >> 
> >> My crystal ball says that either points or polys is empty ;)
> > 
> > is not empty and yes i have two shapefiles from qgis.
> 
> Can I download those files somewhere?
> 
> > what is the error ?
> 
> No idea. Without those files I cannot run your code.

https://www.dropbox.com/s/2693nfi248z0y9q/files.zip?dl=0 with the shapefiles
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[duncan smith]

On 14/01/17 11:18, Xristos Xristoou wrote:
> i want  to create a simple spatial joing using geopandas but i thing so 
> geopandas has bug ?
> 
> 
> 
> geopandas code :
> 
> from geopandas import gpd
> import geopandas
> points = geopandas.GeoDataFrame.from_file('points.shp') # or geojson etc
> polys = geopandas.GeoDataFrame.from_file('polygons.shp')
> pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> 
> error :
> 
> Traceback (most recent call last):
>   File "/home/sarantis/testshapely/sumpointsinsidepolygon/testgeo.py", line 
> 7, in 
> pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
>   File "/usr/local/lib/python2.7/dist-packages/geopandas/tools/sjoin.py", 
> line 57, in sjoin
> r_idx = np.concatenate(idxmatch.values)
> ValueError: need at least one array to concatenate
> 
> and if i change the imports with the some code :
> 
> import geopandas
> import pandas as pd
> import geopandas as gpd
> from geopandas import GeoDataFrame, read_file
> from geopandas.tools import sjoin
> from shapely.geometry import Point, mapping,shape
> import pandas as gpd
> 
> i take that error :
> 
> pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> AttributeError: 'module' object has no attribute 'sjoin'
> 
> 
> any idea why ?
> 


import geopandas as gpd
import pandas as gpd

Does pandas have an attribute 'sjoin'?

Duncan
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[Xristos Xristoou]

Τη Σάββατο, 14 Ιανουαρίου 2017 - 4:30:48 μ.μ. UTC+2, ο χρήστης Peter Otten 
έγραψε:
> Xristos Xristoou wrote:
> 
> > Τη Σάββατο, 14 Ιανουαρίου 2017 - 3:43:10 μ.μ. UTC+2, ο χρήστης Peter Otten
> > έγραψε:
> >> Xristos Xristoou wrote:
> >> 
> >> > i want  to create a simple spatial joing using geopandas but i thing so
> >> > geopandas has bug ?
> >> 
> >> Have you tried the examples on ?
> >> Do they work? If yes, inspect your data, does it have the same format?
> 
> Looks like you chose to ignore the hard part.
>  
> >> > geopandas code :
> >> > 
> >> > from geopandas import gpd
> >> > import geopandas
> >> > points = geopandas.GeoDataFrame.from_file('points.shp') # or geojson
> >> > etc polys = geopandas.GeoDataFrame.from_file('polygons.shp')
> >> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> >> > 
> >> > error :
> >> > 
> >> > Traceback (most recent call last):
> >> >   File "/home/sarantis/testshapely/sumpointsinsidepolygon/testgeo.py",
> >> >   line 7, in 
> >> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> >> >   File
> >> >   "/usr/local/lib/python2.7/dist-packages/geopandas/tools/sjoin.py",
> >> >   line 57, in sjoin
> >> > r_idx = np.concatenate(idxmatch.values)
> >> > ValueError: need at least one array to concatenate
> 
> >> > any idea why ?
> >> 
> >> My crystal ball says that either points or polys is empty ;)
> > 
> > is not empty and yes i have two shapefiles from qgis.
> 
> Can I download those files somewhere?
> 
> > what is the error ?
> 
> No idea. Without those files I cannot run your code.

yes i sent you
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[Peter Otten]

Xristos Xristoou wrote:

> Τη Σάββατο, 14 Ιανουαρίου 2017 - 3:43:10 μ.μ. UTC+2, ο χρήστης Peter Otten
> έγραψε:
>> Xristos Xristoou wrote:
>> 
>> > i want  to create a simple spatial joing using geopandas but i thing so
>> > geopandas has bug ?
>> 
>> Have you tried the examples on ?
>> Do they work? If yes, inspect your data, does it have the same format?

Looks like you chose to ignore the hard part.
 
>> > geopandas code :
>> > 
>> > from geopandas import gpd
>> > import geopandas
>> > points = geopandas.GeoDataFrame.from_file('points.shp') # or geojson
>> > etc polys = geopandas.GeoDataFrame.from_file('polygons.shp')
>> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
>> > 
>> > error :
>> > 
>> > Traceback (most recent call last):
>> >   File "/home/sarantis/testshapely/sumpointsinsidepolygon/testgeo.py",
>> >   line 7, in 
>> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
>> >   File
>> >   "/usr/local/lib/python2.7/dist-packages/geopandas/tools/sjoin.py",
>> >   line 57, in sjoin
>> > r_idx = np.concatenate(idxmatch.values)
>> > ValueError: need at least one array to concatenate

>> > any idea why ?
>> 
>> My crystal ball says that either points or polys is empty ;)
> 
> is not empty and yes i have two shapefiles from qgis.

Can I download those files somewhere?

> what is the error ?

No idea. Without those files I cannot run your code.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[Xristos Xristoou]

Τη Σάββατο, 14 Ιανουαρίου 2017 - 3:43:10 μ.μ. UTC+2, ο χρήστης Peter Otten 
έγραψε:
> Xristos Xristoou wrote:
> 
> > i want  to create a simple spatial joing using geopandas but i thing so
> > geopandas has bug ?
> 
> Have you tried the examples on ?
> Do they work? If yes, inspect your data, does it have the same format?
> 
> > geopandas code :
> > 
> > from geopandas import gpd
> > import geopandas
> > points = geopandas.GeoDataFrame.from_file('points.shp') # or geojson etc
> > polys = geopandas.GeoDataFrame.from_file('polygons.shp')
> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> > 
> > error :
> > 
> > Traceback (most recent call last):
> >   File "/home/sarantis/testshapely/sumpointsinsidepolygon/testgeo.py",
> >   line 7, in 
> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> >   File "/usr/local/lib/python2.7/dist-packages/geopandas/tools/sjoin.py",
> >   line 57, in sjoin
> > r_idx = np.concatenate(idxmatch.values)
> > ValueError: need at least one array to concatenate
> > 
> > and if i change the imports with the some code :
> > 
> > import geopandas
> > import pandas as pd
> > import geopandas as gpd
> > from geopandas import GeoDataFrame, read_file
> > from geopandas.tools import sjoin
> > from shapely.geometry import Point, mapping,shape
> > import pandas as gpd
> > 
> > i take that error :
> > 
> > pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> > AttributeError: 'module' object has no attribute 'sjoin'
> > 
> > 
> > any idea why ?
> 
> My crystal ball says that either points or polys is empty ;)

is not empty and yes i have two shapefiles from qgis.what is the error ?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: geopandas bug ?

[Peter Otten]

Xristos Xristoou wrote:

> i want  to create a simple spatial joing using geopandas but i thing so
> geopandas has bug ?

Have you tried the examples on ?
Do they work? If yes, inspect your data, does it have the same format?

> geopandas code :
> 
> from geopandas import gpd
> import geopandas
> points = geopandas.GeoDataFrame.from_file('points.shp') # or geojson etc
> polys = geopandas.GeoDataFrame.from_file('polygons.shp')
> pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> 
> error :
> 
> Traceback (most recent call last):
>   File "/home/sarantis/testshapely/sumpointsinsidepolygon/testgeo.py",
>   line 7, in 
> pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
>   File "/usr/local/lib/python2.7/dist-packages/geopandas/tools/sjoin.py",
>   line 57, in sjoin
> r_idx = np.concatenate(idxmatch.values)
> ValueError: need at least one array to concatenate
> 
> and if i change the imports with the some code :
> 
> import geopandas
> import pandas as pd
> import geopandas as gpd
> from geopandas import GeoDataFrame, read_file
> from geopandas.tools import sjoin
> from shapely.geometry import Point, mapping,shape
> import pandas as gpd
> 
> i take that error :
> 
> pointInPoly = gpd.sjoin(points, polys, how='left',op='within')
> AttributeError: 'module' object has no attribute 'sjoin'
> 
> 
> any idea why ?

My crystal ball says that either points or polys is empty ;)


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Seen bug in argparse module

[Peter Otten]

Jagdish Choudhary wrote:

> When argument provided from user which doesn't match to right option which
> is mentioned in help , it runs without issue..let me provide an example
> 
> https://docs.python.org/3.3/library/argparse.html
> 
> import argparse
> parser = argparse.ArgumentParser(description='Process some
> integers.')parser.add_argument('integers', metavar='N', type=int,
> nargs='+',
>help='an integer for the
> accumulator')parser.add_argument('--sum', dest='accumulate',
> action='store_const',
>const=sum, default=max,
>help='sum the integers (default: find the max)')
> args = parser.parse_args()print(args.accumulate(args.integers))
> 
> python prog.py -h
> usage: prog.py [-h] [--sum] N [N ...]
> 
> Process some integers.
> 
> positional arguments:
>  N   an integer for the accumulator
> 
> optional arguments:
>  -h, --help  show this help message and exit
>  --sum   sum the integers (default: find the max)
> 
> If user run it like below-
> 
> JAGDISHs-MacBook-Pro:test_python jagdish$ python prog.py 12 3 4 --sum
> 19
> JAGDISHs-MacBook-Pro:test_python jagdish$ python prog.py 12 3 4 --su
> 19
> JAGDISHs-MacBook-Pro:test_python jag

This works as designed:

https://docs.python.org/3.3/library/argparse.html#prefix-matching

Starting with Python 3.5 you can disable this behaviour with 
allow_abbrev=False, see

https://docs.python.org/3.6/library/argparse.html#allow-abbrev

If you think this should have been the default -- I agree.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Re - bug in tokenize module

[Steven D'Aprano]

On Tue, 21 Jun 2016 09:01 am, Harrison Chudleigh wrote:

> Sorry. Only works with text files. But my point is still valid.

What point?

Without context, how are we supposed to know what you're talking about?
We're not mind-readers you know.



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PyPi bug?

[Kev Dwyer]

Nagy László Zsolt wrote:

> Today I have tried to register and upload a new package by executing
> 
> setup.py register
> 
> 
> I was asked if I want to save the creditentials in a .pypirc file and I
> answered yes.
> 
> Next I wanted to run
> 
> setup.py upload
> 
> and I got this error:
> 
> 
> Traceback (most recent call last):
>   File "C:\Python\Projects\some_package\setup.py", line 15, in 
> classifiers=['Topic :: Security', 'Topic :: Internet :: WWW/HTTP'],
>   File "C:\Python35\lib\distutils\core.py", line 148, in setup
> dist.run_commands()
>   File "C:\Python35\lib\distutils\dist.py", line 955, in run_commands
> self.run_command(cmd)
>   File "C:\Python35\lib\distutils\dist.py", line 973, in run_command
> cmd_obj.ensure_finalized()
>   File "C:\Python35\lib\distutils\cmd.py", line 107, in ensure_finalized
> self.finalize_options()
>   File "C:\Python35\lib\distutils\command\upload.py", line 46, in
> finalize_options
> config = self._read_pypirc()
>   File "C:\Python35\lib\distutils\config.py", line 83, in _read_pypirc
> current[key] = config.get(server, key)
>   File "C:\Python35\lib\configparser.py", line 798, in get
> d)
>   File "C:\Python35\lib\configparser.py", line 396, in before_get
> self._interpolate_some(parser, option, L, value, section, defaults, 1)
>   File "C:\Python35\lib\configparser.py", line 445, in _interpolate_some
> "found: %r" % (rest,))
> configparser.InterpolationSyntaxError: '%' must be followed by '%' or
> '(', found
> : *'
> 
> Instead of the many stars, of course there is the password. The problem
> might be that the password contains a % character, and it was
> incorrectly saved by distutils.
> 
> Can somebody please confirm that this is a bug in distutils? The we
> probably have to submit a bug report.
> 
> Thanks,
> 
>Laszlo


Could be http://bugs.python.org/issue20120 ?



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PyPi bug?

[Nagy László Zsolt]

>
> Could be http://bugs.python.org/issue20120 ?
Almost. Except that the password was saved incorrectly by distutils
itself. So maybe the read part should not be fixed, but the write part
should.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: threading bug in strptime

[Akira Li]

Larry Martell  writes:

> We have been trying to figure out an intermittent problem where a
> thread would fail with this:
>
> AttributeError: 'module' object has no attribute '_strptime'
>
> Even though we were importing datetime. After much banging our heads
> against the wall, we found this:
>
> http://code-trick.com/python-bug-attribute-error-_strptime/
>
> The workaround suggested there, to call strptime before starting your
> threads, seems to have fixed the issue.
>
> I thought I'd mention it here in case anyone else is facing this.

I can reproduce it in Python 2 (but not in Python 3) even with
*threading* module:

  #!/usr/bin/env python
  import threading
  import time

  for _ in range(10):
  threading.Thread(target=time.strptime,
   args=("2013-06-02", "%Y-%m-%d")).start()

Don't use *thread* directly (it is even renamed to *_thread* in Python
3, to discourage an unintended usage), use *threading* module instead.

In Python 3.3+, PyImport_ImportModuleNoBlock()  is deprecated
  https://bugs.python.org/issue9260

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pylint bug in python35 b2

[Peter Otten]

Florent Quesselaire wrote:

 i installed python35 b2
 and tested to pip install / easy install pylint plgin
 but i got an :
 
 AttributeError : 'Call' object has no attribute starargs.
 
 i feel this error more python side than pylint side because it is the same
 version who works fine in 3.4.3

That looks like a change in the ast that breaks backwards compatibility 
rather than a bug:

$ python3.4
Python 3.4.0 (default, Jun 19 2015, 14:20:21) 
[GCC 4.8.2] on linux
Type help, copyright, credits or license for more information.
 import ast
 module = ast.parse(f(*x))
 module.body[0].value
_ast.Call object at 0x7f082ce07860
 _.starargs
_ast.Name object at 0x7f082cd8c240
 _.id
'x'
 
$ python3.5
Python 3.5.0b2+ (3.5:9aee273bf8b7+, Jun 25 2015, 09:25:29) 
[GCC 4.8.4] on linux
Type help, copyright, credits or license for more information.
 import ast
 module = ast.parse(f(*x))
 call = module.body[0].value
 call.starargs
Traceback (most recent call last):
  File stdin, line 1, in module
AttributeError: 'Call' object has no attribute 'starargs'
 call.args
[_ast.Starred object at 0x7f6e633cb208]
 call.args[0].value.id
'x'

pylint likely needs an update to support 3.5 anyway as there is some new 
syntax.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pylint bug in python35 b2

[Mark Lawrence]

On 02/07/2015 15:07, Peter Otten wrote:

Florent Quesselaire wrote:


i installed python35 b2
and tested to pip install / easy install pylint plgin
but i got an :

AttributeError : 'Call' object has no attribute starargs.

i feel this error more python side than pylint side because it is the same
version who works fine in 3.4.3


That looks like a change in the ast that breaks backwards compatibility
rather than a bug:



Hardly surprising as there are no guarantees.  From 
https://docs.python.org/3/library/ast.html The abstract syntax itself 
might change with each Python release; this module helps to find out 
programmatically what the current grammar looks like.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Pillow bug?

[kai . peters]

On Tuesday, 24 March 2015 13:15:42 UTC-7, Ian  wrote:
 On Tue, Mar 24, 2015 at 1:52 PM,  Kai wrote:
  Judging from the message archive, the image-sig list is (just about) dead?
 
  Disclaimer: Am a newbie - so anything is possible
 
 
  using 'RGB' works fine
 
  img = Image.new('RGB', (inktile[0], inktile[1]), bgcolor)
 
  using '1' or 'L' does not (see trace below)
 
  img = Image.new('L', (inktile[0], inktile[1]), bgcolor)
 
 
  I change nothing else but the first parameter. If I switch the 'L' back to 
  'RGB' it works again.
 
 
  img  = Image.new('L', (inktile[0], inktile[1]), bgcolor)
File C:\Python27\lib\site-packages\PIL\Image.py, line 2015, in new
  return Image()._new(core.fill(mode, size, color))
  TypeError: an integer is required
 
  The docs say that '1' and 'L' are supported - something broken? Something I 
  do not understand?
 
 What is the value of bgcolor that you're passing in? Per the docs: If
 given, this should be a single integer or floating point value for
 single-band modes, and a tuple for multi-band modes (one value per
 band). So if you're changing from a multi-band mode to a single-band
 mode, you would need to change the color argument as well.

Good catch - that was it.

Thanks much!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pillow bug?

[Ian Kelly]

On Tue, Mar 24, 2015 at 1:52 PM,  kai.pet...@gmail.com wrote:
 Judging from the message archive, the image-sig list is (just about) dead?

 Disclaimer: Am a newbie - so anything is possible


 using 'RGB' works fine

 img = Image.new('RGB', (inktile[0], inktile[1]), bgcolor)

 using '1' or 'L' does not (see trace below)

 img = Image.new('L', (inktile[0], inktile[1]), bgcolor)


 I change nothing else but the first parameter. If I switch the 'L' back to 
 'RGB' it works again.


 img  = Image.new('L', (inktile[0], inktile[1]), bgcolor)
   File C:\Python27\lib\site-packages\PIL\Image.py, line 2015, in new
 return Image()._new(core.fill(mode, size, color))
 TypeError: an integer is required

 The docs say that '1' and 'L' are supported - something broken? Something I 
 do not understand?

What is the value of bgcolor that you're passing in? Per the docs: If
given, this should be a single integer or floating point value for
single-band modes, and a tuple for multi-band modes (one value per
band). So if you're changing from a multi-band mode to a single-band
mode, you would need to change the color argument as well.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Pillow bug?

[Terry Reedy]

On 3/24/2015 3:52 PM, kai.pet...@gmail.com wrote:

Judging from the message archive, the image-sig list is (just about) dead?


PIL and/or pillow should have their own lists.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: anaconda bug?

[memilanuk]

On 03/17/2015 09:51 AM, George Trojan wrote:

On 03/16/2015 11:47 PM, memilanuk wrote:

Might be just you...

monte@machin-shin:~$ python
Python 3.4.3 |Continuum Analytics, Inc.| (default, Mar  6 2015, 12:03:53)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type help, copyright, credits or license for more information.
 import tkinter
 tkinter.Tk()
tkinter.Tk object at 0x7ff4c6fc8160



Just for the heck of it I created a new venv (using conda create -n
test) and tried it again.  Same thing.

How are you creating your venv?

Monte




Hmm. I tried on the different system (Fedora 20), with Python 3.4.2.
Same results:

dilbert@gtrojan python
Python 3.4.2 |Continuum Analytics, Inc.| (default, Oct 21 2014, 17:16:37)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type help, copyright, credits or license for more information.
  import tkinter
  tkinter.Tk()
tkinter.Tk object at 0x7f5713951630
 
dilbert@gtrojan which pyvenv
/usr/local/miniconda3/bin/pyvenv
dilbert@gtrojan pyvenv --system-site-packages ~/test
dilbert@gtrojan source ~/test/bin/activate
(test) dilbert@gtrojan python
Python 3.4.2 |Continuum Analytics, Inc.| (default, Oct 21 2014, 17:16:37)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type help, copyright, credits or license for more information.
  import tkinter
  tkinter.Tk()
Traceback (most recent call last):
   File stdin, line 1, in module
   File /usr/local/miniconda3/lib/python3.4/tkinter/__init__.py, line
1851, in __init__
 self.tk = _tkinter.create(screenName, baseName, className,
interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: Can't find a usable init.tcl in the following
directories:
 /opt/anaconda1anaconda2anaconda3/lib/tcl8.5
/home/gtrojan/test/lib/tcl8.5 /home/gtrojan/lib/tcl8.5
/home/gtrojan/test/library /home/gtrojan/library
/home/gtrojan/tcl8.5.15/library /home/tcl8.5.15/library

This probably means that Tcl wasn't installed properly.
 

I suspect faulty logic: pyvenv does not copy/links the tcl/tk libraries
to the newly created directory. When I run python directly, the second
directory to search is /usr/local/miniconda3/lib/tcl8.5, where conda
puts its tcl version. In virtual environment, the path is replaced and
tkinter fails. So the other fix would be to manually create symlinks
after running pyvenv, or modify Continuum Analytics pyvenv to do that.
There is no issue with pyvenv when Python is built from the source, the
first directory in the path is where tcl is found by configure and that
does not change in virtual environment.

I found another similar bug report here:
https://groups.google.com/a/continuum.io/forum/#!topic/anaconda/Q9xvJT8khTs
Looks this has not been fixed.


Well, it might be worth poking them and seeing if you get any response.

What happens if you create a venv using conda?  Is there some particular 
reason you're using Anaconda but not using its tools for managing 
virtualenvs?  I get it that pyenv should work, but it might be useful to 
isolate it further to tell if its just pyenv, or something else.  I 
don't have any experience with pyenv myself, sorry.


HTH,

Monte


--
Shiny!  Let's be bad guys.

Reach me @ memilanuk (at) gmail dot com

--
https://mail.python.org/mailman/listinfo/python-list

Re: Re: anaconda bug?

[George Trojan]

On 03/16/2015 11:47 PM, memilanuk wrote:

Might be just you...

monte@machin-shin:~$ python
Python 3.4.3 |Continuum Analytics, Inc.| (default, Mar  6 2015, 12:03:53)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type help, copyright, credits or license for more information.
 import tkinter
 tkinter.Tk()
tkinter.Tk object at 0x7ff4c6fc8160



Just for the heck of it I created a new venv (using conda create -n 
test) and tried it again.  Same thing.


How are you creating your venv?

Monte




Hmm. I tried on the different system (Fedora 20), with Python 3.4.2. 
Same results:


dilbert@gtrojan python
Python 3.4.2 |Continuum Analytics, Inc.| (default, Oct 21 2014, 17:16:37)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type help, copyright, credits or license for more information.
 import tkinter
 tkinter.Tk()
tkinter.Tk object at 0x7f5713951630

dilbert@gtrojan which pyvenv
/usr/local/miniconda3/bin/pyvenv
dilbert@gtrojan pyvenv --system-site-packages ~/test
dilbert@gtrojan source ~/test/bin/activate
(test) dilbert@gtrojan python
Python 3.4.2 |Continuum Analytics, Inc.| (default, Oct 21 2014, 17:16:37)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type help, copyright, credits or license for more information.
 import tkinter
 tkinter.Tk()
Traceback (most recent call last):
  File stdin, line 1, in module
  File /usr/local/miniconda3/lib/python3.4/tkinter/__init__.py, line 
1851, in __init__
self.tk = _tkinter.create(screenName, baseName, className, 
interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: Can't find a usable init.tcl in the following 
directories:
/opt/anaconda1anaconda2anaconda3/lib/tcl8.5 
/home/gtrojan/test/lib/tcl8.5 /home/gtrojan/lib/tcl8.5 
/home/gtrojan/test/library /home/gtrojan/library 
/home/gtrojan/tcl8.5.15/library /home/tcl8.5.15/library


This probably means that Tcl wasn't installed properly.


I suspect faulty logic: pyvenv does not copy/links the tcl/tk libraries 
to the newly created directory. When I run python directly, the second 
directory to search is /usr/local/miniconda3/lib/tcl8.5, where conda 
puts its tcl version. In virtual environment, the path is replaced and 
tkinter fails. So the other fix would be to manually create symlinks 
after running pyvenv, or modify Continuum Analytics pyvenv to do that.
There is no issue with pyvenv when Python is built from the source, the 
first directory in the path is where tcl is found by configure and that 
does not change in virtual environment.


I found another similar bug report here: 
https://groups.google.com/a/continuum.io/forum/#!topic/anaconda/Q9xvJT8khTs

Looks this has not been fixed.

George


--
https://mail.python.org/mailman/listinfo/python-list

Re: anaconda bug?

[memilanuk]

Might be just you...

monte@machin-shin:~$ python
Python 3.4.3 |Continuum Analytics, Inc.| (default, Mar  6 2015, 12:03:53)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type help, copyright, credits or license for more information.
 import tkinter
 tkinter.Tk()
tkinter.Tk object at 0x7ff4c6fc8160



Just for the heck of it I created a new venv (using conda create -n 
test) and tried it again.  Same thing.


How are you creating your venv?

Monte


--
Shiny!  Let's be bad guys.

Reach me @ memilanuk (at) gmail dot com

--
https://mail.python.org/mailman/listinfo/python-list

Re: strip bug?

[babyG]

Hello how are you doing please


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: strip bug?

[Rustom Mody]

On Wednesday, February 25, 2015 at 4:35:37 AM UTC+5:30, bay...@gmail.com wrote:
  'http://xthunder'.strip('http://')
 'xthunder'
  'http://thunder'.strip('http://')
 'under'
 
 
 I could understand backslash but forward slash?

Others have answered specifically.
However you probably want to look at urlparse
https://docs.python.org/2/library/urlparse.html
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: strip bug?

[Ian Kelly]

On Tue, Feb 24, 2015 at 4:05 PM,  bayk...@gmail.com wrote:
 'http://xthunder'.strip('http://')
 'xthunder'
 'http://thunder'.strip('http://')
 'under'


This removes all leading and trailing occurrences of the characters in
the string 'http://', not the exact substring 'http://'. For that, use
either the str.replace method or slicing.

 I could understand backslash but forward slash?

I don't understand the question.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: strip bug?

[Irmen de Jong]

On 25-2-2015 0:05, bayk...@gmail.com wrote:
 'http://xthunder'.strip('http://')
 'xthunder'
 'http://thunder'.strip('http://')
 'under'

 
 I could understand backslash but forward slash?
 




 help(.strip)
Help on built-in function strip:

strip(...) method of builtins.str instance
S.strip([chars]) - str

Return a copy of the string S with leading and trailing
whitespace removed.
If chars is given and not None, remove characters in chars instead.
^^



It strips any chars given in the argument, not as a prefix/suffix to remove.

Irmen

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: strip bug?

[Chris Kaynor]

On Tue, Feb 24, 2015 at 3:05 PM, bayk...@gmail.com wrote:

  'http://xthunder'.strip('http://')
 'xthunder'
  'http://thunder'.strip('http://')
 'under'
 

 I could understand backslash but forward slash?


I believe the issue is that str.strip does not do quite what you are
thinking it does, however your message is very unspecific about what you
expect to get. It seems that you except str.strip to remove a sub-string,
while it actually removes the list of specified characters (see
https://docs.python.org/3.4/library/stdtypes.html?highlight=str.strip#str.strip).
As such, you would get the same result from 'http://thunder'.strip('htp:/')
and 'http://thunder'.strip('/thp:') as from your samples.

To get what I am guessing you want (xthunder for the first and thunder
for the second), you should use splicing, likely combined with
str.startswith() (untested code):
url = 'http://thunder/
if url.startswith('http://'):
url = url[7:]
else
# Handle case without the prefix in whatever manner is correct for your
case. Possibly continue, possibly error out.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: strip bug?

[Eduardo]

Well, from the docstring of strip:

--
S.strip([chars]) - string or unicode

Return a copy of the string S with leading and trailing
whitespace removed.
If chars is given and not None, remove characters in chars instead.
If chars is unicode, S will be converted to unicode before stripping
--

In this case 'http://thunder'.strip('http://') is the same as 
'http://thunder'.strip('htp:/') and any character of the string that is passed 
to the method will be removed. 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: locale bug in Python 2.7, 3.3, 3.4 (Win7 64)?

[Mark Lawrence]

On 09/02/2015 15:43, Albert-Jan Roskam wrote:

Hi,

In the locale module we have:
* setlocale, the setter that also returns something
* getlocale, the getter that returns the OS-specific locale tuple (supposedly!)

* getdefaultlocale, the getter that always returns a unix locale tuple

Why are the getlocale() results below sometimes windows-like, sometimes 
unix-like?
It seems that I need to use setlocale(), with only the 'category' parameter, 
right?


ActivePython 3.3.2.0 (ActiveState Software Inc.) based on
Python 3.3.2 (default, Sep 16 2013, 23:11:39) [MSC v.1600 64 bit (AMD64)] on 
win32
Type help, copyright, credits or license for more information.

import locale
locale.getlocale()

(None, None)   # because setocale has not been called yet


# works as expected


locale.setlocale(locale.LC_ALL, )

'Dutch_Netherlands.1252'

locale.getlocale()

('Dutch_Netherlands', '1252')

# bug! locale.setlocale(locale.LC_ALL, german)
'German_Germany.1252'

locale.getlocale()

('de_DE', 'cp1252') # incorect, unix-like!



locale.setlocale(locale.LC_ALL, German_Germany.1252)

'German_Germany.1252'

locale.getlocale()

('de_DE', 'cp1252') # incorect, unix-like!


# bug!


locale.setlocale(locale.LC_ALL, spanish)

'Spanish_Spain.1252'

locale.getlocale()

('es_ES', 'cp1252')   # incorect, unix-like!


# works as expected

locale.setlocale(locale.LC_ALL, italian)

'Italian_Italy.1252'

locale.getlocale()

('Italian_Italy', '1252')# correct!


# ... maybe more?


Regards,

Albert-Jan



There have been loads of bug reports on the issue tracker about locales. 
 I suggest that you take a look to see if there is an open issue about 
this problem.




~~

All right, but apart from the sanitation, the medicine, education, wine, public 
order, irrigation, roads, a

fresh water system, and public health, what have the Romans ever done for us?

~~



Those flaming Romans didn't get the locales right did they? :)

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: locale bug in Python 2.7, 3.3, 3.4 (Win7 64)?

[Albert-Jan Roskam]

- Original Message -

 From: Mark Lawrence breamore...@yahoo.co.uk
 To: python-list@python.org
 Cc: 
 Sent: Monday, February 9, 2015 5:02 PM
 Subject: Re: locale bug in Python 2.7, 3.3, 3.4 (Win7 64)?
 
 On 09/02/2015 15:43, Albert-Jan Roskam wrote:
  Hi,
 
  In the locale module we have:
  * setlocale, the setter that also returns something
  * getlocale, the getter that returns the OS-specific locale tuple 
 (supposedly!)
 
  * getdefaultlocale, the getter that always returns a unix locale tuple
 
  Why are the getlocale() results below sometimes windows-like, sometimes 
 unix-like?
  It seems that I need to use setlocale(), with only the 'category' 
 parameter, right?
 
 
  ActivePython 3.3.2.0 (ActiveState Software Inc.) based on
  Python 3.3.2 (default, Sep 16 2013, 23:11:39) [MSC v.1600 64 bit (AMD64)] 
 on win32
  Type help, copyright, credits or 
 license for more information.
  import locale
  locale.getlocale()
  (None, None)   # because setocale has not been called yet
 
 
  # works as expected
 
  locale.setlocale(locale.LC_ALL, )
  'Dutch_Netherlands.1252'
  locale.getlocale()
  ('Dutch_Netherlands', '1252')
 
  # bug! locale.setlocale(locale.LC_ALL, german)
  'German_Germany.1252'
  locale.getlocale()
  ('de_DE', 'cp1252') # incorect, unix-like!
 
 
  locale.setlocale(locale.LC_ALL, 
 German_Germany.1252)
  'German_Germany.1252'
  locale.getlocale()
  ('de_DE', 'cp1252') # incorect, unix-like!
 
 
  # bug!
 
  locale.setlocale(locale.LC_ALL, spanish)
  'Spanish_Spain.1252'
  locale.getlocale()
  ('es_ES', 'cp1252')   # incorect, unix-like!
 
 
  # works as expected
  locale.setlocale(locale.LC_ALL, italian)
  'Italian_Italy.1252'
  locale.getlocale()
  ('Italian_Italy', '1252')# correct!
 
 
  # ... maybe more?
 
 
  Regards,
 
  Albert-Jan
 
 
 There have been loads of bug reports on the issue tracker about locales. 
   I suggest that you take a look to see if there is an open issue about 
 this problem.


Hi Mark -Thanks! I checked the bug tracker. Lots of locale-related bugs indeed. 
I couldn't find this one though. Strange because what I am doing does not seem 
exotic at all. French, German, Portuguese and Spanish getlocale results are 
incorrect IMHO. Below is little check + output here in case it might be useful 
for others. This is with Python 3.4, 3.3 and 2.7 on Windows 7 64.

from __future__ import print_function
import locale
import collections
import pprint


#https://msdn.microsoft.com/en-us/library/39cwe7zf%28v=vs.80%29.aspx
languages = (chinese czech danish dutch english finnish french german greek 
 hungarian icelandic italian japanese korean norwegian polish 
 portuguese russian slovak spanish swedish turkish)
d = collections.defaultdict(list)
t = collections.namedtuple(Locale, lang setlocale getlocale)
for language in languages.split():
sloc = locale.setlocale(locale.LC_ALL, language)
gloc = locale.getlocale()
record = t(language, sloc, gloc)
if gloc[0][2] == _:
d[unix-like].append(record)
else:
d[windows-like].append(record)
 
pprint.pprint(dict(d))


n:\C:\Miniconda3\python.exe N:\temp\loc.py
{'unix-like': [Locale(lang='french', setlocale='French_France.1252', 
getlocale=('fr_FR', 'cp1252')),
   Locale(lang='german', setlocale='German_Germany.1252', 
getlocale=('de_DE', 'cp1252')),
   Locale(lang='portuguese', setlocale='Portuguese_Brazil.1252', 
getlocale=('pt_BR', 'cp1252')),
   Locale(lang='spanish', setlocale='Spanish_Spain.1252', 
getlocale=('es_ES', 'cp1252'))],
 'windows-like': [Locale(lang='chinese', setlocale=Chinese 
(Simplified)_People's Republic of China.936, getlocale=(Chinese 
(Simplified)_People's Republic of China, '936')),
  Locale(lang='czech', setlocale='Czech_Czech Republic.1250', 
getlocale=('Czech_Czech Republic', '1250')),
  Locale(lang='danish', setlocale='Danish_Denmark.1252', 
getlocale=('Danish_Denmark', '1252')),
  Locale(lang='dutch', setlocale='Dutch_Netherlands.1252', 
getlocale=('Dutch_Netherlands', '1252')),
  Locale(lang='english', setlocale='English_United 
States.1252', getlocale=('English_United States', '1252')),
  Locale(lang='finnish', setlocale='Finnish_Finland.1252', 
getlocale=('Finnish_Finland', '1252')),
  Locale(lang='greek', setlocale='Greek_Greece.1253', 
getlocale=('Greek_Greece', '1253')),
  Locale(lang='hungarian', setlocale='Hungarian_Hungary.1250', 
getlocale=('Hungarian_Hungary', '1250')),
  Locale(lang='icelandic', setlocale='Icelandic_Iceland.1252', 
getlocale=('Icelandic_Iceland', '1252')),
  Locale(lang='italian', setlocale='Italian_Italy.1252', 
getlocale=('Italian_Italy', '1252')),
  Locale(lang='japanese', setlocale='Japanese_Japan.932', 
getlocale=('Japanese_Japan', '932')),
  Locale(lang='korean', setlocale

Re: A bug?

[Denis McMahon]

On Tue, 28 Oct 2014 01:29:28 +, Joshua Landau wrote:

 On 28 October 2014 00:36, Denis McMahon denismfmcma...@gmail.com
 wrote:

 d = [[list(range(1,13))[i*3+j] for j in range(3)] for i in range(4)]
 
 A quick note. Ranges (even 2.7's xrange) are all indexable. The cast to
 a list isn't needed.

Until you apply Chris' slicing trick, and then:

 [list(range(1,13))[i*3:i*3+3] for i in range(4)]
[[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]
 [range(1,13)[i*3:i*3+3] for i in range(4)]
[range(1, 4), range(4, 7), range(7, 10), range(10, 13)]
 

Depends how important it is that you get a list and not a range object ;)

(Thinking back to a recent discussion where someone was taking str(range
(x)) and not getting the expected results)

-- 
Denis McMahon, denismfmcma...@gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A bug?

[Wolfgang Maier]

On 10/27/2014 05:01 PM, uma...@gmail.com wrote:

I use python 3.4.0 version. In the course of developing / running a python 
program, I have encountered a problem. I have reproduced below a simple program 
to bring it out.



d = [[0]*3]*4
dd = [1,2,3,4,5,6,7,8,9,10,11,12]
for i in range(4):

...   for j in range(3): d[i][j] = dd[i*3+j]
...

d

[[10, 11, 12], [10, 11, 12], [10, 11, 12], [10, 11, 12]]



d is not transferred to dd as expected?
Of course I can use 'append'  do my job (less elegantly though).



See 
https://docs.python.org/3/library/stdtypes.html?highlight=list#common-sequence-operations 
under Note 2 .


Also asked and answered multiple times at stackoverflow, e.g., 
http://stackoverflow.com/questions/6688223/




--
https://mail.python.org/mailman/listinfo/python-list

Re: A bug?

[Ian Kelly]

On Mon, Oct 27, 2014 at 10:17 AM, Wolfgang Maier
wolfgang.ma...@biologie.uni-freiburg.de wrote:
 See
 https://docs.python.org/3/library/stdtypes.html?highlight=list#common-sequence-operations
 under Note 2 .

 Also asked and answered multiple times at stackoverflow, e.g.,
 http://stackoverflow.com/questions/6688223/

Also see 
https://docs.python.org/3/faq/programming.html#how-do-i-create-a-multidimensional-list
(probably a better resource than that footnote).
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A bug?

[Denis McMahon]

On Mon, 27 Oct 2014 09:01:57 -0700, uma...@gmail.com wrote:

 I use python 3.4.0 version. In the course of developing / running a
 python program, I have encountered a problem. I have reproduced below a
 simple program to bring it out.
  
  
 d = [[0]*3]*4 dd = [1,2,3,4,5,6,7,8,9,10,11,12]
 for i in range(4):
 ...   for j in range(3): d[i][j] = dd[i*3+j]
 ...
 d
 [[10, 11, 12], [10, 11, 12], [10, 11, 12], [10, 11, 12]]
 
 d is not transferred to dd as expected?
 Of course I can use 'append'  do my job (less elegantly though).

Not sure if this is elegant or not:

d = [[list(range(1,13))[i*3+j] for j in range(3)] for i in range(4)]

but it seems to be a one-line solution for what you're trying to do.

-- 
Denis McMahon, denismfmcma...@gmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A bug?

[Rustom Mody]

On Tuesday, October 28, 2014 6:07:14 AM UTC+5:30, Denis McMahon wrote:
 On Mon, 27 Oct 2014 09:01:57 -0700, umatrp wrote:
  I use python 3.4.0 version. In the course of developing / running a
  python program, I have encountered a problem. I have reproduced below a
  simple program to bring it out.
  d = [[0]*3]*4 dd = [1,2,3,4,5,6,7,8,9,10,11,12]
  for i in range(4):
  ...   for j in range(3): d[i][j] = dd[i*3+j]
  ...
  d
  [[10, 11, 12], [10, 11, 12], [10, 11, 12], [10, 11, 12]]
  d is not transferred to dd as expected?
  Of course I can use 'append'  do my job (less elegantly though).
 Not sure if this is elegant or not:
 d = [[list(range(1,13))[i*3+j] for j in range(3)] for i in range(4)]
 but it seems to be a one-line solution for what you're trying to do.

Neat

More generally for d being a 2-D reshape of dd (which may be anything
as long as the size matches)

 dd = [1,2,3,4,5,6,7,8,9,10,11,12]
 d=[[dd[i*3+j] for j in range(3)] for i in range(4)]
 d
[[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A bug?

[Chris Angelico]

On Tue, Oct 28, 2014 at 12:12 PM, Rustom Mody rustompm...@gmail.com wrote:
 More generally for d being a 2-D reshape of dd (which may be anything
 as long as the size matches)

 dd = [1,2,3,4,5,6,7,8,9,10,11,12]
 d=[[dd[i*3+j] for j in range(3)] for i in range(4)]
 d
 [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]

The inner comprehension should surely be a slice:

 [dd[i*3:i*3+3] for i in range(4)]
[[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A bug?

[Joshua Landau]

On 28 October 2014 00:36, Denis McMahon denismfmcma...@gmail.com wrote:

 d = [[list(range(1,13))[i*3+j] for j in range(3)] for i in range(4)]

A quick note. Ranges (even 2.7's xrange) are all indexable. The cast
to a list isn't needed.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A bug?

[Rustom Mody]

On Tuesday, October 28, 2014 6:56:10 AM UTC+5:30, Chris Angelico wrote:
 On Tue, Oct 28, 2014 at 12:12 PM, Rustom Mody  wrote:
  More generally for d being a 2-D reshape of dd (which may be anything
  as long as the size matches)
 
  dd = [1,2,3,4,5,6,7,8,9,10,11,12]
  d=[[dd[i*3+j] for j in range(3)] for i in range(4)]
  d
  [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]
 
 The inner comprehension should surely be a slice:
 
  [dd[i*3:i*3+3] for i in range(4)]
 [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]

Sweet!
[Something for my class today]

Looks even better this way:

 dd = range(1,13)
 [dd[i*3:i*3+3] for i in range(4)]
[range(1, 4), range(4, 7), range(7, 10), range(10, 13)]
 

In the same vein a transpose:
 [list(dd[i::3]) for i in range(3)]
[[0, 3, 6, 9], [1, 4, 7, 10], [2, 5, 8, 11]]

[Not sure why both are 3; no 4's...]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Parse bug text file

[CM]

Thank you, Chris, Terry, and jmf, for these pointers.  Very helpful.

-CM
-- 
https://mail.python.org/mailman/listinfo/python-list