Re: Share unpickleable object across processes

[dieter]

Israel Brewster  writes:

> I have a Flask/UWSGI web app that serves up web socket connections. When a 
> web socket connection is created, I want to store a reference to said web 
> socket so I can do things like write messages to every connected 
> socket/disconnect various sockets/etc. UWSGI, however, launches multiple 
> child processes which handle incoming connections, so the data structure that 
> stores the socket connections needs to be shared across all said processes. 
> How can I do this?

At low (= Posix) level, socket objects are represented as
ints, representing an "open communication object".

Usually, a child process shares the parent's "open communication object"s
*AT THE TINE OF THE CHILD CREATION*. However, often, this sharing
is broken very quickly: the parent may have set up those objects
to close automatically in the child; the parent may close its
"open communication object" after the spawning of the child.

If two processes have not inherited the same "open communication object"
from a common ancestor, there is no way that they can share
the communication object.


I am quite convinced that UWSGI will
be set up to close "open communication object"s passed on to
child processes. Thus, I see little chance that your current approach
can succeed.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: A use-case for for...else with no break

[Steve D'Aprano]

On Fri, 3 Nov 2017 04:22 pm, Paul Rubin wrote:

> Steve D'Aprano  writes:
>> for x in something():
>> print(x, end='')
> 
> print(''.join(something()))


I hoped that people would recognise a simplified, toy example used only to
illustrate a technique, rather than an exact copy and paste of something I'm
actually doing.

Try this instead:

for s in sequence:
if isinstance(s, bytes):
try:
 s = s.decode.('utf-8')
except UnicodeDecodeError:
 s = s.decode.('latin-1')
print(s, end='')
else:
print()


Without writing a helper function, got a fancy one liner for that too?

Or another example, sometimes you really don't want to wait until the entire
sequence is processed before printing any output.

count = 0
for obj in sequence:
if count == 60:
print(' ', time.asctime())
count = 0
print('.', end='')
expensive_time_consuming_process_that_might_take_minutes_or_hours(obj)
count += 1
else:
del count
print()
print("finished at", time.asctime())


Now honestly, do you think the more complex examples illustrate the point I
was making, and the usefulness of for...else, better than the simple version?

Because I think they make them more complicated and hard to understand, and
distract from the point I am making.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Alexey Muranov]

On Thu, 2017-11-02 at 16:31 +, Jon Ribbens wrote:

> On 2017-11-02, Steve D'Aprano  wrote:

> > On Fri, 3 Nov 2017 12:39 am, Jon Ribbens wrote:

> > > Why would we want to make the language worse? It is fairly
> > > obvious
> > > what 'else' means,

> >
> > Yes, obvious and WRONG.

>
> Nope, obvious and right.
>


I suppose that to continue this way we'd need at some point define 
somehow the meaning of "obvious."





> > > whereas 'then' has an obvious meaning that is in
> > > fact the opposite of what it would actually do.

> >
> > Er... is today opposite day? Because 'then' describes precisely
> > what it
> > actually does.

>
> No, 'then' describes the opposite of what it does. The word 'then'
> implies something that always happens next, whereas 'else' conveys
> the correct meaning, which is something that happens if the course
> of the preceding piece of code did not go as expected.
>


Jon, i get from this that for you, when there is no exception in `try`, 
or no `break` in a loop, the things did not go as expected.  Either we 
need to agree that what is "expected" is subjective, or agree on some 
kind of formal or informal common meaning for it, because i would not 
have put it this way.


'Then' describes what happens next indeed, unless some extraordinary 
situation prevents it from happening, for example:


   try:
   go_to_the_bakery()
   then:
   buy_croissants(2)
   except BakeryClosed:
   go_to_the_grociery()
   buy_baguette(1)
   finally:
   come_back()

I know this is a poor program example (why not to use a boolean return 
value instead of an exception, etc.), and i know that currently in 
Python `except` must precede `else`, it is just to illustrate the 
choice of terms.


Best regards,

Alexey.

--
https://mail.python.org/mailman/listinfo/python-list

Re: A use-case for for...else with no break

[Serhiy Storchaka]

02.11.17 12:10, Steve D'Aprano пише:

Occasionally it is useful to loop over a bunch of stuff in the interactive
interpreter, printing them as you go on a single line:

for x in something():
 print(x, end='')

If you do that, the prompt overwrites your output, and you get a mess:


py> for x in "abcdefgh":
... print(x, end='')
...
py> efghpy>


What the interpreter or configuration do you use? The standard 
interpreter uses '>>> ' as a prompt.


--
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Chris Angelico]

On Fri, Nov 3, 2017 at 8:48 PM, Alexey Muranov  wrote:
> 'Then' describes what happens next indeed, unless some extraordinary
> situation prevents it from happening, for example:
>
>try:
>go_to_the_bakery()
>then:
>buy_croissants(2)
>except BakeryClosed:
>go_to_the_grociery()
>buy_baguette(1)
>finally:
>come_back()
>
> I know this is a poor program example (why not to use a boolean return value
> instead of an exception, etc.), and i know that currently in Python `except`
> must precede `else`, it is just to illustrate the choice of terms.

What is the semantic difference between that code and the same without
the "then:"? The normal behaviour of both Python code and human
instruction sheets is to proceed to the next instruction if nothing
went wrong:

1. Go to the office.
2. Sit down at the desk.
3. Open the third drawer down at your right hand.
4. Take the secret instructions.
5. Read and follow the instructions.
-- if something goes wrong, take out your pistol and kill every zombie
you find --

If there's no desk in the office, you won't continue to the next
steps. But for the normal case, you don't have to say "then".

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Rhodri James]

On 02/11/17 20:24, Chris Angelico wrote:

Thank you. I've had this argument with many people, smart people (like
Steven), people who haven't grokked that all concurrency has costs -
that threads aren't magically more dangerous than other options.


I'm with Steven.  To be fair, the danger with threads is that most 
people don't understand thread-safety, and in particular don't 
understand either that they have a responsibility to ensure that shared 
data access is done properly or what the cost of that is.  I've seen far 
too much thread-based code over the years that would have been markedly 
less buggy and not much slower if it had been written sequentially.


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Jon Ribbens]

On 2017-11-03, Alexey Muranov  wrote:
> 'Then' describes what happens next indeed, unless some extraordinary 
> situation prevents it from happening, for example:
>
> try:
> go_to_the_bakery()
> then:
> buy_croissants(2)
> except BakeryClosed:
> go_to_the_grociery()
> buy_baguette(1)
> finally:
> come_back()
>
> I know this is a poor program example (why not to use a boolean return 
> value instead of an exception, etc.), and i know that currently in 
> Python `except` must precede `else`, it is just to illustrate the 
> choice of terms.

It looks like you're suggesting not just changing the 'else' keyword
to 'then', but changing the syntax completely. The above is certainly
not an improvement on the status quo, as it is completely
counter-intuitive that exceptions in the 'then' clause will not be
caught by the 'except' clauses.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Jon Ribbens]

On 2017-11-03, Steve D'Aprano  wrote:
> On Fri, 3 Nov 2017 03:31 am, Jon Ribbens wrote:
>> No, it's an obvious bug. You have a 'for...else' with no 'break'.
>> Like I said, that should probably be a syntax error.
>
> It should absolutely not be a syntax error. There's no reason for it
> to be a syntax error, except to satisfy some arrogant and foolish
> idea of purity.

It'd be nice if you could be a little less rude. It's not an "arrogant
and foolish idea of purity", it's to help people catch bugs in their
code, and to aid their understanding of the language.

> There are uncountable ways of writing code which is seemingly
> "pointless", and we don't make it a syntax error.

And there's uncountable ways of writing code which we *do* make a
syntax error. I'm not sure what your point is there.

>> No, 'then' describes the opposite of what it does. The word 'then'
>> implies something that always happens next, 
>
> Right, which is what happens with the for...else block.

No. Ok, so look. It's obvious that you and I have different mental
models of the situation here. You're thinking of 'for...else' as two
arbitrary clauses that run consecutively unless the whole thing is
aborted by a 'break', whereas I'm thinking of the 'for' clause as
being a search for a situation that matches a condition and the
'else' clause being what happens if the condition is not matched
(i.e. exactly the same as 'if...else').

Now there's nothing inherently *wrong* with your choice of mental
model, except it's leading you into confusion because my model means
the meaning of the 'else' keyword is intuitive and obvious, and yours
means it's counter-intuitive and confusing. Your suggestion is that
the fix is to change the language, my suggestion is to fix your model.
I'd suggest that changing your mind is easier than changing the
language ;-)

Also, my model has the advantage that if what the 'for' clause is
doing doesn't match the concept of 'searching for a match' then it's
obvious that you shouldn't be using 'for...else' in the first place.
Again, sure, the language doesn't currently strictly enforce the
requirement for a 'break', but nevertheless if you don't have one
you almost certainly have a bug.

>> Yes, I saw that. It's possible you are the only person in the world
>> ever to have done that. It would not make the interactive interpreter
>> 'worse' in the slightest for that silly trick to be lost.
>
> Just because you personally haven't used this technique doesn't make it
> a "silly trick".

It's an incredibly obscure work-around for a different problem,
i.e. an infelicity in the way the interactive prompt parses input
blocks. If the parsing is a genuine issue then the answer is to
fix that, not to look for hacks that almost never help anyway.

> And while I thank you for the complement that I am the cleverest and most
> insightful Python coder in the world,

I didn't say anything even remotely resembling that.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[bartc]

On 03/11/2017 11:49, Jon Ribbens wrote:

On 2017-11-03, Steve D'Aprano  wrote:



Right, which is what happens with the for...else block.


No. Ok, so look. It's obvious that you and I have different mental
models of the situation here. You're thinking of 'for...else' as two
arbitrary clauses that run consecutively unless the whole thing is
aborted by a 'break', whereas I'm thinking of the 'for' clause as
being a search for a situation that matches a condition and the
'else' clause being what happens if the condition is not matched
(i.e. exactly the same as 'if...else').

Now there's nothing inherently *wrong* with your choice of mental
model, except it's leading you into confusion because my model means
the meaning of the 'else' keyword is intuitive and obvious, and yours
means it's counter-intuitive and confusing. Your suggestion is that
the fix is to change the language, my suggestion is to fix your model.
I'd suggest that changing your mind is easier than changing the
language ;-)


I don't think there will be a short keyword that will suit everyone. It 
may be necessary to use this to spell out exactly how it works:


  for i in r:
 pass ...
  after_normal_loop_termination_then:
 pass ...

(And now, there is the possibility of having an additional 'else' clause 
to cover abnormal termination via break. This time what 'else' does is 
more obvious.)


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: A use-case for for...else with no break

[Steve D'Aprano]

On Fri, 3 Nov 2017 09:13 pm, Serhiy Storchaka wrote:

> What the interpreter or configuration do you use? The standard
> interpreter uses '>>> ' as a prompt.

I have this in my Python startup file:

if (sys.version_info[0] >= 3 and os.name == 'posix'
and os.environ['TERM'] in ['xterm', 'vt100']):
# Make the prompt bold in Python 3.
sys.ps1 = '\001\x1b[1m\002py> \001\x1b[0m\002'
sys.ps2 = '\001\x1b[1m\002... \001\x1b[0m\002'
else:
sys.ps1 = 'py> '



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Chris Angelico]

On Fri, Nov 3, 2017 at 10:49 PM, Jon Ribbens  wrote:
> On 2017-11-03, Steve D'Aprano  wrote:
>> On Fri, 3 Nov 2017 03:31 am, Jon Ribbens wrote:
>>> No, it's an obvious bug. You have a 'for...else' with no 'break'.
>>> Like I said, that should probably be a syntax error.
>>
>> It should absolutely not be a syntax error. There's no reason for it
>> to be a syntax error, except to satisfy some arrogant and foolish
>> idea of purity.
>
> It'd be nice if you could be a little less rude. It's not an "arrogant
> and foolish idea of purity", it's to help people catch bugs in their
> code, and to aid their understanding of the language.

That wasn't rudeness. That was an accurate description of the
situation. It is entirely possible for code to be opinionated,
arrogant, foolish, and all those other adjectives that we normally
appropriate to people.

I don't think it would do much for comprehension. And if it's to catch
bugs, that is NOT the job of syntax - it's the job of linters. By all
means, have a linter that says "else clause after loop with no break";
but don't make it *illegal*.

>> There are uncountable ways of writing code which is seemingly
>> "pointless", and we don't make it a syntax error.
>
> And there's uncountable ways of writing code which we *do* make a
> syntax error. I'm not sure what your point is there.

The point is that syntax errors are for things that cannot possibly
make sense, not for things that are likely to be bugs.

There are a very few exceptions, and they're usually things that are
massive bug magnets, or where the semantics are subtly different. And
I can't think of any examples right now other than the way "async def"
forces a function to be a coroutine even if it has no "await"s in it;
note that it does *not* make a syntax error, it simply causes the
semantics to be correct anyway.

>>> No, 'then' describes the opposite of what it does. The word 'then'
>>> implies something that always happens next,
>>
>> Right, which is what happens with the for...else block.
>
> No. Ok, so look. It's obvious that you and I have different mental
> models of the situation here. You're thinking of 'for...else' as two
> arbitrary clauses that run consecutively unless the whole thing is
> aborted by a 'break', whereas I'm thinking of the 'for' clause as
> being a search for a situation that matches a condition and the
> 'else' clause being what happens if the condition is not matched
> (i.e. exactly the same as 'if...else').
>
> Now there's nothing inherently *wrong* with your choice of mental
> model, except it's leading you into confusion because my model means
> the meaning of the 'else' keyword is intuitive and obvious, and yours
> means it's counter-intuitive and confusing. Your suggestion is that
> the fix is to change the language, my suggestion is to fix your model.
> I'd suggest that changing your mind is easier than changing the
> language ;-)

If anything, I would say that Steven's model is at a lower abstraction
layer than yours - though IMO your model is more of an example
use-case than a description of what is actually happening.

>>> Yes, I saw that. It's possible you are the only person in the world
>>> ever to have done that. It would not make the interactive interpreter
>>> 'worse' in the slightest for that silly trick to be lost.
>>
>> Just because you personally haven't used this technique doesn't make it
>> a "silly trick".
>
> It's an incredibly obscure work-around for a different problem,
> i.e. an infelicity in the way the interactive prompt parses input
> blocks. If the parsing is a genuine issue then the answer is to
> fix that, not to look for hacks that almost never help anyway.

TBH I prefer the "if 1:" trick to gather code into a block. But that
requires pre-planning, whereas slapping an "else:" after the loop can
be done after the event.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

python3 byte decode

[Ali Rıza KELEŞ]

Hi,

Yesterday, while working with redis, i encountered a strange case.

I want to ask why is the following `True`

```
"s" is b"s".decode()
```

while the followings are `False`?

```
"so" is b"so".decode()
"som" is b"som".decode()
"some" is b"some".decode()
```

Or vice versa?

I read that `is` compares same objects, not values. So my question is
why "s" and b"s".decode() are same objects, while the others aren't?

My python version is 3.6.3.

Thanks.

-- 
--
Ali Rıza Keleş
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python3 byte decode

[Chris Angelico]

On Fri, Nov 3, 2017 at 8:24 PM, Ali Rıza KELEŞ  wrote:
> Hi,
>
> Yesterday, while working with redis, i encountered a strange case.
>
> I want to ask why is the following `True`
>
> ```
> "s" is b"s".decode()
> ```
>
> while the followings are `False`?
>
> ```
> "so" is b"so".decode()
> "som" is b"som".decode()
> "some" is b"some".decode()
> ```
>
> Or vice versa?
>
> I read that `is` compares same objects, not values. So my question is
> why "s" and b"s".decode() are same objects, while the others aren't?
>
> My python version is 3.6.3.

You shouldn't be comparing string objects with 'is'. Sometimes two
equal strings will be identical, and sometimes they won't. All you're
seeing is that the interpreter happened to notice and/or cache this
particular lookup.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Chris Angelico]

On Fri, Nov 3, 2017 at 10:26 PM, Rhodri James  wrote:
> On 02/11/17 20:24, Chris Angelico wrote:
>>
>> Thank you. I've had this argument with many people, smart people (like
>> Steven), people who haven't grokked that all concurrency has costs -
>> that threads aren't magically more dangerous than other options.
>
>
> I'm with Steven.  To be fair, the danger with threads is that most people
> don't understand thread-safety, and in particular don't understand either
> that they have a responsibility to ensure that shared data access is done
> properly or what the cost of that is.  I've seen far too much thread-based
> code over the years that would have been markedly less buggy and not much
> slower if it had been written sequentially.

Yes, but what you're seeing is that *concurrent* code is more
complicated than *sequential* code. Would the code in question have
been less buggy if it had used multiprocessing instead of
multithreading? What if it used explicit yield points? Multiprocessing
brings with it a whole lot of extra complications around moving data
around. Multithreading brings with it a whole lot of extra
complications around NOT moving data around. Yield points bring with
them the risk of locking another thread out unexpectedly (particularly
since certain system calls aren't async-friendly on certain OSes). All
three models have their pitfalls. It's not that threads are somehow
worse than every other model.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Rhodri James]

On 03/11/17 14:50, Chris Angelico wrote:

On Fri, Nov 3, 2017 at 10:26 PM, Rhodri James  wrote:

On 02/11/17 20:24, Chris Angelico wrote:


Thank you. I've had this argument with many people, smart people (like
Steven), people who haven't grokked that all concurrency has costs -
that threads aren't magically more dangerous than other options.



I'm with Steven.  To be fair, the danger with threads is that most people
don't understand thread-safety, and in particular don't understand either
that they have a responsibility to ensure that shared data access is done
properly or what the cost of that is.  I've seen far too much thread-based
code over the years that would have been markedly less buggy and not much
slower if it had been written sequentially.


Yes, but what you're seeing is that *concurrent* code is more
complicated than *sequential* code. Would the code in question have
been less buggy if it had used multiprocessing instead of
multithreading? What if it used explicit yield points?


My experience with situations where I can do a reasonable comparison is 
limited, but the answer appears to be "Yes".

 Multiprocessing

brings with it a whole lot of extra complications around moving data
around.


People generally understand how to move data around, and the mistakes 
are usually pretty obvious when they happen.  People may not understand 
how to move data around efficiently, but that's a separate argument.


 Multithreading brings with it a whole lot of extra

complications around NOT moving data around.


I think this involves more subtle bugs that are harder to spot.  People 
seem to find it harder to reason about atomicity and realising that 
widely separated pieces of code may interact unexpectedly.


 Yield points bring with

them the risk of locking another thread out unexpectedly (particularly
since certain system calls aren't async-friendly on certain OSes).


I've got to admit I find coroutines straightforward, but I did cut my 
teeth on a cooperative OS.  It certainly makes the atomicity issues 
easier to deal with.


 All

three models have their pitfalls.


Assuredly.  I just think threads are soggier and hard to light^W^W^W^W^W 
prone to subtler and more mysterious-looking bugs.


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Steve D'Aprano]

On Sat, 4 Nov 2017 01:50 am, Chris Angelico wrote:

> On Fri, Nov 3, 2017 at 10:26 PM, Rhodri James  wrote:

>> I'm with Steven.  To be fair, the danger with threads is that most people
>> don't understand thread-safety, and in particular don't understand either
>> that they have a responsibility to ensure that shared data access is done
>> properly or what the cost of that is.  I've seen far too much thread-based
>> code over the years that would have been markedly less buggy and not much
>> slower if it had been written sequentially.
> 
> Yes, but what you're seeing is that *concurrent* code is more
> complicated than *sequential* code. Would the code in question have
> been less buggy if it had used multiprocessing instead of
> multithreading? 

Maybe.

There's no way to be sure unless you actually compare a threading
implementation with a processing implementation -- and they have to
be "equally good, for the style" implementations. No fair comparing the
multiprocessing equivalent of "Stooge Sort" with the threading equivalent
of "Quick Sort", and concluding that threading is better.

However, we can predict the likelihood of which will be less buggy by
reasoning in general principles. And the general principle is that shared
data tends, all else being equal, to lead to more bugs than no shared data.
The more data is shared, the more bugs, more or less.

I don't know if there are any hard scientific studies on this, but experience
and anecdote strongly suggests it is true. Programming is not yet fully
evidence-based.

For example, most of us accept "global variables considered harmful". With few
exceptions, the use of application-wide global variables to communicate
between functions is harmful and leads to problems. This isn't because of any
sort of mystical or magical malignity from global variables. It is because
the use of global variables adds coupling between otherwise distant parts of
the code, and that adds complexity, and the more complex code is, the more
likely we mere humans are to screw it up.

So, all else being equal, which is likely to have more bugs?


1. Multiprocessing code with very little coupling between processes; or

2. Threaded code with shared data and hence higher coupling between threads?


Obviously the *best* threaded code will have fewer bugs than the *worst*
multiprocessing code, but my heuristic is that, in general, the average
application using threading is likely to be more highly coupled, hence more
complicated, than the equivalent using multiprocessing.

(Async is too new, and to me, too confusing, for me to have an opinion on yet,
but I lean slightly towards the position that deterministic task-switching is
probably better than non-deterministic.)



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Alexey Muranov]

On Fri, 2017-11-03 at 22:03 +1100, Chris Angelico wrote:
> On Fri, Nov 3, 2017 at 8:48 PM, Alexey Muranov  com> wrote:
> > 'Then' describes what happens next indeed, unless some
> > extraordinary
> > situation prevents it from happening, for example:
> >
> > try:
> > go_to_the_bakery()
> > then:
> > buy_croissants(2)
> > except BakeryClosed:
> > go_to_the_grociery()
> > buy_baguette(1)
> > finally:
> > come_back()
> >
> > I know this is a poor program example (why not to use a boolean
> > return value
> > instead of an exception, etc.), and i know that currently in Python
> > `except`
> > must precede `else`, it is just to illustrate the choice of terms.
>
> What is the semantic difference between that code and the same
> without the "then:"?

Chris,

the semantic difference is that their meanings/behaviours are not 
identical (i imply that `then` here does what `else` currently does).  
I agree however that from practical viewpoint the difference will 
probably never be observable (unless the person enters the bakery and 
asks for croissants, but during this time the baker exits the bakery 
and closes it to go on vacation).


I can try to think of a better example if you give me any good use-case 
for `else` in `try`.  I have searched on-line, and on StackOverflow in 
particular, but didn't find anything better that this:


 * https://stackoverflow.com/a/6051978

People seem very shy when it comes to giving a real-life example of 
`else` in `try`.


Alexey.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Chris Angelico]

On Sat, Nov 4, 2017 at 2:45 AM, Steve D'Aprano
 wrote:
> So, all else being equal, which is likely to have more bugs?
>
>
> 1. Multiprocessing code with very little coupling between processes; or
>
> 2. Threaded code with shared data and hence higher coupling between threads?
>

Obviously, option 1. But that's "all else being equal". How often can
you actually have your processes that decoupled? And if you can write
your code to be completely (or largely) decoupled, what's to stop you
having your *threads* equally decoupled? You're assuming that "running
in the same memoryspace" equates to "higher coupling", which is no
more proven than any other assertion. Ultimately, real-world code IS
going to have some measure of coupling (you could concoct a scenario
in which requests are handled 100% independent of each other, but even
with a web application, there's going to be SOME connection between
different requests), so all you do is move it around (eg in a web app
scenario, the most common solution is to do all coupling through a
database or equivalent).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Chris Angelico]

On Sat, Nov 4, 2017 at 3:15 AM, Alexey Muranov  wrote:
> On Fri, 2017-11-03 at 22:03 +1100, Chris Angelico wrote:
>> On Fri, Nov 3, 2017 at 8:48 PM, Alexey Muranov > com> wrote:
>> > 'Then' describes what happens next indeed, unless some
>> > extraordinary
>> > situation prevents it from happening, for example:
>> >
>> > try:
>> > go_to_the_bakery()
>> > then:
>> > buy_croissants(2)
>> > except BakeryClosed:
>> > go_to_the_grociery()
>> > buy_baguette(1)
>> > finally:
>> > come_back()
>> >
>> > I know this is a poor program example (why not to use a boolean
>> > return value
>> > instead of an exception, etc.), and i know that currently in Python
>> > `except`
>> > must precede `else`, it is just to illustrate the choice of terms.
>>
>> What is the semantic difference between that code and the same
>> without the "then:"?
>
> Chris,
>
> the semantic difference is that their meanings/behaviours are not identical
> (i imply that `then` here does what `else` currently does).  I agree however
> that from practical viewpoint the difference will probably never be
> observable (unless the person enters the bakery and asks for croissants, but
> during this time the baker exits the bakery and closes it to go on
> vacation).

Okay, so what you're doing is taking the current "try/else" semantics,
and hoisting the "happy path" up to immediately underneath the part
guarded by the try. That's not unreasonable, but also not a huge
advantage. It's basically saying:

try:
do_stuff
except None: # let's pretend
print("all is well")
except ValueError:
print("do_stuff has the wrong value")
finally:
print("we're done")

But if the difference between this and the current layout (with "else"
coming *after* the except blocks) is significant to you, I would
recommend refactoring the try/finally into a context manager:

with car_trip() as trip:
try:
trip.goto(bakery)
buy_croissants(2)
except BakeryClosed:
trip.goto(grocery)
buy_baguette(1)

The context manager takes care of bringing us home unconditionally,
leaving us with cleaner logic.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Grant Edwards]

On 2017-11-03, Chris Angelico  wrote:
> On Sat, Nov 4, 2017 at 2:45 AM, Steve D'Aprano
> wrote:
>> So, all else being equal, which is likely to have more bugs?
>>
>>
>> 1. Multiprocessing code with very little coupling between processes; or
>>
>> 2. Threaded code with shared data and hence higher coupling between threads?
>>
>
> Obviously, option 1. But that's "all else being equal". How often can
> you actually have your processes that decoupled? And if you can write
> your code to be completely (or largely) decoupled, what's to stop you
> having your *threads* equally decoupled? You're assuming that "running
> in the same memoryspace" equates to "higher coupling", which is no
> more proven than any other assertion.

The big difference is that with threaded code you can have accidental
coupling.  With multiprocessing, code you have to explicitly work to
create coupling.

That said, I do a lot of threading coding (in both Python and C), and
have never found it particularly tricky.

It does require that you understand what you're doing and probably
doesn't work well if you're a stack-overflow, cargo-cult,
cut-and-paste programmer.  But then again, what does?

-- 
Grant Edwards   grant.b.edwardsYow! Maybe I should have
  at   asked for my Neutron Bomb
  gmail.comin PAISLEY --

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Ian Kelly]

On Thu, Nov 2, 2017 at 10:27 AM, Israel Brewster  wrote:
>
>> On Nov 1, 2017, at 4:53 PM, Steve D'Aprano  
>> wrote:
>>
>> On Thu, 2 Nov 2017 05:53 am, Israel Brewster wrote:
>>
>> [...]
>>> So the end result is that the thread that "updates" the dictionary, and the
>>> thread that initially *populates* the dictionary are actually running in
>>> different processes.
>>
>> If they are in different processes, that would explain why the second
>> (non)thread sees an empty dict even after the first thread has populated it:
>>
>>
>> # from your previous post
>>> Length at get AC:  54 ID: 4524152200  Time: 2017-11-01 09:41:24.474788
>>> Length At update:  1 ID: 4524152200  Time: 2017-11-01 09:41:24.784399
>>> Length At update:  2 ID: 4524152200  Time: 2017-11-01 09:41:25.228853
>>
>>
>> You cannot rely on IDs being unique across different processes. Its an
>> unfortunately coincidence(!) that they end up with the same ID.
>
> I think it's more than a coincidence, given that it is 100% reproducible. 
> Plus, in an earlier debug test I was calling print() on the defaultdict 
> object, which gives output like "", where 
> presumably the 0x1066467f0 is a memory address (correct me if I am wrong in 
> that). In every case, that address was the same. So still a bit puzzling.

If the empty dict is created before the process is forked then I don't
think it's all that surprising.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Steve D'Aprano]

On Fri, 3 Nov 2017 10:49 pm, Jon Ribbens wrote:

> On 2017-11-03, Steve D'Aprano  wrote:
>> On Fri, 3 Nov 2017 03:31 am, Jon Ribbens wrote:
>>> No, it's an obvious bug. You have a 'for...else' with no 'break'.
>>> Like I said, that should probably be a syntax error.
>>
>> It should absolutely not be a syntax error. There's no reason for it
>> to be a syntax error, except to satisfy some arrogant and foolish
>> idea of purity.
> 
> It'd be nice if you could be a little less rude. 

How else should we respond to the idea that something that has been legal
syntax for over twenty years and is in no way an error should be treated as a
syntax error, just to satisfy the opinion that nobody should write for...else
unless they are performing a search and it contains a break?


>> There are uncountable ways of writing code which is seemingly
>> "pointless", and we don't make it a syntax error.
> 
> And there's uncountable ways of writing code which we *do* make a
> syntax error. I'm not sure what your point is there.

None of these are syntax errors:

if False: do_something()

for x in '': pass

if True: pass

and uncountable other pieces of code which are, from some perspective,
pointless. We don't make them syntax errors because *pointless* doesn't imply
is is an error.

You think that a for...else loop with no break is pointless and "silly", even
when I demonstrate an actual use for it. Too bad. Even if it is "silly", in
your opinion, that still does not make it an error.

Is my point clear now?


>>> No, 'then' describes the opposite of what it does. The word 'then'
>>> implies something that always happens next,
>>
>> Right, which is what happens with the for...else block.
> 
> No.

Yes. It is not a matter of opinion that the `else` clause executes after
execution falls out the bottom of the loop, i.e. the loop runs, THEN the
`else` block runs.

It is an objective fact that in the absence of something which causes
execution to jump outside of the loop altogether (a return, a raise, or a
break) the execution order is loop first, then `else` block.

Both of these two sentences makes sense in English:

"Execution reaches the end of the loop, and THEN the `else` block runs."

"Execution reaches the end of the loop, or ELSE the `else` block runs."

but only the first describes what Python's for...else statement actually does.

This is not just my mental model, it is the actual objective behaviour of the
for...else statement.


> Ok, so look. It's obvious that you and I have different mental 
> models of the situation here. You're thinking of 'for...else' as two
> arbitrary clauses that run consecutively 

Scrub out the "arbitrary", and that is exactly what they are. They aren't
arbitrary: the `else` is optional, but if present it must follow after the
loop, and never the other way around.

The code inside the blocks can, of course, be any legal code we like. There's
no rule that says "only code approved by Jon Ribbens' is allowed", and
there's no rule that says "only searches are allowed".


> unless the whole thing is aborted by a 'break', 

Or return, raise, os._exit, os.abort, or having the process killed by the OS.
For simplicity we can ignore the last three (as well as more exotic
mechanisms such as "pull the power supply out" or "drop a two-tonne weight on
the computer"), but return and raise are common ways of exiting a loop and
should not be ignored or forgotten.


> whereas I'm thinking of the 'for' clause as 
> being a search for a situation that matches a condition and the
> 'else' clause being what happens if the condition is not matched
> (i.e. exactly the same as 'if...else').

That model conflates one specific use-case for the for...else statement with
the semantics of the statement, rather like insisting that "print displays a
greeting to the world" merely because 'print("Hello, World")' is so common.

The for loop does not necessarily perform a search:


count = 1
for obj in sequence:
if count > MAX_OBJECTS:
print("too many objects, halting")
break
process(obj)
else:
print("finished")


According to your mental model, this code is... what? Illegal? Silly?
Impossible? A syntax error?

Obviously wrong and almost certainly buggy?

Even when there is a search, the sense of the search may be reversed from your
mental model:


for request in sequence:
if 'please' not in request:
break
process(request)
else:
print('done')
return
raise SysExit('failure to say please')


It doesn't matter if you can think of alternative ways of writing these code
snippets. They're legal, and they work, and your mental model doesn't cover
them, at all or easily. That means your mental model is at best incomplete.

This is what happens when you treat one use-case for a general purpose
statement as synonymous with the statement itself. `for...else` is not just
for searching, and the `else` clause doesn't necessarily run if the search is
unsuccessful.

You've trapped yourself i

Re: replacing `else` with `then` in `for` and `try`

[Jon Ribbens]

On 2017-11-03, Chris Angelico  wrote:
> On Fri, Nov 3, 2017 at 10:49 PM, Jon Ribbens  
> wrote:
>>> It should absolutely not be a syntax error. There's no reason for it
>>> to be a syntax error, except to satisfy some arrogant and foolish
>>> idea of purity.
>>
>> It'd be nice if you could be a little less rude. It's not an "arrogant
>> and foolish idea of purity", it's to help people catch bugs in their
>> code, and to aid their understanding of the language.
>
> That wasn't rudeness.

Yes, it was.

>> No. Ok, so look. It's obvious that you and I have different mental
>> models of the situation here. You're thinking of 'for...else' as two
>> arbitrary clauses that run consecutively unless the whole thing is
>> aborted by a 'break', whereas I'm thinking of the 'for' clause as
>> being a search for a situation that matches a condition and the
>> 'else' clause being what happens if the condition is not matched
>> (i.e. exactly the same as 'if...else').
>>
>> Now there's nothing inherently *wrong* with your choice of mental
>> model, except it's leading you into confusion because my model means
>> the meaning of the 'else' keyword is intuitive and obvious, and yours
>> means it's counter-intuitive and confusing. Your suggestion is that
>> the fix is to change the language, my suggestion is to fix your model.
>> I'd suggest that changing your mind is easier than changing the
>> language ;-)
>
> If anything, I would say that Steven's model is at a lower abstraction
> layer than yours

Yes, absolutely.

> - though IMO your model is more of an example use-case than a
> description of what is actually happening.

It's a high-level way of thinking about it that avoids confusion and
leads to correct code. Unless you can come up with a *sensible*
use-case that it doesn't cover, it's also a comprehensive way of
thinking about it.

> TBH I prefer the "if 1:" trick to gather code into a block. But that
> requires pre-planning,

Or pressing up-arrow ;-)

> whereas slapping an "else:" after the loop can be done after the event.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Gene Heskett]

On Friday 03 November 2017 10:50:13 Chris Angelico wrote:

> On Fri, Nov 3, 2017 at 10:26 PM, Rhodri James  
wrote:
> > On 02/11/17 20:24, Chris Angelico wrote:
> >> Thank you. I've had this argument with many people, smart people
> >> (like Steven), people who haven't grokked that all concurrency has
> >> costs - that threads aren't magically more dangerous than other
> >> options.
> >
> > I'm with Steven.  To be fair, the danger with threads is that most
> > people don't understand thread-safety, and in particular don't
> > understand either that they have a responsibility to ensure that
> > shared data access is done properly or what the cost of that is. 
> > I've seen far too much thread-based code over the years that would
> > have been markedly less buggy and not much slower if it had been
> > written sequentially.
>
> Yes, but what you're seeing is that *concurrent* code is more
> complicated than *sequential* code. Would the code in question have
> been less buggy if it had used multiprocessing instead of
> multithreading? What if it used explicit yield points? Multiprocessing
> brings with it a whole lot of extra complications around moving data
> around. Multithreading brings with it a whole lot of extra
> complications around NOT moving data around. Yield points bring with
> them the risk of locking another thread out unexpectedly (particularly
> since certain system calls aren't async-friendly on certain OSes). All
> three models have their pitfalls. It's not that threads are somehow
> worse than every other model.
>
> ChrisA

I think that this discussion of threads and threading must be a different 
context than threading as I am using it in linuxcnc.

There, one assigns a function to run in a certain sequence in an assigned 
thread, which there can be several of. There, each thread is assigned a 
repetition rate, and the higher repetition rate stuff can always 
interrupt the slower threaded function in order to get the real time 
stuff done in a usable for the job time frame, and the high frequency 
thread can be as fast as every 25 microseconds on a good motherboard.  
Surprisingly, this seems to be far more board dependent than processor 
dependent, altho its pushing an AMD processor quite hard at 40 
microseconds, while the slower intel atoms can do 25 microseconds with 
about the same effort.

This is where a stepper motor is being stepped by software which didles 
pins on a parport. And it limits how fast you can move the motor 
compared to using an fpga card running at 200 MHz, not because of the 
step rate, but because of the latency of the mobo/cpu combination.

Jitter in step rate issuance is death to high performance with stepper 
motors because torque to do usefull work vanishes when the instant speed 
of the motor is wobbling with that timing jitter.

OTOH, hand controls of a machine using an encoder dial are nicely done in 
a thread running at 100 Hz, far more reliably that I can do it from the 
keyboard on a raspberry pi 3b. Why? The dials data goes into linuxcnc by 
way of a hardware fpga card that talks to the pi over an SPI buss, with 
the pi writing 32 bit packets at 41 megabaud, and reading the results at 
25 megabaud. It doesn't have to get thru that usb2 internal hub in the 
pi that all the other i/o has to go thru. Mouse and keyboard events get 
dropped on the floor, particularly dangerous when its a keyup event that 
gets dropped. The machine keeps moving until it crashes into something, 
often breaking drive parts or cutting tooling, all of which cost real 
money.

My point is that with an interpretor such as hal managing things, threads 
Just Work(TM).

It does of course take a specially built kernel to do that magic.
I'll get me coat.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Jon Ribbens]

On 2017-11-03, Steve D'Aprano  wrote:
> The for loop does not necessarily perform a search:
>
> count = 1
> for obj in sequence:
> if count > MAX_OBJECTS:
> print("too many objects, halting")
> break
> process(obj)
> else:
> print("finished")
>
> According to your mental model, this code is... what? Illegal? Silly?
> Impossible? A syntax error?

That conforms to my model. It's searching for the condition
'count > MAX_OBJECTS'.

> Even when there is a search, the sense of the search may be reversed
> from your mental model:
>
> for request in sequence:
> if 'please' not in request:
> break
> process(request)
> else:
> print('done')
> return
> raise SysExit('failure to say please')

That's searching for the condition "'please' not in request".

> It doesn't matter if you can think of alternative ways of writing these code
> snippets. They're legal, and they work, and your mental model doesn't cover
> them, at all or easily. That means your mental model is at best incomplete.

You've yet to come up with one that it doesn't cover.

> In other words, it was only when I realised that `else` would be better
> written as `then` that I understood the behaviour of the statement. If I read
> it as "else", then the natural interpretation is that the block runs only if
> the loop doesn't run:

You're repeating yourself.

> You'll probably dismiss them as "silly" because they don't meet your mental
> model, and insist that I would be wrong to use them.

As above, you're mistaken.

> Your response to code that doesn't match your mental model is to say
> that it is obviously wrong and probably buggy and should be made
> into a syntax error if possible.

No, it isn't. Try reading again what I actually wrote.

> And my mental model is to treat "else" in this concept as some foreign word,
> perhaps Dutch, a false-friend that actually means "next" but due to some
> awful coincidence happens to look exactly like the English word for "else".

And that's leading you into confusion, as you've demonstrated.

> Ah yes, because it is inconceivable that anyone might have thought of a use
> for for...else without a break.

It's not inconceivable, but nobody has thought of a sensible use so far
(by which I mean one that shows it's a useful feature).

> What matters is that only code that matches your model is allowed.

You do seem to enjoy telling other people what their opinions are.
Frankly, I don't know why I even bother having opinions when you're
here to tell me what they are. In future, I'll just come to you to
find out what I think.

>> It's an incredibly obscure work-around for a different problem,
>
> You mean a different problem to "searching"? Yes indeed it is.

No, I mean a problem that has nothing to do with 'for...else' clauses.
If your door is stuck then you might climb in through a window, but
that doesn't mean that the purpose of windows is to be entrances,
or that windows should be designed to make them better entrances,
or that the problem isn't the stuck door.

>>> And while I thank you for the complement that I am the cleverest and most
>>> insightful Python coder in the world,
>> 
>> I didn't say anything even remotely resembling that.
>
> That was sarcasm.

Yes dear.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Ian Kelly]

On Fri, Nov 3, 2017 at 8:32 AM, Chris Angelico  wrote:
> On Fri, Nov 3, 2017 at 10:49 PM, Jon Ribbens  
> wrote:
>> On 2017-11-03, Steve D'Aprano  wrote:
>>> On Fri, 3 Nov 2017 03:31 am, Jon Ribbens wrote:
 No, it's an obvious bug. You have a 'for...else' with no 'break'.
 Like I said, that should probably be a syntax error.
>>>
>>> It should absolutely not be a syntax error. There's no reason for it
>>> to be a syntax error, except to satisfy some arrogant and foolish
>>> idea of purity.
>>
>> It'd be nice if you could be a little less rude. It's not an "arrogant
>> and foolish idea of purity", it's to help people catch bugs in their
>> code, and to aid their understanding of the language.
>
> That wasn't rudeness. That was an accurate description of the
> situation. It is entirely possible for code to be opinionated,
> arrogant, foolish, and all those other adjectives that we normally
> appropriate to people.

Steve was clearly referring to the coder, not the code. Please stop
defending the use of incivility on this list.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Israel Brewster]

---
Israel Brewster
Systems Analyst II
Ravn Alaska
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
---




> On Nov 3, 2017, at 7:11 AM, Rhodri James  wrote:
> 
> On 03/11/17 14:50, Chris Angelico wrote:
>> On Fri, Nov 3, 2017 at 10:26 PM, Rhodri James  wrote:
>>> On 02/11/17 20:24, Chris Angelico wrote:
 
 Thank you. I've had this argument with many people, smart people (like
 Steven), people who haven't grokked that all concurrency has costs -
 that threads aren't magically more dangerous than other options.
>>> 
>>> 
>>> I'm with Steven.  To be fair, the danger with threads is that most people
>>> don't understand thread-safety, and in particular don't understand either
>>> that they have a responsibility to ensure that shared data access is done
>>> properly or what the cost of that is.  I've seen far too much thread-based
>>> code over the years that would have been markedly less buggy and not much
>>> slower if it had been written sequentially.
>> Yes, but what you're seeing is that *concurrent* code is more
>> complicated than *sequential* code. Would the code in question have
>> been less buggy if it had used multiprocessing instead of
>> multithreading? What if it used explicit yield points?
> 
> My experience with situations where I can do a reasonable comparison is 
> limited, but the answer appears to be "Yes".
> Multiprocessing
>> brings with it a whole lot of extra complications around moving data
>> around.
> 
> People generally understand how to move data around, and the mistakes are 
> usually pretty obvious when they happen.  

I think the existence of this thread indicates otherwise :-) This mistake was 
far from obvious, and clearly I didn't understand properly how to move data 
around *between processes*. Unless you are just saying I am ignorant or 
something? :-)

> People may not understand how to move data around efficiently, but that's a 
> separate argument.
> 
> Multithreading brings with it a whole lot of extra
>> complications around NOT moving data around.
> 
> I think this involves more subtle bugs that are harder to spot.  

Again, the existence of this thread indicates otherwise. This bug was quite 
subtile and hard to spot. It was only when I started looking at how many times 
a given piece of code was called (specifically, the part that handled data 
coming in for which there wasn't an entry in the dictionary) that I spotted the 
problem. If I hadn't had logging in place in that code block, I would have 
never realized it wasn't working as intended. You don't get much more subtile 
than that. And, furthermore, it only existed because I *wasn't* using threads. 
This bug simply doesn't exist in a threaded model, only in a multiprocessing 
model. Yes, the *explanation* of the bug is simple enough - each process "sees" 
a different value, since memory isn't shared - but the bug in my code was 
neither obvious or easy to spot, at least until you knew what was happening.

> People seem to find it harder to reason about atomicity and realising that 
> widely separated pieces of code may interact unexpectedly.
> 
> Yield points bring with
>> them the risk of locking another thread out unexpectedly (particularly
>> since certain system calls aren't async-friendly on certain OSes).
> 
> I've got to admit I find coroutines straightforward, but I did cut my teeth 
> on a cooperative OS.  It certainly makes the atomicity issues easier to deal 
> with.

I still can't claim to understand them. Threads? No problem. Obviously I'm 
still lacking some understanding of how data works in the multiprocessing 
model, however.

> 
> All
>> three models have their pitfalls.
> 
> Assuredly.  I just think threads are soggier and hard to light^W^W^W^W^W 
> prone to subtler and more mysterious-looking bugs.

And yet, this thread exists because of a subtle and mysterious-looking bug with 
multiple *processes* that doesn't exist with multiple *threads*. Thus the point 
- threads are no *worse* - just different - than any other concurrency model.

> 
> -- 
> Rhodri James *-* Kynesim Ltd
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Michael Torrie]

On 11/03/2017 11:44 AM, Jon Ribbens wrote:
> And that's leading you into confusion, as you've demonstrated.

And indeed I've been led into considerable confusion about the else:
clause over the years. Every time I need to use it, I run a python shell
and try it out to remind myself how it works.  However now with your
mental model, Jon, I think I finally have it! Thank you.

I can't think of any normal circumstance where for/else is useful
without a break.  In fact if you have no break you may as well drop the
else entirely, because the block will always execute.  A linter should
definitely flag this.  One reason to leave it as valid syntax is maybe I
am coding up some skeleton code that I plan to fill in later.  So for
now the loop falls through, but at some future point I'll have finished
the searching code and added the break.  But if it were a syntax error I
don't think that'd be any big deal.

Anyway, thank you again, Jon.  Your explanation and model has greatly
cleared up my confusion, and I think if the docs reasoned along the same
lines it would clear up confusion in most programmers.  Certainly it
appears your line of reasoning is what Guido had in mind when he added
the else block.

Sorry Steven, I think Jon's right in this instance and your reasoning
isn't persuasive.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Rhodri James]

On 03/11/17 18:12, Israel Brewster wrote:

On Nov 3, 2017, at 7:11 AM, Rhodri James  wrote:

People generally understand how to move data around, and the mistakes are 
usually pretty obvious when they happen.


I think the existence of this thread indicates otherwise :-) This mistake was 
far from obvious, and clearly I didn't understand properly how to move data 
around *between processes*. Unless you are just saying I am ignorant or 
something? :-)


Ah, but from the point of view of this argument, you didn't make a 
mistake, you made a meta-mistake.  It wasn't that you didn't understand 
how to move data around between processes, it was that you didn't think 
you were moving between processes!  Whether or not you do understand 
remains to be seen :-)


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

Re: python3 byte decode

[Terry Reedy]

On 11/3/2017 5:24 AM, Ali Rıza KELEŞ wrote:

Hi,

Yesterday, while working with redis, i encountered a strange case.

I want to ask why is the following `True`

```
"s" is b"s".decode()
```

while the followings are `False`?

```
"so" is b"so".decode()
"som" is b"som".decode()
"some" is b"some".decode()
```

Or vice versa?

I read that `is` compares same objects, not values. So my question is
why "s" and b"s".decode() are same objects, while the others aren't?


For the same reason as

>>> a = 1
>>> b = 1
>>> a is b
True
>>> a = 1000
>>> b = 1000
>>> a is b
False

For CPython, 'small' ints are cached on startup.  Ditto for 'small' 
strings, which I think includes all 128 ascii chars, and maybe latin1 
chars.  Details depends on the implemention and version.  The main use 
of 'is' for immutable builtins is for developers to test the 
implementation in the test suite.


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Ian Kelly]

On Fri, Nov 3, 2017 at 3:25 PM, Stefan Ram  wrote:
> Jon Ribbens  writes:
>>No, it's an obvious bug. You have a 'for...else' with no 'break'.
>>Like I said, that should probably be a syntax error.
>
>   It should make the syntax of Python much more complicated,
>   when one would try to encode this rule in the /syntax/.
>
>   (Just try to write down the EBNF.)
>
>   It would be more reasonable to call it a constraint.
>   Maybe it could still be a parser error (not a runtime
>   error).

Regardless of whether it's encoded in the EBNF or a constraint on the
resulting parse tree, the eventual outcome would still be called a
syntax error.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Steve D'Aprano]

On Sat, 4 Nov 2017 06:15 am, Michael Torrie wrote:

> In fact if you have no break you may as well drop the
> else entirely, because the block will always execute.

That's incorrect. There are multiple ways to exit a loop that will prevent the
`else` block from executing, `break` is only one.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Thread safety issue (I think) with defaultdict

[Steve D'Aprano]

On Sat, 4 Nov 2017 05:12 am, Israel Brewster wrote:

[...]
>> People generally understand how to move data around, and the mistakes are
>> usually pretty obvious when they happen.
> 
> I think the existence of this thread indicates otherwise :-) This mistake
> was far from obvious, and clearly I didn't understand properly how to move
> data around *between processes*. Unless you are just saying I am ignorant or
> something? :-)

Yes, you were ignorant -- you didn't even realise that you were using
processes, you thought you were using threaded code when it was actually
multiprocessing code. No wonder you got it wrong.

Of course you have a good excuse: the multiprocessing is hidden deep inside
not just the library you were using, but the library *it* was using.

(I don't know how obvious the documentation of the libraries make this --
maybe they're to blame, for not being clear enough -- or maybe you were
simply ignorant about the tools you were using.)

You can't judge multiprocessing code on the basis of bugs caused by assuming
that it was threading code, writing in a threading style with shared data. If
you misuse your tools, that's not the tool's fault.

If anything, we can say that the ultimate error was that you decided to write
in a threaded style without actually using threads: the error was your
(dangerous?) choice to write non-deterministic code using shared data.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Michael Torrie]

On 11/03/2017 07:09 PM, Steve D'Aprano wrote:
> On Sat, 4 Nov 2017 06:15 am, Michael Torrie wrote:
> 
>> In fact if you have no break you may as well drop the
>> else entirely, because the block will always execute.
> 
> That's incorrect. There are multiple ways to exit a loop that will prevent the
> `else` block from executing, `break` is only one.

Such as?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Chris Angelico]

On Sat, Nov 4, 2017 at 1:57 PM, Michael Torrie  wrote:
> On 11/03/2017 07:09 PM, Steve D'Aprano wrote:
>> On Sat, 4 Nov 2017 06:15 am, Michael Torrie wrote:
>>
>>> In fact if you have no break you may as well drop the
>>> else entirely, because the block will always execute.
>>
>> That's incorrect. There are multiple ways to exit a loop that will prevent 
>> the
>> `else` block from executing, `break` is only one.
>
> Such as?

There are many. But other than break, I don't know of any that WOULD
execute the next line of code immediately _after_ the loop.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: replacing `else` with `then` in `for` and `try`

[Michael Torrie]

On 11/03/2017 09:06 PM, Chris Angelico wrote:
> On Sat, Nov 4, 2017 at 1:57 PM, Michael Torrie  wrote:
>> On 11/03/2017 07:09 PM, Steve D'Aprano wrote:
>>> On Sat, 4 Nov 2017 06:15 am, Michael Torrie wrote:
>>>
 In fact if you have no break you may as well drop the
 else entirely, because the block will always execute.
>>>
>>> That's incorrect. There are multiple ways to exit a loop that will prevent 
>>> the
>>> `else` block from executing, `break` is only one.
>>
>> Such as?
> 
> There are many. But other than break, I don't know of any that WOULD
> execute the next line of code immediately _after_ the loop.

Can you be more specific? What are some of these "many" ways of aborting
a loop?  Help a guy out here.

I know, for example, that we have exceptions. But those hardly matter in
this discussion because they wouldn't execute the else clause either.
They'd either be caught elsewhere, or end the program.  sys.exit() can
also terminate a for loop, but it terminates the whole program without
running the else statement.




-- 
https://mail.python.org/mailman/listinfo/python-list

Ideas about how software should behave (was: replacing `else` with `then` in `for` and `try`)

[Ben Finney]

Ian Kelly  writes:

> Steve was clearly referring to the coder, not the code.

Not at all. Steve was referring specifically to the *idea*:

[A ‘for … else’ structure without a ‘break’] should absolutely not
be a syntax error. There's no reason for it to be a syntax error,
except to satisfy some arrogant and foolish idea of purity.

This is clearly a criticism of the idea behind the expectation of a
syntax error. It doesn't refer to the coder, it refers explicitly to the
idea being criticised.

To respond to the criticism of an idea – criticism containing no mention
of the person – as though it “clearly refers to the [person]”, is of
significant concern on a software dicussion forum such as this.

> Please stop defending the use of incivility on this list.

Please stop conflating people, who deserve civility, with ideas. We must
not allow the civility deserved by people, to prevent us from
criticising any ideas — especially not ideas about the behaviour of
software.

-- 
 \  “I used to be a proofreader for a skywriting company.” —Steven |
  `\Wright |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list