Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?

2018-07-13 Thread André Malo 
On Donnerstag, 12. Juli 2018 22:09:41 CEST Antoine Pitrou wrote:
> On Thu, 12 Jul 2018 22:03:30 +0200
> 
> André Malo  wrote:
> > * INADA Naoki wrote:
> > > Is there any real application which marshal.dumps() performance is
> > > critical?
> > 
> > I'm using it for spooling big chunks of data on disk, exactly for the
> > reason that it's faster than pickle.
> 
> Which kind of data is that?

Basically iterators of builtin objects (dicts or tuples of strings or 
numbers). Typically one unit or "row" per dumps() call (they are written one 
after the next and marshal load can easily load them in the same manner).
They're certainly never the same objects (except maybe for dict keys, which 
might be interned)

Cheers,
nd


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accepting PEP 572, Assignment Expressions

2018-07-13 Thread Random832 
On Wed, Jul 11, 2018, at 20:10, Guido van Rossum wrote:
> As anticippated, after a final round of feedback I am hereby accepting PEP
> 572, Assignment Expressions: https://www.python.org/dev/peps/pep-0572/

I know everyone else is probably sick of discussing this (I somehow completely 
missed the discussion until it was almost done) but I have a question...

Why does this not allow assignment to attributes and subscripts? I think this 
is, at least, surprising enough that there should be a rationale section 
explaining it in the PEP. As it is, it's barely even explicitly stated, other 
than the use of 'NAME' in a few places and an offhand mention "and [the 
assignment statement] can assign to attributes and subscripts."
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accepting PEP 572, Assignment Expressions

2018-07-13 Thread Chris Angelico 
On Fri, Jul 13, 2018 at 11:36 PM, Random832  wrote:
> On Wed, Jul 11, 2018, at 20:10, Guido van Rossum wrote:
>> As anticippated, after a final round of feedback I am hereby accepting PEP
>> 572, Assignment Expressions: https://www.python.org/dev/peps/pep-0572/
>
> I know everyone else is probably sick of discussing this (I somehow 
> completely missed the discussion until it was almost done) but I have a 
> question...
>
> Why does this not allow assignment to attributes and subscripts? I think this 
> is, at least, surprising enough that there should be a rationale section 
> explaining it in the PEP. As it is, it's barely even explicitly stated, other 
> than the use of 'NAME' in a few places and an offhand mention "and [the 
> assignment statement] can assign to attributes and subscripts."
>

It was discussed at some length, yes. Assignment to arbitrary targets
would also mean permitting iterable unpacking, which is not desired
("x, y := 3, 4"??), and there weren't enough use-cases for
attribute/item assignment to justify creating a rule of "you can
assign to any single target, but can't unpack". In the future, if such
use-cases are found, the grammar can be expanded.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?

2018-07-13 Thread Christian Tismer 
Well, to my knowledge they did not modify the marshal code.
They are in fact heavily dependent from marshal speed since that
is used frequently to save and restore state of many actors.

But haven't looked further since 2010 ;-)

Btw., why are they considering to make the algorithm slower,
just because someone wants the algorithm stable?

An optional keyword argument would give the stability, and the
default behavior would not be changed at all.

Cheers - Chris


On 12.07.18 12:07, Steve Holden wrote:
> Eve is indeed based on stackless 2, and are well capable of ignoring
> changes they don't think they need (or were when I was working with
> them). At one point I seem to remember they optimised their interpreter
> to use singleton floating-point values, saving large quantities of
> memory by having only one floating-point zero.
> 
> Steve Holden
> 
> On Thu, Jul 12, 2018 at 9:55 AM, Alex Walters  > wrote:
> 
> 
> 
> > -Original Message-
> > From: Python-Dev  > list=sdamon@python.org > On Behalf Of
> Victor Stinner
> > Sent: Thursday, July 12, 2018 4:01 AM
> > To: Serhiy Storchaka mailto:storch...@gmail.com>>
> > Cc: python-dev mailto:python-dev@python.org>>
> > Subject: Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
> > 
> > 2018-07-12 8:21 GMT+02:00 Serhiy Storchaka  >:
> > >> Is there any real application which marshal.dumps() performance is
> > >> critical?
> > >
> > > EVE Online is a well known example.
> > 
> > EVE Online has been created in 2003. I guess that it still uses Python
> 2.7.
> > 
> > I'm not sure that a video game would pick marshal in 2018.
> > 
> 
> EVE doesn't use stock CPython, IIRC.  They use a version of stackless 2,
> with their own patches.  If a company is willing to patch python
> itself, I
> don't think their practices should be cited without more context
> about what
> they actually modified.
> 
> > Victor
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org 
> > https://mail.python.org/mailman/listinfo/python-dev
> 
> > Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/tritium-
> 
> > list%40sdamon.com 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev
> 
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/steve%40holdenweb.com
> 
> 
> 
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com
> 


-- 
Christian Tismer-Sperling:^)   tis...@stackless.com
Software Consulting  : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : http://pyside.org
14482 Potsdam: GPG key -> 0xE7301150FB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?

2018-07-13 Thread Ivan Pozdeev via Python-Dev 
If the use case for stability is only .pyc compilation, I doubt it's 
even relevant 'cuz .pyc's are supposed to be compiled in isolation from 
other current objects (otherwise, they wouldn't be reusable or would be 
invalidated when dependent modules change, neither of which is the 
case), so relevant reference counts should always be the same.

I may be mistaking though.

On 13.07.2018 16:57, Christian Tismer wrote:

Well, to my knowledge they did not modify the marshal code.
They are in fact heavily dependent from marshal speed since that
is used frequently to save and restore state of many actors.

But haven't looked further since 2010 ;-)

Btw., why are they considering to make the algorithm slower,
just because someone wants the algorithm stable?

An optional keyword argument would give the stability, and the
default behavior would not be changed at all.

Cheers - Chris


On 12.07.18 12:07, Steve Holden wrote:

Eve is indeed based on stackless 2, and are well capable of ignoring
changes they don't think they need (or were when I was working with
them). At one point I seem to remember they optimised their interpreter
to use singleton floating-point values, saving large quantities of
memory by having only one floating-point zero.

Steve Holden

On Thu, Jul 12, 2018 at 9:55 AM, Alex Walters mailto:tritium-l...@sdamon.com>> wrote:



 > -Original Message-
 > From: Python-Dev  list=sdamon@python.org > On Behalf Of
 Victor Stinner
 > Sent: Thursday, July 12, 2018 4:01 AM
 > To: Serhiy Storchaka mailto:storch...@gmail.com>>
 > Cc: python-dev mailto:python-dev@python.org>>
 > Subject: Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
 >
 > 2018-07-12 8:21 GMT+02:00 Serhiy Storchaka mailto:storch...@gmail.com>>:
 > >> Is there any real application which marshal.dumps() performance is
 > >> critical?
 > >
 > > EVE Online is a well known example.
 >
 > EVE Online has been created in 2003. I guess that it still uses Python
 2.7.
 >
 > I'm not sure that a video game would pick marshal in 2018.
 >

 EVE doesn't use stock CPython, IIRC.  They use a version of stackless 2,
 with their own patches.  If a company is willing to patch python
 itself, I
 don't think their practices should be cited without more context
 about what
 they actually modified.

 > Victor
 > ___
 > Python-Dev mailing list
 > Python-Dev@python.org 
 > https://mail.python.org/mailman/listinfo/python-dev
 
 > Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/tritium-
 
 > list%40sdamon.com 

 ___
 Python-Dev mailing list
 Python-Dev@python.org 
 https://mail.python.org/mailman/listinfo/python-dev
 
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/steve%40holdenweb.com
 




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/tismer%40stackless.com





___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru


--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accepting PEP 572, Assignment Expressions

2018-07-13 Thread Guido van Rossum 
Also nobody had a use case.

On Fri, Jul 13, 2018 at 6:57 AM Chris Angelico  wrote:

> On Fri, Jul 13, 2018 at 11:36 PM, Random832 
> wrote:
> > On Wed, Jul 11, 2018, at 20:10, Guido van Rossum wrote:
> >> As anticippated, after a final round of feedback I am hereby accepting
> PEP
> >> 572, Assignment Expressions: https://www.python.org/dev/peps/pep-0572/
> >
> > I know everyone else is probably sick of discussing this (I somehow
> completely missed the discussion until it was almost done) but I have a
> question...
> >
> > Why does this not allow assignment to attributes and subscripts? I think
> this is, at least, surprising enough that there should be a rationale
> section explaining it in the PEP. As it is, it's barely even explicitly
> stated, other than the use of 'NAME' in a few places and an offhand mention
> "and [the assignment statement] can assign to attributes and subscripts."
> >
>
> It was discussed at some length, yes. Assignment to arbitrary targets
> would also mean permitting iterable unpacking, which is not desired
> ("x, y := 3, 4"??), and there weren't enough use-cases for
> attribute/item assignment to justify creating a rule of "you can
> assign to any single target, but can't unpack". In the future, if such
> use-cases are found, the grammar can be expanded.
>
> ChrisA
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
-- 
--Guido (mobile)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?

2018-07-13 Thread INADA Naoki 
On Fri, Jul 13, 2018 at 11:46 PM Ivan Pozdeev via Python-Dev
 wrote:
>
> If the use case for stability is only .pyc compilation, I doubt it's even 
> relevant 'cuz .pyc's are supposed to be compiled in isolation from other 
> current objects (otherwise, they wouldn't be reusable or would be invalidated 
> when dependent modules change, neither of which is the case), so relevant 
> reference counts should always be the same.
> I may be mistaking though.
>

Good point!  You're right.

Currently, there is one unstable pyc issue (except frozenset order which can be
fixed by PYTHONHASHSEED).
https://bugzilla.opensuse.org/show_bug.cgi?id=1049186

This is caused by interned string.
Because of interning, reference count can be unstable.

Like that, long objects, tuples and some others are cached and reused
automatically.
But they has refcnt>1 always: reference from object and cache.

So we can use FLAG_REF always for interned string, even if refcnt==1.
Let's try it and wait another issue are found.

Thanks!

-- 
INADA Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

2018-07-13 Thread Python tracker 


ACTIVITY SUMMARY (2018-07-06 - 2018-07-13)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open6706 (-15)
  closed 39153 (+63)
  total  45859 (+48)

Open issues with patches: 2666 


Issues opened (33)
==

#33922: [Windows] Document the launcher's -64 suffix
https://bugs.python.org/issue33922  reopened by eryksun

#34062: Python launcher on Windows does not work with --list or --list
https://bugs.python.org/issue34062  opened by brett.cannon

#34063: binhex REASONABLY_LARGE = 32768 so what is 128000
https://bugs.python.org/issue34063  opened by Jim.Jewett

#34064: subprocess functions with shell=1 pass wrong command to win32 
https://bugs.python.org/issue34064  opened by mandel59

#34067: Problem with contextlib.nullcontext
https://bugs.python.org/issue34067  opened by serhiy.storchaka

#34068: traceback.clear_frames(): Objects/typeobject.c:3086: _PyType_L
https://bugs.python.org/issue34068  opened by serhiy.storchaka

#34070: Superfluous call to isatty in open() when buffering >= 0
https://bugs.python.org/issue34070  opened by Dav1d

#34071: asyncio: repr(task) raises AssertionError for coros which loop
https://bugs.python.org/issue34071  opened by JDLH

#34075: asyncio: We should prohibit setting a ProcessPoolExecutor in w
https://bugs.python.org/issue34075  opened by yselivanov

#34078: Broken CRL functionality in ssl.py
https://bugs.python.org/issue34078  opened by Joe N

#34079: Multiprocessing module fails to build on Solaris 11.3
https://bugs.python.org/issue34079  opened by clallen

#34081: Sphinx duplicate label warning in docs
https://bugs.python.org/issue34081  opened by xtreak

#34082: EnumMeta.__new__ should use enum_class.__new__
https://bugs.python.org/issue34082  opened by rs2

#34084: possible free statically allocated string in compiler when eas
https://bugs.python.org/issue34084  opened by xiang.zhang

#34085: doc Improve wording on classmethod/staticmethod
https://bugs.python.org/issue34085  opened by adelfino

#34086: logging.Handler.handleError regressed in python3
https://bugs.python.org/issue34086  opened by oren

#34087: int(s), float(s) and others may cause segmentation fault
https://bugs.python.org/issue34087  opened by fenrrir

#34088: [EASY] sndhdr.what() throws exceptions on unknown files
https://bugs.python.org/issue34088  opened by Barro

#34089: Remove required (non-optional) modules from Modules/Setup.dist
https://bugs.python.org/issue34089  opened by nascheme

#34090: Python function call optimization: avoid temporary tuple to pa
https://bugs.python.org/issue34090  opened by vstinner

#34091: REPL does not work in msys2, ConEmu terminals on windows
https://bugs.python.org/issue34091  opened by maxnoe

#34093: Reproducible pyc: FLAG_REF is not stable.
https://bugs.python.org/issue34093  opened by inada.naoki

#34095: [2.7] test_idle fails with: /usr/bin/xvfb-run: line 181:  3617
https://bugs.python.org/issue34095  opened by deep42thought

#34096: [2.7] test_audioop.test_max() failed: AssertionError: -2147483
https://bugs.python.org/issue34096  opened by deep42thought

#34097: ZIP does not support timestamps before 1980
https://bugs.python.org/issue34097  opened by petr.viktorin

#34098: multiprocessing.Server swallows original exception traceback
https://bugs.python.org/issue34098  opened by salgado

#34099: Provide debuggers with a way to know that a function is exitin
https://bugs.python.org/issue34099  opened by fabioz

#34100: Same constants in tuples are not merged while compile()
https://bugs.python.org/issue34100  opened by Dan Rose

#34101: PyBuffer_GetPointer() not documented
https://bugs.python.org/issue34101  opened by pitrou

#34102: local variable 'parts' referenced before assignment in feedpar
https://bugs.python.org/issue34102  opened by sshnaidm

#34105: test_socket.test_host_resolution_bad_address fails on Mac OS X
https://bugs.python.org/issue34105  opened by lys.nikolaou

#34106: Add --with-module-config= to 'configure' script
https://bugs.python.org/issue34106  opened by nascheme

#34108: 2to3 munges new lines on Windows
https://bugs.python.org/issue34108  opened by jason.coombs



Most recent 15 issues with no replies (15)
==

#34106: Add --with-module-config= to 'configure' script
https://bugs.python.org/issue34106

#34105: test_socket.test_host_resolution_bad_address fails on Mac OS X
https://bugs.python.org/issue34105

#34101: PyBuffer_GetPointer() not documented
https://bugs.python.org/issue34101

#34099: Provide debuggers with a way to know that a function is exitin
https://bugs.python.org/issue34099

#34098: multiprocessing.Server swallows original exception traceback
https://bugs.python.org/issue34098

#34089: Remove required (non-optional) modules from Modules/Setup.dist
https://bugs.python.org/issue34089

#34085: doc Improve wording on classmethod/static