I need a free LAMP hosting and a iMac

[Benjamin via Python-list]

Hi,

For several months I have searched free web hosing in Google, but have not find 
a satisfying result now. Any body know some good LAMP free web hosting?

And, I have lost job since 2018, my macbook has only 2 intel core, I want to 
buy a new iMac for person programming, but I have only little money, any body 
donate me a iMac with ARM CPU?  or donate some money at
https://buymeacoffee.com/benjamin_yin 

Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Relatively prime integers in NumPy

[Oscar Benjamin via Python-list]

(posting on-list this time)

On Thu, 11 Jul 2024 at 15:18, Popov, Dmitry Yu via Python-list
 wrote:
>
> Dear Sirs.
>
> Does NumPy provide a simple mechanism to identify relatively prime integers, 
> i.e. integers which don't have a common factor other than +1 or -1? For 
> example, in case of this array:
> [[1,5,8],
>   [2,4,8],
>   [3,3,9]]
> I can imagine a function which would return array of common factors along 
> axis 0: [1,2,3]. Those triples of numbers along axis 1 with the factor of1 or 
> -1 would be relatively prime integers.

It sounds like you want the gcd (greatest common divisor) of each row.
The math module can do this:

In [1]: a = [[1,5,8],
   ...:   [2,4,8],
   ...:   [3,3,9]]

In [2]: import math

In [3]: [math.gcd(*row) for row in a]
Out[3]: [1, 2, 3]

NumPy can also do it apparently:

In [10]: np.gcd.reduce(np.transpose(a))
Out[10]: array([1, 2, 3])

https://en.wikipedia.org/wiki/Greatest_common_divisor

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best use of "open" context manager

[Oscar Benjamin via Python-list]

On Sat, 6 Jul 2024 at 11:55, Rob Cliffe via Python-list
 wrote:
>
> Consider this scenario (which I ran into in real life):
>  I want to open a text file and do a lot of processing on the lines
> of that file.
>  If the file does not exist I want to take appropriate action, e.g.
> print an error message and abort the program.
> I might write it like this:
>
> try:
>  with open(FileName) as f:
>  for ln in f:
>  print("I do a lot of processing here")
>  # Many lines of code here .
> except FileNotFoundError:
>  print(f"File {FileName} not found")
>  sys.exit()
>
> but this violates the principle that a "try" suite should be kept small,
> so that only targeted exceptions are trapped,
> not to mention that having "try" and "except" far apart decreases
> readability.

This is catching a targeted exception (FileNotFoundError) so I think
it is fine. If the intention is just to call sys.exit() on error then
I wouldn't worry too much about having too much code in the try. Just
make sure that you do this in any other place where you open a file as
well.

One possible improvement is that you could catch the exception and use
its filename attribute:

except FileNotFoundError as e
 print(f"File {e.filename} not found")

That way if you did catch the wrong FileNotFoundError then at least
you print the correct filename.

Alternatively:

except FileNotFoundError as e
 if e.filename != FileName:
  raise  # re-raise if not the intended exception
 print(f"File {e.filename} not found")

For readability I would just move the many lines of code into a
separate function.

The reason to avoid having too much code in the try mainly applies to
situations where you are going to do something other than call
sys.exit() and the exception is overly generic like ValueError or
TypeError. If the exception can easily be raised by a bug or something
other than the intended cause then it is bad to catch exceptions
around a larger block of code.

If it is expected that the caller of a function might have good reason
to catch the exception and handle it somehow then it is better to make
a dedicated exception class and raise that instead. When there is only
one place in the code that raises a particular exception type and only
one place that catches it then it is usually going to be clear that
you are catching the expected exception.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Couldn't install numpy on Python 2.7

[Oscar Benjamin via Python-list]

On Wed, 12 Jun 2024 at 23:52, Greg Ewing via Python-list
 wrote:
> On 13/06/24 10:09 am, Chris Angelico wrote:
>  > So if anyone
>  > actually does need to use pip with Python 2.7, they probably need to
>  > set up a local server
>
> You should also be able to download a .tar.gz from PyPI and use pip
> to install that. Although you'll have to track down the dependencies
> yourself in that case.

It is almost certainly better to download the wheel (.whl) file rather
than the sdist (.tar.gz) file. Building NumPy from source needs not
just compilers etc but also you first need to build/install a BLAS
library.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Couldn't install numpy on Python 2.7

[Oscar Benjamin via Python-list]

On Wed, 12 Jun 2024 at 23:11, Chris Angelico via Python-list
 wrote:
>
> On Thu, 13 Jun 2024 at 07:57, Oscar Benjamin via Python-list
>  wrote:
> > They are seeing a warning that explicitly says "You can upgrade to a
> > newer version of Python to solve this". I don't know whether that SSL
> > warning is directly connected to pip not finding any versions of numpy
> > but with the available information so far that seems like the first
> > thing to consider.
>
> I think it is; AIUI, with an ancient SSL library, pip is unable to
> download packages safely from the current pypi server. So if anyone
> actually does need to use pip with Python 2.7, they probably need to
> set up a local server, using older encryption protocols (which should
> therefore NOT be made accessible to the internet). Since pip can't
> contact the upstream pypi, there's no available numpy for it to
> install.

I don't know much about SSL and related networking things especially
on Windows. I would be surprised if pip on old Python can't install
from current PyPI though. I imagine that something strange has
happened like a new version of pip running on an old version of Python
or old Python on new OS (or old PyCharm...).

There is no problem using Python 2.7 with pip and PyPI on this Linux
machine but I guess it has a newer SSL library provided by the OS:

$ pip install numpy
DEPRECATION: Python 2.7 reached the end of its life on January 1st,
2020. Please upgrade your Python as Python 2.7 is no longer
maintained. pip 21.0 will drop support for Python 2.7 in January 2021.
More details about Python 2 support in pip can be found at
https://pip.pypa.io/en/latest/development/release-process/#python-2-support
pip 21.0 will remove support for this functionality.
Collecting numpy
  Downloading numpy-1.16.6-cp27-cp27mu-manylinux1_x86_64.whl (17.0 MB)
 || 17.0 MB 14.3 MB/s
Installing collected packages: numpy
Successfully installed numpy-1.16.6

If it is actually the case that pip on Python 2.7 (on Windows) cannot
download from PyPI then an easier option rather than creating a local
server would just be to download the numpy wheels from PyPI using a
browser:

  https://pypi.org/project/numpy/1.15.4/#files

Then you can do

   pip install .\numpy-1.15.4-cp27-none-win_amd64.whl

Using a newer version of Python is still my primary suggestion though.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Couldn't install numpy on Python 2.7

[Oscar Benjamin via Python-list]

On Wed, 12 Jun 2024 at 22:38, AVI GROSS via Python-list
 wrote:
>
> The discussion though was about a specific OP asking if they can fix their
> problem. One solution being suggested is to fix a deeper problem and simply
> make their code work with a recent version of python 3.

The OP has not replied with any explanation as to why they are using
Python 2.7 and has not said whether they have any existing code that
only works with Python 2.7. It is unclear at this point whether there
is any reason that they shouldn't just install a newer version of
Python.

They are seeing a warning that explicitly says "You can upgrade to a
newer version of Python to solve this". I don't know whether that SSL
warning is directly connected to pip not finding any versions of numpy
but with the available information so far that seems like the first
thing to consider.

It is entirely reasonable to start by suggesting to use a newer
version of Python until some reason is given for not doing that.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What is Install-Paths-To in WHEEL file?

[Oscar Benjamin via Python-list]

On Sun, 31 Dec 2023 at 00:35, Left Right  wrote:
>
> It's not for you to choose the way I communicate. There are accepted
> boundaries, and I'm well within those boundaries. Anything beyond that
> is not something I'm even interested in hearing your opinion on.

You might not be interested in my opinion but you might want to
reflect on the fact that although you consider your behavior to be
within "accepted boundaries" the evidence here (and in the forum)
suggests that others do not and so your notion of what is "accepted"
is not universally shared.

I am not going to reply to your other points except to say that if you
want to influence anything then I expect that you would have more
success with a different approach.

To anyone else considering replying in this thread: please don't. I
very much doubt that anything good will happen here.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What is Install-Paths-To in WHEEL file?

[Oscar Benjamin via Python-list]

On Fri, 29 Dec 2023 at 22:38, Left Right via Python-list
 wrote:
>
> > Then your understanding is flat-out wrong. Encouraging participation
> > by everyone DOES mean deleting what is unproductive, offensive, and
> > likely to discourage participation.
>
> I haven't written anything unproductive or offensive. I offered
> constructive criticism with a detailed plan on how to fix the problem.
> The forum owners chose to ban me because they don't like hearing that
> the code they've written is bad. And that's the long and the short of
> it. This has been a pattern in behavior of PyPA members I've
> interacted with so far.

You are conflating several different groups of people. The PyPA are
the people who currently maintain the code for various
libraries/tools. That is very often not the same as the people who
originally wrote the code for the same libraries/tools or for
preceding ones. Neither group is the same as the forum moderators (I
suspect that there is no intersection between the moderators and the
PyPA etc.).

> And whenever they had a chance, they'd use it
> to pretend that the problems I'm talking about don't exist by deleting
> every mention of the problem. That is an example of unproductive and
> offensive behavior because it produces nothing and wastes my time I've
> dedicated to locating, reporting and solving their problem.

Actually you are wasting the time of others by putting across
inaccurate and unhelpful information in a rude way and at the same
time criticising others without really understanding who you are
criticising and for what. Your contribution is unhelpful mostly (but
not exclusively) because of the way that you choose to communicate.

I did not realise earlier what you were referring to but I see now
that I have email notifications with the content of your posts that
were deleted. I am not surprised that they were deleted and that you
were banned because if I was a moderator looking at those then I would
not expect a promising future for your interactions with others in the
forum.

There is some significant irony in you describing the forum as a
"toxic pit" for deleting your posts. I don't always agree with the
moderators and I am not sure that I would have reacted the way that
they did but these threads remind me precisely why moderation
(including deleting posts such as yours) is needed to *prevent* a
forum from turning into a toxic pit.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What is Install-Paths-To in WHEEL file?

[Oscar Benjamin via Python-list]

On Fri, 29 Dec 2023 at 13:04, Left Right via Python-list
 wrote:
>
> Wow. That place turned out to be the toxic pit I didn't expect.
>
> It's a shame that a public discussion of public goods was entrusted to
> a bunch of gatekeepers with no sense of responsibility for the thing
> they keep the keys to.

Here is the discussion referred to:
https://discuss.python.org/t/what-is-install-paths-to-in-wheel-file/42005

I don't see anything "toxic" in that discussion. You asked questions
and people took the time to give clear answers.

The basic answer to your question is that PEP 491 was never completed
and so there is no accepted specification of the Install-Paths-To
feature that it had been intended to introduce. The PEP text itself is
reasonably clear about this and also links to the up to date
specifications:
https://peps.python.org/pep-0491/#pep-deferral

Instead for understanding the wheel format the appropriate document is:
https://packaging.python.org/en/latest/specifications/binary-distribution-format/

That document does not mention Install-Paths-To because it documents
the standards as defined and accepted via the PEP process but PEP 491
was never accepted.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: on writing a number as 2^s * q, where q is odd

[Oscar Benjamin via Python-list]

On Sun, 3 Dec 2023 at 10:25, Julieta Shem via Python-list
 wrote:
>
> Alan Bawden  writes:
> >
> > def powers_of_2_in(n):
> > bc = (n ^ (n - 1)).bit_count() - 1
> > return bc, n >> bc
>
> That's pretty fancy and likely the fastest.

It might be the fastest but it depends how big you expect n to be and
how many trailing zeros you expect. If n is a very large integer then
this does three large integer operations in (n^(n-1)).bit_count().
They are relatively fast operations but all linear in bit size. By
contrast a check like n & 1 is O(1) and half the time proves that no
further steps are necessary.

The mpmath library needs this exact operation and is generally
intended for the large n case so I checked how it is implemented
there:

https://github.com/mpmath/mpmath/blob/f13ea4dc925d522062ac734bd19a0a3cc23f9c04/mpmath/libmp/libmpf.py#L160-L177

That code is:

# Strip trailing bits
if not man & 1:
t = trailtable[man & 255]
if not t:
while not man & 255:
man >>= 8
exp += 8
bc -= 8
t = trailtable[man & 255]
man >>= t
exp += t
bc -= t

The trailtable variable is a pre-initialised list of shifts needed to
remove zeros from an 8-bit integer. The bc variable here is just
bc=man.bit_length() which is redundant but this code predates the
addition of the int.bit_length() method.

In principle this could use a large number of man>>=8 shifts which
would potentially be quadratic in the bit size of man. In practice the
probability of hitting the worst case is very low so the code is
instead optimised for the expected common case of large man with few
trailing zeros.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: txtorcon 23.5.0

[Benjamin Schollnick]

> *Top-posting? Really?

In that case, yes, because the conversion did not depend on what was originally 
said.  Unlike this conversation.

> Actually, a very cursory look through the intarweb would have yielded this -

And that’s the point, if you want to me to consider your package, I should have 
some idea that it’s worth me investing more time into seeing if it can help me. 
 As I mentioned simply stating that it was a “Tor Control management API” would 
have been enough for me to say “That’s not helpful to me”.  On the other hand 
if it said “Django management API/Tool”, I would have gone to the git 
repository and investigated.

Effectively, posting a release notification is an advertisement for the 
package, as well as a notice saying “Here’s some updates”.

When done correctly, it can bring more people to use your package / software…

> While there are one or two graphical package managers (yumex and yumex-dnf 
> that I know of), have a go at the cli manager(s) that may live on your distro 
> and see if you can get pkg info about it. If you're not sure which one(s) you 
> have, try these - Yum, Dnf, Rpm,Apt, Apt-Get, Deb, pacman, dpkg, and zypper 
> for starters. *dpkg is my go-to pkgmgr of choice...
> 
> Also, the names of many linux-centric utility plugins, extensions, and other 
> goodies, while seemingly cryptic, are usually a prominent clue as to what the 
> package is used for. The world of *NIX has never been for the faint of heart, 
> but there are lots of very helping hands in its communities... :)

This has nothing to do with package managers, this has to do with “Why should I 
use this package” or “Should I use this package / software”.  If the release 
notes are being send to people that don’t already have the package then it’s 
worth the couple of sentences or a short paragraph to allow someone that is 
unfamiliar with the package to be able to see if they should investigate the 
package.

Cryptic names maybe cute, but if they are not descriptive, then they are not 
really that helpful other than being unique.

- Benjamin

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: txtorcon 23.5.0

[Benjamin Schollnick]

Meejah,

Sorry, but, would you please consider adding a short description to this of 
what txtorcon is.
It’s great that your announcing the update, and highlighting the changes…if I 
used it, I’d be upgrading it.

But I have absolutely no idea why I would want to investigate using it, because 
there is no description of the package.

Well, that’s not 100% true.  From my knowledge of twisted, I know it’s network 
based, and that’s it.  I can’t tell if it’s a server, or client application.

Even just simply adding your 10,000 feet summary:

txtorcon is an implementation of the control-spec 
<https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt> for Tor 
<https://www.torproject.org/> using the Twisted 
<https://twistedmatrix.com/trac> networking library for Python 
<http://python.org/>.

would have been extremely helpful.

I hope you will take this as the creative criticism that I am offering it as.  

- Benjamin


> On May 18, 2023, at 9:37 PM, mee...@meejah.ca wrote:
> 
> 
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
> 
> I'm happy to announce txtorcon 23.5.0 with the following changes:
> 
>  * twisted.web.client.Agent instances now use the same HTTPS policy
>by default as twisted.web.client.Agent.  It is possible to
>override this policy with the tls_context_factory= argument, the
>equivalent to Agent's contextFactory=
>(Thanks to Itamar Turner-Trauring)
>  * Added support + testing for Python 3.11.
>  * No more ipaddress dependency
> 
> You can download the release from PyPI or GitHub (or of
> course "pip install txtorcon"):
> 
>  https://pypi.python.org/pypi/txtorcon/23.5.0
>  https://github.com/meejah/txtorcon/releases/tag/v23.5.0
> 
> Releases are also available from the hidden service:
> 
>  
> http://fjblvrw2jrxnhtg67qpbzi45r7ofojaoo3orzykesly2j3c2m3htapid.onion/txtorcon-23.5.0.tar.gz
>  
> http://fjblvrw2jrxnhtg67qpbzi45r7ofojaoo3orzykesly2j3c2m3htapid.onion/txtorcon-23.5.0.tar.gz.asc
> 
> You can verify the sha256sum of both by running the following 4 lines
> in a shell wherever you have the files downloaded:
> 
> cat < 93fd80a9dd505f698d0864fe93db8b6a9c1144b5feb91530820b70ed8982651c  
> dist/txtorcon-23.5.0.tar.gz
> 987f0a91184f98cc3f0a7eccaa42f5054063744d6ac15e325cfa666403214208  
> dist/txtorcon-23.5.0-py3-none-any.whl
> EOF
> 
> thanks,
> meejah
> -BEGIN PGP SIGNATURE-
> 
> iQFFBAEBCgAvFiEEnVor1WiOy4id680/wmAoAxKAaacFAmRm0DoRHG1lZWphaEBt
> ZWVqYWguY2EACgkQwmAoAxKAaae/+wgAw3gAm65npc7+yMdGFixNmCd+RUXorJq9
> Hy76hK3BWdtNIA6TZF20QFYs3CX5Vepa0vCJOK1N40fYgxoZTb1/828Zp6Zq2+Gn
> piJGvQ0Z1S95ww7lwSV77o67Xf7PozhLR+k7DaOdY8ugvLb/0Rdp15BykF5DWIo8
> PRgqB8uZ418ebmDLLrYtqYTdlcUMxFTji4CHXc4N55/2hVHiFiuFt59os6kJ3iG1
> u90lmQH8vbDyVF7N6tpgEAdWeb7OdgDbtzhVBdBWHrPg+vDO+UL7WZU8ZjDAcdEr
> YzzmK3fIiCH7ngG2E/VIebiJfrjAA9G4eZXltIm7VcWh5css9MXY1Q==
> =TeQp
> -END PGP SIGNATURE-
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Learning tkinter

[Oscar Benjamin]

On Thu, 18 May 2023 at 10:16, Rob Cliffe via Python-list
 wrote:
>
> I am trying to learn tkinter.
> Several examples on the internet refer to a messagebox class
> (tkinter.messagebox).
> But:
>
> Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:20:19) [MSC v.1925 32
> bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import tkinter
>  >>> tkinter.messagebox
> Traceback (most recent call last):
>File "", line 1, in 
> AttributeError: module 'tkinter' has no attribute 'messagebox'
>  >>>
>
> Why is this?

Do you have a file called tkinter.py in the current directory?

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: An "adapter", superset of an iterator

[Oscar Benjamin]

On Wed, 3 May 2023 at 18:52, Thomas Passin  wrote:
>
> On 5/3/2023 5:45 AM, fedor tryfanau wrote:
> > I've been using python as a tool to solve competitive programming problems
> > for a while now and I've noticed a feature, python would benefit from
> > having.
> > Consider "reversed(enumerate(a))". This is a perfectly readable code,
> > except it's wrong in the current version of python. That's because
> > enumerate returns an iterator, but reversed can take only a sequence type.
>
> Depending on what you want to give and receive, enumerate(reversed(a))
> will do the job here.  Otherwise list() or tuple() can achieve some of
> the same things.

I don't think that is equivalent to the intended behaviour:

reversed(enumerate(a)) # zip(reversed(range(len(a))), reversed(a))
enumerate(reversed(a)) # zip(range(len(a)), reversed(a))

In principle for a sequence input enumerate(a) could be something that
behaves like a sequence and therefore could be reiterated or reversed
etc. The enumerate(a).__reversed__ method could then delegate to
a.__reversed__ and a.__len__ if they exist. This could be confusing
though because the possible behaviour of enumerate(a) would be
different depending on the type of a.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Christoph Gohlke and compiled packages

[Oscar Benjamin]

On Tue, 11 Apr 2023 at 14:55, Mats Wichmann  wrote:
>
> On 4/11/23 06:03, Roel Schroeven wrote:
> > Op 11/04/2023 om 12:58 schreef Chris Angelico:
>
> >> Python itself is fine, but a lot of third-party packages are hard to
> >> obtain. So if you need numpy, for instance, or psycopg2, you might
> >> need to find an alternative source.
> > These days I use pip to install packages, and so far for the things I
> > need it simply works. "pip install numpy" works, same for psycopg2,
> > pillow, pandas, and other packages I use. Conda should work too, for
> > those who use the Anaconda Python distribution. I honestly don't even
> > know how it's done: are there some kind souls who provide the wheels
> > (binary packages) for all those things, or if there is maybe a build
> > farm that does the hard work to make things easy for us.
> >
> > In the past I've used Christoph Gohlke's site and I'm very grateful for
> > the service it provided, but these days I don't really need it anymore,
> > luckily.
>
> The deal really is, the instant a new Python version drops (3.11, 3.12,
> etc.) a million people rush to install it, some of whom should know
> better and be more patient.  3rd party packages are their own projects,
> some have binary wheels ready on Python release day, some soon after,
> some months after.

You can hardly blame a lot of people for doing this. A seb search for
"download python" gives this as the first hit:
https://www.python.org/downloads/

I am guessing that the release process automatically updates that page
so that the minute 3.12 gets released the big yellow button will
suggest downloading 3.12.0 as the default option.

Perhaps it is really package authors who should be getting a release
out that is compatible with Python 3.12 before 3.12 itself is
released. It's tricky though because as a maintainer it makes more
sense to wait until you see the finished 3.12 product before making a
release that is fully tested with it (even if you are testing the
alphas etc in CI and making incremental fixes before 3.12 is
released).

The other option could be changing the downloads page so that it does
not suggest 3.12.0 as the default option until it is clear that at
least some baseline of widely used packages have uploaded compatible
wheels.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Christoph Gohlke and compiled packages

[Oscar Benjamin]

On Tue, 11 Apr 2023 at 12:01, Chris Angelico  wrote:
>
> On Tue, 11 Apr 2023 at 20:15, Jim Schwartz  wrote:
> >
> > What’s the problem now?  Is it with python on windows?  I use python on 
> > windows so I’d like to know. Thanks
> >
>
> Python itself is fine, but a lot of third-party packages are hard to
> obtain. So if you need numpy, for instance, or psycopg2, you might
> need to find an alternative source.

Both numpy and psycopg2 have binary wheels for Windows that can be pip
installed from PyPI. I haven't used psycopg2 myself and I don't use
Windows so I can't say if there is any difficulty using them but I
presume that they can install and run just fine. Certainly the numpy
wheels have been there for Windows for years and work just fine.
Before numpy provided wheels they also provided MSI installers for
Windows anyway so there was always an alternative to Christoph's
stack.

Christoph's Windows binaries project predated the wheel format and so
the alternative options have improved massively since then. I presume
that there are some projects where Christoph's binaries are still the
only non-conda option (besides build yourself). I would not be
surprised if all of those are installable by conda though and we are
probably talking about projects that would seem obscure to most Python
users.

Certainly for the more widely used libraries like numpy installing
binaries with pip is not a problem these days on Windows or other
popular OS. I notice that psycopg2 *only* provides binaries for
Windows and not e.g. OSX or Linux but the list of binaries provided by
numpy is extensive with the current release listing wheels for all
combinations of CPython 3.8-3.11, win32, win amd64, manylinux amd64,
manylinux aarch64, OSX x64, OSX arm64:
https://pypi.org/project/numpy/1.24.2/#files

The difference now since the days when Cristoph started generating and
hosting binaries is that it is typically expected that a project like
numpy should produce its own binary wheels for popular platforms and
host them on PyPI. Of course that is a burden on the numpy maintainers
but tooling for this is much better than it used to be with things
like cibuildwheel, free CI systems including Windows/OSX runners etc.
It is *much* easier for a project to support generating Windows wheels
now than it used to be and to a certain extent it just forms part of
the normal CI setup that a project like numpy would want to have
anyway.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: built-in pow() vs. math.pow()

[Oscar Benjamin]

On Fri, 31 Mar 2023 at 20:24, Peter J. Holzer  wrote:
>
> On 2023-03-31 07:39:25 +0100, Barry wrote:
> > On 30 Mar 2023, at 22:30, Chris Angelico  wrote:
> > > It's called math.pow. That on its own should be a strong indication
> > > that it's designed to work with floats.
> >
> > So long as you know that the math module is provided to give access
> > the C math.h functions.
> >
>
> Well, that's the first line in the docs:
>
> | This module provides access to the mathematical functions defined by
> | the C standard.
>
> Of course a Python programmer may not necessarily know what mathematical
> functions the C standard defines or even what C is.

Or they might know that and might also realise that it is not really
an accurate description of Python's math module e.g. math.factorial,
math.gcd, math.isqrt which are all integer functions. How many of
these 60 names are defined by the C standard:

In [6]: dir(math)
Out[6]:
['__doc__',
 '__loader__',
 '__name__',
 '__package__',
 '__spec__',
 'acos',
 'acosh',
 'asin',
 'asinh',
 'atan',
 'atan2',
 'atanh',
 'ceil',
 'comb',
 'copysign',
 'cos',
 'cosh',
 'degrees',
 'dist',
 'e',
 'erf',
 'erfc',
 'exp',
 'expm1',
 'fabs',
 'factorial',
 'floor',
 'fmod',
 'frexp',
 'fsum',
 'gamma',
 'gcd',
 'hypot',
 'inf',
 'isclose',
 'isfinite',
 'isinf',
 'isnan',
 'isqrt',
 'ldexp',
 'lgamma',
 'log',
 'log10',
 'log1p',
 'log2',
 'modf',
 'nan',
 'perm',
 'pi',
 'pow',
 'prod',
 'radians',
 'remainder',
 'sin',
 'sinh',
 'sqrt',
 'tan',
 'tanh',
 'tau',
 'trunc']

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: built-in pow() vs. math.pow()

[Oscar Benjamin]

On Thu, 30 Mar 2023 at 17:31, Andreas Eisele  wrote:
>
> I sometimes make use of the fact that the built-in pow() function has an 
> optional third argument for modulo calculation, which is handy when dealing 
> with tasks from number theory, very large numbers, problems from Project 
> Euler, etc. I was unpleasantly surprised that math.pow() does not have this 
> feature, hence "from math import *" overwrites the built-in pow() function 
> with a function that lacks functionality. I am wondering for the rationale of 
> this. Does math.pow() do anything that the built-in version can not do, and 
> if not, why is it even there?

It is useful for when you want the pure floating point power which has
an approximately fixed computational cost (unlike integer powers).
Perhaps it would have been better if it was named fpow similar to fsum
vs sum.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Debugging reason for python running unreasonably slow when adding numbers

[Oscar Benjamin]

On Tue, 14 Mar 2023 at 16:27, Alexander Nestorov  wrote:
>
> I'm working on an NLP and I got bitten by an unreasonably slow behaviour in 
> Python while operating with small amounts of numbers.
>
> I have the following code:
>
> ```python
> import random, time
> from functools import reduce
>
> def trainPerceptron(perceptron, data):
>   learningRate = 0.002
>   weights = perceptron['weights']
>   error = 0
>   for chunk in data:
>   input = chunk['input']
>   output = chunk['output']
>
>   # 12x slower than equivalent JS
>   sum_ = 0
>   for key in input:
>   v = weights[key]
>   sum_ += v

In Python something like your task here would usually use something
along the lines of NumPy. Your two innermost loops involve adding up a
subset of numbers from a list chosen using a list of indices. This is
something that numpy can do much more efficiently with its fancy
indexing e.g.:

In [3]: a = np.array([1, 2, 3, 4, 5, 6, 7])

In [4]: b = np.array([0, 3, 5])

In [5]: a[b]
Out[5]: array([1, 4, 6])

In [6]: a[b].sum()
Out[6]: 11

This a[b].sum() operation in your code would be weights[input].sum()
and would be much faster than the loop shown (the speed difference
will be larger if you increase the size of the input array).

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.10 Fizzbuzz

[Oscar Benjamin]

On Tue, 28 Feb 2023 at 20:55, Mats Wichmann  wrote:
>
> On 2/27/23 16:42, Oscar Benjamin wrote:
> > On Mon, 27 Feb 2023 at 21:06, Ethan Furman  wrote:
> >>
> >> On 2/27/23 12:20, rbowman wrote:
> >>
> >>   > "By using Black, you agree to cede control over minutiae of hand-
> >>   > formatting. In return, Black gives you speed, determinism, and freedom
> >>   > from pycodestyle nagging about formatting. You will save time and 
> >> mental
> >>   > energy for more important matters."
> >>   >
> >>   > Somehow I don't think we would get along very well. I'm a little on the
> >>   > opinionated side myself.
> >>
> >> I personally cannot stand Black.  It feels like every major choice it 
> >> makes (and some minor ones) are exactly the
> >> opposite of the choice I make.
> >
> > I agree partially. There are two types of decisions black makes:
> >
> > 1. Leave the code alone because it seems okay or make small modifications.
> > 2. Reformat the code because it violates some generic rule (like line
> > too long or something).
> >
> > I've recently tried Black and mostly for my code it seems to go with 1
> > (code looks okay). There might be some minor changes like double vs
> > single quotes but I really don't care about those. In that sense me
> > and Black seem to agree.
> >
> > However I have also reviewed code where it is clear that the author
> > has used black and their code came under case 2. In that case Black
> > seems to produce awful things. What I can't understand is someone
> > accepting the awful rewrite rather than just fixing the code. Treating
> > Black almost like a linter makes sense to me but accepting the
> > rewrites that it offers for bad code does not.
>
> The amount of disagreement you see here and elsewhere are exactly why
> Black  is like it is - virtually without options.  It doesn't aim to
> solve the challenge of producing The Most Beautiful Code Layout, for
> *you*, or even for any moderately sized group of programmers.  Instead
> it's to remove the bickering:
> 1. we choose to use black for a project.
> 2. black runs automatically
> 3. there is now no need to spend cycles thinking about code-style
> aspects in reviews, or when we're making changes, because black makes
> sure the code aligns with the chosen style (1).

The problem is that although Black runs automatically it doesn't solve
the code problems automatically. Instead it takes something
questionable and produces something worse. If Black just rejected the
author's code and told them to write something better then they
probably would produce something better than what Black produces.

The limitation of Black is that it only reformats but usually at the
point when it does that the option of reformatting is not really the
thing that needs doing. Instead the right option is something like
introducing a new variable to split one statement into two but Black
just goes ahead and reformats without considering that option.

I'm fine with not arguing about what kinds of quotes to use but that
doesn't mean that I'll accept any output from Black without arguing
about the code being improved.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 3.10 Fizzbuzz

[Oscar Benjamin]

On Mon, 27 Feb 2023 at 21:06, Ethan Furman  wrote:
>
> On 2/27/23 12:20, rbowman wrote:
>
>  > "By using Black, you agree to cede control over minutiae of hand-
>  > formatting. In return, Black gives you speed, determinism, and freedom
>  > from pycodestyle nagging about formatting. You will save time and mental
>  > energy for more important matters."
>  >
>  > Somehow I don't think we would get along very well. I'm a little on the
>  > opinionated side myself.
>
> I personally cannot stand Black.  It feels like every major choice it makes 
> (and some minor ones) are exactly the
> opposite of the choice I make.

I agree partially. There are two types of decisions black makes:

1. Leave the code alone because it seems okay or make small modifications.
2. Reformat the code because it violates some generic rule (like line
too long or something).

I've recently tried Black and mostly for my code it seems to go with 1
(code looks okay). There might be some minor changes like double vs
single quotes but I really don't care about those. In that sense me
and Black seem to agree.

However I have also reviewed code where it is clear that the author
has used black and their code came under case 2. In that case Black
seems to produce awful things. What I can't understand is someone
accepting the awful rewrite rather than just fixing the code. Treating
Black almost like a linter makes sense to me but accepting the
rewrites that it offers for bad code does not.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Precision Tail-off?

[Oscar Benjamin]

On Sat, 18 Feb 2023 at 11:19, Peter J. Holzer  wrote:
>
> On 2023-02-18 03:52:51 +, Oscar Benjamin wrote:
> > On Sat, 18 Feb 2023 at 01:47, Chris Angelico  wrote:
> > > On Sat, 18 Feb 2023 at 12:41, Greg Ewing via Python-list
> > > > To avoid it you would need to use an algorithm that computes nth
> > > > roots directly rather than raising to the power 1/n.
> > > >
> > >
> > > It's somewhat curious that we don't really have that. We have many
> > > other inverse operations - addition and subtraction (not just "negate
> > > and add"), multiplication and division, log and exp - but we have
> > > exponentiation without an arbitrary-root operation. For square roots,
> > > that's not a problem, since we can precisely express the concept
> > > "raise to the 0.5th power", but for anything else, we have to raise to
> > > a fractional power that might be imprecise.
> >
> > Various libraries can do this. Both SymPy and NumPy have cbrt for cube 
> > roots:
>
> Yes, but that's a special case. Chris was talking about arbitrary
> (integer) roots. My calculator has a button labelled [x√y], but my
> processor doesn't have an equivalent operation.

All three of SymPy, mpmath and gmpy2 can do this as accurately as
desired for any integer root:

  >>> n = 12345678900

  >>> sympy.root(n, 6)
  10*13717421**(1/6)*3**(1/3)
  >>> sympy.root(n, 6).evalf(50)
  22314431635.562095902499928269233656421704825692573

  >>> mpmath.root(n, 6)
  mpf('22314431635.562096')
  >>> mpmath.mp.dps = 50
  >>> mpmath.root(n, 6)
  mpf('22314431635.562095902499928269233656421704825692572746')

  >>> gmpy2.root(n, 6)
  mpfr('22314431635.562096')
  >>> gmpy2.get_context().precision = 100
  >>> gmpy2.root(n, 6)
  mpfr('22314431635.56209590249992826924',100)

There are also specific integer only root routines like
sympy.integer_nthroot or gmpy2.iroot.

  >>> gmpy2.iroot(n, 6)
  (mpz(22314431635), False)
  >>> sympy.integer_nthroot(n, 6)
  (22314431635, False)

Other libraries like the stdlib math module and numpy define some
specific examples like cbrt or isqrt but not a full root or iroot.
What is lacking is a plain 64-bit floating point routine like:

  def root(x: float, n: int) -> float:
  return x ** (1/n) # except more accurate than this

It could be a good candidate for numpy and/or the math module. I just
noticed from the docs that the math module has a new in 3.11 cbrt
function that I didn't know about which suggests that a root function
might also be considered a reasonable addition in future. Similarly
isqrt was new in 3.8 and it is not a big leap from there to see
someone adding iroot.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Precision Tail-off?

[Oscar Benjamin]

On Sat, 18 Feb 2023 at 01:47, Chris Angelico  wrote:
>
> On Sat, 18 Feb 2023 at 12:41, Greg Ewing via Python-list
>  wrote:
> >
> > On 18/02/23 7:42 am, Richard Damon wrote:
> > > On 2/17/23 5:27 AM, Stephen Tucker wrote:
> > >> None of the digits in RootNZZZ's string should be different from the
> > >> corresponding digits in RootN.
> > >
> > > Only if the storage format was DECIMAL.
> >
> > Note that using decimal wouldn't eliminate this particular problem,
> > since 1/3 isn't exactly representable in decimal either.
> >
> > To avoid it you would need to use an algorithm that computes nth
> > roots directly rather than raising to the power 1/n.
> >
>
> It's somewhat curious that we don't really have that. We have many
> other inverse operations - addition and subtraction (not just "negate
> and add"), multiplication and division, log and exp - but we have
> exponentiation without an arbitrary-root operation. For square roots,
> that's not a problem, since we can precisely express the concept
> "raise to the 0.5th power", but for anything else, we have to raise to
> a fractional power that might be imprecise.

Various libraries can do this. Both SymPy and NumPy have cbrt for cube roots:

  >>> np.cbrt(12345678900.)
  4.979338592181745e+20

SymPy can also evaluate any rational power either exactly or to any
desired accuracy. Under the hood SymPy uses mpmath for the approximate
numerical evaluation part of this and mpmath can also be used directly
with its cbrt and nthroot functions to do this working with any
desired precision.

> But maybe, in practice, this isn't even a problem?

I'd say it's a small problem. Few people would use such a feature but
it would be have a little usefulness for those people if it existed.
Libraries like mpmath and SymPy provide this and can offer a big step
up for those who are really concerned about exactness or accuracy
though so there are already options for those who care. These are a
lot slower though than working with plain old floats but on the other
hand offer vastly more than a math.cbrt function could offer to
someone who needs something more accurate than x**(1/3).

For those who are working with floats the compromise is clear: errors
can accumulate in calculations. Taking the OPs example to the extreme,
the largest result that does not overflow is:

  >>> (123456789. * 10**300) ** (1.0 / 3.0)
  4.979338592181679e+102

Only the last 3 digits are incorrect so the error is still small. It
is not hard to find other calculations where *all* the digits are
wrong though:

  >>> math.cos(3)**2 + math.sin(3)**2 - 1
  -1.1102230246251565e-16

So if you want to use floats then you need to learn to deal with this
as appropriate for your use case. IEEE standards do their best to make
results reproducible across machines as well as limiting avoidable
local errors so that global errors in larger operations are *less
likely* to dominate the result. Their guarantees are only local though
so as soon as you have more complicated calculations you need your own
error analysis somehow. IEEE guarantees are in that case also useful
for those who actually want to do a formal error analysis.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Precision Tail-off?

[Oscar Benjamin]

On Fri, 17 Feb 2023 at 10:29, Stephen Tucker  wrote:
>
> Thanks, one and all, for your reponses.
>
> This is a hugely controversial claim, I know, but I would consider this
> behaviour to be a serious deficiency in the IEEE standard.
[snip]
>
> Perhaps this observation should be brought to the attention of the IEEE. I
> would like to know their response to it.

Their response would be that they are well aware of what you are
saying and knew about this already since before writing any standards.
The basic limitation of the IEEE standard in this respect is that it
describes individual operations rather than composite operations. Your
calculation involves composing operations, specifically:

result = x ** (n / d)

The problem is that there is more than one operation so we have to
evaluate this in two steps:

e = n / d
result = x ** e

Now the problem is that although n / d is correctly rounded e has a
small error because the exact value of n / d cannot be represented. In
the second operation taking this slightly off value of e as the
intended input means that the correctly rounded result for x ** e is
not the closest float to the true value of the *compound* operation.
The exponentiation operator in particular is very sensitive to changes
in the exponent when the base is large so the tiny error in e leads to
a more noticeable relative error in x ** e.

The only way to prevent this in full generality is to to have a system
in which no intermediate inexact operations are computed eagerly which
means representing expressions symbolically in some way. That is what
the SymPy code I showed does:

In [6]: from sympy import cbrt

In [7]: e = cbrt(1234567890)

In [8]: print(e)
1000*123456789**(1/3)

In [9]: e.evalf(50)
Out[9]: 49793385921817.447440261250171604380899353243631762

Because the *entire* expression is represented here *exactly* as e it
is then possible to evaluate different parts of the expression
repeatedly with different levels of precision and it is necessary to
do that for full accuracy in this case. Here evalf will use more than
50 digits of precision internally so that at the end you have a result
specified to 50 digits but where the error for the entire expression
is smaller than the final digit. If you give it a more complicated
expression then it will use even more digits internally for deeper
parts of the expression tree because that is what is needed to get a
correctly rounded result for the expression as a whole.

This kind of symbolic evaluation is completely outside the scope of
what the IEEE floating point standards are for. Any system based on
fixed precision and eager evaluation will show the same problem that
you have identified. It is very useful though to have a system with
fixed precision and eager evaluation despite these limitations. The
context for which the IEEE standards are mainly intended (e.g. FPU
instructions) is one in which fixed precision and eager evaluation are
the only option.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Precision Tail-off?

[Oscar Benjamin]

On Tue, 14 Feb 2023 at 07:12, Stephen Tucker  wrote:
>
> Hi,
>
> I have just produced the following log in IDLE (admittedly, in Python
> 2.7.10 and, yes I know that it has been superseded).
>
> It appears to show a precision tail-off as the supplied float gets bigger.
>
> I have two questions:
> 1. Is there a straightforward explanation for this or is it a bug?
> 2. Is the same behaviour exhibited in Python 3.x?
>
> For your information, the first 20 significant figures of the cube root in
> question are:
>49793385921817447440
>
> Stephen Tucker.
> --
> >>> 123.456789 ** (1.0 / 3.0)
> 4.979338592181744
> >>> 1234567890. ** (1.0 / 3.0)
> 49793385921817.36

You need to be aware that 1.0/3.0 is a float that is not exactly equal
to 1/3 and likewise the other float cannot have as many accurate
digits as is suggested by the number of zeros shown. Therefore you
should compare what exactly it means for the numbers you really have
rather than comparing with an exact cube root of the number that you
intended. Here I will do this with SymPy and calculate many more
digits than are needed. First here is the exact cube root:

In [29]: from sympy import *

In [30]: n = 1234567890

In [31]: cbrt(n).evalf(50)
Out[31]: 49793385921817.447440261250171604380899353243631762

So that's 50 digits of the exact cube root of the exact number and the
first 20 match what you showed. However in your calculation you use
floats so the exact expression that you evaluate is:

In [32]: e = Pow(Rational(float(n)), Rational(1.0/3.0), evaluate=False)

In [33]: print(e)
1234567888830049821836693930508288**(6004799503160661/18014398509481984)

Neither base or exponent is really the number that you intended it to
be. The first 50 decimal digits of this number are:

In [34]: e.evalf(50)
Out[34]: 49793385921817.360106660998131166304296436896582873

All of the digits in the calculation you showed match with the first
digits given here. The output from the float calculation is correct
given what the inputs actually are and also the available precision
for 64 bit floats (53 bits or ~16 decimal digits).

The reason that the results get further from your expectations as the
base gets larger is because the exponent is always less than 1/3 and
the relative effect of that difference is magnified for larger bases.
You can see this in a series expansion of a^x around x=1/3. Using
SymPy again:

In [37]: a, x = symbols('a, x')

In [38]: print(series(a**x, x, Rational(1, 3), 2))
a**(1/3) + a**(1/3)*(x - 1/3)*log(a) + O((x - 1/3)**2, (x, 1/3))

You can see that the leading relative error term from x being not
quite equal to 1/3 is proportional to the log of the base. You should
expect this difference to grow approximately linearly as you keep
adding more zeros in the base.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: HTTP server benchmarking/load testing in Python

[Benjamin Schollnick]

> On Jan 26, 2023, at 11:02 AM, Grant Edwards  wrote:
> 
> On 2023-01-26, Thomas Passin  wrote:
>> On 1/25/2023 7:38 PM, Peter J. Holzer wrote:
>>> On 2023-01-25 16:30:56 -0500, Thomas Passin wrote:
>>>> Great!  Don't forget what I said about potential overheating if you
>>>> hit the server with as many requests as it can handle.
>>> 
>>> Frankly, if you can overheat a server by hitting it with HTTP requests,
>>> get better hardware and/or put it into a place with better airflow.
>>> 
>> 
>> Frankly, if you have a server-grade machine then well and good but if 
>> you are running a nice quiet consumer grade laptop - my development 
>> machine - you need to be careful.
> 
> A properly designed laptop with a non-broken OS will not overheat
> regardless of the computing load you throw at it. The fan might get
> annoying loud, but if it overheats either your hardware or OS needs
> to be fixed.

Exactly.  

But what he might be thinking about is Thermal Throttling, which I keep seeing 
people attribute
to overheating….  

Overheating is not thermal throttling, it’s the OS and CPU protecting 
themselves from overheating.
Usually because the manufacturer didn’t add enough cooling to keep the system 
cool enough with a continuous load.  (Which to be honest, almost no laptop 
designers do, because they assuming you are going to be having a spiky load 
instead…  

- Benjamin

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ok, I feel stupid, but there must be a better way than this! (finding name of unique key in dict)

[Oscar Benjamin]

On Fri, 20 Jan 2023 at 17:30, Dino  wrote:
>
> let's say I have this list of nested dicts:
>
> [
>{ "some_key": {'a':1, 'b':2}},
>{ "some_other_key": {'a':3, 'b':4}}
> ]
>
> I need to turn this into:
>
> [
>{ "value": "some_key", 'a':1, 'b':2},
>{ "value": "some_other_key", 'a':3, 'b':4}
> ]

You want both the key and the value so you can use items():

In [39]: L = [
...:{ "some_key": {'a':1, 'b':2}},
...:{ "some_other_key": {'a':3, 'b':4}}
...: ]

In [40]: [{"value": k, **m} for l in L for k, m in l.items()]
Out[40]:
[{'value': 'some_key', 'a': 1, 'b': 2},
 {'value': 'some_other_key', 'a': 3, 'b': 4}]

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Does one have to use curses to read single characters from keyboard?

[Oscar Benjamin]

On Sun, 11 Dec 2022 at 15:55, Chris Green  wrote:
>
> Is the only way to read single characters from the keyboard to use
> curses.cbreak() or curses.raw()?  If so how do I then read characters,
> it's not at all obvious from the curses documentation as that seems to
> think I'm using a GUI in some shape or form.
>
> All I actually want to do is get 'Y' or 'N' answers to questions on
> the command line.
>
> Searching for ways to do this produces what seem to me rather clumsy
> ways of doing it.

What you are asking for is known as getch. On Windows Python's msvcrt
module provides precisely this function. There are ways to do it on
non-Windows platforms but nothing as simple as the getch function is
in the stdlib. Some modules and recipes are available which I guess it
what you've already seen:

https://pypi.org/project/getch/
https://github.com/joeyespo/py-getch
https://stackoverflow.com/questions/510357/how-to-read-a-single-character-from-the-user

This seems to be the most well maintained option:
https://pypi.org/project/readchar/

I've often thought that getch was a good candidate for the stdlib.
There are plenty of recipes around but it would ideally just be
available as a cross platform function. Using curses would have been
overkill in any of my use cases where I really just wanted getch to
make a more efficient interface for a terminal program having some
limited interactivity. Actually slightly more than getch is readchar's
readkey which also works for pressing non-character keys. There were
times in the past where I might have used that if I'd known about it.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Parallel(?) programming with python

[Oscar Benjamin]

On Mon, 8 Aug 2022 at 19:01, Andreas Croci  wrote:
>
> tI would like to write a program, that reads from the network a fixed
> amount of bytes and appends them to a list. This should happen once a
> second.
>
> Another part of the program should take the list, as it has been filled
> so far, every 6 hours or so, and do some computations on the data (a FFT).
>
> Every so often (say once a week) the list should be saved to a file,
> shorthened in the front by so many items, and filled further with the
> data coming fom the network. After the first saving of the whole list,
> only the new part (the data that have come since the last saving) should
> be appended to the file. A timestamp is in the data, so it's easy to say
> what is new and what was already there.
>
> I'm not sure how to do this properly: can I write a part of a program
> that keeps doing its job (appending data to the list once every second)
> while another part computes something on the data of the same list,
> ignoring the new data being written?
>
> Basically the question boils down to wether it is possible to have parts
> of a program (could be functions) that keep doing their job while other
> parts do something else on the same data, and what is the best way to do
> this.

Why do these "parts of a program" need to be part of the *same*
program. I would write this as just two separate programs. One
collects the data and writes it to a file. The other periodically
reads the file and computes the DFT.

Note that a lot of the complexity discussed in other posts to do with
threads and locks etc comes from the supposed constraint that this
needs to be done with threads or something else that can work in
parallel *within the same program*. If you relax that constraint the
problem becomes a lot simpler.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why isn't there a built-in product()?

[Oscar Benjamin]

On Thu, 7 Jul 2022 at 22:55, Michael F. Stemper
 wrote:
>
> sum() is wonderful.
>
>   >>> nums = [1,2,3]
>   >>> sum(nums)
>   6
>   >>> product(nums)
>   Traceback (most recent call last):
> File "", line 1, in 
>   NameError: name 'product' is not defined
>   >>>
>
> I understand that there is no product() or prod(). Does anybody
> here know why that was not included in the language? It seems
> as if it would be useful, so there must have been some rationale
> for that decision.

There is math.prod:

  >>> from math import prod
  >>> prod([1, 2, 3, 4])
  24

https://docs.python.org/3/library/math.html#math.prod

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Automatic Gain Control in Python?

[Benjamin Schollnick]

> I doubt you even need to write any code to do that. Sox can normalize
> audio levels in files with a single command.

Correct.  

My phrasing was slightly misleading.

There are plenty of applications that should do that.  I was thinking of one 
that Marco Arment makes, but I can’t locate it for reference.

- Benjamin



> On May 29, 2022, at 3:18 PM, Grant Edwards  wrote:
> 
> On 2022-05-29, Benjamin Schollnick  wrote:
> 
>> Why not just right a 3rd party package to normalize the audio levels
>> in the digital file?  It’ll be faster, and probably easier than
>> trying to do it in real time…
> 
> I doubt you even need to write any code to do that. Sox can normalize
> audio levels in files with a single command.
> 
> http://sox.sourceforge.net/
> 
> --
> Grant
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Automatic Gain Control in Python?

[Benjamin Schollnick]

Well, maybe a different approach might be better?

https://www.amazon.com/LEMONKTV-Control-Device-Karaoke-Machine/dp/B07H49JB8S 
<https://www.amazon.com/LEMONKTV-Control-Device-Karaoke-Machine/dp/B07H49JB8S>

I’m unclear on how bringing the audio track into a computer to examine detect 
the audio level, and then adjusting the volume out of the Smart Speaker is 
really doing more than adding complexity.

An all analog solution might be the better route, although finding something 
that is inexpensive might be an issue as well.

- Benjamin



> On May 29, 2022, at 11:32 AM, Steve GS  wrote:
> 
> No, not a digital file. I am playing the podcast/broadcast live to the 
> community using a separate network of smart speakers (Amazon Echo). Commands 
> are being sent hourly through a speaker to the SS from an excel program. I 
> want to monitor the audio level between the line-out of the SS and the input 
> to another computer which then records the audio using Audacity for a single 
> replay during the week.
> 
> I think my first post should have started “Fasten your seat belts, it is 
> going to be a bumpy night…)
>  
>  
> Genie: You have three wishes.
> Me: I wish I had more wishes.
> Genie: You cannot wish for more wishes.
> Me: I wish I could.
>  
> From: Benjamin Schollnick  <mailto:bscholln...@schollnick.net>> 
> Sent: Sunday, May 29, 2022 11:18 AM
> To: Steve GS mailto:Gronicus@SGA.Ninja>>
> Cc: Richard Damon  <mailto:rich...@damon-family.org>>; Python  <mailto:python-list@python.org>>
> Subject: Re: Automatic Gain Control in Python?
>  
> Okay, you are capturing the audio stream as a digital file somewhere, correct?
>  
> Why not just right a 3rd party package to normalize the audio levels in the 
> digital file?  It’ll be faster, and probably easier than trying to do it in 
> real time…
>  
> eg. 
> https://campus.datacamp.com/courses/spoken-language-processing-in-python/manipulating-audio-files-with-pydub?ex=8
>  
> <https://campus.datacamp.com/courses/spoken-language-processing-in-python/manipulating-audio-files-with-pydub?ex=8>
>  
>> Normalizing an audio file with PyDub
>> 
>> Sometimes you'll have audio files where the speech is loud in some portions 
>> and quiet in others. Having this variance in volume can hinder transcription.
>> Luckily, PyDub's effects module has a function called normalize() which 
>> finds the maximum volume of an AudioSegment, then adjusts the rest of the 
>> AudioSegment to be in proportion. This means the quiet parts will get a 
>> volume boost.
>> You can listen to an example of an audio file which starts as loud then goes 
>> quiet, loud_then_quiet.wav, here 
>> <https://assets.datacamp.com/production/repositories/4637/datasets/9251c751d3efccf781f3e189d68b37c8d22be9ca/ex3_datacamp_loud_then_quiet.wav>.
>> In this exercise, you'll use normalize() to normalize the volume of our 
>> file, making it sound more like this 
>> <https://assets.datacamp.com/production/repositories/4637/datasets/f0c1ba35ff99f07df8cfeee810c7b12118d9cd0f/ex3_datamcamp_normalized_loud_quiet.wav>.
> or
>  
> https://stackoverflow.com/questions/57925304/how-to-normalize-a-raw-audio-file-with-python
>  
> <https://stackoverflow.com/questions/57925304/how-to-normalize-a-raw-audio-file-with-python>
> 
> 
>  
> - Benjamin
> 
> 
>> On May 29, 2022, at 11:04 AM, Steve GS > <mailto:Gronicus@SGA.Ninja>> wrote:
>>  
>>>> From your description, your fundamental problem is you are trying to 
>>>> automatically "control" things that weren't designed to be automatically 
>>>> controlled in the way you are attempting.
>> 
>> How so? I am sending commands to a smart speaker and it plays podcasts and 
>> broadcasts.
>> How is this a violation of SS design?
>> 
>> ===
>> 
>>>> The smart speaker assumes the "user" will adjust the volume either with 
>>>> the controls or with verbal commands, So things will be a bit "clunky" in 
>>>> your results as you command the smart speaker volume level.
>> 
>> So, you want me to sit here for every hour of the weekend and monitor the 
>> audio levels for a result that will get, at best, one replay when I believe 
>> it can be automated.
>> 
>> ===
>> 
>>>> Yes, you have an automated system that does most of what you want, but it 
>>>> is based on pieces not designed to be automated in this way, and you are 
>>>> running into the limitations caused by that.
>> 
>> Again, what limitations 

Re: Automatic Gain Control in Python?

[Benjamin Schollnick]

Okay, you are capturing the audio stream as a digital file somewhere, correct?

Why not just right a 3rd party package to normalize the audio levels in the 
digital file?  It’ll be faster, and probably easier than trying to do it in 
real time…

eg. 
https://campus.datacamp.com/courses/spoken-language-processing-in-python/manipulating-audio-files-with-pydub?ex=8
 
<https://campus.datacamp.com/courses/spoken-language-processing-in-python/manipulating-audio-files-with-pydub?ex=8>

Normalizing an audio file with PyDub

Sometimes you'll have audio files where the speech is loud in some portions and 
quiet in others. Having this variance in volume can hinder transcription.

Luckily, PyDub's effects module has a function called normalize() which finds 
the maximum volume of an AudioSegment, then adjusts the rest of the 
AudioSegment to be in proportion. This means the quiet parts will get a volume 
boost.

You can listen to an example of an audio file which starts as loud then goes 
quiet, loud_then_quiet.wav, here 
<https://assets.datacamp.com/production/repositories/4637/datasets/9251c751d3efccf781f3e189d68b37c8d22be9ca/ex3_datacamp_loud_then_quiet.wav>.

In this exercise, you'll use normalize() to normalize the volume of our file, 
making it sound more like this 
<https://assets.datacamp.com/production/repositories/4637/datasets/f0c1ba35ff99f07df8cfeee810c7b12118d9cd0f/ex3_datamcamp_normalized_loud_quiet.wav>.

or

https://stackoverflow.com/questions/57925304/how-to-normalize-a-raw-audio-file-with-python
 
<https://stackoverflow.com/questions/57925304/how-to-normalize-a-raw-audio-file-with-python>


- Benjamin

> On May 29, 2022, at 11:04 AM, Steve GS  wrote:
> 
>>> From your description, your fundamental problem is you are trying to 
>>> automatically "control" things that weren't designed to be automatically 
>>> controlled in the way you are attempting.
> 
> How so? I am sending commands to a smart speaker and it plays podcasts and 
> broadcasts.
> How is this a violation of SS design?
> 
> ===
>>> The smart speaker assumes the "user" will adjust the volume either with the 
>>> controls or with verbal commands, So things will be a bit "clunky" in your 
>>> results as you command the smart speaker volume level.
> 
> So, you want me to sit here for every hour of the weekend and monitor the 
> audio levels for a result that will get, at best, one replay when I believe 
> it can be automated.
> 
> ===
>>> Yes, you have an automated system that does most of what you want, but it 
>>> is based on pieces not designed to be automated in this way, and you are 
>>> running into the limitations caused by that.
> 
> Again, what limitations of the SS am I violating? It is designed to receive 
> commands and play the audio.
> Also, what makes you think that you know how my program is based?
> 
> ===
>>> Yes, you could split the aux-out to bring it into another computer to 
>>> listen to the sound level, and then using a sound input package get samples 
>>> of what is playing, and analyze that data to get an average volume, and 
>>> then issues the command to adjust the volume level.
> 
> Hmmm, is that not my original question? Are you suggesting to monitor the 
> audio, sense it for volume changes and apply those changes to the original 
> audio? One thing that may have to happen is a timed-delay to all for the AGC 
> to work.   This may require a correlation circuit.
> 
> ==
>>> What you seem to be missing is that you could get the podcasts from a 
>>> browser, and all a browser is is a program. It isn't that much work to 
>>> write a rudimentary browser in python, especially if you don't actually 
>>> need to display the results to a user, but are only trying to automate a 
>>> particular task.
> 
> Writing my own browser in Python might work. Do you have a sample one that I 
> could twerk to fit my needs?
> I would have to be able to invoke it and an hour later devoke it least I end 
> up with multiple audio channels playing.
> 
> Either way, I would still need an AGC program which was my original question. 
>  
> 
> ===
>>> You seem to feel strongly invested in your current code base, which is 
>>> understandable, but it seems you have reached a point where you don't want 
>>> to live with the limitations CAUSED by that system. 
> 
> The changes in volume are not CAUSED by my program. The want to fix them is a 
> new development to improve the product. The volume fluctuations are causes, 
> or neglections, by the engineers at the sources of podcasts and broadcasts. 
&

Re: C is it always faster than nump?

[Oscar Benjamin]

On Sat, 26 Feb 2022 at 03:10, Dennis Lee Bieber  wrote:
>
> On Fri, 25 Feb 2022 23:06:57 + (UTC), Avi Gross 
> declaimed the following:
>
> >I do have to wonder if anyone ever considered adding back enough 
> >functionality into base Python to make some additions less needed. Is there 
> >any reason the kind of structures used by so many languages cannot be made 
> >part of python such as a vector/array that holds exactly one kind of data 
> >structure and not force use of things like a list when that is more than is 
> >needed?
> >
> https://docs.python.org/3/library/array.html
>
> seems to fit the criteria...

The stdlib array module is basically unused in comparison to NumPy.
The capabilities of the array module do not meet the needs for most
users who want to do anything useful with arrays.

The intention in creating NumPy  (in the NumPy/SciPy split) was that
it might be possible that NumPy could be merged into core Python.
Unfortunately that didn't come to be.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: C is it always faster than nump?

[Oscar Benjamin]

On Fri, 25 Feb 2022 at 23:13, Barry  wrote:
>
> > On 25 Feb 2022, at 23:00, Richard Damon  wrote:
> >
> > On 2/25/22 2:47 PM, Chris Angelico wrote:
> >>> On Sat, 26 Feb 2022 at 05:49, Richard Damon  
> >>> wrote:
> >>> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
>  Hi,
>  a lot of people think that C (or C++) is faster than python, yes I agree,
>  but I think that's not the case with numpy, I believe numpy is faster 
>  than
>  C, at least in some cases.
> 
> >>> My understanding is that numpy is written in C, so for it to be faster
> >>> than C, you are saying that C is faster that C.
> >> Fortran actually, but ultimately, they both get compiled to machine code.
> >
> > Looking at the Github repo I see:
> >
> > Languages:
> > Python.  62.5%
> > C   35.3%
> > C++.   1.0%
> > Cython.   0.9%
> > Shell.   0.2%
> > Fortran.   0.1%
>
> I assume that this is just for bumpy and not for all its dependencies.
> That will add a lot of Fortran and c++ I expect.

NumPy links with BLAS/LAPACK that will do the heavy lifting for common
linear algebra operations. Multiple different BLAS libraries can be
used with NumPy and those libraries might be written in Fortran and
might also involve some hand-crafted assembly for particular
architectures. Some explanation of NumPy's BLAS/LAPACK support is
here:
https://numpy.org/devdocs/user/building.html

By default if you install NumPy from conda (or at least if you install
the Anaconda distribution) then I think that NumPy will use the Intel
MKL library:
https://en.wikipedia.org/wiki/Math_Kernel_Library
As I understand it the core of MKL is Fortran and is compiled with
Intel's ifortran compiler but some parts might be C/C++. MKL is also
the same library that is used by Matlab for its own heavy lifting.
It's not sufficiently free to be used in NumPy's PyPI wheels though.

If you install the precompiled NumPy wheels with pip from PyPI then I
think those are statically linked with OpenBLAS:
https://github.com/xianyi/OpenBLAS
Again I think the core of OpenBLAS is Fortran but there's some C in there.

In the pre-wheel days the situation was that NumPy provided installer
files for Windows that would give binaries linked with ATLAS (also
Fortran):
https://en.wikipedia.org/wiki/Automatically_Tuned_Linear_Algebra_Software

I think at some point NumPy used to use the OSX Accelerate library but
the page I linked above says that's now deprecated. I don't know
anything about Accelerate but I wouldn't be surprised to hear that it
was a bunch of old Fortran code!

If you build NumPy from source without having any BLAS/LAPACK
libraries then I think it uses its own backup version of these that is
written in C but not as well optimised. This used to be the default
for a pip install on Linux in pre-wheel times.

Many operations in NumPy don't actually use BLAS/LAPACK and for those
parts the heavy lifting is all done in NumPy's own C code.

Lastly SciPy which is very often used together with NumPy does have a
load of Fortran code. As I understand it at some point NumPy and SciPy
were divided from the previous numerical Python libraries and there
was a deliberate split so that the Fortran code all ended up in SciPy
rather than NumPy.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Best way to check if there is internet?

[Benjamin Schollnick]

>>> Thanks for the in-between. I really like the Python comunity as,
>>> even though it's a 'scripting' language,
> 
> And we like you, even though you're only a ...
> 
> In English, a statement like that is considered rather insulting.
> 
>> To me, it's a programming language. In my usage, a "script" is a
>> bunch of OS commands.
> 
> Exactly. A script is a set of commands that you would have normally
> entered by hand at a command prompt. But, after having gone through
> the process a few times, you decided to shove them into a file to save
> on typing.
> 
> Python is a programming language. Period.

Exactly.  In my opinion “It’s a scripting language”, is a devaluation of the 
language.
It’s an attempt to throw python in to a “trivial” classification.  

Heck, the same argument can be made of Bash, PHP, Perl, and a few other 
languages as well.

How many “scripts” have been throw quickly together in Perl, or PHP?  

Quite a damn few, yet, would anyone call Wordpress a “script”?

It’s effectively damning with faint praise.

- Benjamin


-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46736] Generate HTML 5 with SimpleHTTPRequestHandler.list_directory

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 0bb40a42d71873ea267aace8c92a02d66fe36dc2 by Dong-hee Na in branch 
'main':
closes bpo-46736: SimpleHTTPRequestHandler now uses HTML5. (GH-31533)
https://github.com/python/cpython/commit/0bb40a42d71873ea267aace8c92a02d66fe36dc2


--
nosy: +benjamin.peterson
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue46736>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45459] Limited API support for Py_buffer

[Benjamin Peterson]

Benjamin Peterson  added the comment:

clang doesn't like the typedef forward-decl:

In file included from ../cpython/Modules/_ctypes/_ctypes.c:108:
In file included from ../cpython/Include/Python.h:43:
../cpython/Include/object.h:109:3: warning: redefinition of typedef 'PyObject' 
is a C11 feature [-Wtypedef-redefinition]
} PyObject;
  ^
../cpython/Include/pybuffer.h:23:24: note: previous definition is here
typedef struct _object PyObject;
   ^
1 warning generated.

--
nosy: +benjamin.peterson

___
Python tracker 
<https://bugs.python.org/issue45459>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Pypy with Cython

[Oscar Benjamin]

On Thu, 3 Feb 2022 at 23:16, Greg Ewing  wrote:
>
> On 4/02/22 5:07 am, Albert-Jan Roskam wrote:
> > On Feb 3, 2022 17:01, Dan Stromberg  wrote:
> >
> > What profiler do you recommend
>
> If it runs for that long, just measuring execution time should
> be enough. Python comes with a "timeit" module to help with
> that, or you can use whatever your OS provides (e.g. the
> "time" shell command in unices).

Yes, exactly. It's important to distinguish between timing or
benchmarking as compared to profiling.

When you use a profiler it does not usually give an accurate
representation of the time taken by the same code when the profiler is
not running because of the overhead added by the profiler itself. The
purpose of the profiler is instead to give lots of information that
can help you to *think* about how to make the code faster (e.g. what
proportion of time is spent in different functions or how many times
is a function called etc). This information is useful for considering
at a *high level* what parts of the code could be improved to make it
faster by e.g. calling a particular function less or making that
function faster.

The timeit module can be used to time micro-operations i.e. things
that take nanoseconds or maybe milliseconds. It repeats an operation
in a loop which is often needed to get reliable timings for things
that are otherwise too fast to reliably time from a single run. It can
give information that helps to identify good approaches to try at a
*low level* e.g. when optimising a single line of code.

If you want to *evaluate* whether or not a change actually makes your
*whole* program faster you should just run your code normally but time
how long it takes (which is what the unix "time" command does). You
can also use time.time() from Python for this. Profilers and timeit
help to identify ways to speed up your code but should not be used as
the way to assess the final impact of the changes you make to a long
running program.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue46626] expose IP_BIND_ADDRESS_NO_PORT linux socket option

[Benjamin Peterson]

New submission from Benjamin Peterson :


New changeset 1aa6be06c4cb7f04a340adb1c7b16b89803ef254 by Benjamin Peterson in 
branch 'main':
closes bpo-46626: Expose IP_BIND_ADDRESS_NO_PORT socket option. (GH-31106)
https://github.com/python/cpython/commit/1aa6be06c4cb7f04a340adb1c7b16b89803ef254


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue46626>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46626] expose IP_BIND_ADDRESS_NO_PORT linux socket option

[Benjamin Peterson]

Change by Benjamin Peterson :


--
keywords: +patch
pull_requests: +29289
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31106

___
Python tracker 
<https://bugs.python.org/issue46626>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46626] expose IP_BIND_ADDRESS_NO_PORT linux socket option

[Benjamin Peterson]

Change by Benjamin Peterson :


--
components: Library (Lib)
nosy: benjamin.peterson
priority: normal
severity: normal
status: open
title: expose IP_BIND_ADDRESS_NO_PORT linux socket option
type: enhancement
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46626>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46253] C API documentation of Py_UNICODE_* character properties macros use Py_UNICODE instead of Py_UCS4

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 43c5c1369cb21f08a1dc1d63923c3586b883e3e8 by Julian Gilbey in 
branch 'main':
closes bpo-46253: Change Py_UNICODE to Py_UCS4 in the C API docs to match the 
current source code (GH-30387)
https://github.com/python/cpython/commit/43c5c1369cb21f08a1dc1d63923c3586b883e3e8


--
nosy: +benjamin.peterson
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue46253>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38522] Py_USING_MEMORY_DEBUGGER is referenced in docs but not present in code

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 9f0e40fae5191c3e3ed6109bd2e2f97aa0ac8d64 by Miss Islington (bot) 
in branch '3.10':
closes bpo-38522 docs: remove references to Py_USING_MEMORY_DEBUGGER (GH-30284) 
(GH-30295)
https://github.com/python/cpython/commit/9f0e40fae5191c3e3ed6109bd2e2f97aa0ac8d64


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue38522>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Call julia from Python: which package?

[Oscar Benjamin]

On Fri, 17 Dec 2021 at 23:11, Chris Angelico  wrote:
>
> On Sat, Dec 18, 2021 at 10:01 AM Oscar Benjamin
>  wrote:
> >
> > On Fri, 17 Dec 2021 at 22:40, Chris Angelico  wrote:
> > >
> > > On Sat, Dec 18, 2021 at 9:24 AM Oscar Benjamin
> > >  wrote:
> > > > When I timed the result in Julia and in Python I found that the Julia
> > > > code was slower than the Python code. Of course I don't know how to
> > > > optimise Julia code so I asked one of my colleagues who does (and who
> > > > likes to proselytise about Julia). He pointed me to here where the
> > > > creator of Julia says "BigInts are currently pretty slow in Julia":
> > > > https://stackoverflow.com/questions/37193586/bigints-seem-slow-in-julia#:~:text=BigInts%20are%20currently%20pretty%20slow,that%20basic%20operations%20are%20fast.
> > > > I should make clear here that I used the gmpy2 library in Python for
> > > > the basic integer operations which is a wrapper around the same gmp
> > > > library that is used by Julia. That means that the basic integer
> > > > operations were being done by the same gmp library (a C library) in
> > > > each case. The timing differences between Python and Julia are purely
> > > > about overhead around usage of the same underlying C library.
> > >
> > > Point of note: "Python actually uses a hybrid approach where small
> > > integer values are represented inline and only when values get too
> > > large are they represented as BigInts" might be sorta-kinda true for
> > > Python 2, but it's not true for Python 3. In all versions of CPython
> > > to date, all integers are objects, they're not "represented inline"
> > > (there *are* some languages that do this intrinsically, and I think
> > > that PyPy can sometimes unbox integers, but CPython never will); and
> > > the performance advantage of Py2's machine-sized integers clearly
> > > wasn't worth recreating in Py3. (It would be possible to implement it
> > > in Py3 as an optimization, while still having the same 'int' type for
> > > both, but nobody's done it.)
> > >
> > > So if Python's integers are faster than Julia's, it's not because
> > > there's any sort of hybrid approach, it's simply because CPython is
> > > more heavily optimized for that sort of work.
> > >
> > > (That said: I have no idea whether a StackOverflow answer from 2016 is
> > > still applicable.)
> >
> > To be clear: I wasn't using Python's int type. I used the gmpy2.mpz
> > type which is precisely the same mpz from the same gmp library that
> > Julia uses. I'm told by my colleague that Julia has a lot of overhead
> > when using "heap types" which is possibly the cause of the problem.
>
> Ah, interesting. What's the advantage of using mpz instead of Python's
> builtin int?

In this particular context the advantage was to give parity to the two
languages I was profiling but in general gmpy2/flint have faster large
integer arithmetic than Python's int type:

In [13]: from gmpy2 import mpz

In [14]: nums_int = [3**i for i in range(1000)]

In [15]: nums_mpz = [mpz(3)**i for i in range(1000)]

In [16]: def prod(nums):
...: result = nums[0]
...: for num in nums[1:]:
...: result *= num
...: return result
...:

In [17]: %time ok = prod(nums_int)
CPU times: user 384 ms, sys: 12 ms, total: 396 ms
Wall time: 398 ms

In [18]: %time ok = prod(nums_mpz)
CPU times: user 124 ms, sys: 0 ns, total: 124 ms
Wall time: 125 ms

That's somewhat significant but the big difference for SymPy in using
gmpy2 (as an optional dependency) is the mpq rational type which is
significantly faster for small rational numbers:

In [19]: from gmpy2 import mpq

In [20]: from fractions import Fraction

In [21]: nums_mpq = [mpq(i, 3) for i in range(1000)]

In [22]: nums_frac = [Fraction(i, 3) for i in range(1000)]

In [23]: %time ok = sum(nums_mpq)
CPU times: user 0 ns, sys: 0 ns, total: 0 ns
Wall time: 633 µs

In [24]: %time ok = sum(nums_frac)
CPU times: user 8 ms, sys: 0 ns, total: 8 ms
Wall time: 10.2 ms

For some slow operations SymPy is about 30x faster when gmpy2 is installed.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Call julia from Python: which package?

[Oscar Benjamin]

On Fri, 17 Dec 2021 at 22:40, Chris Angelico  wrote:
>
> On Sat, Dec 18, 2021 at 9:24 AM Oscar Benjamin
>  wrote:
> > When I timed the result in Julia and in Python I found that the Julia
> > code was slower than the Python code. Of course I don't know how to
> > optimise Julia code so I asked one of my colleagues who does (and who
> > likes to proselytise about Julia). He pointed me to here where the
> > creator of Julia says "BigInts are currently pretty slow in Julia":
> > https://stackoverflow.com/questions/37193586/bigints-seem-slow-in-julia#:~:text=BigInts%20are%20currently%20pretty%20slow,that%20basic%20operations%20are%20fast.
> > I should make clear here that I used the gmpy2 library in Python for
> > the basic integer operations which is a wrapper around the same gmp
> > library that is used by Julia. That means that the basic integer
> > operations were being done by the same gmp library (a C library) in
> > each case. The timing differences between Python and Julia are purely
> > about overhead around usage of the same underlying C library.
>
> Point of note: "Python actually uses a hybrid approach where small
> integer values are represented inline and only when values get too
> large are they represented as BigInts" might be sorta-kinda true for
> Python 2, but it's not true for Python 3. In all versions of CPython
> to date, all integers are objects, they're not "represented inline"
> (there *are* some languages that do this intrinsically, and I think
> that PyPy can sometimes unbox integers, but CPython never will); and
> the performance advantage of Py2's machine-sized integers clearly
> wasn't worth recreating in Py3. (It would be possible to implement it
> in Py3 as an optimization, while still having the same 'int' type for
> both, but nobody's done it.)
>
> So if Python's integers are faster than Julia's, it's not because
> there's any sort of hybrid approach, it's simply because CPython is
> more heavily optimized for that sort of work.
>
> (That said: I have no idea whether a StackOverflow answer from 2016 is
> still applicable.)

To be clear: I wasn't using Python's int type. I used the gmpy2.mpz
type which is precisely the same mpz from the same gmp library that
Julia uses. I'm told by my colleague that Julia has a lot of overhead
when using "heap types" which is possibly the cause of the problem.

The other library I referenced (flint) does have a super-optimised
approach for handling small integers natively as machine types at the
same time as supporting large integers and even includes hand-written
assembly for the most basic operations. That library is significantly
faster than Python's int or gmpy2/gmp.

In any case what it shows is that Julia does not have reduced overhead
*in general* (i.e. for all situations) as compared to Python. I
expected that but I also thought that this particular example would be
within the range of things where Julia out-performed Python and that
turned out not to be the case.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Call julia from Python: which package?

[Oscar Benjamin]

On Fri, 17 Dec 2021 at 15:04, Albert-Jan Roskam  wrote:
>
> Hi,
>
> I have a Python program that uses Tkinter for its GUI. It's rather slow so I 
> hope to replace many or all of the non-GUI parts by Julia code. Has anybody 
> experience with this? Any packages you can recommend? I found three 
> alternatives:
>
> * https://pyjulia.readthedocs.io/en/latest/usage.html#
> * https://pypi.org/project/juliacall/
> * https://github.com/JuliaPy/PyCall.jl

Like others here I don't have any particular advice to offer for the
actual question you have asked. Probably you would get better answers
on a Julia mailing list.

Before you consider rewriting any of your Python code in Julia I
suggest trying it out to see if you can actually demonstrate a proof
of concept that it can even be faster for your particular problem.
Also like others here I suggest that you explore the options for
speeding things up within Python. Putting together a Frankenstein
application that's half in Julia and half in Python might be worth it
but that approach has significant costs so make sure you are clear
that there are going to be benefits before you go down that road.

I do have a very small amount of experience using Julia. As a
maintainer of SymPy I was very interested by Symbolics.jl which was
created in an attempt to be faster than SymPy. I spent some time going
through tutorials for Julia and making toy programs and then I tried
out Symbolics.jl. I also had various discussions with those involved
in Symbolics.jl on Julia mailing lists and GitHub.

Based on my experience with Julia the immediate conclusion I have that
is relevant for you is this:

Do *not* presume that writing your code in Julia rather than Python
will make it faster.

There is a lot of hype about Julia and its proselytisers will claim
that it "runs at the speed of C". That might be sort of true for
certain specific kinds of code or particular kinds of problems but it
is certainly not true in general that Julia runs at the speed of C. In
fact I have seen plenty of basic cases where Julia is slower than
Python.

The other thing that I have seen plenty of in discussions about
performance in both Python and Julia is poor use of benchmark results.
The big culprit is timeit and its equivalent btime in Julia. When I
tried to discuss the speed of operations in Julia it was suggested
that I should use btime but there are so many different levels of
caching in Julia (at least in Symbolics.jl) that the btime results are
meaningless. It seems to be common practice in the Julia community to
refer to btime results just as it is in Python to refer to timeit
results.

I think that there is a specific class of problems where Julia is
significantly faster than (CPython) which in practice is mostly around
intensive non-vectorisable floating point calculations. Note that
these are the same kind of things that can be made a lot faster if you
use PyPy or Numba or any of the related options that can be done
without rewriting your existing Python code in another language. In
Julia just as in PyPy/Numba etc it's important for the speed gain that
you can actually reduce your operations down to low-level machine
types like float64, int32 etc.

I tried writing a matrix multiplication routine in Julia to see how it
would compare with Python. I used a straight-forward dumb
implementation in both languages. I wanted to compare with something
like this which is pure Python:
https://github.com/sympy/sympy/blob/88ed7abb488da615b007dd2ed5404312caef473c/sympy/polys/matrices/dense.py#L85-L91
Note that in that particular application it isn't possible to use
something like int32 because it's part of a symbolic library that
performs exact calculations that can easily overflow any standard
machine types e.g. the determinant of a 20x20 matrix of integers
between 0 and 100 easily overflows the range of a 64 bit integer:

>>> from sympy import *
>>> det(randMatrix(20, 20))
1389363438512397826059139369892638115688

When I timed the result in Julia and in Python I found that the Julia
code was slower than the Python code. Of course I don't know how to
optimise Julia code so I asked one of my colleagues who does (and who
likes to proselytise about Julia). He pointed me to here where the
creator of Julia says "BigInts are currently pretty slow in Julia":
https://stackoverflow.com/questions/37193586/bigints-seem-slow-in-julia#:~:text=BigInts%20are%20currently%20pretty%20slow,that%20basic%20operations%20are%20fast.
I should make clear here that I used the gmpy2 library in Python for
the basic integer operations which is a wrapper around the same gmp
library that is used by Julia. That means that the basic integer
operations were being done by the same gmp library (a C library) in
each case. The timing differences between Python and Julia are purely
about overhead around usage of the same underlying C library.

The Julia code was 2x slower than Python in this task. By comparison
flint is a library that 

[issue45582] Rewrite getpath.c in Python

[Benjamin Peterson]

Change by Benjamin Peterson :


--
nosy: +benjamin.peterson
nosy_count: 7.0 -> 8.0
pull_requests: +28131
pull_request: https://github.com/python/cpython/pull/29907

___
Python tracker 
<https://bugs.python.org/issue45582>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: pyinstaller wrong classified as Windows virus

[Benjamin Schollnick]

>> Windows Defender has a setting to also use “Reputation Scoring”.
>> What that simply means is that WDef will report back a hash to microsoft 
>> which is then checked to see if it is known.  If it is known, then it has a 
>> reputation and based off that reputation Defender will either allow it to 
>> run or not.
>> But if there is no reputation (eg no one has ever run it), that’s 
>> suspicious.  And that’s what you are running into.
>> You can submit the EXE to the defender team, which should allow it to 
>> operate properly without any issue.
>>  - Benjamin
> 
> sure... "that's suspicious". Unless you're a developer compiling your own 
> code. In which case every fresh build will be something "unknown". You have 
> to set every setting you can find to "developer mode" to help with this kind 
> of thing, and sometimes it's still not enough.

Understandable, and Steve Gibson (GRC.com <http://grc.com/>, creator of 
Spinrite) has mentioned that he has had this problem with every Beta of his 
applications.

I agree completely with you, but that’s how Microsoft has set things up.

- Benjamin


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: pyinstaller wrong classified as Windows virus

[Benjamin Schollnick]

Windows Defender has a setting to also use “Reputation Scoring”.

What that simply means is that WDef will report back a hash to microsoft which 
is then checked to see if it is known.  If it is known, then it has a 
reputation and based off that reputation Defender will either allow it to run 
or not.

But if there is no reputation (eg no one has ever run it), that’s suspicious.  
And that’s what you are running into.  

You can submit the EXE to the defender team, which should allow it to operate 
properly without any issue.

- Benjamin



> On Nov 29, 2021, at 1:57 PM, Barry  wrote:
> 
> 
> 
>> On 29 Nov 2021, at 00:03, anthony.flury via Python-list 
>>  wrote:
>> 
>> 
>> On 26/11/2021 07:13, Ulli Horlacher wrote
>>>> But consider another possibility that your compiler software is compromised
>>> Then https://www.python.org/ftp/python/3.10.0/python-3.10.0-amd64.exe
>>> is infected. I doubt this.
>> 
>> But you aren't using python3.10 to 'compile' the code to the executable that 
>> windows complains about: you are using pyinstaller, which if memory serves 
>> is a 3rd party application.
>> 
>> I assume that you have no problem running the script without pyinstaller ?
>> 
>> so Might pyinstaller be compromised in some way ?
> 
> Not likely.
> 
> On windows pyinstall, and other tools like it, create .exe files on windows.
> I would guess it’s that .exe that is triggering the malware detector false 
> positive.
> 
> Barry
>> 
>> 
>>> 
>>>> Is this happening to only one set of code?
>>> This is happening SOMETIMES, not always. With the SAME source code. When I
>>> call pyinstaller often enough, then the virus scanner is quiet. In about 1
>>> of 20 compile runs.
>>> 
>>> 
>>> 
>> -- 
>> Anthony Flury
>> *Moble*: +44 07743 282707
>> *Home*: +44 (0)1206 391294
>> *email*: anthony.fl...@btinternet.com <mailto:anthony.fl...@btinternet.com>
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Why so fast a web framework?

[Benjamin Schollnick]

>> Sometimes this group reminds me of a certain large company I worked for.
>> If they didn't have a solution addressing a problem, they'd pretend it
>> didn't matter and belittle anyone who questioned that version of reality.
>> 
> 
> That's not strictly true; what's happening here is that someone's
> published a cool-looking bar graph but nobody knows what it really
> means. I don't feel the need to delve into everyone's benchmarks to
> explain why Python is purportedly worse. If someone uses those kinds
> of numbers to decide which programming language to use, they have
> bigger problems.

If you dig a bit, the benchmark is scary…
As in stupid-scary.

It consists of, 7 tests, and then a composite score is generated:

JSON Serialization - In this test, each response is a JSON serialization of a 
freshly-instantiated object that maps the key message to the value Hello, World!
Single Query - In this test, each request is processed by fetching a single row 
from a simple database table. That row is then serialized as a JSON response.
Multiple Queries - In this test, each request is processed by fetching multiple 
rows from a simple database table and serializing these rows as a JSON 
response. The test is run multiple times: testing 1, 5, 10, 15, and 20 queries 
per request. All tests are run at 512 concurrency.
Cached Queries - In this test, each request is processed by fetching multiple 
cached objects from an in-memory database (the cache having been populated from 
a database table either as needed or prior to testing) and serializing these 
objects as a JSON response. The test is run multiple times: testing 1, 5, 10, 
15, and 20 cached object fetches per request. All tests are run at 512 
concurrency. Conceptually, this is similar to the multiple-queries test except 
that it uses a caching layer.
Fortunes - In this test, the framework's ORM is used to fetch all rows from a 
database table containing an unknown number of Unix fortune cookie messages 
(the table has 12 rows, but the code cannot have foreknowledge of the table's 
size). An additional fortune cookie message is inserted into the list at 
runtime and then the list is sorted by the message text. Finally, the list is 
delivered to the client using a server-side HTML template. The message text 
must be considered untrusted and properly escaped and the UTF-8 fortune 
messages must be rendered properly.  Whitespace is optional and may comply with 
the framework's best practices.
data Updates - This test exercises database writes. Each request is processed 
by fetching multiple rows from a simple database table, converting the rows to 
in-memory objects, modifying one attribute of each object in memory, updating 
each associated row in the database individually, and then serializing the list 
of objects as a JSON response. The test is run multiple times: testing 1, 5, 
10, 15, and 20 updates per request. Note that the number of statements per 
request is twice the number of updates since each update is paired with one 
query to fetch the object. All tests are run at 512 concurrency.  The response 
is analogous to the multiple-query test. 
plain text - In this test, the framework responds with the simplest of 
responses: a "Hello, World" message rendered as plain text. The size of the 
response is kept small so that gigabit Ethernet is not the limiting factor for 
all implementations. HTTP pipelining is enabled and higher client-side 
concurrency levels are used for this test (see the "Data table" view).

Here, I instead benchmark my django gallery app, using Apache Bench, and so 
forth.  I guess I’ve been over-achieving…

I have to admit, though, that these benchmarks certainly allow everyone to 
play.  

431 cherrypy587 0.0%(0.0%)

Even cherrypy with it’s 587 per second replies with plain-text.  

The tasks seem deceptively (?) simple?  

But looking closer, the data table for each task, gives more details.  For 
example the plain text is run 4 different times, at 4 different client-side 
concurrency levels are used…  But the levels are: 256, 1024, 4096, and 16384.   
That can’t be the concurrency/thread count??!?!?!??  I can believe 1,000 - 
3,000, outrageously high, but believable.  But 16K worth of 
concurrency/threads?  I doubt that Wikipedia even has to dial it that high?

I have to give them points for providing API latency, and framework overhead….  

- Benjamin



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Does loading PDB slow down execution?

[Benjamin Schollnick]

> On Oct 27, 2021, at 1:01 PM, Unixnut  wrote:
> 
> On 06/10/2021 18:30, Dieter Maurer wrote:
>> Unixnut wrote at 2021-10-3 22:03 +0100:
>>> If I run a python3 program with "import pdb" in the code, would it
>>> execute slower than without loading the debugger?
>> Importing `pdb` does not slow down the application significantly
>> (it just adds the import time but does not otherwise affect the
>> application).
>> Only when you execute code under debugger control, the
>> execution is significantly slowed down (because the debugger
>> gets informed (via callbacks) about important "event"s (entering
>> a line, function call, function return, exception) during
>> the execution).
> Excellent, many thanks for confirming. I can leave the execution running then.

But it’s pointless to import pdb, if you aren’t going to use it.

I would suggest that a best practice would be to only import pdb, if and when 
you were going to be performing the debugging.
(So maybe set a boolean constant, that would cause the import, and debugging 
code to be executed.

-- 
https://mail.python.org/mailman/listinfo/python-list

[issue45429] [Windows] time.sleep() should use CREATE_WAITABLE_TIMER_HIGH_RESOLUTION

[Benjamin Szőke]

Benjamin Szőke  added the comment:

A similar solution was introduced in VirtualBox some months ago. Soon, i could 
get back my Windows 10 developing PC and i can try this things.

https://www.virtualbox.org/browser/vbox/trunk/src/VBox/Runtime/r3/win/timer-win.cpp#L312

--

___
Python tracker 
<https://bugs.python.org/issue45429>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45429] [Windows] time.sleep() should use CREATE_WAITABLE_TIMER_HIGH_RESOLUTION

[Benjamin Szőke]

Change by Benjamin Szőke :


--
nosy: +Livius

___
Python tracker 
<https://bugs.python.org/issue45429>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30238] 2to3 doesn't detect or fix Exception indexing

[Benjamin Peterson]

Benjamin Peterson  added the comment:

See the superseding issue; 2to3 is deprecated and headed towards deletion.

--

___
Python tracker 
<https://bugs.python.org/issue30238>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45479] clean up Py_UniversalNewlineFgets

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 9ce9cfe595d64e3081e69de7296042cc54bccf18 by Benjamin Peterson in 
branch 'main':
bpo-45479: Futher simplify Py_UniversalNewlineFgets. (GH-28967)
https://github.com/python/cpython/commit/9ce9cfe595d64e3081e69de7296042cc54bccf18


--

___
Python tracker 
<https://bugs.python.org/issue45479>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45479] clean up Py_UniversalNewlineFgets

[Benjamin Peterson]

Change by Benjamin Peterson :


--
pull_requests: +27255
pull_request: https://github.com/python/cpython/pull/28967

___
Python tracker 
<https://bugs.python.org/issue45479>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45479] clean up Py_UniversalNewlineFgets

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 160c38df7fc7ba22dc687879c387bf643ffc3398 by Benjamin Peterson in 
branch 'main':
closes bpo-45479: Degunkify Py_UniversalNewlineFgets. (GH-28965)
https://github.com/python/cpython/commit/160c38df7fc7ba22dc687879c387bf643ffc3398


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue45479>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45479] clean up Py_UniversalNewlineFgets

[Benjamin Peterson]

Change by Benjamin Peterson :


--
keywords: +patch
pull_requests: +27254
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28965

___
Python tracker 
<https://bugs.python.org/issue45479>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45479] clean up Py_UniversalNewlineFgets

[Benjamin Peterson]

New submission from Benjamin Peterson :

Py_UniversalNewlineFgets has a bunch of dead code and general gunk that should 
be removed.

--
assignee: benjamin.peterson
components: IO
messages: 403970
nosy: benjamin.peterson
priority: normal
severity: normal
status: open
title: clean up Py_UniversalNewlineFgets
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue45479>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: sum() vs. loop

[Oscar Benjamin]

On Mon, 11 Oct 2021 at 23:00, Christian Gollwitzer  wrote:
>
> Am 10.10.21 um 10:49 schrieb Steve Keller:
> > I have found the sum() function to be much slower than to loop over the
> > operands myself:
> >
> > def sum_products(seq1, seq2):
> >  return sum([a * b for a, b in zip(seq1, seq2)])
> >
> > def sum_products2(seq1, seq2):
> >  sum = 0
> >  for a, b in zip(seq1, seq2):
> >  sum += a * b
> >  return sum
> >
> > In a program I generate about 14 million pairs of sequences of ints each
> > of length 15 which need to be summed.  The first version with sum() needs
> > 44 seconds while the second version runs in 37 seconds.
>
> The first version constructs a list, sums it up and throws the list
> away, while the second version only keeps the running sum in memory. How
> about a generator expression instead, i.e.
>
>
>  sum((a * b for a, b in zip(seq1, seq2)))

What really matters in cases like this is interpreter overhead.
Typically a generator expression has slightly more overhead compared
to a list comprehension. You get a slightly slower per-item speed in
exchange for O(1) memory consumption.

A list comprehension like

result = sum([expr for foo in bar])

Translates internally to something like:

def _func():
data = []
for foo in bar:
data.append(foo)
return foo

_stuff = _func()
result = sum(_stuff)

Running this really does bring in the overhead of a function call
_func() because it needs to create a frame with locals and so on.
However it is only the cost of a single function call. Afterwards if
the elements in the list _stuff are built in types like int etc with
builtin __add__ methods then sum(_stuff) does not involve the
interpreter at all: it's a built-in function operating on a built-in
container of built-in objects.

With a generator expression like

result = sum(expr for foo in bar)

This translates internally to

def _genfunc():
for foo in bar:
yield foo

_gen = _genfunc()
result = sum(_gen)

Now the _genfunc() function call simply creates the generator and sets
up its frame which I think is more or less equivalent to the overhead
of the _func() function call. However the sum(_gen) line is no longer
a pure built-in operation. Internally each time sum calls
_gen.__next__() the execution frame for _genfunc() is reactivated so
that the interpreter can execute the bytecodes taking the frame from
one yield to the next. This is almost the overhead of a function call
for each item in bar although this is often unnoticeable in practice
and other factors can affect this.

The fastest version eliminates the interpreter altogether at least
when operating on pure built-in elements:

In [7]: nums1 = nums2 = list(range(10**5))

In [10]: %timeit sum([a*b for a, b in zip(nums1, nums2)])
9.83 ms ± 21.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [11]: %timeit sum((a*b for a, b in zip(nums1, nums2)))
10.3 ms ± 109 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [12]: from operator import mul

In [13]: %timeit sum(map(mul, nums1, nums2))
7.25 ms ± 33 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Of course if the elements being multiplied and summed have pure-Python
__add__/__mul__ methods or the container has a pure Python
__iter__/__next__ then any of those methods will typically dominate
the runtime over any of the overheads considered here.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux

[Benjamin Szőke]

Benjamin Szőke  added the comment:

In other words, using absolute timeout can eliminate the systematic error of 
desired sleep time.

--

___
Python tracker 
<https://bugs.python.org/issue21302>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux

[Benjamin Szőke]

Benjamin Szőke  added the comment:

It is not true that there are no benefits. Absolute timeout using can reduce 
the overhead time of any variable and object intialization cost before the 
WaitForMultipleObjects() which will perform the real sleeping via blocking wait 
in pysleep(). GetSystemTimePreciseAsFileTime() must be call at the first line 
as much as it can in pysleep(). This is the same implementation in Linux via 
clock_nanosleep().

So, to using absolute timeout and GetSystemTimePreciseAsFileTime() can improves 
the accuracy of the desired sleep time. For example if sleep = 2.0 sec then 
real relative sleep time = 2.001234 sec, but absolute sleep time = 2.12 sec.

Benefits are in not the technicaly backgorund, rather it is in the usecase.

--

___
Python tracker 
<https://bugs.python.org/issue21302>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux

[Benjamin Szőke]

Benjamin Szőke  added the comment:

Absolute timeout implementation via SetWaitableTimer() and 
GetSystemTimePreciseAsFileTime() is always better because it can reduce the 
"waste time" or "overhead time" what is always exist in any simple interval 
sleep implementation. Moreover, it is the only one which is eqvivalent with 
clock_nanosleep() implementation of Linux which is now the most state of the 
art implementation for precise sleeping.

So, my opinion is that absolute timeout implementation could be the most modern 
and sustainable for future python.

--

___
Python tracker 
<https://bugs.python.org/issue21302>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux

[Benjamin Szőke]

Benjamin Szőke  added the comment:

https://www.python.org/downloads/windows/
"Note that Python 3.10.0 cannot be used on Windows 7 or earlier."

vstinner: Is it true that Windows 7 is not supported OS anymore? In this case 
we do not need to care about Windows 7 and earlier Windows OS compatibility and 
it is time to use nicely GetSystemTimePreciseAsFileTime() in time.time() and 
time.sleep() as absolute sleeping because it is available since Windows 8.

--

___
Python tracker 
<https://bugs.python.org/issue21302>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Different "look and feel" of some built-in functions

[Oscar Benjamin]

On Sat, 25 Sept 2021 at 02:11, Oscar Benjamin 
wrote:

> On Sat, 25 Sept 2021 at 02:01, Chris Angelico  wrote:
>
>> On Sat, Sep 25, 2021 at 10:56 AM Oscar Benjamin
>>  wrote:
>> >
>> > On Sat, 25 Sept 2021 at 00:37, Greg Ewing 
>> > wrote:
>> > > I suppose they could be fiddled somehow to make it possible, but
>> > > that would be turning them into special cases that break the rules.
>> > > It would be better to provide separate functions, as was done with
>> > > sum().
>> > >
>> >
>> > A separate union function would be good. Even in a situation where all
>> > inputs are assured to be sets the set.union method fails the base case:
>> >
>> > >>> sets = []
>> > >>> set.union(*sets)
>> > Traceback (most recent call last):
>> >   File "", line 1, in 
>> > TypeError: descriptor 'union' of 'set' object needs an argument
>> >
>> > In the case of intersection perhaps the base case should be undefined.
>> >
>>
>> Rather than calling the unbound method, why not just call it on an
>> empty set? That defines your base case as well.
>>
>> set().union(*sets)
>>
>
> That is indeed what I do but it seems unnatural compared to simply
> union(*sets). It shouldn't be necessary to create a redundant empty set
> just to compute the union of some other sets. If there was a union function
> then I don't think I would ever use the union method.
>

I just hit up against a problem using this with mypy:

$ cat y.py
from typing import Set, Tuple

class Tree:
_children: 'Tuple[Tree]'

@property
def children(self) -> 'Tuple[Tree]':
return self._children

@property
def leaves(self) -> 'Set[Tree]':
return set().union(*(child.leaves for child in self.children))

$ mypy y.py
y.py:12: error: Incompatible return value type (got "Set[]",
expected "Set[Tree]")
y.py:12: error: Argument 1 to "union" of "set" has incompatible type
"*Generator[Set[Tree], None, None]"; expected "Iterable[]"
Found 2 errors in 1 file (checked 1 source file)

It seems that mypy is okay with set.union(*stuff) but not
set().union(*stuff).

I can fix it with:

@property
def leaves(self) -> 'Set[Tree]':
empty: 'Set[Tree]' = set()
return empty.union(*(child.leaves for child in self.children))

That's just horrible though. Why should I give a name and type hint for the
completely redundant empty set object?

Does anyone know of a better way to hint this?

-- 
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux

[Benjamin Szőke]

Benjamin Szőke  added the comment:

Do you have any information about when will be it released in 3.11?

--

___
Python tracker 
<https://bugs.python.org/issue21302>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Different "look and feel" of some built-in functions

[Oscar Benjamin]

On Sat, 25 Sept 2021 at 02:16, Chris Angelico  wrote:

> On Sat, Sep 25, 2021 at 11:11 AM Oscar Benjamin
>  wrote:
> >
> > On Sat, 25 Sept 2021 at 02:01, Chris Angelico  wrote:
> >>
> >> On Sat, Sep 25, 2021 at 10:56 AM Oscar Benjamin
> >>  wrote:
> >> >
> >> > On Sat, 25 Sept 2021 at 00:37, Greg Ewing <
> greg.ew...@canterbury.ac.nz>
> >> > wrote:
> >> > > I suppose they could be fiddled somehow to make it possible, but
> >> > > that would be turning them into special cases that break the rules.
> >> > > It would be better to provide separate functions, as was done with
> >> > > sum().
> >> > >
> >> >
> >> > A separate union function would be good. Even in a situation where all
> >> > inputs are assured to be sets the set.union method fails the base
> case:
> >> >
> >> > >>> sets = []
> >> > >>> set.union(*sets)
> >> > Traceback (most recent call last):
> >> >   File "", line 1, in 
> >> > TypeError: descriptor 'union' of 'set' object needs an argument
> >> >
> >> > In the case of intersection perhaps the base case should be undefined.
> >> >
> >>
> >> Rather than calling the unbound method, why not just call it on an
> >> empty set? That defines your base case as well.
> >>
> >> set().union(*sets)
> >
> >
> > That is indeed what I do but it seems unnatural compared to simply
> union(*sets). It shouldn't be necessary to create a redundant empty set
> just to compute the union of some other sets. If there was a union function
> then I don't think I would ever use the union method.
> >
>
> Maybe, but if you start with a set, then you define the base case, and
> it also is quite happy to take non-set arguments:
>
> >>> set().union([1,2,3], map(int, "234"), {3:"three",4:"four",5:"five"})
> {1, 2, 3, 4, 5}


Actually it should be union(sets) rather than union(*sets). A more
realistic example is when you need the union of sets given by something
like a generator expression so it looks like:

items = set().union(*(f(x) for x in stuff))

With a union function it should look like:

items = union(f(x) for x in stuff)

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Different "look and feel" of some built-in functions

[Oscar Benjamin]

On Sat, 25 Sept 2021 at 02:01, Chris Angelico  wrote:

> On Sat, Sep 25, 2021 at 10:56 AM Oscar Benjamin
>  wrote:
> >
> > On Sat, 25 Sept 2021 at 00:37, Greg Ewing 
> > wrote:
> > > I suppose they could be fiddled somehow to make it possible, but
> > > that would be turning them into special cases that break the rules.
> > > It would be better to provide separate functions, as was done with
> > > sum().
> > >
> >
> > A separate union function would be good. Even in a situation where all
> > inputs are assured to be sets the set.union method fails the base case:
> >
> > >>> sets = []
> > >>> set.union(*sets)
> > Traceback (most recent call last):
> >   File "", line 1, in 
> > TypeError: descriptor 'union' of 'set' object needs an argument
> >
> > In the case of intersection perhaps the base case should be undefined.
> >
>
> Rather than calling the unbound method, why not just call it on an
> empty set? That defines your base case as well.
>
> set().union(*sets)
>

That is indeed what I do but it seems unnatural compared to simply
union(*sets). It shouldn't be necessary to create a redundant empty set
just to compute the union of some other sets. If there was a union function
then I don't think I would ever use the union method.


--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Different "look and feel" of some built-in functions

[Oscar Benjamin]

On Sat, 25 Sept 2021 at 00:37, Greg Ewing 
wrote:

> On 25/09/21 10:15 am, Steve Keller wrote:
> > BTW, I like how the min() and max() functions allow both ways of being
> > called.
>
> That wouldn't work for set.union and set.intersection, because as
> was pointed out, they're actually methods, so set.union(some_seq)
> is a type error:
>
>  >>> a = {1, 2}
>  >>> b = {3, 4}
>  >>> set.union([a, b])
> Traceback (most recent call last):
>File "", line 1, in 
> TypeError: descriptor 'union' for 'set' objects doesn't apply to a
> 'list' object
>
> I suppose they could be fiddled somehow to make it possible, but
> that would be turning them into special cases that break the rules.
> It would be better to provide separate functions, as was done with
> sum().
>

A separate union function would be good. Even in a situation where all
inputs are assured to be sets the set.union method fails the base case:

>>> sets = []
>>> set.union(*sets)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: descriptor 'union' of 'set' object needs an argument

In the case of intersection perhaps the base case should be undefined.

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux

[Benjamin Szőke]

Change by Benjamin Szőke :


--
pull_requests: +26917
pull_request: https://github.com/python/cpython/pull/28526

___
Python tracker 
<https://bugs.python.org/issue21302>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45262] crash if asyncio is used before and after re-initialization if using python embedded in an application

[Benjamin Schiller]

New submission from Benjamin Schiller :

We have embedded Python in our application and we deinitialize/initialize the 
interpreter at some point of time. If a simple script with a thread that sleeps 
with asyncio.sleep is loaded before and after the re-initialization, then we 
get the following assertion in the second run of the python module:

"Assertion failed: Py_IS_TYPE(rl, _Type), file 
D:\a\1\s\Modules_asynciomodule.c, line 261"

Example to reproduce this crash: 
https://github.com/benjamin-sch/asyncio_crash_in_second_run

--
components: asyncio
messages: 402412
nosy: asvetlov, benjamin-sch, yselivanov
priority: normal
severity: normal
status: open
title: crash if asyncio is used before and after re-initialization if using 
python embedded in an application
type: crash
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue45262>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux

[Benjamin Szőke]

Change by Benjamin Szőke :


--
pull_requests: +26754
pull_request: https://github.com/python/cpython/pull/28341

___
Python tracker 
<https://bugs.python.org/issue21302>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45190] unicode 14.0 upgrade

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 024fda47d40b8cee77ac1cd3d31ee549edc11986 by Benjamin Peterson in 
branch 'main':
closes bpo-45190: Update Unicode data to version 14.0.0. (GH-28336)
https://github.com/python/cpython/commit/024fda47d40b8cee77ac1cd3d31ee549edc11986


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue45190>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45190] unicode 14.0 upgrade

[Benjamin Peterson]

Change by Benjamin Peterson :


--
keywords: +patch
pull_requests: +26749
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28336

___
Python tracker 
<https://bugs.python.org/issue45190>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45190] unicode 14.0 upgrade

[Benjamin Peterson]

New submission from Benjamin Peterson :

Unicode 14.0 is expected on September 14. We'll need to do the usual table 
regenerations.

--
assignee: benjamin.peterson
components: Unicode
messages: 401747
nosy: benjamin.peterson, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: unicode 14.0 upgrade

___
Python tracker 
<https://bugs.python.org/issue45190>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux

[Benjamin Szőke]

Benjamin Szőke  added the comment:

Can you review my final implementation?
https://github.com/python/cpython/pull/28111

--
versions: +Python 3.11

___
Python tracker 
<https://bugs.python.org/issue21302>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: on floating-point numbers

[Oscar Benjamin]

On Fri, 3 Sept 2021 at 13:48, Chris Angelico  wrote:
>
> On Fri, Sep 3, 2021 at 10:42 PM jak  wrote:
> >
> > Il 03/09/2021 09:07, Julio Di Egidio ha scritto:
> > > On Friday, 3 September 2021 at 01:22:28 UTC+2, Chris Angelico wrote:
> > >> On Fri, Sep 3, 2021 at 8:15 AM Dennis Lee Bieber  
> > >> wrote:
> > >>> On Fri, 3 Sep 2021 04:43:02 +1000, Chris Angelico 
> > >>> declaimed the following:
> > >>>
> >  The naive summation algorithm used by sum() is compatible with a
> >  variety of different data types - even lists, although it's documented
> >  as being intended for numbers - but if you know for sure that you're
> >  working with floats, there's a more accurate algorithm available to
> >  you.
> > 
> > >>> math.fsum([7.23, 8.41, 6.15, 2.31, 7.73, 7.77])
> >  39.6
> > >>> math.fsum([8.41, 6.15, 2.31, 7.73, 7.77, 7.23])
> >  39.6
> > 
> >  It seeks to minimize loss to repeated rounding and is, I believe,
> >  independent of data order.
> > >>>
> > >>> Most likely it sorts the data so the smallest values get summed first,
> > >>> and works its way up to the larger values. That way it minimizes the 
> > >>> losses
> > >>> that occur when denormalizing a value (to set the exponent equal to 
> > >>> that of
> > >>> the next larger value).
> > >>>
> > >> I'm not sure, but that sounds familiar. It doesn't really matter
> > >> though - the docs just say that it is an "accurate floating point
> > >> sum", so the precise algorithm is an implementation detail.
> > >
> > > The docs are quite misleading there, it is not accurate without further 
> > > qualifications.
> > >
> > > 
> > > 
> > >
> >
> > https://en.wikipedia.org/wiki/IEEE_754
>
> I believe the definition of "accurate" here is that, if you take all
> of the real numbers represented by those floats, add them all together
> with mathematical accuracy, and then take the nearest representable
> float, that will be the exact value that fsum will return. In other
> words, its accuracy is exactly as good as the final result can be.

It's as good as it can be if the result must fit into a single float.
Actually the algorithm itself maintains an exact result for the sum
internally using a list of floats whose exact sum is the same as that
of the input list. In essence it compresses a large list of floats to
a small list of say 2 or 3 floats while preserving the exact value of
the sum.

Unfortunately fsum does not give any way to access the internal exact
list so using fsum repeatedly suffers the same problems as plain float
arithmetic e.g.:
>>> x = 10**20
>>> fsum([fsum([1, x]), -x])
0.0

--
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue21302] time.sleep (floatsleep()) should use clock_nanosleep() on Linux

[Benjamin Szőke]

Change by Benjamin Szőke :


--
nosy: +Livius
nosy_count: 5.0 -> 6.0
pull_requests: +26552
pull_request: https://github.com/python/cpython/pull/28111

___
Python tracker 
<https://bugs.python.org/issue21302>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33930] Segfault with deep recursion into object().__dir__

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset d7f5796a1ec7ba223f6a844d7580559abef05238 by Miss Islington (bot) 
in branch '3.8':
bpo-33930: Fix typo in the test name. (GH-27735)
https://github.com/python/cpython/commit/d7f5796a1ec7ba223f6a844d7580559abef05238


--

___
Python tracker 
<https://bugs.python.org/issue33930>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33930] Segfault with deep recursion into object().__dir__

[Benjamin Peterson]

Change by Benjamin Peterson :


--
pull_requests: +26216
pull_request: https://github.com/python/cpython/pull/27736

___
Python tracker 
<https://bugs.python.org/issue33930>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33930] Segfault with deep recursion into object().__dir__

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset f08e6d1bb3c5655f184af88c6793e90908bb6338 by Benjamin Peterson in 
branch 'main':
bpo-33930: Fix typo in the test name. (#27733)
https://github.com/python/cpython/commit/f08e6d1bb3c5655f184af88c6793e90908bb6338


--

___
Python tracker 
<https://bugs.python.org/issue33930>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33930] Segfault with deep recursion into object().__dir__

[Benjamin Peterson]

Change by Benjamin Peterson :


--
nosy: +benjamin.peterson
nosy_count: 14.0 -> 15.0
pull_requests: +26213
pull_request: https://github.com/python/cpython/pull/27733

___
Python tracker 
<https://bugs.python.org/issue33930>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44871] Threading memory leak

[Benjamin Peterson]

Change by Benjamin Peterson :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Memory leak/high usage on copy in different thread

___
Python tracker 
<https://bugs.python.org/issue44871>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39091] _PyErr_CreateException() must check that the result is an exception (CPython Segfault in 5 lines of code)

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 83ca46b7784b7357d82ec47b33295e09ed7380cb by Noah in branch 'main':
closes bpo-39091: Fix segfault when Exception constructor returns non-exception 
for gen.throw. (#17658)
https://github.com/python/cpython/commit/83ca46b7784b7357d82ec47b33295e09ed7380cb


--
nosy: +benjamin.peterson
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39091>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37715] 2to3 set default encoding

[Benjamin Peterson]

Change by Benjamin Peterson :


--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue37715>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41311] Add a function to get a random sample from an iterable (reservoir sampling)

[Oscar Benjamin]

Oscar Benjamin  added the comment:

I was contacted by someone interested in this so I've posted the last version 
above as a GitHub gist under the MIT license:
https://gist.github.com/oscarbenjamin/4c1b977181f34414a425f68589e895d1

--

___
Python tracker 
<https://bugs.python.org/issue41311>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Optimizing Small Python Code

[Benjamin Schollnick]

How would you measure the steps that it takes?

- Benjamin



> On Jun 22, 2021, at 7:04 PM, Greg Ewing  wrote:
> 
> On 23/06/21 3:03 am, Kais Ayadi wrote:
>> for n in range(1, 7):
>> print (n)
>> for x in range(0, n):
>> print(" ", x)
>> can this code be more optimised?
> 
> Optimised for what? Readability? Execution speed? Code size?
> Memory usage?
> 
> -- 
> Greg
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How Do I Get A Bug In Multiprocessing Fixed?

[Oscar Benjamin]

On Fri, 18 Jun 2021 at 15:27, Michael Boom  wrote:

> The below issue is pretty serious and it is preventing me from using a
> system I wrote on a larger scale.  How do I get this bug fixed?  Thanks.
> https://bugs.python.org/issue43329

On Fri, 18 Jun 2021 at 06:07, Alexander Neilson 
wrote:

>
> I am very interested to note that a report filed in February hasn't had at
> least one person take a brief look at it and post a note or set a status
> like "needs more info" etc.
>

Personally I don't find that surprising. I just looked at the issue very
quickly and I couldn't immediately tell if the problem was a bug in
multiprocessing or a mistake in the code shown. Just figuring that out
would take more than the very little time I was prepared to spend looking
at it so I moved on. If the OP hopes that someone else will use their
limited time to fix this issue for them then they should do what they can
to make it as quick as possible for that other person. The first part of
that is making clear why it should be considered a bug in the first place.
Don't expect the reader to spend time deciphering your code to figure that
out.

If there is a bug in multiprocessing then the issue should be clearer about
what it is or why it should be considered a bug. Which function is
documented as doing something that apparently does not work in this case?
Why should it be expected to work?

-- 
Oscar
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue39247] dataclass defaults and property don't work together

[Benjamin Lee]

Benjamin Lee  added the comment:

> I'm not sure "alias" feels quite right, as it only applies to __init__ (if 
> I'm understanding it correctly).

Maybe `init_alias` might be a better name. In any case, this would support 
private variables in dataclasses.

--

___
Python tracker 
<https://bugs.python.org/issue39247>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39247] dataclass defaults and property don't work together

[Benjamin Lee]

Benjamin Lee  added the comment:

Would this issue not be trivially resolved if there was a way to specify alias 
in the dataclasses field? I.e.:

_uploaded_by: str = dataclasses.field(alias="uploaded_by", default=None, 
init=False)

Ultimately, the main goal is to make it so that the generated __init__ 
constructor does

self._uploaded_by = uploaded_by

but with current implementation, there is no aliasing so the default __init__ 
constructor is always:

self._uploaded_by = _uploaded_by

--
nosy: +UnHumbleBen

___
Python tracker 
<https://bugs.python.org/issue39247>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17681] Work with an extra field of gzip and zip files

[Benjamin Sergeant]

Benjamin Sergeant  added the comment:

type Header struct {
Comment string// comment
Extra   []byte// "extra data"
ModTime time.Time // modification time
Namestring// file name
OS  byte  // operating system type
}

This is what the header/extra things look like for reference.

--

___
Python tracker 
<https://bugs.python.org/issue17681>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17681] Work with an extra field of gzip and zip files

[Benjamin Sergeant]

Benjamin Sergeant  added the comment:

There is a comment field too which would be nice to support.

The Go gzip module has a Header class that describe all the metadata. I see in 
3.8 mtime was made configurable, so hopefully we can add comment and extra.

https://golang.org/pkg/compress/gzip/#Header

For our purpose we'd like to put arbitrary stuff in a gzip file but it is 
complicated to do so, I might use the patch here and apply to the python gzip 
module, but that feels a bit  hackish.

--
nosy: +Benjamin.Sergeant

___
Python tracker 
<https://bugs.python.org/issue17681>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: neoPython : Fastest Python Implementation: Coming Soon

[Benjamin Schollnick]

> Why? The currently extant Python implementations contribute to climate change 
> as they are so inefficient; 

That same argument can be made for every triple-AAA video game, game console, 
etc.

Python is more efficient for some problem sets, and Python is less efficient 
for other problem sets.

Please feel free to come out with NeoPython.  When you are done, and it is 
backward compatible with existing Python code, I’ll be happy to benchmark it 
against Native python.  But don’t blame Python for global climate change.  
There are plenty of bigger “causations” to Global climate change, then a 
freakin’ computer programming language.

- Benjamin


-- 
https://mail.python.org/mailman/listinfo/python-list

[issue42422] types.CodeType() has no bytecode verifier

[Benjamin Peterson]

Change by Benjamin Peterson :


--
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue42422>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43834] Use context manager in StringIO example

[Benjamin Peterson]

Benjamin Peterson  added the comment:

I agree that closing or using a context manager with StringIO (or BytesIO) is 
not something one normally has to do, so it doesn't need to be in the example.

--
resolution:  -> rejected
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue43834>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43602] Include Decimal's in numbers.Real

[Oscar Benjamin]

Oscar Benjamin  added the comment:

I've never found numbers.Real/Complex to be useful. The purpose of the ABCs 
should be that they enable you to write code that works for instances of any 
subclass but in practice writing good floating point code requires knowing 
something e.g. the base, precision, max exponent etc of the type. Also many 
implementations like Decimal have contexts and rounding control etc that need 
to be used and the ABC gives no way to know that or to do anything with it.

The main thing that is useful about the Rational/Integer ABCs is that they 
define the numerator and denominator attributes which makes different 
implementations interoperable by providing exact conversion. If Real was 
presumed to represent some kind of floating point type then an analogous 
property/method would be something that can deconstruct the object in an exact 
way like:

mantissa, base, exponent = deconstruct(real)

You would also need a way to handle nan, inf etc. Note that as_integer_ratio() 
is not suitable because it could generate enormous integers unnecessarily e.g. 
Decimal('1E+1').as_integer_ratio().

Instead the Real ABC only defines conversion to float. That's useful in the 
sense that you can write code for float and pass in some other floating point 
type and have everything reduce to float. You don't need an ABC for that though 
because __float__ does everything. In practice most alternate "real" number 
implementations exist precisely to be better than float in some way by either 
having greater range/precision or a different base but anything written for the 
Real ABC is essentially reduced to float as a lowest common (inexact) 
denominator.

--

___
Python tracker 
<https://bugs.python.org/issue43602>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43850] unreproducible bytecode: set order depends on random seed for compiled bytecode

[Benjamin Peterson]

Benjamin Peterson  added the comment:

Let's keep any discussion on the preëxisting issue for this.

--
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed
superseder:  -> support reproducible Python builds

___
Python tracker 
<https://bugs.python.org/issue43850>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: not able to download PyAudio

[Benjamin Schollnick]

> On Apr 2, 2021, at 4:40 AM, ᗷᑌᑎᑎY  wrote:
> 
>   Hello  everyone
>   I am not able to download PyAudio. I tried by typing pip install in
>   PyAudio in cmd but it show's no compatible version availbale. What should
>   I do? .

There seems to be some confusion, which I can understand…  

“No compatible version”, simply means that Pip was unable to find a version of 
 for your version of python.

Please note, that does not mean that a version *EXISTS*, just that it could not 
find a version for your platform.

So go to https://pypi.org <https://pypi.org/> and do a search.  

- Benjamin


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: .title() - annoying mistake

[Benjamin Schollnick]

>> That's not true for digraphs where there is a third, distinct and
>> different "title" case. I think the doc should state that the initial
>> character is converted to titlecase. A parenthentical statement that
>> titlecase is usually but not always equal to uppercase would be nice,
>> but the current statement is obsolete and not correct in all, um...
>> cases.
> 
> Fair enough, but the trouble is that getting too pedantic in a
> docstring just makes it read like IBM documentation. :)

And actually conversely makes it harder to keep updating, because you’ll need 
to document every and all edge-cases, and then need to know when one of those 
edge cases breaks, etc.  

The core concept is documented, and it’s pretty straight-forward.  

I’m sorry, but it’s as if he’s arguing for the sake of arguing.  It’s starting 
to feel very unproductive, and unnecessary.

- Benjamin


-- 
https://mail.python.org/mailman/listinfo/python-list