[pypy-dev] Re: Windows zip file for pypy3.10-v7.3.16 is not completed

[Matti Picus]

On 24/04/2024 15:30, yanglei wrote:

Hi Matti,

Thanks for your prompt reply, I have downloaded it again, but it's 
still the same issue.

Maybe you can check it again, thanks.

Yanglei



Try again? I purged the CDN cache with `curl -X PURGE 
https://downloads.python.org/pypy/pypy3.10-v7.3.16-win64.zip` and now I 
get the correct download.


Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Windows zip file for pypy3.10-v7.3.16 is not completed

[Matti Picus]

On 24/04/2024 14:20, yanglei wrote:


Hi PyPy developer team,

Glad that you have released the new version for pypy3.10-v7.3.16 
recently, but there is a little problem for this windows zip file.


Maybe this file pypy3.10-v7.3.16-win64.zip 
(https://downloads.python.org/pypy/pypy3.10-v7.3.16-win64.zip) is not 
completed, the sha256 value is different from you wrote on the website 
(https://www.pypy.org/checksums.html).


Official Website sha256:
e08415a2f35c6ecf2342b504bdfde11e4c5eca3fc5ef7fd2214ff064a5a54396 
 pypy3.10-v7.3.16-win64.zip


Actual sha256:
95ff7c3f23098307de9bea5ea75f47c6c4d615c07bee2e841aecef10e751286b 
pypy3.10-v7.3.16-win64.zip


Hope you can check with this issue, thanks in advance.



Thanks for the heads-up. The file upload indeed failed, and the zip-file 
on the download site was incomplete. I have re-uploaded the file, and 
the sha356sum hashes should now match.


Matti


___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Release of pypy 7.3.16

[Matti Picus]

I have released PyPy v7.3.16, which includes

- bugfixes for the garbage collector

- updates to the upstream stdlib for security issues


Please try it out and report any problems.

Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Contribute a RISC-V 64 JIT backend

[Matti Picus]

On 16/1/24 07:02, Logan Chien wrote:


Hi,

I have good news: the RISC-V backend can pass as many unit tests as 
the AArch64 backend.  I got vmprof and codemap working this weekend.  
I also completed a full translation and got a workable pypy executable.


I have two questions now:

1. Are there other test suites that I can check for the correctness?
2. How do we measure the performance?  Do we have a command line that 
can run all benchmarks?


Thank you in advance.

Regards,
Logan

p.s. All changes are at: https://github.com/loganchien/pypy/tree/rv64



Very cool.

1: Eventually we would want a buildbot worker [0] using either actual 
hardware or qemu. Using qemu might be too slow to be practical. It could 
be based off the aarch64 docker file [1] as a template. For now, you 
could follow manually the different steps in a pyp-c-jit- 
buildbot run [2]: click on the stdio link for each step to follow the 
workflow. The "app-level -A", "extra tests", "lib-python" would give an 
indication of how compatible the rpython code is, and the "pypyjit" 
tests would give an indication of how well the JIT code generation 
follows the other platforms. A deeper compliance test would be to run 
the binary in the workflows from the binary-testing repo [6] against 
some common python libraries.



2: Theses [3] are the benchmarks that feed speed.pypy.org. They are run 
by a buildbot worker [4]. The step of interest is 9, where the command 
line is the top line of [5]. This generates a json results file and also 
some textual output. You will want an additional run with cpython for a 
baseline.



Matti


[0] 
https://foss.heptapod.net/pypy/buildbot/-/blob/branch/default/README_BUILDSLAVE?ref_type=heads


[1] 
https://foss.heptapod.net/pypy/buildbot/-/blob/branch/default/docker/DockerfileArm64?ref_type=heads


[2] 
https://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64/builds/9219 
for example


[3] https://foss.heptapod.net/pypy/benchmarks

[4] 
https://buildbot.pypy.org/builders/jit-benchmark-linux-x86-64/builds/4056


[5] 
https://buildbot.pypy.org/builders/jit-benchmark-linux-x86-64/builds/4056/steps/shell_9/logs/stdio


[6] https://github.com/pypy/binary-testing

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Release v7.3.15 of PyPy

[Matti Picus]

I have prepared a new release of PyPy. This is mainly a bugfix release 
to fix backward compatibility in c-extensions that use the buffer 
protocol. It also includes chages from our move to Git/Github. The 
pre-release tarballs are now on https://buildbot.pypy.org/pypy/ (until 
the CDN percolates them out to https://downloads.python.org/pypy/), the 
checksums are on a Pr to pypy.org at 
https://deploy-preview-110--keen-mestorf-442210.netlify.app/checksums. 
The release notes are at 
https://doc.pypy.org/en/latest/release-v7.3.15.html.



Please test out the source tarballs, they have changed since they are 
from a git repo.



I plan to release officially later today.

Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Contribute a RISC-V 64 JIT backend

[Matti Picus]

On 8/1/24 10:03, Maciej Fijalkowski wrote:

Hi Logan

Very cool you are interested in that! It's often useful to hang out on
IRC as you can ask questions directly. I have not taken any looks at
all, but can you tell me what kind of setup does one need for testing
it? Are you using real hardware or emulation?

The approach of starting with tests and getting translation done later
is very much what we have done in the past.

Best,
Maciej

On Mon, 8 Jan 2024 at 09:42, Logan Chien  wrote:

Hi,

I forgot to include the link in my previous email.

If you want to have a look on my prototype, you can find it here:
https://github.com/loganchien/pypy/tree/rv64

Thanks.

Regards,
Logan



On Sun, Jan 7, 2024 at 5:18 PM Logan Chien  wrote:

Hi all,

I would like to contribute a RISC-V 64 JIT backend for RPython.  I have made 
some progress at the end of 2023.

## Status

My prototype can pass the test cases below:

* test_runner.py
* test_basic.py and almost all test_ajit.py related tests (except 
test_rvmprof.py)
* test_zrpy_gc_boehm.py

I am still working on test_zrpy_gc.py though (p.s. I can pass this if I disable 
malloc inlining).

I haven't done a full translation yet.

## Logistic

I wonder how you would like to review the patches?  I have roughly 73 pending 
commits.  Each commit has a specific reason for change and corresponding test 
cases (if applicable).

Is it better to just send one GitHub Pull Request containing all of them?

Or, do you prefer one commit per Pull Request?

Thank you.

Regards,
Logan



Exciting, thanks!

I find IRC too temporary: it is hard to search through. This can be both 
an advantage and a disadvantage. Maybe since we have moved development 
efforts to github we could try out the github discussions platform. I 
opened it up at https://github.com/orgs/pypy/discussions. Of course you 
are welcome to use IRC if you are comfortable with it.



In addition to Maciej's questions:  is there only one compilation target 
or would the backend need to know about the different ISA extensions?



As for patch review and merging: we have a history of long-lived 
branches in PyPy. Two examples: the windows 64-bit branch was only 
merged once it was quite ready, and led to only small breakage of the 
main branches. I recently merged the hpy0.9 branch too early, and the 
failing tests masked some other py3.9 failures until I got it under 
control, so it would have been better to hold off until it was more 
completely finished. Something tho think about.


Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Moving to github

[Matti Picus]

On 31/12/23 00:55, Matti Picus wrote:


On 28/12/23 10:20, Matti Picus wrote:
Now that 7.3.14 has been released, I would like to move the canonical 
repo for pypy and rpython to github. Reasons:


...



- I have created the repo at github.com/pypy/pypy. It is still private 
so only some of you will be able to see it. Please take a look at a 
few issues and branches and make sure things look OK.



- I also wrote a draft blog post as a PR 
https://github.com/pypy/pypy.org/pull/108, the preview is at 
https://deploy-preview-108--keen-mestorf-442210.netlify.app/blog/. 
Please comment on the PR.



- I couldn't figure out how to add a message to the heptapod issue 
tracker, but I did disable issue and PR creation at heptapod. Project 
developers can still create issues, but not non-project members.



I will make this all public before year's end unless someone spots a 
problem. There is a bit of redundancy in branch names. The 
git-remote-hg script uses branches/XXX when converting a branch name, 
but heptapod/gitlab use branch/XXX (without the "es") so the repo has 
both names for many branches. We will have to decide which one to keep 
tracking.


Matti



The repo is live, and the issue tracker is open. The post on 
https://www.pypy.org/posts/2023/12/pypy-moved-to-git-github.html has 
more information.


Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Moving to github

[Matti Picus]

On 28/12/23 10:20, Matti Picus wrote:
Now that 7.3.14 has been released, I would like to move the canonical 
repo for pypy and rpython to github. Reasons:


...



- I have created the repo at github.com/pypy/pypy. It is still private 
so only some of you will be able to see it. Please take a look at a few 
issues and branches and make sure things look OK.



- I also wrote a draft blog post as a PR 
https://github.com/pypy/pypy.org/pull/108, the preview is at 
https://deploy-preview-108--keen-mestorf-442210.netlify.app/blog/. 
Please comment on the PR.



- I couldn't figure out how to add a message to the heptapod issue 
tracker, but I did disable issue and PR creation at heptapod. Project 
developers can still create issues, but not non-project members.



I will make this all public before year's end unless someone spots a 
problem. There is a bit of redundancy in branch names. The git-remote-hg 
script uses branches/XXX when converting a branch name, but 
heptapod/gitlab use branch/XXX (without the "es") so the repo has both 
names for many branches. We will have to decide which one to keep tracking.


Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Moving to github

[Matti Picus]

Now that 7.3.14 has been released, I would like to move the canonical 
repo for pypy and rpython to github. Reasons:


- foss.heptapod.net is not well tracked in google/bing/duckduckgo 
search, so people find it hard to search for issues in the project


- since the site has tightened its spam control, we get reports that 
users create issues only to have them flagged as spam


- open source has become synonymous with github, and we are too small to 
change that


- Much of the current development comes as a reaction to fixing issues. 
Tracking interlocking issues is easier if all the code is on the same 
platform


- The FAQ [2] presents two arguments against the move. Github notes [3] 
solves much of point (1), although not entirely. But the main problem is 
point (2), it turns out that __not__ moving to github is an impediment 
to contribution and issue reporting.


- As development effort winds down, github is a better archive for PyPy 
than foss.heptapod.net. I cannot predict the future, but if there is a 
move to revive the project, or to move to another platform, I believe 
github will provide a better jumping-off point as well. And since the 
repo at foss.heptapod.net will not be deleted, even if I am wrong I 
expect the effort to port the issues and commits from github will be 
manageable.


- People who wish to continue to use mercurial can add a cron job github 
action to pull the changes from foss.heptapod.net across to github


- github is more resource rich than foss.heptapod.net. We can add CI 
jobs to replace some of our aging buildbot infrastructure (still using 
buildbot 0.8 and python2).



Technique:


These steps will be done in a private repo

- I will convert the repo to git and add a note to most of the commits 
(where I can) which will allow using "git log --notes=branch" to 
determine which branch a commit came from. More details at [0]


- I will convert the issues and PRs to github via [1]. Using a private 
repo prevents spamming issue authors with emails about the transfer.



Then I will make the repo public:

- Move it to github.com/pypy/pypy

- Freeze the issue tracker at 
https://foss.heptapod.net/pypy/pypy/-/issues, and add a message that 
development has moved


- Write a blog post

- Modify the links in the documentation

- Activate the github action in [3] to add a branch note to each git commit


Anything else? Any suggestions to make the transition easier?

Matti


[0] https://gist.github.com/mattip/b6752c164a075c2aa53f4069e9c30573

[1] https://github.com/piceaTech/node-gitlab-2-github

[2] 
https://doc.pypy.org/en/default/faq.html#why-doesn-t-pypy-use-git-and-move-to-github


[3] https://github.com/Julian/named-branch-action

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Release of v7.3.14

[Matti Picus]

I have released v7.3.14 of pypy2.7, pypy3.9, and pypy3.10. Please check 
it out. The main new feature is support for HPy 0.9. There are also 
bugfixes and smaller enhancements.



Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] merge of hpy-0.9

[Matti Picus]

I merged the hpy-0.9 branch to py3.9. There are a few untranslated 
failures, and more extra_tests post-translation failures. Most of the 
extra_test failures are in the debug/trace/hpy_devel/importing tests 
which cannot really be checked untranslated. Help is welcome to try to 
debug if the failures are due to using pytest4.0, to something in the 
test harnesses, or whether there is an actual problem with the hpy 
backend implementation.



Additionally, the branch crashes when importing NumPy from the hpy 
branch https://github.com/hpyproject/numpy-hpy. This is being tracked in 
issue https://github.com/hpyproject/numpy-hpy/issues/15. It seems the 
custom trace function necessary to keep the W_Root object associated 
with a HPyField is not being called. Again: help here is welcome.



I can meet up on zoom or google meet if anyone would like to dive in and 
wants to talk about the problems.


Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Debug of PyPy segmentation fault / read of coredump

[Matti Picus]

On 12/10/23 17:20, Yevhenii Yatchenko wrote:

Greetings!


I'm working on a project related to SIP stack where one of our python 
interpreters is PyPy.


We faced an issue on PyPy 7.3.1 / Python 3.6.9 (OS - Oracle Linux 
Server release 8.8) where the whole PyPy process crushes with 
"segmentation fault" for unknown reason. I've tried to search a 
suitable way how to read the acquired coredump of the PyPy crushed 
process on the internet, but there are no any.



Could you please advice maybe you know how to Debug such situation?

Perhaps there is a way to read such coredumps of crushed processes?


P.S.: If additional info is required - feel free to ask.


TIA


Most often such segfaults are due to c-extension modules, and should 
print some type of RPython debug exception to stderr. Do you have access 
to the stdout/stderr of the process? Is there any more information? We 
no longer support that version. The last version of python3.6 was 
v7.3.3, maybe updating to that would help. If you can update further 
that might also help.


Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] PyPy bugfix release 7.3.13

[Matti Picus]

I am going to create a new release 7.3.13 to fix a problems in the conda 
feedstock. I improved the support for multiphase c-extension 
initialization immediately after the 7.3.12 release, and tried to merge 
pieces of it to the conda-forge feedstock but it has become too many 
patches to maintain.



If there is anything you think I should add to the release, please file 
an issue and mark it with the 7.3.13 milestone. If the CFFI 1.16 release 
happens before I get too deep into the PyPy release process, I will try 
to update it. I had hoped to get most of hpy0.9 into this release, but 
there is still a lot to do.



Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Improve symlink support for DirEntry on Windows

[Matti Picus]

On 22/9/23 04:19, cpbu...@gmail.com wrote:


I have an account on foss.heptapod.net, @cpburnz, but how do I request access? I don't 
see an option on the Project members page to request permissions. On that page, it says: 
"Members can be added by project Maintainers or Owners".

Thanks, I try those pytest.py methods.



I sent you an invite.

Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Improve symlink support for DirEntry on Windows

[Matti Picus]

On 15/9/23 03:19, cpbu...@gmail.com wrote:

Hi PyPy devs,

In one of my projects I noticed tests involving symlinked directories on 
Windows failing on PyPy 3.9/10 (v7.3.12), but succeeding on CPython 3.9/10. 
After digging into `pypy/module/posix/interp_scandir.py`, it looks like 
`W_DirEntry` has an incomplete implementation for Windows. The 
`W_DirEntry.is_*()` methods only return results for the symlink (the 
*follow_symlinks* argument is ignored via `.check_mode()`). The 
`W_DirEntry.stat()` method only returns the `stat_result` for the target file 
(ignoring the *follow_symlinks* via `.get_stat_or_lstat()`).

I have a proof of concept fixing the Windows implementation based upon the 
adjacent POSIX code. It passes the previously skipped tests `test_stat4`, and 
`test_dir3-6` in `pypy/module/posix/test/test_scandir.py`. My modification to 
`test_scandir.py` is kind of janky to support Python 2.7 on Windows because it 
doesn't natively support `os.symlink()`. The test uses 
`pypy.module.posix.interp_posix.symlink` when `os.symlink` is unavailable.

I'm not sure how to run `test_scandir.py` after compiling/translationing, and 
what other tests I should be running. I can successfully compile these 
modifications on the py3.10 branch on Windows 10 and Linux (Manjaro, an Arch 
Linux derivative).

The Contributing Guidelines recommends that I start with the mailing list 
before making a random merge request. What's the next step in possibly getting 
this fix reviewed?

Regards,
Caleb Burns



Thanks for picking this up. You are welcome to push a branch to 
https://foss.heptapod.net/pypy/pypy. If you have not yet been granted 
permissions, you can register and request them at 
https://foss.heptapod.net/pypy/pypy/-/project_members.



We have a number of pytest extensions to be able to run tests both pre- 
and post- translation. In order to run the current tests 
post-translation, you can run with -A



python2 pytest.py -A --python= 
pypy\module\posix\test\test_scandir.py



For extra points, we are trying to move tests to the newer apptest 
syntax. For an example, see apptest_posix.py. Then the translated tests 
can be run directly with pypy3 (or with CPython)



 pytest.py -D pypy\module\posix\test\apptest_scandir.py


Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Is PyPy MinGW support broken?

[Matti Picus]

On 19/6/23 10:45, Julian Waters wrote:

I'm working with the MinGW team; I can create a patch to help, but a 
quick low down on how the build_executable_cache and related logic 
works would help me out a little


Also: Is MinGW not used by the core team?

best**regards,
Julian

Thanks for giving PyPy a try. That code is a little obscure, but the 
goal is to figure out the size in C of different rpython types. So it 
creates a C file with various printf lines, compiles it, and checks the 
stdout. When I run it on linux here is the first C file (I put `import 
pdb;pdb.set_trace()` just before the line that fails to stop the process 
and look around) called gcctest.c that is created:



$ cat /tmp/usession-py3.9-0/gcctest.c

// includes
#include 
#include 
#include 
#include 
#include 



// checking code
int main(void)
{
   printf("sizeof __int128_t=%ld\n", (long)sizeof(__int128_t));
   return (0);
}


Next to that source file is the object file and the executable. Did the 
compilation succeed? What happens when you run the executable?



The "core windows team" is me, and I admit I have not tried to use mingw 
to build PyPy. There are some instructions here [0] but I don't think 
anyone has actually tried that in a long time so there is probably 
significant bitrot. Corrections are welcome


In order to build on windows, you must use PyPy2.7, a regular CPython 
2.7 will not work, see [1] for the justification.



Matti

[0] https://doc.pypy.org/en/latest/windows.html#using-the-mingw-compiler

[1] https://doc.pypy.org/en/latest/windows64.html#windows64

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Is PyPy MinGW support broken?

[Matti Picus]

On 19/6/23 10:08, Julian Waters wrote:

When I try to build PyPy with MinGW, I get the following:

...
It seems as if the ask_gcc function is broken for MinGW, and I can't 
tell why at a first glance. Is a fix possible?



Someone who uses mingw would have to suggest a patch.

Matti



___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: v7.3.12 has been released

[Matti Picus]

On 16/6/23 09:50, Phyo Arkar Lwin wrote:


Hi guys , grats!
Long time i haven't use pypy. Question , does pypy support pyo3 now ? 
There are a lot of rust-python binded libs now and i cannot use them  
, when i tried last year.
I am contribution to development of litestar framework which use 
rusted python libs , which i coin't get pypy to run with it.


Looking forward to use pypy again and we are going to get into tech 
empower benchmarks .


Regards

Phyo.



There is support in PyO3 for PyPy. Cryptography uses PyO3 and ships a 
PyPy wheel. Perhaps a github issue on the relevant project's repo would 
be a better place to discuss this.



I see this litestar issue [0] points to orjson, which led me to this 
issue in orjson [1], which was closed by adding this to their FAQ [2]: 
"Will it support PyPy? Probably not." On the other hand, taht was three 
years ago and the maintainer did express a willingness for other 
contributors to work on support for PyPy. Do you know what exactly 
doesn't work and what is the failure? I see there is support in litestar 
for using alternative json libraries, and truthfully PyPy has a pretty 
good reader built in, and PyPy works best when it does not have to call 
out to non-python modules.



Matti

[0] https://github.com/litestar-org/litestar/issues/851

[1] https://github.com/ijl/orjson/issues/90

[2] https://github.com/ijl/orjson#will-it-support-pypy

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] v7.3.12 has been released

[Matti Picus]

  
  
PyPy v7.3.12, which includes python 2.7, 3.9, 3.10 had been
  released. Please try it out.



Matti

  

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: PSF hosted PyPy infrastructure

[matti picus]

On Thu, Jun 1, 2023 at 7:48 PM Simon Cross  wrote:

> Hello Ee,
>
> > When I returned, I found that a discussion had ensued that presumed a
> lot of intent and was honestly hurtful.
>
> Maciej did express his concerns in an unhelpful way, and I'm sorry you
> were on the receiving end. I share some of Maciej's concerns about
> some of the PSF's stances, but that conversation didn't benefit
> anyone. For what it's worth, I don't think Maciej intended to presume
> anything about your intent.
>
> I can't apologise on Maciej's behalf, but thank you for writing here
> even though the conversation on IRC was shitty.
>
> Regarding the DNS, could you give me an example of the kind of setup
> that would make things better for you? And what kinds of
> infrastructure changes are you making? Is it just that servers move
> around? Or do nameservers also move? Would delegating NS records for
> certain domains to the right place help?
>
> If we just need to delegate some NS records to PSF nameservers, I
> don't foresee anyone having strong objections.
>
> I'm also not a DNS expert, so please excuse and correct anything above
> that is naive, incorrect or just won't work.
>
> Regards & many thanks for all your assistance over the years,
> Simon
>

Ee: we are waiting for a reply ...

Matti
___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Question: Is there any faster way to run benchmarks on pypy/module/cpyext module?

[Matti Picus]

On 29/5/23 07:37, hzy15610046011 via pypy-dev wrote:


Thanks for your answer!

I read the link [0] and run such benchmarks, finding that slow performance on 
allocate_tuple still exists now. On my cpu, it was 0.65s for cpython3.8, and 
7.6s for PyPy 3.9. I am wondering if this could lead to the undesired low 
performance on pandas.

As for my code pattern, I am trying to add support for PyPy in my simulation 
library, and the workflow of my project was like that:

1. Read data from excel files
2. Perform CPU-intensive computations with an object-oriented program involving 
a number of objects.
3. Write simulation data to a SQLite database by Pandas, just using `pd.to_sql`.

When running on PyPy, step 2 was more than 8 times faster than CPython 
interpreter. However, step 3 were 3~5 times slower.

I have found that it was not a problem caused by the sqlite3 library inside 
PyPy, because on a pure-python SQLite3 program,  PyPy was 1.2x~2x faster than 
CPython. So this problem might be due to the C-API performance problem when 
calling pandas.

As far as I know, (1) the easiest way is to rewrite a pure-python table IO 
library instead of pandas, because there were just few functions in pandas that 
had been imported into my project. (2) But if one day the performance of pandas 
on PyPy could be better (about 0.5x~0.8x of that on CPython), the better idea 
should be continuing using Pandas, because most of the python programmer knows 
it.

Could you please give me some suggestion about what I should do to solve this 
problem? Should I choose way (1) to implement a pure-python table library, or 
had better wait for (2)? Also I am interested in PyPy project itself, and 
wondering if improving performance for `Py_BuildValue` is feasible. Thanks!

Hou



I would think a properly written pure-python solution (1) could out 
perform any c-extension on PyPy, but I don't think such a project exists.



For (2), in the long term there is HPy [0]. In the short term, there are 
many possible optimizations we could do for cpyext. Are you sure 
Py_BuildValue is the top of the list, i.e. did you profile `pd.to_sql` 
and that came out as a very common and slower-than-cpython function?



Matti


[0] https://hpyproject.org/

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Question: Is there any faster way to run benchmarks on pypy/module/cpyext module?

[Matti Picus]

On 28/5/23 21:55, hzy15610046011 via pypy-dev wrote:


Hello pypy experts!

What I want to do is to run a performance test on the cpyext module.  The 
translation procedure took about 1 hour on my computer. After I 
modifying some code( just a slight modification under pypy/modules/cpyext), the 
whole translation procedure seemed to restart completely, taking another one hour.

I found pytest could do unit test, but it seems to be run on a different 
interpreter such as Cpython2 or another pre-built pypy interpreter.

So I was wondering if there is any way to run performance benchmarks partially, 
without compiling the whole project, honestly thanks!

-- zy hou



The short answer is "you need to do a complete translation in order to 
benchmark". But you can run some microbenchmarks like in this blog post 
[0] and use kcachegrind to look for inefficiencies if you know what you 
are looking for.


Is there some particular code pattern you would like to speed up?

Matti


[0] 
https://www.pypy.org/posts/2018/09/inside-cpyext-why-emulating-cpython-c-8083064623681286567.html#avoiding-unnecessary-roundtrips


___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] PyPy v7.3.12rc2 release candidate available

[Matti Picus]

I have uploaded 3 versions of python: 2.7, 3.9, 3.10 (first time 
release) as the PyPy7.3.12rc2 release candidate.

Release note: https://doc.pypy.org/en/latest/release-v7.3.12.html


Downloads: https://downloads.python.org/pypy/
The aarch64 3.9 rc2 is missing, I am rebuilding it now.

checksums: https://www.pypy.org/checksums.html

I will be updating the https://downloads.python.org/pypy/versions.json 
file so these can be used in github actions/azure CI soon.


Please try these out and report any problems.
Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] PyPy v7.3.12rc1 release candidate available

[Matti Picus]

I have uploaded 3 versions of python: 2.7, 3.9, 3.10 (first time 
release) as the PyPy7.3.12rc1 release candidate.

Release note: https://doc.pypy.org/en/latest/release-v7.3.12.html


Downloads: https://downloads.python.org/pypy/


checksums: https://www.pypy.org/checksums.html (once the site regenerates)


I will be updating the https://downloads.python.org/pypy/versions.json 
file so these can be used in github actions/azure CI after the weekend.


Please try these out and report any problems.
Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: New release cycle started

[Matti Picus]

Thanks for the offer. I will contact you privately.

Matti


On 4/5/23 10:23, Joshua Szepietowski wrote:
I have zero supporting knowledge, but I wouldn't mind learning if you 
can train me.


On Wed, May 3, 2023, 9:17 PM Matti Picus  wrote:

I think the py3.10 branch has reached a point where we can start a
new
release cycle. We last released Dec 29, 2022, and soon after
created the
py3.10 branch. Work on this progressed quickly, due in no small
part to
earlier work done on the interpreter/parser to handle the new 3.10
features.


Those who read the release notes will notice I am looking for
someone to
take over as release manager. I have been releasing PyPy since
2014, it
is time to pass the job on. Anyone interested could pair-program
with me
through this release cycle, so now is the time to volunteer.


Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: joshs...@gmail.com


___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] New release cycle started

[Matti Picus]

I think the py3.10 branch has reached a point where we can start a new 
release cycle. We last released Dec 29, 2022, and soon after created the 
py3.10 branch. Work on this progressed quickly, due in no small part to 
earlier work done on the interpreter/parser to handle the new 3.10 features.



Those who read the release notes will notice I am looking for someone to 
take over as release manager. I have been releasing PyPy since 2014, it 
is time to pass the job on. Anyone interested could pair-program with me 
through this release cycle, so now is the time to volunteer.



Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Windows support needs work

[Matti Picus]

While it basically works, PyPy's support for windows is not great:


- The winconsoleio crashes and has been disabled (the winconsoleio-debug 
branch was last touched in July 2020), leaving PyPy with a suboptimal 
console experience. The unicode rendering does not work properly, so 
things like "rich" print out random ascii art. [0]



- There is a problem with the posix interfaces. Things like 
os.listdir(dir-as-bytes) return slightly different results on CPython 
and on PyPy. This has made it difficult to support the missing 
`os.symlink` and `os.readlink`, which are becoming more popular as 
support for symlinks in windows improves. [1]



- The only reason we still have unicode strings in RPython is to support 
windows (see  uses of space.realunicode_w). This makes even starting a 
RPython3 port much more difficult. It probably also slows down windows 
performance.



- The work to create a windows installer was never finished (the 
windowsinstaller branch was last touched in Nov 2019)



- There are builds of PyPy on windows at conda-forge, but they seem to 
be almost broken [2]



How can we find contributors or sponsors to improve PyPy support for 
windows?


Matti


[0] issues 2999, 3767, 3768,

[1] issues 3225, 2753, 3506, 3535, 3641, 3479, 3434

[2] https://github.com/conda-forge/pypy3.6-feedstock/issues/101

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Using pypy sandboxing with Python 3.x

[Matti Picus]

On 2/2/23 13:27, Sidrah Sayyad wrote:

Hi team,

I came across the PyPy documentation at 
https://doc.pypy.org/en/latest/sandbox.html and was particularly 
interested in trying out the sandboxing feature.
However, the documentation seems to be old and probably works for 
python2.x.
I downloaded PyPy 3.8 from https://www.pypy.org/download.html for 
Linux x86 64 bit.

Is sandboxing available here ? If yes, how can one use it ?

Regards,
Sidrah.



Did you see the warning at the top of the page?

"This describes the old, unmaintained version. A new version is in 
progress in the |sandbox-2| and |py3.6-sandbox-2| branches and in the 
sandboxlib  repo. Please see 
its description here: 
https://mail.python.org/pipermail/pypy-dev/2019-August/015797.html 



Also note that python 3.7+ requires the _thread module, which may be a 
consideration in escaping the sandbox."


Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Dropping py3.8, working on py3.10

[Matti Picus]

We have begun work on py3.10. I changed the buildbot configuration to 
build default (py2.7), py3.9, py3.10 instead of default (py2.7), py3.8, 
py3.9. Builds for py3.8 (or any branch) can be kicked off manually. 
Unless there is significant push-back, or some latent urgent bug appears 
on py3.8, I do not plan on making another py3.8 release.



Help is always welcome, the py3.10 work is at a stage of "death by 1000 
paper cuts" where there are many small details that need to be changed. 
One thing people could do is to peruse the lib-python failures (testing 
py3.10 on the upstream stdlib tests) for obvious [2] problems. You can 
see the log file by looking at the latest builds [0] -> clicking on the 
pypy-c-jit-linux-x86-64 link -> shell_16 lib-python test failed -> 
stdio. There are some hints for parsing that output on the wiki [1].



Matti


[0] https://buildbot.pypy.org/summary?branch=py3.10

[1] https://foss.heptapod.net/pypy/pypy/-/wikis/Analysing-lib-python-errors

[2] what may be obvious to you may be gibberish to someone else, which 
is why we need everyone's expertise


___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: rpython tests

[Matti Picus]

On 30/12/22 14:24, Oliver Margetts wrote:

Hello all,

Are the tests in the rpython/ directory expected to pass when run via 
`pypy pytest.py rpython/`? And do you have a current CI setup for 
running these tests?


With zero modifications to the codebase, I'm seeing quite a few test 
failures in the CI pipelines over here: 
https://github.com/olliemath/pypy/actions/runs/3803555068/jobs/6470041568 
Some of these are spurious (I think the runners don't have enough 
memory) but quite a few are reproducible locally. I'm interested in 
getting them up to scratch (i.e. passing), but don't have a sufficient 
baseline to go off of.


For background: I'm currently playing around with RPython 
(specifically adding "from __future__ import print_function", see 
https://github.com/olliemath/pypy/pull/1). As part of that I decided 
to use github workflows to test my code using the unofficial github 
mirror of pypy. To make sure I don't break anything I would ideally 
start from a green pipeline and iterate.


Best,
Oliver



The rpython tests should pass with pypy2. There is some tricky code in 
rpython/rtyper/lltypesystem/ll2ctypes to allow untranslated tests to 
run, it may need some tweaks since it is sometimes not clear what is run 
on the host python and what is run interpreted.


As I commented on the PR, I had a rpython3 branch that was meant to get 
the document build to use python3 instead of python2 since readthedocs 
will someday quit supporting python2. There was some dicusssion about 
it, which I cannot find, and it never made it to a real PR even though I 
started a  rpython3-for-merge branch to merge the non-controversial 
parts of rpython3, but never merged it.



Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Release of PyPy v7.3.11

[Matti Picus]

I have released a bug-fix v7.3.11 of PyPy with python2.7, 3,8, 3..9. The 
complete release note is at 
https://doc.pypy.org/en/latest/release-v7.3.11.html. Looking back at all 
the blog posts with the release tag 
https://www.pypy.org/categories/release.html, it seems there were 40 
releases since my first 2.4.0 in 2014.



Plans for 2023 include python3.10 and maybe python 3.11, updating HPy, 
and more performance enhancements. I also want to find a way to give 
more information about memory allocations to help people understand if 
there are leaks.



Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] PyPy v7.3.10 release candidate available

[Matti Picus]

I have uploaded 3 versions of python: 2.7, 3.8, 3.9 (no longer beta 
quality) as the first release candidate of PyPy7.3.10. They are named 
rc3 because I made mistakes when tagging the repo and did not want to 
change the tags.


Release note: https://doc.pypy.org/en/latest/release-v7.3.10.html


Downloads: https://downloads.python.org/pypy/


checksums: https://www.pypy.org/checksums.html


As soon as the CDN network updates the 
https://downloads.python.org/pypy/versions.json file, you can use them 
in github actions CI.



Highlights from the release notes:
- A release of Apple Silicon M1 arm64 versions. This work was sponsored 
by an anonymous donor and is tested on our buildbots.

- Many improvements to the basic interpreter to make it faster
- The conda-forge community has built over 1000 packages for PyPy3.8 and 
3.9, making it easier than ever to use PyPy.
- Update the packaged OpenSSL to 1.1.1s and apply applicable secruity 
fixes from CPython 3.9.15 to PyPy2.7

- Update the HPy backend in PyPy3.8 and PyPY3.9 to 0.0.4

Please try these out and report any problems.
Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: PyPy Winter Sprint 2019 + cpyext

[Matti Picus]

On 4/2/19 10:36, Stefan Beyer wrote:

Hey everyone!

Those of you who attended or followed one of the last two winter 
sprints in Leysin might know me, the others probably won't. I'm 
(still) a Master's student writing my thesis on PyPy's current issue 
with cross-heap cycles when using cpyext. The main point is, that they 
are not collected and stay around as floating garbage, even if they 
are not reachable any more. Correct me if I'm wrong, but I didn't 
notice anyone else working on that topic (and/or committing something) 
since I've picked it up two years ago (yeah, I know, that long ago...).


I saw that you would be working on "cpyext performance and 
completeness" during the current winter sprint this week and thought 
that this might also concern my thesis. So I thought I'll give you an 
update.


I recently pushed my current (non-optimized, breaking-some-tests, but 
more or less working) implementation of the "CPython-style" 
GC-extension for cpyext. It is still in an early stage and not all 
cases (legacy and non-legacy-finalizers, weakrefs) are handled. 
However, I picked up some pace during the last couple of weeks and I'm 
determined to finish this implementation during the following weeks 
(some quirks with the tests have been haunting me, but I think I 
figured most of them out by now). After that, I will implement a 
second alternative implementation (to compare to, mostly for the sake 
of my thesis), which will take some more time. I also added a couple 
of fancy test cases (the so called "dot-tests"), so that we can test 
complex object graphs a little bit easier (and also because it was 
kind of cool to have another language inside PyPy/RPython, even if it 
was only for the tests...), with more test cases to come (the current 
ones are a bit messy). None of the new changes concerning low-latency 
applications are currently integrated, but that should not be too hard.


I guess it won't make much sense for me to join you at this year's 
winter sprint, especially as I won't be able to get there before 
Thursday, but I might be able to join you over the IRC channel (or 
some other form of communication if you like). If there is anything 
that is worth discussing please let me know! Also feel free to comment 
on my code, but beware that I might change some things once I try to 
do some optimizations (probably not many, but at least fix the worst 
issues) and make it a little bit more readable. You can find my code 
on the cpyext-gc-cycle 
branch.


Looking forward to hearing from you!

Greetings,
Stefan



Hi Stefan. It has been a while, I hope all is good by you.


Recently there was some interest in looking at the work you did in the 
cpyext-gc-cycle branch toward getting it merged into PyPy. Did you ever 
get any further than the published code? I merged the current py3.8 
branch into cpyext-gc-cycle and immediately ran into problems (see below).



It would be great to keep pushing this forward. Would you like to pick 
this up again or review changes as needed?



Any thoughts?

Matti



N.B: PyUnicodeObject has tp_itemsize==0 when in reality it sometimes 
allocates extra room for data [0]. My fix in [1] didn't help.


[0] https://foss.heptapod.net/pypy/pypy/-/issues/3772

[1] 
https://foss.heptapod.net/pypy/pypy/-/commit/2d53e90b42a803bfac8ca1fe7b8f70433deed89e


___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Question about pypy rpython and jitlogs

[Matti Picus]

I think you are looking for the JIT "threshold" option [1], which can be 
specified as



pypy --jit threshold=200


to get the JIT to consider a loop "hot" after it has been hit 200 times. 
The default is 1000



Matti

[0] 
https://doc.pypy.org/en/latest/jit_help.html?highlight=trace_eagerness#advanced-jit-options




On 24/9/22 14:04, Pavel Durov wrote:

Hi,

I'm not sure that's the best place to ask questions, but I couldn't 
find other mediums or documentation.


I have a question regarding pypy jit logs:

Sometimes when I run an RPython program that runs very quickly, I 
don't get any jit logs produced, but when I intentionally increase the 
execution time (by adding more loops, for example), I do get the logs.


What is the mechanics behind jit logs? Is there some kind of a hot 
threshold that needs to be reached? Where can I read more about it?


My stack overflow question from a couple of weeks ago:
https://stackoverflow.com/questions/73665618/rpython-jit-logs 
noredirect=1#comment130093714_73665618


I moved on from this problem since I was able to produce logs 
eventually, but I would like to understand what's happening behind the 
scenes.


Thank you,
Pavel

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: matti.pi...@gmail.com

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: The certificate for www.pypy.org is invalid

[matti picus]

You are right, the problem is local. My DNS cache seems to have gotten
messed up. I have switched to another machine until I figure out what
is going on.
Matti


On Mon, Sep 12, 2022 at 9:56 PM Matt Billenstein  wrote:
>
> I see a Lets Encrypt cert there:
>
> https://ismycertexpired.com/check?domain=www.pypy.org
>
> m
>
> On Mon, Sep 12, 2022 at 10:46:27PM -0400, matti picus wrote:
> > Something changed over the last few days. I now get this message when 
> > trying to
> > figure out what is wrong with the https://www.pypy.org:
> >
> > Websites prove their identity via certificates. Firefox does not trust this
> > site because it uses a certificate that is not valid for www.pypy.org. The
> > certificate is only valid for the following names: api.lookc.app, lookc.app
> >
> > The certificate is valid from Sept 2 to Dec 2 2022, which makes me think
> > someone updated it lately.
> >
> > Matti
>
> > ___
> > pypy-dev mailing list -- pypy-dev@python.org
> > To unsubscribe send an email to pypy-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/pypy-dev.python.org/
> > Member address: m...@vazor.com
>
>
> --
> Matt Billenstein
> m...@vazor.com
> https://vazor.com
___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] The certificate for www.pypy.org is invalid

[matti picus]

Something changed over the last few days. I now get this message when
trying to figure out what is wrong with the https://www.pypy.org:

Websites prove their identity via certificates. Firefox does not trust this
site because it uses a certificate that is not valid for www.pypy.org. The
certificate is only valid for the following names: api.lookc.app, lookc.app

The certificate is valid from Sept 2 to Dec 2 2022, which makes me think
someone updated it lately.

Matti
___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Contribute to complement the original documentation

[Matti Picus]

On 7/9/22 12:24, Rangga Kusuma Dinata wrote:

Hi, there. My name is Rangga. I knew about PyPy several years ago, but 
I stumbled upon some online judge problem that made me review this 
blog . So I install PyPy but actually, I don't 
find any guide on documentation about how to use it properly for the 
first time especially when you already have Python on your machine. 
So, I may have to take look once more to get it. The thing is, I'm 
already excited about this project and want to contribute to your 
documentation/blog so those people who are excited like me didn't get 
their excitement down just because your documentation is not clear 
enough and not friendly to the newcomer.


I will take my time and take notes about my POV as a newcomer once I 
found how to contribute properly to this awesome project so PyPy will 
get the recognition it deserves. Thank you for all the contributors 
and the hard work they have done. You inspire me to be a contributor 
as well, especially on the open source project. I'm writing this email 
with a burning passion! Haha


My best regard,
Rangga aka Retiago Drago



Hi Rannga and welcome. As you pointed out the blog lives at 
https://www.pypy.org/blog/, the source for that is the github repo 
https://github.com/pypy/pypy.org/#readme. The PyPy documentation lives 
at https://doc.pypy.org/en/latest/, the source for that is the "doc" 
directory of the source code 
https://foss.heptapod.net/pypy/pypy/-/tree/branch/default/pypy/doc. 
Since PyPy is compatible with CPython, your experience actually using it 
should be very similar to using CPython. Where the two differ is 
documented at https://doc.pypy.org/en/latest/cpython_differences.html. 
You may be interested in looking at the FAQ as well at 
https://doc.pypy.org/en/latest/faq.html



Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Performance tuning, GC logs

[matti picus]

On Sat, Aug 27, 2022 at 12:42 AM AJ R  wrote:

> Hi,
>
> What's the recommended way to improve my program's performance, in
> particular wrt garbage collection? I see jitviewer and vmprof, but I'm not
> sure if either is still maintained. I've noticed speed improvements after
> playing with the gc nursery size, but that was trial-and-error.
>
> Thanks.
>
> Albert-Jan
>

You may be interested in this blog post [0] around improving gc latency,
with some information about how to benchmark that particular problem. There
is also the gc.get_stats() function might help you figure out what is going
on. For more help we would need to know what exactly you mean by "speed
improvements" and where you see bottlenecks in your program.

Matti

[0]
https://www.pypy.org/posts/2019/01/pypy-for-low-latency-systems-613165393301401965.html
[1] https://doc.pypy.org/en/latest/gc_info.html
___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: 32bit Windows build

[Matti Picus]

On 10/8/22 17:07, Matti Picus wrote:
>Any chance of making a 32bit Windows build of PyPy2.7-v7.3.9 
available for download?



That is a big ask now that we have moved to 64-bit builds. We lack the 
infrastructure to support two windows builds, so we have not been 
running the 32-bit buildbot ever since we started up the 64-bit one. 
The changes in the https://foss.heptapod.net/pypy/externals repo. the 
win64_14x branch to update libffi, expat, openssl, bz2, and sqlite3 
should be redone for 32-bit windows in the win32_14x branch. Then 
there may be more fixes needed.



Matti



I updated the libffi and kicked off a build, without updating the rest 
of the extensions. It succeeded, so now there is a win32 tarball on 
https://buildbot.pypy.org/nightly/trunk/ for pypy2.7. Please update us 
with how it goes: is the newer version better/faster/stronger? Do you do 
anything that would require updating the rest of the external libraries?


Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: 32bit Windows build

[Matti Picus]

>Any chance of making a 32bit Windows build of PyPy2.7-v7.3.9 available 
for download?



That is a big ask now that we have moved to 64-bit builds. We lack the 
infrastructure to support two windows builds, so we have not been 
running the 32-bit buildbot ever since we started up the 64-bit one. The 
changes in the https://foss.heptapod.net/pypy/externals repo. the 
win64_14x branch to update libffi, expat, openssl, bz2, and sqlite3 
should be redone for 32-bit windows in the win32_14x branch. Then there 
may be more fixes needed.



Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: How can we make Python's C-API more friendly for cpyext?

[Matti Picus]

On 29/4/22 20:20, encu...@gmail.com wrote:


Hello,
I'm figuring out a strategy for the future of Python's C-API, and as one of the 
goals I'd like to make it friendly for non-C languages. I'd like to ask you for 
any suggestions, pain points or comments on how to better do that.
Would you like to help?
I understand PyPy solved a lot of these issues and won't get too much benefit, 
but I'd like to improve things for future projects.
I'm also aware of HPy, a long-term solution. I'd like to meet in the middle.

I started a [Google 
doc](https://docs.google.com/document/d/1fDfF7JanOEyQ9awhKE87-Ajp3H4k7qXL3cmyzPMoNPQ/edit#)
 to collect notes. If you don't mind Google, please feel free to comment there, 
but I'll be happy to discuss here as well. If you have a link to a 
document/rant, I'd love to go through it.

In particular, I'd like to ask if any of the following would help, or would 
have helped in the past:
- Regularity in strong/borrowed references (always returning strong references 
and taking borrowed ones)
- Regularity in error handling (all functions should have a dedicated return 
value, an exception should be set set iff that value is returned)
- Avoiding preprocessor macros
- Avoiding bit fields
- Avoiding enums
- Exposing API/ABI information in a structured format, rather than C headers
Is anything else missing from this list?

Thank you for your time!



Thanks for reaching out. One person's opinion to get the discussion going:


The first two points about regularity would help.


Preprocessor macros are problematic where we need to reproduce the exact 
defines used by the "configure" front end to compilation.



I don't think the bit fields, or enums are too hard for PyPy to handle, 
we use cpyparser to parse the C syntax and I think those are handled 
correctly.



I am not sure what you mean by "structured format", for us C structs are 
fine, some XML or JSON constructs would probably be harder to use.



One thing not on the list is assuming that objects can be resurrected 
during deallocation. The PyPy garbage collector cannot resurrect 
objects. This is less a problem for pure-python code than for C 
extensions that implement complicated functionality inside tp_dealloc, 
or when calling PyObject_CallFinalizerFromDealloc (which is not 
supported on PyPy at all).



Another thing would be better code examples as snippets of code to test 
the interfaces that are part of the stable API. For instance, I have 
been unable to write such a sample for PyCFunction_New [0]. Having such 
snippets as part of a PEP to add a new interface would make it much 
easier to implement new functionality. The VectorCall functionality 
would have been much easier to emulate if the spec was completely 
described by code snippets.


Matti


[0] https://foss.heptapod.net/pypy/pypy/-/issues/3601

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: buildbot master upgrade to support python3 workers?

[Matti Picus]

On 6/4/22 11:16, Matt Billenstein wrote:


On Wed, Apr 06, 2022 at 10:57:13AM +0300, Matti Picus wrote:

On 6/4/22 00:59, Matt Billenstein via pypy-dev wrote:


Hi, python2 is really long in the tooth - when are we planning to upgrade the
buildbot master? I cannot install python2 from homebrew on macos any longer,
the system python2 is really really old...

thx

m


We are using buildbot 0.8.8. CPython is using 3.5.0. When I looked into
upgrading, the first thing that caught my eye is that buildbot now requires
a database (ours currently stores pickled files on disk), so we would need
to get some kind of database going on the host machine. Then we would need
to try upgrading the versions, there are upgrade guides [0] starting from
0.9 all the way up to 4.0

I'm naive, but why would you need to upgrade? Why not just create a new master
and start fresh?

m



We could start fresh. We do have some customized pages that would have 
to be updated: summary (hard, it includes log parsing and building 
custom HTML), nightly (easy), maybe some others. So moving to a newer 
buildbot would be nice, but it is quite a bit of work.



We still require python2 for running untranslated tests. However (1) the 
macos buildbot is currently not running untranslated tests, (2) pypy2 
should JustWork for these tests, and if not we should fix that.



Maybe you could run the buildbot worker with PyPy2? I tried doing that 
with the windows worker, but it turned out some of the dependencies were 
missing on windows. Perhaps they are available for macOS?



Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: buildbot master upgrade to support python3 workers?

[Matti Picus]

On 6/4/22 00:59, Matt Billenstein via pypy-dev wrote:


Hi, python2 is really long in the tooth - when are we planning to upgrade the
buildbot master? I cannot install python2 from homebrew on macos any longer,
the system python2 is really really old...

thx

m



We are using buildbot 0.8.8. CPython is using 3.5.0. When I looked into 
upgrading, the first thing that caught my eye is that buildbot now 
requires a database (ours currently stores pickled files on disk), so we 
would need to get some kind of database going on the host machine. Then 
we would need to try upgrading the versions, there are upgrade guides 
[0] starting from 0.9 all the way up to 4.0



Matti


[0] https://buildbot.readthedocs.io/en/latest/manual/upgrading/index.html

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: How have you found compiling in OpenSSL and other libraries into your relocatable builds?

[Matti Picus]

I think there are two different topics here:

- macOS, windows: PyPy is not doing a good job with the native 
installers. We should be following CPython and provide real packages, 
not a tarball. This is issue 3697 [0] for macOS and issue 2894 [5] for 
windows.



- linux: (aarch64, x86_64, i686): we create a portable build of PyPy. It 
starts in a docker file based on manylinux2014 [1] (note the various 
install scripts to get updated versions of some libraries). There is an 
additional wrinkle in that some of the python modules we ship are based 
on cffi c-extensions (like _ssl, _gdbm, lzma). For these we have an 
additional build step to download and build the relevant support library 
on every build [6]. Then when packaging, we use  ldd to check 
dependencies [2], set RPATH appropriately, and bundle those into the 
tarball [4]. We make sure that not only libpypy3-c.so is checked with 
ldd, but also the c-extensions [3]. There is a tradeoff between putting 
libraries in the Docker file and rebuilding the buildbots or building 
them each time, and openssl is actually on both build paths (probably 
this could be simplified). CI caching could probably be used to lessen 
the burden, we have a different build/test workflow from CPython where 
we typically build only once a day.



[0] https://foss.heptapod.net/pypy/pypy/-/issues/3697

[1] https://foss.heptapod.net/pypy/buildbot/-/tree/branch/default/docker

[2] 
https://foss.heptapod.net/pypy/pypy/-/blob/branch/default/pypy/tool/release/make_portable.py#L39


[3] 
https://foss.heptapod.net/pypy/pypy/-/blob/branch/default/pypy/tool/release/make_portable.py#L119


[4] 
https://foss.heptapod.net/pypy/pypy/-/blob/branch/default/pypy/tool/release/package.py#L292


[5] https://foss.heptapod.net/pypy/pypy/-/issues/2894

[6] 
https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.8/lib_pypy/pypy_tools/build_cffi_imports.py#L65



As for complaints about OpenSSL versions: we release 2-3 times a year so 
can respond "quickly" if security concerns arise. My impression, based 
on the little feedback we get, is that shipping portable 
manylinux2014-based builds solves more end user problems than it causes. 
Sophisticated end users can always rebuild to suit their purposes.



Matti


On 1/4/22 01:47, matt--- via pypy-dev wrote:

I think you have to bundle on macOS - Apple ships a really old openssl.

I think cpython on macOS does this already? Might be good to be consistently 
bundling the same thing in Linus as well.

M

--
Matt Billenstein
m...@vazor.com


On Mar 31, 2022, at 3:36 PM, Brett Cannon  wrote:

I noticed that https://www.pypy.org/download.html says that "the linux binaries 
ship an OpenSSL library". Have you found that to be an issue in any way? Have users 
been accepting of it?

I ask as I'm contemplating trying to get relocatable builds for CPython on 
Linux (to start), and I suspect the OpenSSL question is going to be one of the 
bigger/stickier discussion points and having some real-world 
experience/information on the topic would probably help.
___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: m...@vazor.com

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] PyPy v7.3.9 was released

[Matti Picus]

I released PyPy v7.3.9.


This is mainly a security update, but we also fixed a number of PyPy3.9 
edge cases and cleaned up a performance regression. You can read more 
about it on blog post 
https://www.pypy.org/posts/2022/03/pypy-v738-release.html The downloads 
are at https://downloads.python.org/pypy/, and checksums at  
https://www.pypy.org/checksums.html



Unfortunately readthedocs is not building the latest version of the 
release note.  You can read the raw one at 
https://foss.heptapod.net/pypy/pypy/-/blob/branch/default/pypy/doc/release-v7.3.9.rst. 
  Hopefully that will get cleaned up soon.



Please report any other problems.


Matti

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

[pypy-dev] Re: Building a C extension that uses PyDictKeysObject on Pypy 3

[Matti Picus]

On 12/3/22 23:20, Marco Sulla wrote:

Hello. Sorry for posting in the dev mailing list, I did not found an user list 
for Pypy.

I developed a C extension for CPython 3.6+. This extension uses 
PyDictKeysObject. I tried to enable Pypy in cibuildwheel, but I get a 
compilation error for PyDictKeysObject.

What object can be used by C extension in Pypy instead of PyDictKeysObject?


This is the correct place to post, welcome. PyDictKeysObject is a 
CPython implementation detail, and not meant to be used even with 
CPython. The declaration is found in the cpython/Include/cpython/ 
directory. In general, we would recommend not writing c-extension 
modules for PyPy, rather providing a pure python version of your library 
since in 99% of cases it will be faster than going through the C-API 
emulation [0]. If you really have to use C, we would recommend you use 
CFFI [1] or CPPYY [2].



Matti



[0] 
https://www.pypy.org/posts/2018/09/inside-cpyext-why-emulating-cpython-c-8083064623681286567.html


[1] https://cffi.readthedocs.io/en/latest/

[2] https://cppyy.readthedocs.io/en/latest/

___
pypy-dev mailing list -- pypy-dev@python.org
To unsubscribe send an email to pypy-dev-le...@python.org
https://mail.python.org/mailman3/lists/pypy-dev.python.org/
Member address: arch...@mail-archive.com

Re: [pypy-dev] PYPY- Trace Limit

[Matti Picus]

On 7/3/22 22:19, Aksam Lwanga wrote:

Am working  on a person project I would like to know how meta trace 
limit configured for pypy.any assistance needed is highly appreciated.


On Mon, Mar 7, 2022, 3:43 PM Aksam Lwanga  wrote:

Hello , I would like to understand how to set pypy trace limit .
Any link available for use will highly help me.




Hi Aksam. You can see the various ways to influence the JIT via `pypy 
--jit help`. There are explanations for some of these parameters here 
https://rpython.readthedocs.io/en/latest/jit/index.html. There are also 
python-level hooks you can use, they are documented here 
https://doc.pypy.org/en/latest/jit-hooks.html#jit-hooks-1. You can also 
use the PYPYLOG to record activity/./


/
/

Take a look at these tools and let us know if you need more help.

Matti
//

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy3.7-v7.3.8 broke binary compatibility

[matti picus]

Once again, I broke ABI compatibility with the new release. I re-added
a field to PyDateTime_CAPI that should have been part of the original
PyPy3.7 release, but was missed, and so should be left out of PyPy3.7
v7.3.x. This causes segfaults when using PyPy3.7-v7.3.8 on wheels
built for PyPy3.7-v7.3.7 and older. Please do not build packages with
PyPy3.7-v7.3.8 for general release because they will not work on older
PyPy3.7.

What is doubly annoying is this is exactly why I needed to release
v7.3.7 over v7.3.6. PRs welcome to suggest a test for this. A "hg
diff" is a start, but it picks up refactoring and valid additions as
well, so it needs to be a bit smarter. Maybe something with CFFI?

Matti
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy 7.3.8 release

[Matti Picus]

I have released PyPy v7.3.8. Thanks to all who contributed and tested 
and for moving PyPy forward.



Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Differences between pypy2 and 3 and cffi

[Matti Picus]

On Tue, Feb 15, 2022 at 3:36 AM Ioannis Foufoulas  
wrote:


   Hi,
   I have an embedded function:

   @ffi.def_extern()
   def toint(input,insize,result):
    for i in range(insize):
        x = ffi.string(input[i])
        result[i] =  float(x) if x else 0
      return 1

   It seems that this runs around 30-40% percent slower in PyPy3 than in 2.

   Input is a char** c array and result is a float c array. The
   difference is not in ffi.string
   If having `result[i] = 1 if x else 0` it runs same in both versions.
   Any ideas?

   ___
   pypy-dev mailing list
   pypy-dev@python.org
   https://mail.python.org/mailman/listinfo/pypy-dev




I converted this to an issue 
https://foss.heptapod.net/pypy/pypy/-/issues/3682 with some timings and 
analysis of what might be happening.


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyCodeObject incompatibility

[Matti Picus]

On 15/2/22 05:15, Dmitry Kovner wrote:

Hi, again! First of all, thanks for the fast reply! The answer helped 
me a lot. However, during the porting of the extension to the awesome 
PyPy, I've got more questions:
1. The extension uses opcodes from cPython's opcode.h header file. 
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/opcode.h) 
There is no such file in the implementation of C API in PyPy. What 
could you recommend as the best alternative?
2. The extension uses some fields of PyThreadState structure. For 
example, its frame field 
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L59). 
Is it right to use PyObject_GetAttrString() to access that field in 
PyPy C API?
3. There is no function PyFrame_FastToLocals 
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Objects/frameobject.c#L931) 
in the C API of PyPy. Are there any alternatives?
4. Functions PyEval_SetTrace(), PyEval_SetProfile(), 
PyFrame_GetLineNumber() are defined only in stubs.py in PyPy. 
(https://foss.heptapod.net/pypy/pypy/-/blob/branch/py3.7/pypy/module/cpyext/stubs.py#L918) 
Is there a way to get that information in PyPy except patching of its 
source code?
5. Are there any analogues of all PyTrace_* constants 
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/cpython/pystate.h#L26) 
in the PyPy C API?
6. There is no function PyFrame_Check() 
(https://github.com/python/cpython/blob/8a84aef0123bd8c13cf81fbc3b5f6d45f96c2656/Include/frameobject.h#L53) 
in the PyPy C API. Are there any alternatives?


I'm really sorry for the long question! I hope the list of the 
problems described above is full enough to port the extension and I 
will have no more questions. :)


Best regards, Dmitrii



Even if PyPy could expose the interfaces you desire (and it would be a 
lot of work to do that), you would lose out on speed since the C-API on 
PyPy is much much slower than pure python. If you really want to port 
your extension, you might want to rethink what it is doing and refactor 
it to do so in a PyPy-friendly way: pure python + CFFI where needed to 
interface to libraries that expose C interfaces. The C-API is a layer we 
bolted on top of PyPy to support popular libraries like Cython and 
NumPy, it is not meant for projects that probe CPython internals.



Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy v7.3.8 rc2 now available

[Matti Picus]

PyPy v7.2.8 rc2 is now available for testing. Thanks to all who tried 
out rc1. There are a few bugfixes. The sys.version_info now also 
correctly reports



This is a release of 4 versions of python: 2.7, 3.7, 3.8 (no longer 
beta) and 3.9 (beta quality).


Release note: https://doc.pypy.org/en/latest/release-v7.3.8.html

Downloads: https://downloads.python.org/pypy

Checksums: https://www.pypy.org/checksums.html

Please try these out and report any problems. Note that we are having 
issues with the ARM8-64bit buildbot, so we are not releasing binary 
packages for that platform. Hopefully we will have an alternative for 
the final release.


Matti
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Installation layout of the PyPy 3.8 Fedora package

[Matti Picus]

On 27/1/22 14:59, Miro Hrončok wrote:

...
The only remaining thing is libpypy3.9-c.so in /usr/bin.
I'd like to move it to /usr/lib64 where it will be found.

Technically, Fedora forbids an unversioned so file there.

https://docs.fedoraproject.org/en-US/packaging-guidelines/#_downstream_so_name_versioning 



CPython has libpython3.9.so.1.0.

$ objdump -p /usr/lib64/libpython3.9.so.1.0 | grep 'SONAME'
  SONAME   libpython3.9.so.1.0

Do you think PyPy can do this as well? Or is there a reason not to?

Thanks!



(Previously replied privately, sorry about that).


I tried moving the so to /usr/lib64/libpypy3-c.so.1.0 and adding a 
symlink to /usr/lib64/libpypy3-c.so.


After running ldconfig to pick up the new file, I could run pypy3-c, and 
it seems venv and virtualenv do not complain about missing the shared 
object.



So it seems it JustWorks.


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyPy v7.3.8 rc1 now available

[Matti Picus]

Downloads: https://downloads.python.org/pypy


On 27/1/22 01:03, Matti Picus wrote:
This is a release of 4 versions of python: 2.7, 3.7, 3.8 (no longer 
beta) and 3.9 (beta quality).



Release note: https://doc.pypy.org/en/latest/release-v7.3.8.html


Downloads: https://


checksums: https://www.pypy.org/checksums.html


As soon as the CDN network updates the 
https://downloads.python.org/pypy/versions.json file, you can use them 
in github actions CI.



Please try these out and report any problems. Note that we are having 
issues with the ARM8-64bit buildbot, so we are not releasing binary 
packages for that platform. Hopefully we will have an alternative for 
the final release.


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy v7.3.8 rc1 now available

[Matti Picus]

This is a release of 4 versions of python: 2.7, 3.7, 3.8 (no longer 
beta) and 3.9 (beta quality).



Release note: https://doc.pypy.org/en/latest/release-v7.3.8.html


Downloads: https://


checksums: https://www.pypy.org/checksums.html


As soon as the CDN network updates the 
https://downloads.python.org/pypy/versions.json file, you can use them 
in github actions CI.



Please try these out and report any problems. Note that we are having 
issues with the ARM8-64bit buildbot, so we are not releasing binary 
packages for that platform. Hopefully we will have an alternative for 
the final release.


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Running PyPy on top of CPython

[Matti Picus]

Hi Omer. I think much of what Armin wrote is already in the 
documentation like 
https://doc.pypy.org/en/latest/contributing.html#architecture and 
https://doc.pypy.org/en/latest/contributing.html#test-driven-development. 
Could you see where things can be improved?


Matti


On 11/1/22 4:17 pm, Omer Katz wrote:

Hi Armin,

Would you mind adding the above to the documentation?
I think it'll be useful for newcomers.

Best Regards,
Omer Katz

‫בתאריך יום ב׳, 10 בינו׳ 2022 ב-18:03 מאת ‪Armin Rigo‬‏ 
<‪armin.r...@gmail.com ‬‏>:‬


Hi,

On Mon, 10 Jan 2022 at 15:56, M A mailto:teammember0...@gmail.com>> wrote:
> I think you are confused. I just read in PyPy's documentation
that PyPy could be run from CPython. This sounds like something
that could help save me time by seeing if my changes work. I'm am
not sure why you think I am ignoring the tests. Yes I have tried
them. I am seeing if there is a more efficient way of trying out
my code without having to wait a long time of PyPy to recompile.

Sorry for not explaining myself clearly.  I've been trying all along
to tell you that you don't need to recompile PyPy, ever. As long as
not all the tests I listed are passing, then there is basically no
point.  (except that it feels good to see PyPy half-working instead of
not working at all, of course, but don't try to debug that)

The documentation says "PyPy can be run on top of CPython": that's
almost what all the tests are doing.  They run not the whole PyPy, but
instead some small example RPython code (sometimes written as RPython
in the test, sometimes at a lower level).  The point is that they test
the JIT backend by running it as normal Python code, on top of
CPython.  When working on the JIT backend, you don't want to run the
whole PyPy on top of CPython running with the JIT; while possible in
theory, it is far too slow to be practical.  Instead, run the tests,
which test the same thing but with a small bit of RPython code instead
of the many-thousand-of-lines-of-code of PyPy.

Let me try to explain myself in a different way.
What we're trying to do here is fix a JIT compiler emitting machine
code, at the level of RPython instead of Python.  That means that any
error is likely to cause not just a problem for some specific Python
function which you can find by running Python code; instead, bugs
cause random hard-to-debug crashes that show up at half-random
points, with no obvious one-to-one correspondence to what Python code
is running.  This could be caused for example by the wrong registers
being used or accidentally overwritten in some cases, leading to a
nonsense value propagating further, until the point of crash. Most
tests listed above check all basic cases and problems we encountered
so far.  The last test is producing and running random exemples, in an
attempt to find the random rare-case issues.  It has been the case
that this last test found a remaining bug after running for hours, but
I think that after half a day we can conclude that there is no bug
left to be found.  In the past, there were very rare bugs that went
uncaught by random testing too; these were a real pain to debug---I
once spent more than two weeks straight on a pair of them, running
"gdb" inside a translated PyPy, hacking at the generated C code to
dump more information to try to figure it out.  But these bugs were
more than just in the JIT backend (e.g. some interaction between the
JIT and the GC), and they are fixed now.  I'm telling this story to
explain why I do not recommend going down that path for the simpler
bugs that are already caught by the tests!

So, once more, I recommend working on this by running the tests, and
fixing them, until all tests pass.  Once all tests pass (and not
before) you can try to translate the whole PyPy, and at this point it
will likely work on the first try.

Sorry for not managing to get this message across to you in the past.
I hope I have done so now.


A bientôt,

Armin.
___
pypy-dev mailing list
pypy-dev@python.org 
https://mail.python.org/mailman/listinfo/pypy-dev



___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Running PyPy on top of CPython

[Matti Picus]

On 10/1/22 4:31 am, M A wrote:

Well I am developing for PyPy and was hoping to try my code out without having 
to build PyPy first.



You asked this question on the issue tracker as well. Running PyPy on 
top of CPython is not a replacement for getting stress tests in the 
issue  comment [0] to pass.



[0] https://foss.heptapod.net/pypy/pypy/-/issues/3566#note_167193

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Running PyPy on top of CPython

[Matti Picus]

On 9/1/22 9:50 pm, M A wrote:

Hello, I read here (https://rpython.readthedocs.io/en/latest/rpython.html) that 
PyPy is runnable on top of CPython. How do I start PyPy so it runs in CPython?



python2 pypy/bin/pyinteractive.py


You can get help with

python2 pypy/bin/pyinteractive.py --help


It takes ~20 secs to startup. To drop into rpython, hit ctrl-c, to 
return to interpreted PyPy on top of CPython hit ctrl-d



In rpython you have access to "space".



Matti


___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Installation layout of the PyPy 3.8 Fedora package

[Matti Picus]

On 2/12/21 8:45 pm, Miro Hrončok wrote:


Hello PyPy folks,

I co-maintain the PyPy and CPython interpreter packages in Fedora



Thanks for supporting PyPy


We basically took the advice from 
https://doc.pypy.org/en/latest/build.html




That documentation may now need some updating.

Several different versions of pypy (e.g. 3.6 and 3.7) could be 
installed alongside each other, as long as only one of them "owned" 
/usr/bin/pypy3.



This e.g. means that the standard library is installed in:

 /usr/lib64/pypy3.8/lib/pypy3.8/

Or the header files are in:

 /usr/lib64/pypy3.8/include/pypy3.8/

Double namespacing like this seems a bit overengineered, and since the 
idea was to resemble CPython, I was aiming at this setup:

...



It seems reasonable that PyPy should follow CPython's lead in layout.


Which would be the exact copy of our CPython setup | sed s/python/pypy/


Indeed, that is the goal.



However, I found 4 problems:

1) The PyPy install scheme only has "lib", not "lib64"

I think we could sort this by a tiny patch that we used to carry in 
CPython 3.8 before configurable platlibdir was introduced in 3.9.


If it is part of CPython 3.9, it should be part of PyPy 3.9 as well. Can 
you check that the latest nightly at 
https://buildbot.pypy.org/nightly/py3.9/ indeed no longer needs that change?




2) The /usr/include/PyPy.h file is *not* namespaced with PyPy Python 
version


I don't know if we even need to ship this file. It seems deprecated. 
Maybe it can be in /usr/include/pypy3.8/PyPy.h?


I was confused by this as well. As Armin pointed out, it was part of the 
older cffi embedding code, and I think it can be dropped. See 
https://foss.heptapod.net/pypy/pypy/-/issues/3608


3) /usr/bin/pypy3.8 is a symbolic link to /usr/bin/pypy3

I would expect it to be the other way around



This makes sense. For testing, we need pypy3, but can create it as a 
symlink already as part of the build.




4) The /usr/bin/libpypy3-c.so file is *not* namespaced and seems 
misplaced


This is the one thing I cannot wrap my head around. I do not wish to 
install this file "globally" to /usr/bin



This also makes sense and some ideas were floated in other responses to 
this thread. It would be best if someone with a plan could open an issue 
for discussion, and even better a PR.




So, I am asking on this list: How should we package PyPy 3.8 for 
Fedora? Do we need to install it to /usr/lib64/pypy3.8/ prefix even 
thou most (but not all) of the directories within would contain a 
"pypy3.8" subdirectory? Or is there a better option I do not yet see?


While it is too late for pypy3.8, now is the time to tweak pypy3.9. I am 
not sure what support you expect from PyPy to recreate the same layout 
as the one used in CPython. Are there patches we can upstream?



There are other repackagers: debian, BSD, and conda-forge, so any 
changes would need to be done with an eye toward not breaking their 
workflow as well.



Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] How to fix pypy3 runtime error ?

[Matti Picus]

On 29/11/21 7:38 am, Hao Wang wrote:

Dear devs:

I encountered the following error but didn't know how to fix it. 
Please help.


cpyext, the emulation layer, detected that while it is calling
an object's tp_dealloc, the C code calls back a function that
tries to recreate the PyPy version of the object.  Usually it
means that tp_dealloc calls some general PyXxx() API.  It is
a dangerous and potentially buggy thing to do: even in CPython
the PyXxx() function could, in theory, cause a reference to the
object to be taken and stored somewhere, for an amount of time
exceeding tp_dealloc itself.  Afterwards, the object will be
freed, making that reference point to garbage.
>>> PyPy could contain some workaround to still work if
you are lucky, but it is not done so far; better fix the bug in
the CPython extension.
>>> This object is of type 'pandas._libs.parsers.TextReader'
Aborted

Bravo !
Hao Wang



You don't show any code so it is difficult to guess what is going on. In 
general, PyPy is more sensitive to closing resources after use rather 
than depending on the garbage collector to release them when an object 
is collected. You could try closing the parser before it goes out of scope.



Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] pip installation for pypy is slow

[Matti Picus]

On 29/11/21 6:01 am, Hao Wang wrote:


Dear pypy-devs,

The pip installation of pypy (pypy -m pip install scipy, for example) 
is slower than python-pip clean. Is there a way to fix this problem ?


Bravo!
Hao Wang



Thanks for giving PyPy a try. When pre-compiled binary wheels are not 
available, python must compile from source. This takes time. We have 
been making an effort to convince project maintainers to provide binary 
wheels for PyPy. SciPy has a complicated workflow and a skeleton support 
team. That together with the fact that currently PyPy + SciPy is much 
slower to run CI (small snippets of code run in tests are not PyPy's 
strong point, and the c-api emulation layer makes things worse) mean 
that SciPy is hesitant to add support for PyPy, see the PR where the CI 
runs were removed [0] in 2019.



Our long-term proposal to solve this problem is to get pybind11 (for c++ 
code), cython (for c code), and f2py (for fortran code) to generate 
wrappers using HPy [1] instead of the C-API, which will speed up testing 
with PyPy.  A second possibility is that the scientific python community 
will move away from its heavy dependency on the CPython C-API and more 
towards using higher-level protocols [4]. Both of these efforts will 
take many years unless heavily sponsored.



A more specific short-term solution for your installation problem is to 
use conda-forge's miniforge [3]. This will solve the installation 
problem, but PyPy will still be slow on scipy-heavy workflows. The 
conda-forge project supplies binary packages for pypy3.7, and is 
discussing pypy3.8 support [2]:



conda create -n pypy3.7 pypy

conda activate pypy3.7

conda install scipy


Matti


[0] https://github.com/scipy/scipy/pull/10085

[1] https://hpyproject.org/

[2] https://github.com/conda-forge/conda-forge-pinning-feedstock/issues/2089

[3] https://github.com/conda-forge/miniforge

[4] https://labs.quansight.org/blog/2021/11/pydata-extensibility-vision/

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] How to compile a file with rpython

[Matti Picus]

On 22/11/21 7:06 pm, M A wrote:



On Nov 22, 2021, at 1:08 AM, Dan Stromberg  wrote:

  
Why do you desire an executable?


Last I heard, a program would perform well on Pypy's JIT, and that using 
rpython was almost always unnecessary.

If you need an executable, you could also try Cython or Nuitka.

Well I'm working on code for PyPy. If I could unit test my code my development 
speed would increase a lot. It takes a long time for PyPy to build on my 
computer.
___



The way to unit test code is with


python2 pytest.py rpython/path/to/test/test_file.py


This is explained in the testing section of the "how to contribute" [0]. 
If that document is not clear, please help us improve it.



Matti


[0] https://doc.pypy.org/en/latest/contributing.html#testing

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] How to compile a file with rpython

[Matti Picus]

On 21/11/21 9:16 pm, M A wrote:


How do I make a program using rpython?

Say I have this file:

def main():
 for i in range(100):
 print("hello world"),
 
if __name__ == "__main__":

 main()

How would I turn it into an executable file using rpython?



The canonical starting place for rpython is this page [0] in the rpython 
documentation. It is a bit old, so please raise an issue if something 
does not work right. There are some examples in 
rpython/translator/goal/, the targetnopstandalone.py one is close to 
what you are trying to do. If you call



python2 rpython/bin/rpython -O2 
rpython/translator/goal/targetnopstandalone.py



you will get an executable in 
$tmp/usession*/testing_1/targetnopstandalone-c that will print "hello world"



Matti


[0] https://rpython.readthedocs.io/en/latest/getting-started.html

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] macos buildbot offline for a few days

[Matti Picus]

On 12/11/21 12:36 am, Matt Billenstein via pypy-dev wrote:

Starting tomorrow morning PST, I'm moving, so it should be back sometime
Saturday evening or perhaps Monday if I have trouble with the internet
self-install.

thx

m



Thanks for the heads up and thanks for allowing us to use your machine. 
I hope the transition goes smoothly.


Matti


___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy 7.3.7 bugfix release

[Matti Picus]

I mistakenly added fields to the middle of PyDateTimeAPI and 
PyFrameObject between the 7.3.5 release and the 7.3.6 release. This 
broke the promise of ABI compatibility between all the 7.3.x releases 
for the python 3.7 version.


Python 3.8 had a different problem connected with the values of 
INCLUDEPY and LIBDIR in _sysconfidata that led to cases of inability to 
build c-extension modules.


I have released PyPy 7.3.7 to fix these issues. I have also extended the 
weekly testing in https://github.com/pypy/binary-testing 
 which hopefully will catch ABI 
breakage before a release.


Please do not use the 7.3.6 releases.

Sorry for the trouble,

Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] What to call the latest pypy3.8 release?

[Matti Picus]

I broke C-ABI compatibility with the latest 7.3.6 release of PyPy. Also 
this is hard to talk about since we have python3.7 and pypy7.3.6 (soon 
to be pypy7.3.7).


For 3.7 (probably the last fully supported release, only security and 
large bug fixes planned), I will back out the C-ABI breakage so that 
existing pp37-pypy73 binary wheels will continue working and make a 
7.3.7 release, see the abi-compat branch.


I am not sure how to progress with the python3.8 branch, under the 
assumption that there will not be another 3.7 release. On the one hand, 
purists would argue "we changed the ABI, we need to increment the 
version to 7.4.0". On the other hand, since this is the first release of 
python3.8, the wheel tag will become pp38-pypy73 which is unambiguous, 
so we could just leave the pypy version 7.3.x until we break ABI 
compatibility with 7.3.6.


Opinions? We should decide soon since packagers will begin releasing 
wheels/distros soon.


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Problem installing python modules on pypy3

[Matti Picus]

On 17/10/21 3:36 pm, Kostas Christofakis wrote:

Hello guys,

I have a problem installing 2 modules on pypy and contacting you to 
find a solution. The issue is this:
I am trying to pip install the python-binance module and discum module 
on pypy3, either with a virtual environment or in pypy folder with 
root. In both cases I get errors and the installation fails. That is 
what happens on the binance case :

...
    python/_brotli.cc:2:10: fatal error: Python.h: No such file or 
directory

        2 | #include 
          |          ^~
    compilation terminated.
    error: command 'gcc' failed with exit status 1
    
ERROR: Command errored out with exit status 1: 
/root/pypy-venv/bin/pypy3 -u -c 'import io, os, sys, setuptools, 
tokenize; sys.argv[0] = 
'"'"'/tmp/pip-install-eqv3vkyi/brotli_4726248b5b3440c6b70b40e51592e1b0/setup.py'"'"'; 
__file__='"'"'/tmp/pip-install-eqv3vkyi/brotli_4726248b5b3440c6b70b40e51592e1b0/setup.py'"'"';f 
= getattr(tokenize, '"'"'open'"'"', open)(__file__) if 
os.path.exists(__file__) else io.StringIO('"'"'from setuptools import 
setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', 
'"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' 
install --record /tmp/pip-record-dk6qnjak/install-record.txt 
--single-version-externally-managed --compile --install-headers 
/root/pypy-venv/include/site/python3.7/brotli Check the logs for full 
command output.


*Any suggestions or ways to make pypy compatible with those 2 modules ?*
*
*
*Btw I have python-dev and everything installed, the modules get 
installed with pip3.9 and python3.9 commands but not on pyp, when i 
try pip3.7 or pip installation i just get the above errors .*




Maybe you got your pypy3 from the one in the debian/ubuntu distro, and 
did not do "sudo apt install pypy3-dev" (not python3-dev)?


Matti


___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy 7.3.6 has been released

[Matti Picus]

https://www.pypy.org/posts/2021/10/pypy-v736-release.html

Thanks to all. Let the bug reports begin.

The docs website release notice with the complete changelog

https://doc.pypy.org/en/latest/release-v7.3.6.html

The uploads are in place

https://downloads.python.org/pypy/

The checksums are at the usual place

https://www.pypy.org/checksums.html

Matti
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy v7.3.6rc3

[Matti Picus]

Hot on the heels of rc2, tarballs for rc3 are up. I messed up the venv 
fixes, so now hopefully it is really fixed.


Additionally, in the two days since rc2 the following were added to 
python3.8:


- A bug when reporting import errors was fixed (issue 3573)

- Performance for some of the quadratic behaviour in enums was improved 
(issue 3571)


- When building the CFFI extension modules, fix the way we add the 
include path for pypy C headers (until setuptools can catch up to the 
new layout)


And for both 3.7 and 3.8

- Added HOST_GNU_TYPE to sysconfigdata (and sysconfig) which is useful 
for cross-compiling


- Add a PYPY_MULTIARCH env variable to allow overriding the value of 
HOST_GNU_TYPE when translating on posix


- Added the beginning of a table of possible environment variables used 
during translation to https://doc.pypy.org/en/latest/build.html (I now 
see it is mis-formatted)


Matti

On 6/10/21 6:29 pm, Matti Picus wrote:
Tarballs for the second release candidate for PyPy 7.3.6, with 
python2.7, python3.7 and python3.8 are available at 
https://downloads.python.org/pypy/. The checksums are available at 
https://www.pypy.org/checksums.html


Thanks to all who tried out rc1 and reported problems, and special 
thanks to those who helped fix them via code, suggestions, or fixes to 
other projects.


The release note is at 
https://doc.pypy.org/en/latest/release-v7.3.6.html. Please take a look 
and suggest improvements.


Changes since rc1:

- Bugs with venv were fixed for windows (now it copies python.exe) and 
python3.8.


- The HPy backend was updated to 0.0.3

- For python3.8, the contents of the lib_pypy directory is now 
packaged into the stdlib directory, there is no more separation. This 
simplifies downstream packaging and brings us more in line with CPython.


- Issue 3555 about string formatting was fixed

- Issue 3463, about a bug in the PPC JIT code was fixed

- The _ssl cffi extension module uses a context manager to manage 
buffer allocations


- Unboxed map dicts now round-trip data correctly

- Suggestions from syntax errors in python3.8 was restored after 
mistakenly being deleted. This feature was backported from Python3.11 
to Python3.7 a while ago.


- Implement os.add_dll_directory for windows on python3.8 and change 
the flags used in LoadLibraryEx accordingly.


- Fix the value of sysconfig.get_config_var('LDLIBRARY') for darwin

Please try it out.

Matti


___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Speed of numpy vs numpypy

[Matti Picus]

On 9/10/21 12:31 am, Werner Heisenberg wrote:

Hi everyone

I read the pypy FAQ and some blog posts, which leaves me puzzled and 
this is why I'm asking here:


I'm doing data science and use numpy as only library in the code 
I'm talking about.
A few years ago I used pypy to great success to speed up an algorithm 
of mine without much development effort.
I had to rewrite the code a bit so numpypy was digesting it but it 
worked great.


Now I am trying to do the same thing and numpypy seems to have been 
dropped in favor of complete support of numpy. In theory this is great 
- however now my algorithm is at least 4x slower with pypy. Is numpypy 
is no longer around or is there a way to install it and try the code 
with it?


Is the numpy support by support of the c extension which the JIT 
cannot optimise? Does this make pypy slower than Cpython? There have 
been some blog posts about progress to make it faster from 2017 but 
then nothing.


So is there anything I can do?

I do not want to write all the numpy stuff I need in plain python as 
this would take a lot of time and then I would be so close to C, that 
I could implement everything in C straight away.


Best regards
Nikolai


___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev



It turns out the approach in numpypy was not sustainable with the 
current level of resources available to PyPy. We have a hard enough time 
trying to catch up with CPython, and are only now releasing a PyPy3.8, 
when CPython has recently released 3.10. Keeping up with NumPy, with 
their twice-early release schedule, in addition to the other maintenance 
tasks for PyPy, was too much, given our very limited budget for ongoing 
maintenance. We ended up disabling the numpypy extension for python3+ 
since the code no longer translated. As Dan Stromberg mentioned, 
currently we view HPy as the way forward: we predict it will allow the 
calls to C code to be closer to the speed of CPython. Note this is a 
more modest goal than numpypy, since numpypy could have provided 
Numba-like performance.


I guess the answer to "what can you do" could be anything from

- help up move HPy forward

- run an effort to revive numpypy (this will have to be a long-term 
commitment)


- get involved in ongoing maintenance of PyPy which would free up other 
resources.


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy v7.3.6rc2

[Matti Picus]

Tarballs for the second release candidate for PyPy 7.3.6, with 
python2.7, python3.7 and python3.8 are available at 
https://downloads.python.org/pypy/. The checksums are available at 
https://www.pypy.org/checksums.html


Thanks to all who tried out rc1 and reported problems, and special 
thanks to those who helped fix them via code, suggestions, or fixes to 
other projects.


The release note is at 
https://doc.pypy.org/en/latest/release-v7.3.6.html. Please take a look 
and suggest improvements.


Changes since rc1:

- Bugs with venv were fixed for windows (now it copies python.exe) and 
python3.8.


- The HPy backend was updated to 0.0.3

- For python3.8, the contents of the lib_pypy directory is now packaged 
into the stdlib directory, there is no more separation. This simplifies 
downstream packaging and brings us more in line with CPython.


- Issue 3555 about string formatting was fixed

- Issue 3463, about a bug in the PPC JIT code was fixed

- The _ssl cffi extension module uses a context manager to manage buffer 
allocations


- Unboxed map dicts now round-trip data correctly

- Suggestions from syntax errors in python3.8 was restored after 
mistakenly being deleted. This feature was backported from Python3.11 to 
Python3.7 a while ago.


- Implement os.add_dll_directory for windows on python3.8 and change the 
flags used in LoadLibraryEx accordingly.


- Fix the value of sysconfig.get_config_var('LDLIBRARY') for darwin

Please try it out.

Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy v7.3.6rc1 is available for testing

[Matti Picus]

The release candidates for pypy v7.3.6rc1 for python2.7, python3.7, and 
python3.8 are up. This is our first release of 3.8. This release also 
includes a backend for HPy0.0.2 (may be upgraded to 0.0.3 by the final 
release). This release has a number of enhancements and speed-ups, as 
well as the usual bug fixes. The win64 rollout on conda is going well, 
over 600 packages are already built. Many of the bug fixes in this 
version were discovered while building those packages.



Thanks to all who contributed code, comments, or otherwise support PyPy.


The downloads are at https://downloads.python.org/pypy/

and the checksums https://www.pypy.org/checksums.html

Please try them out. The directory layout of the 3.8 release has 
changed, so I hope it still builds c-extension modules on your platform.



The release notice https://doc.pypy.org/en/latest/release-v7.3.6.html

As always, edits are welcome.

Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Python 3.7.10[pypy-7.3.5-final] OperationError in multiprocessing

[Matti Picus]

On 18/7/21 11:43 am, Amirouche wrote:

Same behavior with recent 3.7 and 3.8 nightly builds:

- pypy-c-jit-102762-cfc38b565a4e-linux64.tar.bz2 (3.7)

- pypy-c-jit-102728-ace3d170113d-linux64.tar.bz2 (3.8)

By the way the same code runs fine with CPython 3.9.2



Our issue tracker is at https://foss.heptapod.net/pypy/pypy/-/issues.

There is issue https://foss.heptapod.net/pypy/pypy/-/issues/3317, is 
that similar?


The nightly py3.7 branch is probably better to test against than the 
as-yet-unreleased py3.8.


Does the code run without erroring on CPython 3.7.10


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Autotools integration / How to determine .so filename extension

[Matti Picus]

On 10/7/21 4:44 pm, Roland Lutz wrote:
How do I determine the correct filename extension for PyPy extension 
modules?




The canonical way is to use sysconfig


pypy -c "import sysconfig;print(sysconfig.get_config_var('EXT_SUFFIX'))"

.pypy37-pp73-x86_64-linux-gnu.so


On python2.7 use 'SO'

~/oss/pypy2.7-HEAD/bin/pypy -c "import 
sysconfig;print(sysconfig.get_config_var('SO'))"


.pypy-73.so


If you wish to cross-compile or do not want to run python to find the 
values, pypy7.3.5+ on linux now ship a CPython compatible file with all 
the values pre-computed. The file is


/lib_pypy/_sysconfigdata__linux*.py

Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Subscribed to mailing list, but don't see how to submit a question

[Matti Picus]

On 6/7/21 12:01 pm, Armin Rigo wrote:


Hi,

You write to the mailing list by writing to pypy-dev@python.org 
.


Your question: How do you break into running pypy3 code, to stop 
execution?


Like CPython, it's supposed to be ctrl-c or Ctrl-break. Maybe it 
didn't work on some older versions on windows. If you are using 
windows, try upgrading to the latest version.


A bientôt,
Armin




Maybe relevant https://foss.heptapod.net/pypy/pypy/-/issues/3454 where 
ctrl-c does not work sometimes on windows


A small reproducer would be nice - it is difficult to determine what is 
going on in the scenario mentioned in the issue.


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pandas/Numpy in Ubuntu

[Matti Picus]

On 3/7/21 10:04 pm, Александр Рауд wrote:

the python-dev is installed:

~# locate Python.h

/usr/include/python2.7/Python.h

/usr/include/python3.8/Python.h



This is insufficient. You need to follow the instructions and install 
*pypy3-dev*, not python3-dev. This will install header files in 
/usr/lib/pypy3.



Unless you are interested in compiling the entire scientific python 
stack from source I would recommend you use conda. They provide 
pre-compiled binary packages.



conda create -n pypy3.7 -c conda-forge pypy

conda activate pypy3.7

# now your prompt should show (pypy3.7)$

conda install -c conda-forge pandas


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] help needed from tk/tcl experts

[Matti Picus]

Our cffi-backed tk/tcl implementation is failing the test_idel tests [0] 
on python3.7, and the failures persist in the 3.8 branch.


Could someone who understands the way it is supposed to work take a look?

There is something tricky about passing unicode strings and kwargs that 
we don't have right.


This is pure python + cffi + tk/tcl, no RPython needed, no long 
translation times.



If you wish to help but don't know how to start, let us know, anyone is 
welcome to pitch in.



Thanks

Matti


[0] 
http://buildbot.pypy.org/summary/longrepr?testname=unmodified=pypy-c-jit-linux-x86-64=7734=lib-python.3.test.test_idle


___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] How much money do you need to implement the 4+GB RAM (true 64bit) for pypy?

[Matti Picus]

On 2/6/21 11:48 pm, super smart wrote:

My AI program can only use 4GB of RAM in pypy. How much in CAD would 
you need to implement the 64 bit solution?


Pypy has made my code so fast, that C++ is possibly not worth writing 
in ATM.




Please donate at https://opencollective.com/pypy


If you are using windows, we now have a first 64-bit version available. 
We still need to


- complete feature parity with CPython (winconsoleio, audit events, 
faulthandler)


- improve the installer and virtualenv compatibility

- make sure performance is equivalent to linux, especially around IO


If you cannot contribute directly to help us work on these, perhaps you 
could sponsor a directed effort. Sums on the order of $10,000 would go a 
long way toward enabling PyPy developers to work on the project.



One thing that would really help us is to get the word out via user 
stories like yours. Do you have a link to a blog post about your 
experience or would you like to write one for https://pypy.org/blog?



Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] [External] pypy3 Multiprocessing Socket Issue

[Matti Picus]

On 20/5/21 7:45 pm, Moulton, Tom via pypy-dev wrote:

I'm using pypy3.7-v7.3.4-linux64 (Python v3.7.10) but have also 
tried pypy3.7-v7.3.5rc1-linux64 
& pypy-c-jit-102427-77787b8f4c49-linux64.  I'm seeing an issue that 
does NOT occur when using regular Python 3.7 (CPython v3.7.10).

...
OSError: [Errno 24] Too many open files

...
Thoughts on how I might overcome this issue so I can see if using 
pypy3 over CPython 3 garners better performance for my specific 
application?


Regards,

Tom Moulton



Without seeing your code it is difficult to be specific. Typically this 
is caused by not closing allocated resources. CPython uses reference 
counting, and so it implicitly releases resources when they go out of 
scope. PyPy does not. Are you calling socket.close() or fid.close() when 
you are done with them? Also see this section of our documentation 
https://doc.pypy.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies



Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] 7.3.5 rc3 tarballs are up

[Matti Picus]

I have uploaded rc3 candidates to

https://downloads.python.org/pypy/ and the checksums can be found in the 
PR to pypy.org


https://608f07a183d23c00083f6115--keen-mestorf-442210.netlify.app/download_advanced.html#checksums 



The changes are in the release note.

https://doc.pypy.org/en/latest/release-v7.3.5.html:


- The new windows 64-bit builds improperly named c-extension modules 
with the same extension as the 32-bit build (issue 3443)


- Use the windows-specific PC/pyconfig.h rather than the posix one

- A change to the python 3.7 sysconfig.get_config_var('LIBDIR') was 
wrong, leading to problems finding libpypy3-c.so for embedded PyPy 
(issue 3442).


- Instantiate distutils.command.install schema for PyPy-specific 
implementation_lower


- Four upstream (CPython) security patches were applied

- When assigning the full slice of a list, evaluate the rhs before 
clearing the list (issue 3440)


- On Python2, PyUnicode_Contains accepts bytes as well as unicode.

- Update the packaged sqlite3 to 3.35.5 on windows. While not a bugfix, 
this seems like an easy win.



Added in this rc3:

---


- Fix the return type for |_Py_HashDouble| which impacts 64-bit windows


- Delay thread-checking logic in greenlets until the thread is actually 
started (continuation of issue 3441 
)



- Fix for json-specialized dicts (issue 3460 
)



- Specialize |ByteBuffer.setslice| which speeds up binary file reading 
by a factor of 3



- Test and fix a broken code path in _sqlite3 (issue 3432)


Please try them out
Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] 7.3.5 rc2 tarballs are up.

[Matti Picus]

On 14/5/21 11:08 pm, Michał Górny wrote:

I think the fix for:

https://foss.heptapod.net/pypy/pypy/-/issues/3432

is missing too.


Thanks. Done in 77787b8f4c49

Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] 7.3.5 rc2 tarballs are up.

[Matti Picus]

On 7/5/21 11:59 am, Oliver Margetts wrote:
Hi, did this fix https://foss.heptapod.net/pypy/pypy/-/issues/3441 
 make it into the RC?


From my testing, it still seems to be a problem.

Best,
Oliver



Thanks, I missed that but put it in now. It will be part of rc3, which 
will come out once I figure out what is going wrong with cpyext and win64.


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] 7.3.5 rc2 tarballs are up.

[Matti Picus]

Following the rc1 I have uploaded rc2 candidates to

https://downloads.python.org/pypy/ and the checksums can be found in the 
PR to pypy.org


https://608f07a183d23c00083f6115--keen-mestorf-442210.netlify.app/download_advanced.html#checksums

The changes are in the release note.

https://doc.pypy.org/en/latest/release-v7.3.5.html:


- The new windows 64-bit builds improperly named c-extension modules 
with the same extension as the 32-bit build (issue 3443)



- Use the windows-specific PC/pyconfig.h rather than the posix one


- A change to the python 3.7 sysconfig.get_config_var('LIBDIR') was 
wrong, leading to problems finding libpypy3-c.so for embedded PyPy 
(issue 3442).



- Instantiate distutils.command.install schema for PyPy-specific 
implementation_lower


- Four upstream (CPython) security patches were applied


- When assigning the full slice of a list, evaluate the rhs before 
clearing the list (issue 3440)


- On Python2, PyUnicode_Contains accepts bytes as well as unicode.

- Update the packaged sqlite3 to 3.35.5 on windows. While not a bugfix, 
this seems like an easy win.




Please try them out
Matti
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] release candidates for bugfix v7.3.5 are available

[Matti Picus]

In order to fix some problems with the 7.3.4 release, I am releasing a 
7.3.5 bugfix. The rc1 candidates are available at 
https://downloads.python.org/pypy/ and the checksums can be found in the 
PR to pypy.org 
https://608f07a183d23c00083f6115--keen-mestorf-442210.netlify.app/download_advanced.html#checksums



The changes are (as can be seen in the release note 
https://doc.pypy.org/en/latest/release-v7.3.5.html):



- The new windows 64-bit builds improperly named c-extension modules 
with the same extension as the 32-bit build (issue 3443)


- A change to the python 3.7 sysconfig.get_config_var('LIBDIR') was 
wrong, leading to problems finding libpypy3-c.so for embedded PyPy 
(issue 3442).


- Two upstream (CPython) security patches were applied: BPO 42988 to 
remove pydoc.getfile and BPO 43285 to not trust the PASV response in ftplib.
- When assigning the full slice of a list, evaluate the rhs before 
clearing the list (issue 3440)

- On Python2, PyUnicode_Contains accepts bytes as well as unicode.
- Update the packaged sqlite3 to 3.35.5 on windows. While not a bugfix, 
this seems like an easy win.




Please try out the release candidates soon so I can finish the release.


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] Please do not create win64 wheels with the latest 7.3.4 release: bugfix needed.

[Matti Picus]

  
  
It turns out there is a problem with the win64 release: the
  c-extension platform name is win32. It should be win_amd64. 
This was reported by cgohlke in
  https://foss.heptapod.net/pypy/pypy/-/issues/3443.



I will be releasing a 7.3.5 bugfix soon. In the meantime, please
  do not upload win64 wheels to PyPI built with the 7.3.4 release:
  they will be tagged as appropriate for win32 which will cause
  problems for people who have locally built a 32-bit windows
  pypy3.7 and use "pip install". 


Note we did not officially release a 32-bit windows, so the
  damage so far might be minimal.



Matti

  

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] pypy tests on aarch64 musl libc

[Matti Picus]

On 19/4/21 1:12 pm, Thomas Liske wrote:

Hi,

I'm trying to contribute the packaging of pypy to Alpine Linux. I got 
the bootstrapping for x86 x86_64 and s390x working.

On aarch64 two of the pypy tests are failing[1]:

- TestLogParser.test
- TestMicroNumPy.test_reduce_logical_and

[1] https://gitlab.alpinelinux.org/liske/aports/-/jobs/367904#L4261


Are that known issues, is there anything I could do to help to fix 
them? I don't know if my packaging attempt will be accepted for 
Alpine. Running all tests without failures should not make it worse ;-)



Regards,
Thomas



Here are the buildbot results on the Linux aarch64 builds of the default 
(python2.7) branch 
http://buildbot.pypy.org/summary?category=aarch64=default. Those 
failures are known.


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy v7.3.4 is out

[Matti Picus]

PyPy v7.3.4 is out. The release note is here


https://doc.pypy.org/en/latest/release-v7.3.4.html


The blog post should appear soon. The downloads are available on


https://pypy.org/download.html 


or directly from


https://downloads.python.org/pypy/


Now on to the next release.

Here are a few things it would be nice to push forward, there is work 
progressing on some of these and contributions are welcome:


- Python3.8

- Close the gap on known slowdowns:

   - A faster _pickle module implemented in RPython

   - A faster json writer

   - Better handling of too-many-traces in the JIT (for both too large 
loops and too much specialization)


- Making HPy viable for some small c-extensions

- Fix these packages so they work with PyPy: hdf5 [0], grpcio [1]


Matti

[0] https://github.com/h5py/h5py/issues/1591

[1] https://github.com/grpc/grpc/issues/4221

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy7.3.4rc2 tarballs are available

[Matti Picus]

The second release candidate for PyPy7.3.4 is up at
https://downloads.python.org/pypy/.

People using cibuildwheel or github actions to test PyPy should be able to use 
this in their test runs once the 
https://downloads.python.org/pypy/versions.json file updates (it may take a few 
hours to percolate out)


The sha256 checksums can be found in the PR to the pypy.org web site at


https://github.com/pypy/pypy.org/pull/13/files#diff-622ff636d3511de2e408d8b4134b4b825743385cb6c8d9bbadf6dc5b0c887f0eR491


or via the preview at


https://deploy-preview-13--keen-mestorf-442210.netlify.app/download_advanced.html#checksums


The draft release note is up at


https://doc.pypy.org/en/latest/release-v7.3.4.html


Highltights:
- we now have a win64 build (and are dropping the win32 one)
- there are a number of JIT performance enhancements
- overall better compatibility with CPython.
- ARM64 for linux is now built on top of debian9 (manylinux2014) and uses 
"portable" packaging

Please try it out and report issues on IRC or on the issue tracker. Comments to 
improve the release note are welcome too.

Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Is pypy still locked to 4GB RAM ? (64 bit Windows)

[Matti Picus]

There are two versions: a 32-bit one that is limited to 4GB, and a 
64-bit one that is not. We will be releasing the 64-bit one soon, the 
release candidate is already available on 
https://downloads.python.org/pypy/. You want the one that has win64 in 
the name.



Please try it out and tell us how it worked for you. Note if you want to 
use any c-extension packages like NumPy and lxml you will have to 
compile them yourself.



Matti


On 1/4/21 10:32 pm, Steven Mathews wrote:
Is pypy still locked to 4GB RAM? Or fully 64bit Windows now? The docs 
say 'finally done' but doesn't seem to specify if it means 
compatibility-only or can handle over 4GB now.




___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] PyPy 7.3.4rc1 tarballs are up

[Matti Picus]

The first release candidate for PyPy7.3.4 is up at 
https://downloads.python.org/pypy/. People using cibuildwheel or github 
actions to test PyPy should be able to use this in their test runs.



The sha256 checksums can be found in the PR to the pypy.org web site at


https://github.com/pypy/pypy.org/pull/13/files#diff-622ff636d3511de2e408d8b4134b4b825743385cb6c8d9bbadf6dc5b0c887f0eR491


or via the preview at


https://deploy-preview-13--keen-mestorf-442210.netlify.app/download_advanced.html#checksums


The draft release note is up at


https://doc.pypy.org/en/latest/release-v7.3.4.html


This is an incrementally better release: there are no ground-breaking 
changes. However there are a number of JIT preformance enhancements, and 
overall better compatibility with CPython.



Please try it out and report issues on IRC or on the issue tracker.


Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] Release 7.3.4

[Matti Picus]

I would like to release new versions of PyPy for python2.7 and 
python3.7. I think 3.7 has matured to the point that we can drop both 
the "beta" label and the python3.6 release. The python3.8 version is not 
yet ready for an "alpha" label.



There are three issues (one about running "pypy -Wonce ..." and warnings 
inside a decorator, and two having to do with sqlite3) marked as 
blockers [0]. Are there more blockers? Anyone want to take a look at 
fixing them?



Matti


[0] https://foss.heptapod.net/pypy/pypy/-/milestones/3

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] moving blog posts to pypy.org

[Matti Picus]

On 2/28/21 10:51 AM, Matti Picus wrote:


I opened a repo on https://github.com/pypy/pypy.org. It now completes 
the points at the end of my mail: preview for PRs via netlify, uses 
utterances for new blog posts. Please take a look at the deploy 
preview https://keen-mestorf-442210.netlify.app, especially the new 
Blog dropdown and links.



Matti



The move is ready. I have

- made a PR to the python salt recipe to pull the static site from the 
new repo


https://github.com/python/psf-salt/pull/205


- created a blog post for morepypy.blogspot.com, here is the preview

https://www.blogger.com/blog/post/edit/preview/3971202189709462152/2449361080661885989


- made a parallel PR to the new repo with the same content, here is the PR

https://github.com/pypy/pypy.org/pull/7

and the preview (comments should work when it goes live)

https://deploy-preview-7--keen-mestorf-442210.netlify.app/posts/2021/03/pypys-blog-has-moved.html


Last chance to stop the move is now. Comments or fixes/typos to the blog 
post(s) are also welcome.


Matti


___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] moving blog posts to pypy.org

[Matti Picus]

  
  
On 2/28/21 12:06 PM, anatoly techtonik wrote:


  Why not GitLab Pages? Or Heptapod doesn't support it yet?

How to comment on these posts?




Heptapod does not support gitlab pages. 
You cannot comment on the old posts imported from mprepypy, but
  see the first PR for a new blog post[0] where there is a comment
  form at the bottom.


Matti
[0]
https://deploy-preview-3--keen-mestorf-442210.netlify.app/posts/2021/02/28/benchmarking-sphinx.html

  

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] moving blog posts to pypy.org

[Matti Picus]

On 2/3/21 12:54 AM, Matti Picus wrote:


I imported the blog posts and comments to pypy/pypy.org on the 
blogposts branch. If you want a preview, you can update to the branch, 
and run "make build" which should create a nikola virtualenv, install 
the sidebar plugin, build the site, and put a sidebar into the blog 
post pages.



The blogspot site served us well for 13 years, it is time to move on.

The operative change will be that from now on blog posts become a 
merge request to the pypy.org repo, using the nikola workflow:


- nikola new_post 

- edit the post in RST or markdown or jupyter notebook or ...

- add tags and a blurb

- merge it to default


The repo has a CI job to build and push the site, so it is no longer 
necessary to render locally and commit the pages to /public, they will 
be rebuilt on a merge.



We may want to move this repo to github to get a nice preview in 
github pages, and to use the utterances system for comments based on 
github issues.



Any thoughts?

Matti



I opened a repo on https://github.com/pypy/pypy.org. It now completes 
the points at the end of my mail: preview for PRs via netlify, uses 
utterances for new blog posts. Please take a look at the deploy preview 
https://keen-mestorf-442210.netlify.app, especially the new Blog 
dropdown and links.



Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] moving blog posts to pypy.org

[Matti Picus]

  
  
I imported the blog posts and comments to pypy/pypy.org on the
  blogposts branch. If you want a preview, you can update to the
  branch, and run "make build" which should create a nikola
  virtualenv, install the sidebar plugin, build the site, and put a
  sidebar into the blog post pages.


The blogspot site served us well for 13 years, it is time to move
  on. 
The operative change will be that from now on blog posts become a
  merge request to the pypy.org repo, using the nikola workflow:
- nikola new_post 
- edit the post in RST or markdown or jupyter notebook or ...
- add tags and a blurb

- merge it to default


The repo has a CI job to build and push the site, so it is no
  longer necessary to render locally and commit the pages to
  /public, they will be rebuilt on a merge.



We may want to move this repo to github to get a nice preview in
  github pages, and to use the utterances system for comments based
  on github issues.


Any thoughts?

Matti

  

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Some web benchmarks

[Matti Picus]

On 1/26/21 7:07 PM, Oliver Margetts wrote:


Hello all,

not sure where to post this, so I'll put it here. We recently wrote a 
blog post where we benchmarked some python web frameworks: 
https://suade.org/dev/12-requests-per-second-with-python/


One of the big conclusions was: if you want faster code, use pypy! I 
was pleasantly surprised as I was under the impression that web wasn't 
pypy's strongest area (I'm not sure where from). I wondered if you 
would consider adding something as simple as a "Hello, World" 
benchmark to speed.pypy.org ?


Aside from that, thanks for all of your efforts on making pypy fast!



Thanks!. I liked the post. We have a simple django benchmark in the 
suite which shows PyPy to be about 4x faster for templating[0], I don't 
think it actually sets up for answering requests. There is a need for 
more complete site benchmarking like in your post.



You may be interested in a deep dive[1] into profiling one django app 
with PyPy.



Matti


[0] 
https://foss.heptapod.net/pypy/benchmarks/-/blob/branch/default/unladen_swallow/performance/bm_django.py#L34


[1] https://lincolnloop.com/blog/faster-django-sites-pypy/

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] PyPy.org 2021 redesign suggestion

[Matti Picus]

On 1/1/21 11:57 PM, Panos Laganakos wrote:

Hello,

I was going over the PyPy website the other day, and something kept 
bugging me. While the project itself is in great condition really 
amazing at what it does, the website felt that it wasn't giving that 
information to the average visitor.


So, I took a stab at it, with a basic mockup:
https://www.dropbox.com/s/hnabjxy4aybfp1y/pypy.org-0.3.png?dl=0

And here is an annotated version:
https://www.dropbox.com/s/r817xirkhcm9dks/pypy.org-0.3-annotations.png?dl=0
(hover over the image to see the annotations)

And an announcement header one:
https://www.dropbox.com/s/3fnvjcdm4aak7ys/pypy.org-0.3-announcement.png?dl=0

Let me know what you think.


--
Pãnoș
https://panoslaganakos.com



Thanks for the suggestions. That is quite an overhaul! I took the 
liberty of opening an issue on the repo that builds the website 
https://foss.heptapod.net/pypy/pypy.org/-/issues/7. The site is built 
using Nikola and a pypy-specific theme. Would you be up to trying to 
implement parts/all of the design as merge requests to that repo? Then 
we could fine tune the ideas iteratively.


Matti


___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Pickling generators

[Matti Picus]

According to issue 3150 
https://foss.heptapod.net/pypy/pypy/-/issues/3150, this is on purpose, 
and brings us into feature compatibility with CPython:



Python 3.8.6 | packaged by conda-forge | (default, Oct  7 2020, 19:08:05)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> def f():
... yield 10
...
>>> gen = f()
>>> pickle.dumps(gen)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: cannot pickle 'generator' object

I guess we could improve the error message to be more helpful. Do you 
have a concrete use case for this?


Matti.


On 12/11/20 1:55 PM, Ioannis Foufoulas wrote:

Hi,
While in PyPy was possible to pickle a generator function and resume after 
unpicking this does not happen with PyPy3:

File "/snap/pypy3/72/lib-python/3/pickle.py", line 942, in save_global
 (obj, module_name, name))
pickle.PicklingError: Can't pickle : it's not found as 
builtins.generator

Is this a bug or this feature is removed in PyPy3?

Thanks,
Yannis
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Re: [pypy-dev] Switching nightly builds from py3.6 to py3.7

[Matti Picus]

On 11/27/20 7:03 PM, Ronan Lamy wrote:


On 26/11/2020 18:16, Matti Picus wrote:
Now that the re fixes are in for py3.7, I think our primary branch 
for development should be py3.7, with backports to py3.6 if needed. 
PEP 494 for python 3.6 has that version in a security-fixes-only 
stage, so we should adopt a similar strategy (although some bug-fixes 
may be needed, especially around utf8 problems that are still 
cropping up).


I think it's a bad idea to reverse the direction of merges now. We 
still have quite a few fixes coming in for 3.6.


Toward that end, I have updated the buildbot configuration to build 
nightlies off the py3.7 branch instead of the py3.6 branch. So if you 
wish to obtain a newer build of py3.6, you will need to manually 
trigger the buildbots.


+1 though ideally we would just run both as nightlies.



The fixes so far for 3.6 (after the last release) have been

- merge from default to fix default attributes in XML (issue )

- merges from default to support the py3.7 regex changes

- merge from default of rpy-cparser (code rearrangement)

- fixing _crypt import

- fix utf_8_decode(... final=False) (issue 3348)

- merge workaround to detect failed imports intests from default, which 
exposed missing _opcode builtin module



The bug fixes were for XML, utf_8_decode, and _crypt, which could have 
been backported from py3.7 to py3.6 instead of forward merged from py3.6 
to py3.7.



Since we depend on contributed machines for our buildbot machines, I do 
not feel comfortable running 3 nightly builds. To be fair, maybe only 
the macOSx (contributed by someone who also runs CPython builds) and the 
windows (which I run on a VM at home) machines would notice the extra 
load, but it still seems an imposition.



So if you accept the limitation of 2 nightly builds on windows and 
macOSx, which would you choose? Also take into account after we merge 
the win64 branch, windows runs may become unstable for a while.



Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev

[pypy-dev] Switching nightly builds from py3.6 to py3.7

[Matti Picus]

Now that the re fixes are in for py3.7, I think our primary branch for 
development should be py3.7, with backports to py3.6 if needed. PEP 494 
for python 3.6 has that version in a security-fixes-only stage, so we 
should adopt a similar strategy (although some bug-fixes may be needed, 
especially around utf8 problems that are still cropping up).



Toward that end, I have updated the buildbot configuration to build 
nightlies off the py3.7 branch instead of the py3.6 branch. So if you 
wish to obtain a newer build of py3.6, you will need to manually trigger 
the buildbots.



This does not change the status of the default branch.


Please let me know if this was too fast and you feel we should continue 
to use py3.6 as our primary branch for development.



Matti

___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev