Hello, I have a python dev team that I support in our local jenkins/artifactory environment. Recently this team switched over from an old local pypi legacy repository to a current pypi remote/virtual of pypi.org. So we could get them updated libraries and such. Anyway, I am banging my head to explain the time taken for reading the simple index. This index is about 11MB, takes 1 second to download itself, but then it takes ~30 seconds for setuptools to scan this and find the package it's looking for. Multiple this by the ~170 libraries they have in requirements, and the build has gone from 30minutes to almost 3 hours. And 90% comes from reading the simple index for each dependency.
Here's a sample output with timestamps 00:57:00.036 Searching for tenacity==4.8.0 00:57:00.037 Reading https://repo1.local/artifactory/api/pypi/py-pypi-virt/simple/ 00:57:27.650 Reading https://repo1.local/artifactory/api/pypi/py-pypi-virt/simple/tenacity/ 00:57:27.881 Downloading https://repo1.local/artifactory/api/pypi/py-pypi-virt/packages/packages/fc/e9/5499018e0d420f8d03a215c310ee7bc6e1a7e84adaa63f6ea208e864bdb6/tenacity-4.8.0-py2.py3-none-any.whl#sha256=efcf0672547f52fd49f96c2c1912e0f0e77d78a6630823aad54f99227a3c332d 00:57:28.037 Best match: tenacity 4.8.0 00:57:28.037 Processing tenacity-4.8.0-py2.py3-none-any.whl 00:57:28.038 Installing tenacity-4.8.0-py2.py3-none-any.whl to /home/build/workspace/workflows-pr/workflows-extension/.eggs 00:57:28.069 writing requirements to /home/build/workspace/workflows-pr/workflows-extension/.eggs/tenacity-4.8.0-py2.7.egg/EGG-INFO/requires.txt 00:57:28.114 Installed /home/build/workspace/workflows-pr/workflows-extension/.eggs/tenacity-4.8.0-py2.7.egg Note the time on the first Reading. I'm hoping there is some obvious simple solution to make this faster. Any ideas or help would be greatly appreciated? Thanks -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-le...@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/O7M274ZYQMMU3WT52PY2YIVJMCNOQJIR/
