> Do we have good pypi caching?

Building Python SDK harness containers takes 2 mins each (times 4, the
number of versions) on my machine, even if nothing has changed. But we're
already paying that cost, so I don't think splitting the jobs should make
it any worse. (https://issues.apache.org/jira/browse/BEAM-8277 if anyone
has any ideas)

Kyle Weaver | Software Engineer | github.com/ibzib | kcwea...@google.com


On Wed, Sep 25, 2019 at 11:21 AM Pablo Estrada <pabl...@google.com> wrote:

> Thanks Chad, and thank you for notifying on the dev list.
>
> On Wed, Sep 25, 2019 at 10:59 AM Kenneth Knowles <k...@apache.org> wrote:
>
>> Nice.
>>
>> Do we have good pypi caching? If not this could add a lot of overhead to
>> our already-backed-up CI queue. (btw I still think your change is good, and
>> just makes proper caching more important)
>>
>> Kenn
>>
>> On Tue, Sep 24, 2019 at 9:55 PM Chad Dombrova <chad...@gmail.com> wrote:
>>
>>> Hi all,
>>> I'm working to make the CI experience with python a bit better, and my
>>> current initiative is splitting up the giant Python PreCommit job into 5
>>> separate jobs into separate jobs for Lint, Py2, Py3.5, Py3.6, and Py3.7.
>>>
>>> Around 11am Pacific time tomorrow I'm going to initiate the seed jobs,
>>> at which point all PRs will start to run the new precommit jobs.  It's a
>>> bit of a chicken-and-egg scenario with testing this, so there could be
>>> issues that pop up after the seed jobs are created, but I'll be working to
>>> resolve those issues as quickly as possible.
>>>
>>> If you run into problems because of this change, please let me know on
>>> the github PR.
>>>
>>> Here's the PR: https://github.com/apache/beam/pull/9642
>>> Here's the Jira: https://issues.apache.org/jira/browse/BEAM-8213#
>>>
>>> The upshot is that after this is done you'll get better feedback on
>>> python test failures!
>>>
>>> Let me know if you have any concerns.
>>>
>>> thanks,
>>> chad
>>>
>>>

Reply via email to