Thanks a lot Yifan! This work will likely be a great improvement in Beam developer velocity. INFRA team seems to be overbooked, and things like installing additional dependency took them significant time to be resolved in the past. By having more control and context over what needs to be installed on Jenkins workers, we should be able to move faster.
On Tue, Apr 9, 2019 at 11:44 PM Connell O'Callaghan <[email protected]> wrote: > Yifan thank you for this update and progressing this replacement!!! > > On Tue, Apr 9, 2019, 11:14 PM Yifan Zou <[email protected]> wrote: > >> Thanks, Pablo. The new workers use custom image in the boot disk and it's >> easy to reboot without re-imaging. Means we will no longer need assistance >> from Infra to reconnect an offline node. Dockerizing the environment would >> be helpful to make changes on the environment, such as installing/updating >> a beam-required package. >> >> >> On Tue, Apr 9, 2019 at 2:39 PM Pablo Estrada <[email protected]> wrote: >> >>> Thanks for the updates Yifan. I am sure this process has been difficult, >>> and I appreciate the good communication, and that this didn't really affect >>> the workflow of anyone to validate the new setup for nodes. >>> >>> I imagine that once we move to dockerizing the testing environment, it >>> will be much simpler to restart machines that are having trouble? >>> Thanks again! >>> -P. >>> >>> On Tue, Apr 9, 2019 at 2:23 PM Yifan Zou <[email protected]> wrote: >>> >>>> Hello, >>>> >>>> I have some good news about our Jenkins nodes. We're now having 7 new >>>> nodes <https://builds.apache.org/label/beam/> online named as >>>> "apache-beam-jenkins-[1, 2, 4, 7, 8, 9, 12]", which substitute the old >>>> broken agents "beam[1, 2, 4, 7, 8, 9, 12]". This helps to reduce the job >>>> waiting queue that let your tests getting executed quickly. They're >>>> starting up and accepting jobs. There is no action needed on your end. >>>> >>>> I'll keep the remaining old agents running for one more week just in >>>> case it won't fully block the development works if any unexpected issues on >>>> the new agents. Once the new agents are stable and verified, I'll move >>>> forward to connect the rest agents and remove old set. The playbook is also >>>> on the way. >>>> >>>> For more background and information of the Jenkins updates, please see: >>>> One pager: >>>> https://docs.google.com/document/d/1c38IPrF94PZC-ItGZgmAgAKrgmC1MGA6N6nkK0cL6L4/edit?ts=5ca54b3e# >>>> Environment verification sheet: >>>> https://docs.google.com/spreadsheets/d/1MDL6vy_0iaFSZeWQ-4JWKlRiZ5WFdDVjJh6Xvczgld0/edit#gid=0 >>>> Previous Thread on dev@: >>>> https://lists.apache.org/thread.html/7b9863b241b37484f321d8812e2ad10d8f054ec720aec4b98efe0446@%3Cdev.beam.apache.org%3E >>>> >>>> Thanks. >>>> >>>> Regards. >>>> Yifan Zou >>>> >>>> >>>>
