Re: please read: current state and the future of the apache spark build system

2021-07-28 Thread shane knapp ☠
3 months later, i have some updates! TLDR1: we're shutting jenkins down at the end of 2021. > > this is still the goal, exact shutdown date TBD. > long term (until EOY): > * decide what the future of spark builds and releases will look like > - do we need jenkins? > - if we do, who's

Re: please read: current state and the future of the apache spark build system

2021-04-15 Thread Yikun Jiang
Much thanks for your work on infra @Shane. Especially, we (I and @huangtianhua) got really much help from you when make Arm CI work. [1] > prepare jenkins worker ansible configs and stick in the spark repo https://github.com/apache/spark/pull/32178 I take a quick glance on it, it seems it

Re: please read: current state and the future of the apache spark build system

2021-04-14 Thread Holden Karau
Thanks Shane for keeping the build infra structure running for all of these years :) I've got some Kubernetes infra on AS399306 down in HE in Fremont but it's also perhaps not of the newest variety, but so far no disk failures or anything like that (knock on wood of course). The catch is it's on

Re: please read: current state and the future of the apache spark build system

2021-04-14 Thread shane knapp ☠
> > medium term (in 6 months): > * prepare jenkins worker ansible configs and stick in the spark repo > - nothing fancy, but enough to config ubuntu workers > - could be used to create docker containers for testing in > THE CLOUD > > fwiw, i just decided to bang this out today:

please read: current state and the future of the apache spark build system

2021-04-07 Thread shane knapp ☠
this will be a relatively big update, as there are many many moving pieces with short, medium and long term goals. TLDR1: we're shutting jenkins down at the end of 2021. TLDR2: i know we're way behind on pretty much everything. most of the hardware is at or beyond EOL, and random systemic