Re: moving the spark jenkins job builder repo from dbricks --> spark

2019-01-24 Thread shane knapp
looking here: https://dist.apache.org/repos/dist/dev/spark/3.0.0-SNAPSHOT-2019_01_24_10_34-69dab94-docs/ and here: https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-master-docs/5312/console this does confirm that these artifacts are indeed created by the packaging docs

Re: moving the spark jenkins job builder repo from dbricks --> spark

2019-01-24 Thread Sean Owen
Are these docs builds creating the SNAPSHOT docs builds at https://dist.apache.org/repos/dist/dev/spark/ ? I think from a thread last month, these aren't used and should probably just be stopped. On Thu, Jan 24, 2019 at 3:34 PM shane knapp wrote: > > revisiting this thread from october... sorry

Re: moving the spark jenkins job builder repo from dbricks --> spark

2019-01-24 Thread shane knapp
revisiting this thread from october... sorry for the delay in getting around to this until now, but the jenkins job builder configs (and associated apache credentials stored in there) are *directly* related to the work i'm doing here: https://issues.apache.org/jira/browse/SPARK-26565

Re: moving the spark jenkins job builder repo from dbricks --> spark

2018-10-17 Thread shane knapp
On Wed, Oct 17, 2018 at 10:25 AM Yin Huai wrote: > Shane, Thank you for initiating this work! Can we do an audit of jenkins > users and trim down the list? > > re pruning external (spark-specific) users w/shell and jenkins login access: we can absolutely do this. limiting logins for EECS

Re: moving the spark jenkins job builder repo from dbricks --> spark

2018-10-17 Thread Yin Huai
Shane, Thank you for initiating this work! Can we do an audit of jenkins users and trim down the list? Also, for packaging jobs, those branch snapshot jobs are active (for example, https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-master-maven-snapshots/ for publishing

Re: moving the spark jenkins job builder repo from dbricks --> spark

2018-10-10 Thread shane knapp
> > Not sure if that's what you meant; but it should be ok for the jenkins > servers to manually sync with master after you (or someone else) have > verified the changes. That should prevent inadvertent breakages since > I don't expect it to be easy to test those scripts without access to > some

Re: moving the spark jenkins job builder repo from dbricks --> spark

2018-10-10 Thread Marcelo Vanzin
Thanks for doing this. The more things we have accessible to the project members in general the better! (Now there's that hive fork repo somewhere, but let's not talk about that.) On Wed, Oct 10, 2018 at 9:30 AM shane knapp wrote: >> > * the JJB templates are able to be run by anyone w/jenkins

Re: moving the spark jenkins job builder repo from dbricks --> spark

2018-10-10 Thread shane knapp
hey everyone! just for visibility, after some lengthy conversations w/some PMC members (mostly sean and josh) about the location of the jenkins job builder temples being in a private, databricks repo, we've decided to move them in to the main apache spark repo.