Glad I could help! The fix is merged now: https://github.com/awleblang/impala-setup/commit/1fcff44887de76fdb5f86948635c073d0e615f1c
On Thu, May 25, 2017 at 10:00 AM, Laszlo Gaal <laszlo.g...@cloudera.com> wrote: > Thanks a lot for fixing this, Sailesh, this has been bugging me for a while > when spinning up cloud-based workstations. > > On Thu, May 25, 2017 at 6:18 PM, Jim Apple <jbap...@cloudera.com> wrote: > > > Nice find, Sailesh! > > > > On Wed, May 24, 2017 at 6:39 PM, Sailesh Mukil <sail...@cloudera.com> > > wrote: > > > The issue was that there was a bug in the install.sh script from the > > > impala-setup/ repo that always assumed that the repo would reside in > > > /home/$user. > > > > > > The Jenkins AMIs seem to have had a very old checkout (8 months old) at > > > /home/ubuntu/ which was being used for the GVOs. The Jenkins job would > > make > > > a fresh checkout in /tmp/ and run install.sh from that checkout. But > due > > to > > > the above bug, the script would change its sources to the old repo in > > > /home/ubuntu. So, the new dependencies added to the project didn't get > > > picked up. I've submitted a fix, and will redo the GVO once it's merged > > > into impala-setup. > > > > > > Thanks for your help Michael and Jim! > > > > > > On Wed, May 24, 2017 at 4:58 PM, Michael Brown <mi...@cloudera.com> > > wrote: > > > > > >> "from-scratch" is a misnomer: only the first build to run on a given > > node > > >> is truly from-scratch, because running impala-setup takes effect > > >> system-wide, and is run once, when the node is created. > > >> > > >> For http://jenkins.impala.io:8080/job/ubuntu-14.04-from-scratch/1371/ > , > > >> the > > >> worker's build history suggests it has been up roughly 17 hours. > > >> http://jenkins.impala.io:8080/computer/ub1404-c4.4xl-gp2%20( > > >> i-0d76efbc8de26926c)/builds > > >> This means your recent change hasn't taken effect on "older" workers. > > >> > > >> Any new workers that get created should get the change. > > >> > > >> > > >> > > >> > > >> > > >> On Wed, May 24, 2017 at 4:34 PM, Sailesh Mukil <sail...@cloudera.com> > > >> wrote: > > >> > > >> > Thanks Michael and Jim, > > >> > > > >> > It looks like adding to bin/bootstrap_build.sh makes the > > ub14-build-only > > >> > job work fine, however, updating the impala-setup still doesn't seem > > to > > >> > work. I submitted a pull request that was merged by Dimitris here: > > >> > https://github.com/awleblang/impala-setup/commits/master > > >> > > > >> > Also, the impala-setup chef logs don't seem to get printed in the > > jenkins > > >> > console output. So I'm not sure if it's actually being run or not. > > >> > > > >> > On Wed, May 24, 2017 at 3:37 PM, Michael Brown <mi...@cloudera.com> > > >> wrote: > > >> > > > >> > > Thanks Jim; I had forgotten bin/bootstrap_build.sh. > > >> > > > > >> > > On Wed, May 24, 2017 at 3:14 PM, Jim Apple <jbap...@cloudera.com> > > >> wrote: > > >> > > > > >> > > > I'd recommend against #2 using the node setup to install Impala > > >> > > > dependencies. The only reason it installs openjdk is that > Jenkins > > >> > > > needs Java to talk to it. All of the other Impala dependencies > are > > >> > > > installed in the jobs themselves. > > >> > > > > > >> > > > On Wed, May 24, 2017 at 3:11 PM, Michael Brown < > > mi...@cloudera.com> > > >> > > wrote: > > >> > > > >> Is there a way to add a new dependency to these machines? > > >> > > > > > > >> > > > > Our builds use two worker labels: > > >> > > > > 1. ubuntu14.04-c4.4xlarge-gp2 > > >> > > > > 2. ub14-build-only > > >> > > > > > > >> > > > > 1 uses https://github.com/awleblang/impala-setup, and I think > > you > > >> > > should > > >> > > > > add libffi to there. It's something you should do anyway so > new > > >> users > > >> > > get > > >> > > > > set up with the right dependencies. > > >> > > > > > > >> > > > > 2 is a different story, but I see that at > > >> > > > > http://jenkins.impala.io:8080/configure that worker is > already > > >> > > > installing a > > >> > > > > JDK via apt-get. You could use a similar pattern to install > > libffi. > > >> > > > > > >> > > > > >> > > > >> > > >
