I tried to run it from the shell (ssh'd in) on the master node of a newly spun up EMR cluster which used the AWS EMR 4.x.
On Fri, Jul 1, 2016 at 10:33 PM, Paul Mogren <pmog...@commercehub.com> wrote: > Did you by chance attempt to run it outside of EMR? AWS is known to > provide a local install of EMR integration libraries on the nodes, but not > publish them to OSS repositories. > > This was derived from the old > https://github.com/awslabs/emr-bootstrap-actions/tree/master/drill and > worked for us fairly recently. > > > > > On Jul 1, 2016, at 12:54 PM, David Kincaid <kincaid.d...@gmail.com> > wrote: > > > > Thanks, Paul. This does look like a good place to start. Unfortunately, > it > > fails right off the bat due to the emr/common library not being > available. > > Not being a Ruby guy, I'm not sure where to go from here. Is there some > > package that I can easily install to get that library? > > > > - Dave > > > > Here's the error for reference: > > /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in > > `require': cannot load such file -- emr/common (LoadError) > > from > > /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in > > `require' > > from ./setup-drill:39:in `<main>' > > > > > > On Wed, Jun 29, 2016 at 4:04 PM, Paul Mogren <pmog...@commercehub.com> > > wrote: > > > >> Dave, > >> > >> This is a little bit old/outdated but may be better than others you > found: > >> > >> > https://github.com/commercehub-oss/watershed/blob/master/watershed/resources/s3/emr/exec/setup_drill > >> > >> We haven’t gotten around to updating the overall Watershed project for > EMR > >> 4, which is managed differently, nor more recent versions of Drill. > Mostly > >> because it still meets our needs. But that has discouraged us from > >> announcing the project, so I’d really like to get it done one of these > days. > >> > >> Paul Mogren > >> > >> > >> On 6/20/16, 10:29 AM, "David Kincaid" <kincaid.d...@gmail.com> wrote: > >> > >> I was wondering if anyone had an up to date bootstrap script that > installs > >> and configures Drill on an AWS EMR cluster. I'm looking for one that > will > >> work with the stock Amazon EMR version and not a MapR EMR cluster. I've > had > >> too much trouble in the past with the MapR EMR bootstrap scripts > themselves > >> failing. Nowadays I stick to the stock Amazon EMR. I've done some web > >> searching, but all I can seem to find are pretty old, outdated or in > some > >> cases non-existent scripts. So I know people have tried it in the past. > I'm > >> very new to Drill, so having something to start with at least would be a > >> huge help. > >> > >> Thanks, > >> > >> Dave > >> > >> > >> >
