Could be a EMR 3 vs 4 difference.
> On Jul 2, 2016, at 8:51 AM, David Kincaid <kincaid.d...@gmail.com> wrote: > > I tried to run it from the shell (ssh'd in) on the master node of a newly > spun up EMR cluster which used the AWS EMR 4.x. > > On Fri, Jul 1, 2016 at 10:33 PM, Paul Mogren <pmog...@commercehub.com> > wrote: > >> Did you by chance attempt to run it outside of EMR? AWS is known to >> provide a local install of EMR integration libraries on the nodes, but not >> publish them to OSS repositories. >> >> This was derived from the old >> https://github.com/awslabs/emr-bootstrap-actions/tree/master/drill and >> worked for us fairly recently. >> >> >> >>>> On Jul 1, 2016, at 12:54 PM, David Kincaid <kincaid.d...@gmail.com> >>> wrote: >>> >>> Thanks, Paul. This does look like a good place to start. Unfortunately, >> it >>> fails right off the bat due to the emr/common library not being >> available. >>> Not being a Ruby guy, I'm not sure where to go from here. Is there some >>> package that I can easily install to get that library? >>> >>> - Dave >>> >>> Here's the error for reference: >>> /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in >>> `require': cannot load such file -- emr/common (LoadError) >>> from >>> /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in >>> `require' >>> from ./setup-drill:39:in `<main>' >>> >>> >>> On Wed, Jun 29, 2016 at 4:04 PM, Paul Mogren <pmog...@commercehub.com> >>> wrote: >>> >>>> Dave, >>>> >>>> This is a little bit old/outdated but may be better than others you >> found: >> https://github.com/commercehub-oss/watershed/blob/master/watershed/resources/s3/emr/exec/setup_drill >>>> >>>> We haven’t gotten around to updating the overall Watershed project for >> EMR >>>> 4, which is managed differently, nor more recent versions of Drill. >> Mostly >>>> because it still meets our needs. But that has discouraged us from >>>> announcing the project, so I’d really like to get it done one of these >> days. >>>> >>>> Paul Mogren >>>> >>>> >>>> On 6/20/16, 10:29 AM, "David Kincaid" <kincaid.d...@gmail.com> wrote: >>>> >>>> I was wondering if anyone had an up to date bootstrap script that >> installs >>>> and configures Drill on an AWS EMR cluster. I'm looking for one that >> will >>>> work with the stock Amazon EMR version and not a MapR EMR cluster. I've >> had >>>> too much trouble in the past with the MapR EMR bootstrap scripts >> themselves >>>> failing. Nowadays I stick to the stock Amazon EMR. I've done some web >>>> searching, but all I can seem to find are pretty old, outdated or in >> some >>>> cases non-existent scripts. So I know people have tried it in the past. >> I'm >>>> very new to Drill, so having something to start with at least would be a >>>> huge help. >>>> >>>> Thanks, >>>> >>>> Dave >>
