Could be a EMR 3 vs 4 difference.
> On Jul 2, 2016, at 8:51 AM, David Kincaid wrote:
>
> I tried to run it from the shell (ssh'd in) on the master node of a newly
> spun up EMR cluster which used the AWS EMR 4.x.
>
> On Fri, Jul 1, 2016 at 10:33 PM, Paul Mogren
&g
)
> from
> /usr/share/ruby/vendor_ruby/2.0/rubygems/core_ext/kernel_require.rb:55:in
> `require'
> from ./setup-drill:39:in `'
>
>
> On Wed, Jun 29, 2016 at 4:04 PM, Paul Mogren
> wrote:
>
>> Dave,
>>
>> This is a little bit old/outdate
more recent versions of Drill. Mostly because
it still meets our needs. But that has discouraged us from announcing the
project, so I’d really like to get it done one of these days.
Paul Mogren
On 6/20/16, 10:29 AM, "David Kincaid" wrote:
I was wondering if anyone had an up to date
It looks like this bug fix has stalled because of a dependency upgrade that has
stalled. This is preventing me from introducing Drill in a context that I find
interesting. Any idea how/when this can be addressed?
Thanks,
Paul
On 3/1/16, 9:37 AM, "Paul Mogren" wrot
;CTO and Co-Founder, Dremio
>
>On Mon, Feb 29, 2016 at 2:47 PM, Paul Mogren
>wrote:
>
>> Hello,
>>
>> I just tried Drill with MS SQL Server and I found that Drill treats table
>> names case-sensitively, contrary to
>> https://drill.apache.org/docs/lexical-
m for users and existing SQL
scripts that expect table names to be case-insensitive.
This works: select * from mysandbox.dbo.AD_Role
This does not work: select * from mysandbox.dbo.ad_role
I will paste the verbose error log below.
Regards,
Paul Mogren
2016-02-29 17:43:15,754 [292b367c-61e3
Amazon S3 allows object names to contain characters, such as colon(:),
that when used in URI¹s need to be percent-encoded.
When Drill is directed to read such an object either when pointed
directly at an encoded object name, or upon encountering the object in a
folder scan the encoding is disca
ot; wrote:
>When is 1.2 planned for release?
>
>Sent from my iPhone
>
>> On Aug 18, 2015, at 2:53 PM, Paul Mogren
>>wrote:
>>
>> I looked at JIRA and it seems this fix is sitting around waiting for a
>> 1.2.0 release to roll around, which looks like
I looked at JIRA and it seems this fix is sitting around waiting for a
1.2.0 release to roll around, which looks like it is still going to be a
while longer. Could we please have a point release for this?
Thanks,
Paul
On 7/28/15, 2:42 PM, "Daniel Barclay" wrote:
>Parth Chandra wrote:
>> Yes a
:24 PM, "Paul Mogren" wrote:
>In the storage plugin configuration that you have for S3, the connection
>URL can be like “s3n://accessKeyId:secretKeyId@bucket”.
>AFAIK, there is nothing stopping you from creating multiple such
>configurations pointing to the same bucket with
Stefan,
You might be interested in http://tachyon-project.org
On 7/14/15, 1:12 PM, "Stefán Baxter" wrote:
>Hi,
>
>Thank you.
>
>I was not suggesting this to be a part of Drill, only asking if any
>experience exist in this area. :)
>
>I'm trying to evaluate S3-almost-only vs. HDFS so your poi
ay?
>
>
>
>On Mon, Jul 13, 2015 at 11:46 PM, Paul Mogren
>wrote:
>
>> Your question seems to be about clients passing credentials to a Drill
>> query which are then passed through to resources as they are accessed. I
>> don¹t think you are going to find tha
Your question seems to be about clients passing credentials to a Drill
query which are then passed through to resources as they are accessed. I
don¹t think you are going to find that. You can put the credentials in the
URL in storage plugin configuration instead, if that helps. Storage
plugins can
I see that Drill 1.1.0 declares support for Hive 1.0, which is not yet
provided by Amazon EMR. Any chance Hive 0.13 will still work? Can you
characterize when 0.13 would or would not work?
In general I think users will want to upgrade Drill much more frequently
than they are able to upgrade Hive.
your case you provided 1 (as part of hive.metastore.uris). Can you
>double check if that is the correct one.
>
>Also you need provide fs.default.name and other s3 related settings in
>Hive
>storage plugin config.
>
>Thanks
>Venki
>
>On Fri, Jun 26, 2015 at 3:12 PM
I have scoured the Drill website and mailing list, and Google, and have
come up with no advice. Can you help?
I started up an EMR cluster with AWS Hive 0.13.1 installed,
started the metastore service: hive/bin/hive ‹service metastore,
created a table:
CREATE TABLE apachelog (
host STRING,
ID
a that
was written very recently to S3 by another process. The other process also
has to write via EMRFS, not directly to S3, in order to get that benefit.
On 6/18/15, 11:24 AM, "Paul Mogren" wrote:
>Following up. Ted gave sound advice regarding reading S3 vs HDFS, but
>didn¹t
Anybody working on, or interested in, supporting DynamoDB?
/HdfsConfigurationUpdater.java
On 6/18/15, 4:08 PM, "Paul Mogren" wrote:
>Maybe another way to go is to copy Jets3tNativeFileSystemStore and any
>necessary dependent classes to another name, modify, and register it under
>a different URL scheme (not s3n)
>
>
>
>
>On 6/18/15, 3:54 PM
Maybe another way to go is to copy Jets3tNativeFileSystemStore and any
necessary dependent classes to another name, modify, and register it under
a different URL scheme (not s3n)
On 6/18/15, 3:54 PM, "Paul Mogren" wrote:
>Thanks.
>
>
>I tried to follow up on the upcom
(Thread.java:745) ~[na:1.7.0_71]
On 6/18/15, 11:28 AM, "Ted Dunning" wrote:
>On Thu, Jun 18, 2015 at 8:24 AM, Paul Mogren
>wrote:
>
>> Following up. Ted gave sound advice regarding reading S3 vs HDFS, but
>> didn¹t address EMRFS specifically. Here is what I ha
:
https://bitbucket.org/jmurty/jets3t/issue/163/provide-support-for-aws-iam-i
nstance-roles
-Paul
On 5/26/15, 2:15 PM, "Paul Mogren" wrote:
>Thank you. This kind of summary advice is helpful to getting started.
>
>
>
>
>On 5/22/15, 6:37 PM, "Ted Dunning" w
I¹m also interested in having an option to ³continue on error² without
having advance knowledge of where errors lie.
On 5/26/15, 4:26 PM, "Andries Engelbrecht"
wrote:
>The query will typically fail. What source data are you looking at that
>may cause this issue?
>
>One way of working around
On 6/1/15, 12:14 PM, "Matt" wrote:
>Segmenting data into directories in HDFS would require clients to
>structure queries accordingly, but would there be benefit in reduced
>query time by limiting scan ranges?
Yes. I am just a newbie user, but I have already seen that work with
localFS and S3; I
ng
>directly from S3.
>
>Exactly how these trade-offs pan out requires some careful thought and
>considerable knowledge of your workload.
>
>
>
>On Fri, May 22, 2015 at 3:22 PM, Paul Mogren
>wrote:
>
>> > When running Drill in AWS EMR, can anyone advise as to
> When running Drill in AWS EMR, can anyone advise as to the advantages
>and disadvantages of having Drill access S3 via EMRFS vs. directly?
Also, a third option: an actual HDFS not backed by S3
When running Drill in AWS EMR, can anyone advise as to the advantages and
disadvantages of having Drill access S3 via EMRFS vs. directly?
Paul
As support for AWS S3 is advertised and seems to be a common way to try
Drill with existing data, it would be nice to have S3 support fully
built-in. Having to search for and follow multi-step documentation on a
blog post, to enable Jets3t including a protocol scheme error corrected
only by a thi
An imperfect storage plugin configuration - which I strongly suspect
includes even a good configuration pointing at an unavailable resource -
made all Drill queries fail, even against the default cp storage. I
expected that only those queries that reference the faulty storage
configuration would
29 matches
Mail list logo
