describe for parquet, parquet-tools or other alternatives

Hi, I'm wondering what tools people are using to get metadata and structure information from Parquet files. Describe can not be used for that and parquet-tools seem to be the only viable option. Are there any other alternatives that I should know about? Regards, -Stefan

Query : Regarding drill jdbc with big csv file.

Hi, I am using "MapR-Sandbox-For-Apache-Drill-1.0.0-4.1.0.ova" and copy the 10GB CSV(220 columns and 5 millions rows) file in /mapr/demo.mapr.com/data. I am running my application on Apache tomcat server and connecting it through JDBC to DrillBit. I am getting error that says OutOfMemory(Stac

Re: describe for parquet, parquet-tools or other alternatives

Hi Stefan, parquet-meta and parquet-schema in parquet-tools has always been enough for me. Is there anything you're looking for that is not available using parquet-tools? Parth On Tue, Jul 21, 2015 at 3:07 AM, Stefán Baxter wrote: > Hi, > > I'm wondering what tools people are using to get met

Re: Query : Regarding drill jdbc with big csv file.

To avoid the error you need to increase the Drill direct memory setting in the drill-env.sh file, in /opt/mapr/drill/drill-1.0.0/conf For the tomcat console/logs you need to configure the java app server settings in tomcat. Don't have it in front of me, but it will be under tomcat-home/conf or

Re: describe for parquet, parquet-tools or other alternatives

thank you parth, No, I just wanted to make sure that I headed down the right path :). Regards, -Stefan On Tue, Jul 21, 2015 at 3:08 PM, Parth Chandra wrote: > Hi Stefan, > > parquet-meta and parquet-schema in parquet-tools has always been enough for > me. Is there anything you're looking for

Hangout happening now

Come join the Drill community as we discuss what has been happening lately and what is in the pipeline. All are welcome, if you know about Drill, want to know more or just want to listen in. Link: https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc Thanks

Re: Drill not picking up a UDF

Hi, I have created this content, hope this will help people to develop new function: http://tgrall.github.io/blog/2015/07/21/apache-drill-how-to-create-a-new-function/ Regards Tug @tgrall On Mon, Jul 20, 2015 at 2:28 PM, Ted Dunning wrote: > No apologies needed. We welcome constructive criti

Re: describe for parquet, parquet-tools or other alternatives

Here step by step: How to build and use parquet-tools to read parquet files Thanks, Hao On Tue, Jul 21, 2015 at 8:12 AM, Stefán Baxter wrote: > thank you parth, > > No, I just wanted to make sure that I headed down the

Re: describe for parquet, parquet-tools or other alternatives

thank you :) On Tue, Jul 21, 2015 at 5:25 PM, Hao Zhu wrote: > Here step by step: > How to build and use parquet-tools to read parquet files > > > > Thanks, > Hao > > On Tue, Jul 21, 2015 at 8:12 AM, Stefán Baxter > wr

The mysterious life of mr. Trim (aka. the value displacement trick)

Hi, Here is small trick I think you will like :). - With this minimal dataset as /tmp/test.json: {"dimensions":{"adults":"A"}} - Running this: select lower(p.dimensions.budgetLevel) as `field1`, lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test.json` as p; - To no sur

Re: The mysterious life of mr. Trim (aka. the value displacement trick)

Well, After some more testing it appears that this has nothing to do with trim. (any non existing nested-value will be pushed aside) select p.dimensions.budgetLevel as `field1`, lower(p.dimensions.adults) as `field2` from dfs.tmp.`/test/0_0_0.parquet` as p; also returns: +-+-+ |

Re: The mysterious life of mr. Trim (aka. the value displacement trick)

Hi, all kidding aside then this renders Drill+Parquet unusable for us at the moment. If there is a quick fix then please let me know. :) Regards, -Stefan On Tue, Jul 21, 2015 at 5:37 PM, Stefán Baxter wrote: > Well, > > After some more testing it appears that this has nothing to do with trim

Re: The mysterious life of mr. Trim (aka. the value displacement trick)

Whoa, something strange is going on. Expected select * from dfs.tmp.test2; +-+ | dimensions| +-+ | {"adults":"A"} | +-+ select p.dimensions as b from dfs.tmp.test2 p; +-+ |b| +-+ | {"adults":"A"

Re: The mysterious life of mr. Trim (aka. the value displacement trick)

done :) On Tue, Jul 21, 2015 at 6:16 PM, Jacques Nadeau wrote: > Whoa, something strange is going on. > > Expected > select * from dfs.tmp.test2; > +-+ > | dimensions| > +-+ > | {"adults":"A"} | > +-+ > > select p.dimensions as b from dfs.tm

Re: The mysterious life of mr. Trim (aka. the value displacement trick)

Hi, I would be super thankful if someone could fix this for us as it's impeding our devaluation and testing (Assuming a fix is fairly doable). I would be more than happy to take a look at it myself but I'm nowhere near having the required understanding of your inner workings. (I have built the pro

Re: The mysterious life of mr. Trim (aka. the value displacement trick)

I'm taking a look at the JIRA you filed :DRILL-3533. If I could not find a fix soon, I'll ask someone in our team to take a further look. On Tue, Jul 21, 2015 at 2:50 PM, Stefán Baxter wrote: > Hi, > > I would be super thankful if someone could fix this for us as it's impeding > our devaluatio

Re: The mysterious life of mr. Trim (aka. the value displacement trick)

Thank you! :) On Tue, Jul 21, 2015 at 10:26 PM, Jinfeng Ni wrote: > I'm taking a look at the JIRA you filed :DRILL-3533. If I could not find a > fix soon, I'll ask someone in our team to take a further look. > > > On Tue, Jul 21, 2015 at 2:50 PM, Stefán Baxter > wrote: > > > Hi, > > > > I woul