Re: Dedupping json records based on nested value

2017-08-31 Thread Aman Sinha
Hi Francois, I haven't read the details of your use case but just want to make sure you have looked at the nested data functions and ruled them out for your requirement: https://drill.apache.org/docs/nested-data-functions/ -Aman On Thu, Aug 31, 2017 at 8:23 AM, François Méthot

Re: Dedupping json records based on nested value

2017-08-31 Thread François Méthot
I manage to implement a single UDF that returns a copy of a MapHolder input var, it allowed me to figure how to use SingleMapReaderImpl input and ComplexWriter as out. I tried to move that approach into an aggregation function that looks like the snippet below. I want to return the first

Dedupping json records based on nested value

2017-08-30 Thread François Méthot
Hi, Congrat for the 1.11 release, we are happy to have our suggestion implemented in the new release (automatic HDFS block size for parquet files). It seems like we are pushing the limit of Drill with new type query...(I am learning new SQL trick in the process) We are trying to aggregate a