Re: Classpath scanning & udfs

2016-01-12 Thread rahul challapalli
Adding the drill-module.conf file in the drill-conf directory with the custom package names worked for me. Thanks for the assistance Jason, Julien and Jacques. - Rahul On Tue, Jan 12, 2016 at 8:09 AM, Jason Altekruse wrote: > Copying info over from Slack. > > For anyone who finds this thread, e

Re: Classpath scanning & udfs

2016-01-12 Thread Jason Altekruse
Copying info over from Slack. For anyone who finds this thread, empty drill-module.conf files do not cause problems. The issue was a misunderstanding about the function of drill-override.conf. Values cannot be added to existing property lists using this file, it is designed for overriding the defa

Re: Classpath scanning & udfs

2016-01-11 Thread rahul challapalli
Sure! On Mon, Jan 11, 2016 at 11:06 AM, Jason Altekruse wrote: > The error you posted originally is from deserializing a storage plugin > config, not the drill-override or drill-module files. That being said, we > find the list of available storage plugins using classpath scanning. > > I don't k

Re: Classpath scanning & udfs

2016-01-11 Thread Jason Altekruse
The error you posted originally is from deserializing a storage plugin config, not the drill-override or drill-module files. That being said, we find the list of available storage plugins using classpath scanning. I don't know why the inclusion or exclusion of contents inside of a module.conf file

Re: Classpath scanning & udfs

2016-01-11 Thread rahul challapalli
Julien, I have an empty drill-module.conf file in the udf jar file. Below are the contents of my global drill-override.conf file drill.classpath.scanning.packages : ${?drill.classpath.scanning.packages} [ org.apache.drill.udfs ] drill.exec: { cluster-id: "rahul_cluster_com-drillbits", zk.conn

Re: Classpath scanning & udfs

2016-01-11 Thread Julien Le Dem
Yes I believe that should work: - add an empty drill-module.conf in the root of the udf jar - add the package to drill.classpath.scanning.packages in the drill conf (possibly using drill-override.conf) However if you are adding the drill-module.conf file to the jar, you might as well add the pac

Re: Classpath scanning & udfs

2016-01-11 Thread rahul challapalli
Just to be sure, If I have an empty drill-module.conf in the root of my udf jar, then there is no way to add the package information to the global drill-override.conf file? On Mon, Jan 11, 2016 at 10:26 AM, Julien Le Dem wrote: > You are correct: > The jar containing the UDFs should have a drill

Re: Classpath scanning & udfs

2016-01-11 Thread Jinfeng Ni
If what Rahul described is true, I think at minimum we should document the backward compatible issue in Drill new release doc. For today, drill doc seems to still use the old way [1] [1] https://drill.apache.org/docs/adding-custom-functions-to-drill/ On Mon, Jan 11, 2016 at 10:17 AM, rahul chall

Re: Classpath scanning & udfs

2016-01-11 Thread Julien Le Dem
You are correct: The jar containing the UDFs should have a drill-module.conf at the root adding your package to the property drill.classpath.scanning.packages for scanning drill.classpath.scanning.packages : ${?drill.classpath.scanning.packages} [ my.package.containing.my.udfs ] Jars th

Re: Classpath scanning & udfs

2016-01-11 Thread rahul challapalli
Thanks for your reply Jason. If we cannot override the global configuration file, then for existing UDF's we have to re-compile them by modifying the drill-module.conf file. If so our UDF's are not backward compatible. Appreciate it if someone can confirm this. - Rahul On Mon, Jan 11, 2016 at 9:

Re: Classpath scanning & udfs

2016-01-11 Thread Jason Altekruse
Rahul, The error message you are seeing is in reading a storage plugin configuration file. I am planning to fix these kinds of messages to actually direct users at the file that is failing parsing. I have seen this in the past when the classpath was incorrect and one of the plugins (like Hbase) wa

Classpath scanning & udfs

2016-01-08 Thread rahul challapalli
Before 1.2, my udfs project contained an empty drill-override.conf file and I used to update the drill-override.conf on all the drillbits to specify the package of my UDF. This is no longer working for me. I tried a few things and below is how my drill-override.conf file looks now drill.classpath.