Alok, Can you put your UDF in a git repo that I can access?
I am happy to build you a modified version in exchange for you helping to document the somewhat obscure constraints on how a UDF needs to be written. Sound like a good trade? n On Thu, Jun 25, 2015 at 3:21 PM, Alok Tanna <[email protected]> wrote: > I got the full JDK version now I am getting below errors while building the > JAR file > > [root@ip-172-31-34-166 simple-drill-functions]# > /home/ec2-user/apache-maven-3.3. > 3/bin/mvn package > [INFO] Scanning for projects... > [INFO] > [INFO] > ------------------------------------------------------------------------ > [INFO] Building simple-drill-function 1.0-SNAPSHOT > [INFO] > ------------------------------------------------------------------------ > [INFO] > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > simple-dri ll-function --- > [WARNING] Using platform encoding (UTF-8 actually) to copy filtered > resources, i .e. build is platform > dependent! > [INFO] Copying 1 resource > [INFO] > [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ > simple-drill- function --- > [WARNING] File encoding has not been set, using platform encoding UTF-8, > i.e. bu ild is platform dependent! > [INFO] Compiling 4 source files to > /home/ec2-user/simple-drill-functions/target/ > classes > [INFO] ------------------------------------------------------------- > [ERROR] COMPILATION ERROR : > [INFO] ------------------------------------------------------------- > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > eAdd.java:[3,33] error: package > org.apache.drill.exec.expr does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > eAdd.java:[4,45] error: package > org.apache.drill.exec.expr.annotations does not > exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > eAdd.java:[5,62] error: package > org.apache.drill.exec.expr.annotations.FunctionT > emplate does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > eAdd.java:[6,62] error: package > org.apache.drill.exec.expr.annotations.FunctionT > emplate does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > eAdd.java:[7,45] error: package > org.apache.drill.exec.expr.annotations does not > exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > eAdd.java:[8,45] error: package > org.apache.drill.exec.expr.annotations does not > exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > eAdd.java:[9,41] error: package > org.apache.drill.exec.expr.holders does not exis > t > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > eAdd.java:[10,41] error: package > org.apache.drill.exec.expr.holders does not exi > st > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > eAdd.java:[22,45] error: cannot find > symbol > [ERROR] class SimpleAdd > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/SimpleAdd.jav > a:[21,5] error: cannot find symbol > [ERROR] class SimpleAdd > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/SimpleAdd.jav > a:[24,8] error: cannot find symbol > [ERROR] class IntIntAdd > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/SimpleAdd.jav > a:[26,8] error: cannot find symbol > [ERROR] class IntIntAdd > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/SimpleAdd.jav > a:[28,8] error: cannot find symbol > [ERROR] class IntIntAdd > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[3,3 > 3] error: package > org.apache.drill.exec.expr does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > ava:[4,45] error: package > org.apache.drill.exec.expr.annotations does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > ava:[5,45] error: package > org.apache.drill.exec.expr.annotations does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > ava:[6,45] error: package > org.apache.drill.exec.expr.annotations does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > ava:[7,41] error: package > org.apache.drill.exec.expr.holders does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > ava:[8,41] error: package > org.apache.drill.exec.expr.holders does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > ava:[9,50] error: package > org.apache.drill.exec.vector.complex.writer does not e > xist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > ava:[64,43] error: cannot find symbol > [ERROR] class Zip > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[63, > 5] error: cannot find symbol > [ERROR] class Zip > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[66, > 8] error: cannot find symbol > [ERROR] class IntZip2 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[68, > 8] error: cannot find symbol > [ERROR] class IntZip2 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[71, > 18] error: package BaseWriter does not > exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > ava:[82,43] error: cannot find symbol > [ERROR] class Zip > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[81, > 5] error: cannot find symbol > [ERROR] class Zip > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[84, > 8] error: cannot find symbol > [ERROR] class IntZip3 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[86, > 8] error: cannot find symbol > [ERROR] class IntZip3 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[88, > 8] error: cannot find symbol > [ERROR] class IntZip3 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[91, > 18] error: package BaseWriter does not > exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > ava:[102,43] error: cannot find symbol > [ERROR] class Zip > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[101 > ,5] error: cannot find symbol > [ERROR] class Zip > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[104 > ,8] error: cannot find symbol > [ERROR] class IntZip4 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[106 > ,8] error: cannot find symbol > [ERROR] class IntZip4 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[108 > ,8] error: cannot find symbol > [ERROR] class IntZip4 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[110 > ,8] error: cannot find symbol > [ERROR] class IntZip4 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[113 > ,18] error: package BaseWriter does > not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > ava:[124,45] error: cannot find symbol > [ERROR] class Zip > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[123 > ,5] error: cannot find symbol > [ERROR] class Zip > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[126 > ,8] error: cannot find symbol > [ERROR] class FloatZip2 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[128 > ,8] error: cannot find symbol > [ERROR] class FloatZip2 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[131 > ,18] error: package BaseWriter does > not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > ava:[147,45] error: cannot find symbol > [ERROR] class Zip > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[146 > ,5] error: cannot find symbol > [ERROR] class Zip > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[149 > ,8] error: cannot find symbol > [ERROR] class FloatZip3 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[151 > ,8] error: cannot find symbol > [ERROR] class FloatZip3 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[153 > ,8] error: cannot find symbol > [ERROR] class FloatZip3 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[156 > ,18] error: package BaseWriter does > not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > ava:[172,45] error: cannot find symbol > [ERROR] class Zip > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[171 > ,5] error: cannot find symbol > [ERROR] class Zip > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[174 > ,8] error: cannot find symbol > [ERROR] class FloatZip4 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[176 > ,8] error: cannot find symbol > [ERROR] class FloatZip4 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[178 > ,8] error: cannot find symbol > [ERROR] class FloatZip4 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[180 > ,8] error: cannot find symbol > [ERROR] class FloatZip4 > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[183 > ,18] error: package BaseWriter does > not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > um.java:[3,33] error: package > org.apache.drill.exec.expr does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > um.java:[4,45] error: package > org.apache.drill.exec.expr.annotations does not ex > ist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > um.java:[5,45] error: package > org.apache.drill.exec.expr.annotations does not ex > ist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > um.java:[6,45] error: package > org.apache.drill.exec.expr.annotations does not ex > ist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > um.java:[7,41] error: package > org.apache.drill.exec.expr.holders does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > um.java:[8,41] error: package > org.apache.drill.exec.expr.holders does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > um.java:[9,35] error: package > org.apache.drill.exec.vector does not exist > [ERROR] > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > um.java:[27,39] error: cannot find > symbol > [ERROR] class ListSum > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListSum.java: > [26,5] error: cannot find symbol > [ERROR] class ListSum > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListSum.java: > [29,8] error: cannot find symbol > > > ---------------------------------------------------- > *Alok Tanna | **Business Analyst* > > > c 215-410-8236 | e [email protected] | w eXert Digital > <http://www.exertdigital.com/> > https://www.facebook.com/exertdigital > https://www.linkedin.com/company/exert-digital > > On Thu, Jun 25, 2015 at 2:24 AM, Ted Dunning <[email protected]> > wrote: > > > As Vince says, get a full JDK, version 7 or above. > > > > On Wed, Jun 24, 2015 at 2:17 PM, Vince Gonzalez < > [email protected]> > > wrote: > > > > > Try using the JDK. > > > > > > Also, here's a POM: > > > https://github.com/vicenteg/DrillJDBCExample/blob/master/pom.xml > > > > > > On Wed, Jun 24, 2015 at 2:10 PM, Alok Tanna <[email protected]> > > > wrote: > > > > > > > I was able to install the Drill dependence locally but now I am > getting > > > > complication error > > > > > > > > INFO] Changes detected - recompiling the module! > > > > [WARNING] File encoding has not been set, using platform encoding > > UTF-8, > > > > i.e. build is platform dependent! > > > > [INFO] Compiling 4 source files to > > > > /home/ec2-user/simple-drill-functions/target/classes > > > > [INFO] ------------------------------------------------------------- > > > > [ERROR] COMPILATION ERROR : > > > > [INFO] ------------------------------------------------------------- > > > > [ERROR] No compiler is provided in this environment. Perhaps you are > > > > running on a JRE rather than a JDK? > > > > [INFO] 1 error > > > > [INFO] ------------------------------------------------------------- > > > > [INFO] > > > > > > ------------------------------------------------------------------------ > > > > [INFO] BUILD FAILURE > > > > [INFO] > > > > > > ------------------------------------------------------------------------ > > > > [INFO] Total time: 1.484 s > > > > [INFO] Finished at: 2015-06-24T18:04:05+00:00 > > > > [INFO] Final Memory: 7M/17M > > > > [INFO] > > > > > > ------------------------------------------------------------------------ > > > > [ERROR] Failed to execute goal > > > > org.apache.maven.plugins:maven-compiler-plugin:3.0:compile > > > > (default-compile) on project simple-drill-function: Compilation > failure > > > > [ERROR] No compiler is provided in this environment. Perhaps you are > > > > running on a JRE rather than a JDK? > > > > [ERROR] -> [Help 1] > > > > [ERROR] > > > > [ERROR] To see the full stack trace of the errors, re-run Maven with > > the > > > -e > > > > switch. > > > > [ERROR] Re-run Maven using the -X switch to enable full debug > logging. > > > > > > > > > > > > Thanks, > > > > Alok Tanna > > > > > > > > ---------- Forwarded message ---------- > > > > From: Alok Tanna <[email protected]> > > > > Date: Wed, Jun 24, 2015 at 1:16 PM > > > > Subject: Re: Custom Functions > > > > To: [email protected] > > > > > > > > > > > > Vince, If possible can you share your POM.xml file . > > > > > > > > > > > > > > > > > > > > On Wed, Jun 24, 2015 at 1:16 PM, Alok Tanna <[email protected] > > > > > > wrote: > > > > > > > > > Vince, If possible can you share your POM.xml file . > > > > > > > > > > > > > > > On Wed, Jun 24, 2015 at 12:47 PM, Vince Gonzalez < > > > > [email protected] > > > > > > wrote: > > > > > > > > > >> Alok, > > > > >> > > > > >> I had to install the Drill dependencies locally since they're not > in > > > > maven > > > > >> central (DRILL-284 < > https://issues.apache.org/jira/browse/DRILL-284 > > > >). > > > > >> > > > > >> I did the following with drill-1.0.0: > > > > >> > > > > >> mvn install:install-file > > > > >> > > -Dfile=/opt/apache-drill-1.0.0/jars/drill-java-exec-1.0.0-rebuffed.jar > > > > >> -DgroupId=org.apache.drill.exec -DartifactId=drill-java-exec > > > > >> -Dversion=1.0.0 -Dpackaging=jar -DgeneratePom=true > > > > >> mvn install:install-file > > > > >> > -Dfile=/opt/apache-drill-1.0.0/jars/drill-common-1.0.0-rebuffed.jar > > > > >> -DgroupId=org.apache.drill -DartifactId=drill-common > -Dversion=1.0.0 > > > > >> -Dpackaging=jar -DgeneratePom=true > > > > >> > > > > >> > > > > >> Worked for me. > > > > >> > > > > >> > > > > >> On Wed, Jun 24, 2015 at 12:35 PM, Alok Tanna < > > [email protected] > > > > > > > > >> wrote: > > > > >> > > > > >> > Thank you Ted for the below information. I would be perfectly > fine > > > to > > > > >> use > > > > >> > the Java API. > > > > >> > I tried running the Simple Drill function . > > > > >> > https://github.com/mapr-demos/simple-drill-functions > > > > >> > > > > > >> > But then when I am trying to run the package , I am getting > below > > > > error > > > > >> . > > > > >> > I am also attaching the detailed error file . > > > > >> > > > > > >> > [WARNING] The POM for > > > org.apache.drill.exec:drill-java-exec:jar:1.0.0 > > > > is > > > > >> > missing > > > > >> > , no dependency information available > > > > >> > > > > > >> > [ERROR] Failed to execute goal on project simple-drill-function: > > > Could > > > > >> not > > > > >> > resol > > > > >> > ve dependencies for project > > > > >> > com.mapr:simple-drill-function:jar:1.0-SNAPSHOT: Cou > > > > >> > ld not find artifact > > org.apache.drill.exec:drill-java-exec:jar:1.0.0 > > > > in > > > > >> > central > > > > >> > (https://repo.maven.apache.org/maven2) -> [Help 1] > > > > >> > > > > > >> > > > > > >> > Thanks, > > > > >> > Alok Tanna > > > > >> > eXertDigital > > > > >> > > > > > >> > > > > > >> > On Wed, Jun 24, 2015 at 12:42 AM, Ted Dunning < > > > [email protected]> > > > > >> > wrote: > > > > >> > > > > > >> >> Yes and no. > > > > >> >> > > > > >> >> It would be pretty easy to build a Drill function that calls > out > > to > > > > >> Jython > > > > >> >> code. It should be relatively easy to pass a Python function > > name > > > in > > > > >> as > > > > >> >> one of the arguments as well. > > > > >> >> > > > > >> >> The issues with this approach are: > > > > >> >> > > > > >> >> 1) kinda ugly because you aren't calling your code directly > > > > >> >> > > > > >> >> 2) Jython isn't quite Python > > > > >> >> > > > > >> >> I have no idea how to do it specifically, but it is plausible > > that > > > > you > > > > >> >> could build something that automatically generates the > interlude > > > > >> function > > > > >> >> for you in Java that calls your Python code. Python annotations > > > might > > > > >> be > > > > >> >> the cat's pajamas for this. Or not. Hard to say. > > > > >> >> > > > > >> >> The Java API that Drill provides is not a traditional API in > the > > > > normal > > > > >> >> sense of the word because Drill actually snarfs the source code > > and > > > > >> >> inlines > > > > >> >> it into Drill-generated code. It wouldn't understand your > Python > > > > code > > > > >> at > > > > >> >> all. > > > > >> >> > > > > >> >> It might be possible to teach Drill how to understand how to > > > > transform > > > > >> >> Python UDF's directly into something that Drill could use, but > > that > > > > >> would > > > > >> >> definitely be a bit ambitious. > > > > >> >> > > > > >> >> How utterly clean an interface do you need? > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> On Tue, Jun 23, 2015 at 10:10 PM, Alok Tanna < > > > > [email protected]> > > > > >> >> wrote: > > > > >> >> > > > > >> >> > Can we create Custom Functions using Python in Drill ? > > > > >> >> > > > > > >> >> > I do understand Drill provides a high performance Java API > with > > > > >> >> interfaces > > > > >> >> > so that we can implement to develop simple and aggregate > custom > > > > >> >> functions. > > > > >> >> > > > > > >> >> > Thanks, > > > > >> >> > Alok Tanna > > > > >> >> > > > > > >> >> > > > > >> > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > >
