----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26769/#review56783 -----------------------------------------------------------
common/src/main/java/org/apache/drill/common/logical/data/Join.java <https://reviews.apache.org/r/26769/#comment97191> Fix for LogicalPlan De-Serialization bug. DRILL-1535. exec/java-exec/src/test/resources/pigparsing/sample-script-micro.txt <https://reviews.apache.org/r/26769/#comment97195> Not neeeded for Web interface now. Needed for file input though. Yes drill has capability to run PigScript from file directly. Just need an interface to expose this functionality. exec/java-exec/src/test/resources/store/text/data/nations_caratseparated.txt <https://reviews.apache.org/r/26769/#comment97196> For test with custom delimiters provided in PigStorage('^') exec/pom.xml <https://reviews.apache.org/r/26769/#comment97193> Some issue with jdbc-all: Obfuscation error. Not able to break it yet. pom.xml <https://reviews.apache.org/r/26769/#comment97188> Downgraded Sqlline for bringing JLine 1.0 Back. Need to consider Julian's suggestion for having custom Sqlline with both newer and older JLine support. protocol/src/main/java/org/apache/drill/exec/proto/beans/QueryType.java <https://reviews.apache.org/r/26769/#comment97197> Missed few TABS again. Will make sure to take care. - Yash Sharma On Oct. 15, 2014, 7:15 p.m., Yash Sharma wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/26769/ > ----------------------------------------------------------- > > (Updated Oct. 15, 2014, 7:15 p.m.) > > > Review request for drill, Aditya Kishore, Jinfeng Ni, Mehant Baid, and > Timothy Chen. > > > Repository: drill-git > > > Description > ------- > > Dev Patch - Only for review - Not Commit ready - > ------------------------------------------------- > This review comment is only for getting review comments. Current dev is just > a poc without best practices in place. Would be implementing all the comments > as I keep adding any code. > > - Support for Load: Reads text files only. Picks delimeter provided in > PigStorage(). Default \t. > - Support for store: Only dumps on --SCREEN-- for now. > - Support for LIMIT. > - Support for JOIN: supports all joins by Pig apart from FULL OUTER(not > supported by drill currently). Only supports alias based joins not index > based($0 etc). > - Does not support un terminated statements. ie. Operator that does not > output/used in SINK operator. > > Known Issues: > - Sqlline Hack: Need to take a call on this. One solution is suggested by > Julian to have a custom Sqlline with both versions of JLine - so as to have > compatibility with Pig's Sqlline > - Drill's jdbc-all's build failure: Not able to pass through jdbc-all's > Obfuscation failure error. Solution suggested by Aditya did not help me > (removed jetty's jars along with other dependencies). Any tips would be > appreciated here. > > > Diffs > ----- > > > common/src/main/java/org/apache/drill/common/expression/UnquotedFieldReference.java > PRE-CREATION > common/src/main/java/org/apache/drill/common/logical/data/Join.java df7e583 > exec/java-exec/pom.xml d05e4c6 > > exec/java-exec/src/main/java/org/apache/drill/exec/pigparser/parser/ExtendedPigServer.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/pigparser/parser/PigLatinParser.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/pigparser/parser/PigParserUtil.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/pigparser/parser/PigParsingException.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/pigparser/parser/PigPlanTranslator.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/pigparser/planconverter/TestPigParsing.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillImplementor.java > 7fff6e9 > > exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/QueryWrapper.java > 7bf3909 > > exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java > 0a34a22 > exec/java-exec/src/main/resources/rest/query/query.ftl 5033aca > exec/java-exec/src/test/resources/pigparsing/sample-script-micro.txt > PRE-CREATION > exec/java-exec/src/test/resources/pigparsing/sample-script-mini.txt > PRE-CREATION > exec/java-exec/src/test/resources/pigparsing/sample-script.txt PRE-CREATION > exec/java-exec/src/test/resources/pigparsing/test-pig-logical-plan.json > PRE-CREATION > > exec/java-exec/src/test/resources/store/text/data/nations_caratseparated.txt > PRE-CREATION > exec/java-exec/src/test/resources/store/text/data/nations_tsv.txt > PRE-CREATION > exec/jdbc-all/pom.xml 349366b > exec/pom.xml f27320e > pom.xml 70abb75 > protocol/src/main/java/org/apache/drill/exec/proto/UserBitShared.java > 4f01053 > protocol/src/main/java/org/apache/drill/exec/proto/beans/QueryType.java > 6c8f3c5 > protocol/src/main/java/org/apache/drill/exec/proto/beans/RpcType.java > 3f1f9fd > protocol/src/main/protobuf/UserBitShared.proto 3ad9a09 > > Diff: https://reviews.apache.org/r/26769/diff/ > > > Testing > ------- > > Testing done on Drill Rest Interface only via PigLatin script. > Sample Pig Script: > ----------------------------------------------------------------------------------- > regions = LOAD > '/home/yash/work/git/drill-win-forked/incubator-drill/exec/java-exec/src/test/resources/store/text/data/regions.csv' > USING PigStorage(',') AS > ( > zone_id : int, > zone_name : chararray, > comment : chararray > ); > > nations = LOAD > '/home/yash/work/git/drill-win-forked/incubator-drill/exec/java-exec/src/test/resources/store/text/data/nations_caratseparated.txt' > USING PigStorage('^') AS > ( > nation_id : int, > nation_name : chararray, > zone_id : int, > comment : chararray > ); > > r = limit regions 5; > r1 = limit r 5; > r = limit r1 2; > > n = limit nations 3; > n1 = limit n 3; > n = limit n1 2; > -- JOIN Currently only works with named types > e = JOIN r BY zone_id left outer , n BY zone_id; > f = limit e 5; > > store f into 'some_location'; > ----------------------------------------------------------------------------------- > Logical Plan Generated: > ----------------------------------------------------------------------------------- > { > "head" : { > "version" : 1, > "generator" : { > "type" : > "org.apache.drill.exec.pigparser.parser.PigPlanTranslator$PlanTranslator", > "info" : "drill-pig-parser" > }, > "type" : "APACHE_DRILL_LOGICAL", > "options" : null, > "queue" : 0, > "resultMode" : "EXEC" > }, > "storage" : { > "cp" : { > "type" : "file", > "enabled" : true, > "connection" : "file:///", > "workspaces" : null, > "formats" : { > "delim^" : { > "type" : "text", > "extensions" : [ null ], > "delimiter" : "^" > }, > "csv" : { > "type" : "text", > "extensions" : [ "csv" ], > "delimiter" : "," > } > } > } > }, > "query" : [ { > "op" : "scan", > "@id" : 1, > "storageengine" : "cp", > "selection" : { > "format" : { > "type" : "named", > "name" : "csv" > }, > "files" : [ > "file:/home/yash/work/git/drill-win-forked/incubator-drill/exec/java-exec/src/test/resources/store/text/data/regions.csv" > ] > } > }, { > "op" : "project", > "@id" : 2, > "input" : 1, > "projections" : [ { > "ref" : "`zone_id`", > "expr" : "`columns`[0]" > }, { > "ref" : "`zone_name`", > "expr" : "`columns`[1]" > }, { > "ref" : "`comment`", > "expr" : "`columns`[2]" > } ] > }, { > "op" : "limit", > "@id" : 3, > "input" : 2, > "first" : 0, > "last" : 5 > }, { > "op" : "limit", > "@id" : 4, > "input" : 3, > "first" : 0, > "last" : 5 > }, { > "op" : "limit", > "@id" : 5, > "input" : 4, > "first" : 0, > "last" : 2 > }, { > "op" : "project", > "@id" : 6, > "input" : 5, > "projections" : [ { > "ref" : "`r::zone_name`", > "expr" : "`zone_name`" > }, { > "ref" : "`r::zone_id`", > "expr" : "`zone_id`" > }, { > "ref" : "`r::comment`", > "expr" : "`comment`" > } ] > }, { > "op" : "scan", > "@id" : 7, > "storageengine" : "cp", > "selection" : { > "format" : { > "type" : "named", > "name" : "delim^" > }, > "files" : [ > "file:/home/yash/work/git/drill-win-forked/incubator-drill/exec/java-exec/src/test/resources/store/text/data/nations_caratseparated.txt" > ] > } > }, { > "op" : "project", > "@id" : 8, > "input" : 7, > "projections" : [ { > "ref" : "`nation_id`", > "expr" : "`columns`[0]" > }, { > "ref" : "`nation_name`", > "expr" : "`columns`[1]" > }, { > "ref" : "`zone_id`", > "expr" : "`columns`[2]" > }, { > "ref" : "`comment`", > "expr" : "`columns`[3]" > } ] > }, { > "op" : "limit", > "@id" : 9, > "input" : 8, > "first" : 0, > "last" : 3 > }, { > "op" : "limit", > "@id" : 10, > "input" : 9, > "first" : 0, > "last" : 3 > }, { > "op" : "limit", > "@id" : 11, > "input" : 10, > "first" : 0, > "last" : 2 > }, { > "op" : "project", > "@id" : 12, > "input" : 11, > "projections" : [ { > "ref" : "`n::zone_id`", > "expr" : "`zone_id`" > }, { > "ref" : "`n::nation_id`", > "expr" : "`nation_id`" > }, { > "ref" : "`n::comment`", > "expr" : "`comment`" > }, { > "ref" : "`n::nation_name`", > "expr" : "`nation_name`" > } ] > }, { > "op" : "join", > "@id" : 13, > "left" : 6, > "right" : 12, > "conditions" : [ { > "relationship" : "==", > "left" : "`r::zone_id`", > "right" : "`n::zone_id`" > } ], > "type" : "LEFT" > }, { > "op" : "project", > "@id" : 14, > "input" : 13, > "projections" : [ { > "ref" : "`r::zone_name`", > "expr" : "`r::zone_name`" > }, { > "ref" : "`r::zone_id`", > "expr" : "`r::zone_id`" > }, { > "ref" : "`r::comment`", > "expr" : "`r::comment`" > }, { > "ref" : "`n::zone_id`", > "expr" : "`n::zone_id`" > }, { > "ref" : "`n::nation_id`", > "expr" : "`n::nation_id`" > }, { > "ref" : "`n::comment`", > "expr" : "`n::comment`" > }, { > "ref" : "`n::nation_name`", > "expr" : "`n::nation_name`" > } ] > }, { > "op" : "limit", > "@id" : 15, > "input" : 14, > "first" : 0, > "last" : 5 > }, { > "op" : "store", > "@id" : 16, > "input" : 15, > "target" : null, > "storageEngine" : "--SCREEN--" > } ] > } > ----------------------------------------------------------------------------------- > > > Thanks, > > Yash Sharma > >
