Re: Review Request 30313: Fixing Mongo join issue when * is selected
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30313/ --- (Updated March 3, 2015, 3:54 p.m.) Review request for drill, Jacques Nadeau and Jinfeng Ni. Changes --- Fixing review comments Repository: drill-git Description --- Fixing Mongo join issue when * is selected. Diffs (updated) - contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java da9670154e38b6248e826b03dd71ec396ddef5f4 Diff: https://reviews.apache.org/r/30313/diff/ Testing --- Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits. Thanks, Kamesh B
Re: Review Request 30313: Fixing Mongo join issue when * is selected
On Marzo 3, 2015, 3:10 p.m., Mariano Ruiz wrote: This patch solve the issue #DRILL-2352 reported by me. Thanks! Ups, I made some extra tests, and found some errors. I not sure if they are related to this patch. Please check my comments in the bug reported [here](https://issues.apache.org/jira/browse/DRILL-2352?focusedCommentId=14345256page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14345256). - Mariano --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30313/#review74955 --- On Marzo 3, 2015, 3:54 p.m., Kamesh B wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30313/ --- (Updated Marzo 3, 2015, 3:54 p.m.) Review request for drill, Jacques Nadeau and Jinfeng Ni. Repository: drill-git Description --- Fixing Mongo join issue when * is selected. Diffs - contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java da9670154e38b6248e826b03dd71ec396ddef5f4 Diff: https://reviews.apache.org/r/30313/diff/ Testing --- Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits. Thanks, Kamesh B
Would someone be able to lead the hangout today?
I will not have time this morning to lead the hangout, anyone with topics to discuss is welcome to still attend and post minutes to the list. If anyone knows they can make it feel free to raise your hand for leading the meeting. You don't have to do prep, just collect discussion topics from the attendees, go around and introduce any new people and jump right into the discussions. Notes can be high level with a quick description of the issue, features, etc. https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc Thanks, Jason Altekruse
Re: Would someone be able to lead the hangout today?
I did not see Yash's message before I sent this, please see his message for more info. On Tue, Mar 3, 2015 at 8:15 AM, Jason Altekruse altekruseja...@gmail.com wrote: I will not have time this morning to lead the hangout, anyone with topics to discuss is welcome to still attend and post minutes to the list. If anyone knows they can make it feel free to raise your hand for leading the meeting. You don't have to do prep, just collect discussion topics from the attendees, go around and introduce any new people and jump right into the discussions. Notes can be high level with a quick description of the issue, features, etc. https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc Thanks, Jason Altekruse
Re: Would someone be able to lead the hangout today?
Thanks Jason. I will be there to take the notes and share them across. On 3 Mar 2015 9:52 pm, Jason Altekruse altekruseja...@gmail.com wrote: I did not see Yash's message before I sent this, please see his message for more info. On Tue, Mar 3, 2015 at 8:15 AM, Jason Altekruse altekruseja...@gmail.com wrote: I will not have time this morning to lead the hangout, anyone with topics to discuss is welcome to still attend and post minutes to the list. If anyone knows they can make it feel free to raise your hand for leading the meeting. You don't have to do prep, just collect discussion topics from the attendees, go around and introduce any new people and jump right into the discussions. Notes can be high level with a quick description of the issue, features, etc. https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc Thanks, Jason Altekruse
Re: Join us for the Hangout today
Attendees : Anh, Byron, Krystine, Xiao, Shadi, Shudhakar, Norris, Alexander, Yash Meeting minutes - Anh : - Exploring Drill for his use case on SQL native files on shared drive. Anh do share any issues you face with Drill or any part of Drill you would like to have more info on. Krystine : - Looking forward to write documents on Rest interface and Cassandra storage. Shadi : - Looking to enhance the Analytic capabilities of Drill adding new Operators. Shadi please feel free to share your questions and someone would be able to answer them here. Norris Alexander : - Working on the ODBC Driver - Driver for windows/mac is in a completed state, and we can expect a review request soon once its done with the linux env support. Yash : - Working on the suggestions on Cassandra Storage. Regards On Tue, Mar 3, 2015 at 11:31 PM, Yash Sharma yash...@gmail.com wrote: The hangout is on. On Tue, Mar 3, 2015 at 8:17 PM, Yash Sharma yash...@gmail.com wrote: Hi Drillers, Lets meet up for the monthly G+ Hangout usual time (10 AM PDT) - to discuss stuffs we are working on. I would be talking on the suggestions provided on Cassandra Storage and how we can allow Drill-Cassandra interoperability. Would encourage new developers join and share what they are planning to work on. Link : https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc See you all there.
Review Request 31691: DRILL-1385.4.patch.txt: rebase squashed single patch for scalar replacement with option handling cleanup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31691/ --- Review request for drill and Jacques Nadeau. Bugs: DRILL-1385 https://issues.apache.org/jira/browse/DRILL-1385 Repository: drill-git Description --- Cleaned up option handling. This includes using finals, making member variables private whenever possible, and some formatting. - fixed a bug in the string formatting for the double range validator - OptionValidator, OptionValue, and their implementations now conspire not to allow the creation of malformed options because the OptionType has been added to validator calls to handle OptionValues that are created on demand. Started with updated byte code rewrite from Jacques Fixed several problems with scalar value replacement: - use consistent ASM api version throughout - stop using deprecated ASM methods (actually causes bugs) - visitMethodInsn() - added a couple of missing super.visitEnd()s - fixed a couple of minor FindBugs issues - accounted for required stack size increases when replacing holders for longs and doubles - added accounting for frame offsets to cope with long and double local variables and value holder members - fixed a few minor bugs found with FindBugs - stop using carrotlabs' hash map lget() method on shared constant data - fixed an incorrect use of DUP2 on objectrefs when copying long or double holder members into locals - fixed a problem with redundant POP instructions left behind after replacem - fixed a problem with incorrect DUPs in multiple assignment statements - fixed a problem with DUP_X1 replacement when handling constants in multiple assignment statements - fixed a problem with non-replaced holder member post-decrements - don't replace holders passed to static functions as out parameters (common with Accessors on repeated value vectors) - increased the maximum required stack size when transferring holder members locals - changed the code generation block type mappings for constants for external sorts - fixed problems handling constant and non-constant member variables in operator classes - in general, if a holder is assigned to or from an operator member variab it can't be replaced (at least not until we replace those as well) - Use a derived ASM Analyzer (MethodAnalyzer) and Frame (AssignmentTrackingFrame) in order to establish relationships between assignments of holders through chains of local variables. This effective back-propagates non-replaceability attributes so that if a holder variable that can't be replaced is assigned to from another holder variable, that second one cannot be replaced either, and so on through longer chains of assignments. - code for dumping generated source code - MergeAdapter dumps before and after results of scalar replacement (if it's on) - fixed some problems in ReplacingBasicValue by replacing HashSet with IdentityHashMap - made loggers private - added a retry strategy for scalar replacement if a scalar replacement code rewriting fails, then this will try to regenerate the bytecode again without the scalar replacement. - bytecode verification is always on now (required for the retry strategy) - use system option to determine whether scalar replacement should be used - default option: if scalar replacement fails, retry without it - force replacement on or off - unit tests for the retry strategy are based on a single known failure case covered by DRILL-2326. - add tests TestConvertFunctions to test the three scalar replacement options for the failing test case (testVarCharReturnTripConvertLogical) - made it possible to set a SYSTEM option as a java property in Drillbit - added a command line argument to force scalar replacement to be on during testing in the rootmost pom.xml In the course of this, added increased checking of intermediate stages of code rewriting, as well as logging of classes that cause failures. - work around a bug in ASM's CheckClassAdapter that doesn't allow for checking of inner classes Added comments, tidied up formatting, and added final in a number of place Diffs - common/src/main/java/org/apache/drill/common/config/DrillConfig.java dee8a8a common/src/main/java/org/apache/drill/common/util/PathScanner.java 6223777 exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 5efcce8 exec/java-exec/src/main/java/org/apache/drill/exec/compile/AbstractClassCompiler.java f4a3cc9 exec/java-exec/src/main/java/org/apache/drill/exec/compile/AsmUtil.java PRE-CREATION
[jira] [Created] (DRILL-2362) QueryInfo collection grows without bound
Chris Westin created DRILL-2362: --- Summary: QueryInfo collection grows without bound Key: DRILL-2362 URL: https://issues.apache.org/jira/browse/DRILL-2362 Project: Apache Drill Issue Type: Bug Components: Execution - Flow Affects Versions: 0.7.0 Reporter: Chris Westin Assignee: Chris Westin Fix For: Future We collect query profile information for analysis purposes, but we keep it forever. At this time, for a few queries, it isn't a problem. But as users start putting Drill into production, automated use via other applications will make this grow quickly. We need to come up with a retention policy mechanism, with suitable settings administrators can use, and implement it so that this data can be cleaned up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-2316) Docs Enhancement: Data Sources and File Formats, Basics Tutorial
[ https://issues.apache.org/jira/browse/DRILL-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn resolved DRILL-2316. -- Resolution: Fixed Review #31459 completed, patch committed. Docs Enhancement: Data Sources and File Formats, Basics Tutorial Key: DRILL-2316 URL: https://issues.apache.org/jira/browse/DRILL-2316 Project: Apache Drill Issue Type: Task Affects Versions: 0.7.0 Reporter: Kristine Hahn Assignee: Kristine Hahn Fix For: 1.0.0 Hive types, Hive UDF, Parquet, JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Loggers should be private
Seems like a good consensus to go private. Unless any other objections, this seems decided. On Tue, Mar 3, 2015 at 10:50 AM, Chris Westin cwes...@maprtech.com wrote: I've been making them private as I go -- that's how this came up, form a code review. On Mon, Mar 2, 2015 at 8:49 PM, Hanifi Gunes hgu...@maprtech.com wrote: Agreed. - Does anyone object to making all the loggers in the Drill codebase private from now on? Does this include making existing ones private as well? It would be nice if those who are touching a piece of code make non-private loggers private. -Hanifi On Mon, Mar 2, 2015 at 8:03 PM, Steven Phillips sphill...@maprtech.com wrote: I have no objection to making the loggers private, and your reasons make sense to me. But I do wonder if there was a reason for not making them private to begin with. On Mon, Mar 2, 2015 at 12:27 PM, Aditya a...@apache.org wrote: Sounds good to me. On Mon, Mar 2, 2015 at 11:28 AM, Chris Westin chriswesti...@gmail.com wrote: I've noticed in the Drill codebase that loggers are being declared static final, but not private. Based on my past experience, this doesn't match common practice. Loggers should be private: (1) It's common practice. I've never seen otherwise in the past. A quick search turned up these examples in the top search results: - http://www.vogella.com/tutorials/Logging/article.html - http://examples.javacodegeeks.com/core-java/util/logging/java-util-logging-example/ (2) It prevents accidentally using the logger in a derived class. I've been sent off to the wrong place a few times when looking at logs for the Drill code base, because some derived classes have (accidentally?) used their super-classes' loggers. Since the loggers aren't private, this isn't prevented. This gave me the wrong information about where to go looking for messages associated with a problem. (3) Loggers are meant to be associated with exactly one class; that's why they take the class as an argument. Since they're not meant to be used by another class, there's no reason for them to have greater visibility. (There certainly aren't any examples of code that use OtherClass.logger) In order to reduce the fragility of code, we reduce the visible surface of classes (aka encapsulation), and that means giving everything the least visibility that it requires -- in this case, private. (4) Private loggers reveal when they aren't being used, at least in IDEs. When they're private, and they aren't used, we can comment them out // private final static Logger logger = ; and avoid allocating them when we don't need them. If we need them, then we can uncomment them. Does anyone object to making all the loggers in the Drill codebase private from now on? Chris Westin -- Steven Phillips Software Engineer mapr.com
[jira] [Resolved] (DRILL-2336) configuration storage plugin docs update
[ https://issues.apache.org/jira/browse/DRILL-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn resolved DRILL-2336. -- Resolution: Fixed Review #31655 completed, patch committed. configuration storage plugin docs update Key: DRILL-2336 URL: https://issues.apache.org/jira/browse/DRILL-2336 Project: Apache Drill Issue Type: Task Components: Documentation Affects Versions: 0.7.0 Reporter: Kristine Hahn Assignee: Kristine Hahn Fix For: 1.0.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2364) JSON Data Model Reference 2nd draft
Kristine Hahn created DRILL-2364: Summary: JSON Data Model Reference 2nd draft Key: DRILL-2364 URL: https://issues.apache.org/jira/browse/DRILL-2364 Project: Apache Drill Issue Type: Task Components: Documentation Affects Versions: 1.0.0 Reporter: Kristine Hahn Assignee: Kristine Hahn Fix For: 1.0.0 Update MD docs with review feedback from Jason and Andries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 30051: DRILL-1908: new window function implementation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30051/ --- (Updated March 3, 2015, 10 p.m.) Review request for drill and Jacques Nadeau. Changes --- patch rebased to master Bugs: DRILL-1908 https://issues.apache.org/jira/browse/DRILL-1908 Repository: drill-git Description --- In order to fix DRILL-1487 a complete rewrite of the StreamingWindowFrameRecordBatch was needed. This patch adds a new WindowFrameRecordBatch that correctly handles window functions with or without order by clauses. This code still lacks support for frame clauses and may be optimized to reduce unneeded frame computations. Diffs (updated) - common/src/main/java/org/apache/drill/common/logical/data/AbstractBuilder.java 28424a5 common/src/main/java/org/apache/drill/common/logical/data/Window.java 6dba77c contrib/data/pom.xml 86075f2 contrib/data/window-test-data/pom.xml PRE-CREATION exec/java-exec/pom.xml 28eca2b exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 5efcce8 exec/java-exec/src/main/java/org/apache/drill/exec/opt/BasicOptimizer.java 5288f5d exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/WindowPOP.java 17738ee exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/OverFinder.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/StreamingWindowFrameBatchCreator.java 9b8929f exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/StreamingWindowFrameRecordBatch.java 87209eb exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/StreamingWindowFrameTemplate.java e2c7e9e exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/StreamingWindowFramer.java 9588cef exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/WindowFrameBatchCreator.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/WindowFrameRecordBatch.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/WindowFrameTemplate.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/WindowFramer.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java 3b7adca exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/StreamingWindowPrel.java f1a8bc0 exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/StreamingWindowPrule.java 00c20b2 exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/WindowPrel.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/WindowPrule.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DefaultSqlHandler.java 232778a exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java 3d3e96f exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/window/TestWindowFrame.java a9d2ef8 exec/java-exec/src/test/java/org/apache/drill/exec/sql/TestWindowFunctions.java 6eff6db Diff: https://reviews.apache.org/r/30051/diff/ Testing --- Unit tests are available to test window functions in mulitple scenarios: - b1.p1: single batch with a single partition - b1.p2: 2 batches, each containing a different parition - b2.p4: 2 batches and 4 partitions, one partition has rows in both batches - b3.p2: 3 batches and 2 partitions, one partition includes the whole 2nd batch and has rows in 3 batches - b4.p4: 4 batches and 4 partitions, the partitions are arranged to test an edge case: the 2nd time innerNext() is called, WindowFrameRecordBatch has enough saved batches to call it's framer.doWork() without the need to call next(incoming) All tests, except the last one, come in 2 variations: with and without order by clause all unit tests pass. functional, sf100 and customer tests don't add any new failures Thanks, abdelhakim deneche
Re: Review Request 31521: fix broken links in gh-pages-master docs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31521/#review75057 --- Ship it! Looks good! - Bridget Bridget On March 3, 2015, 8:43 p.m., Kristine Hahn wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31521/ --- (Updated March 3, 2015, 8:43 p.m.) Review request for drill and Bridget Bridget. Bugs: DRILL-2331 https://issues.apache.org/jira/browse/DRILL-2331 Repository: drill-git Description --- When we moved files to the Drill site, links that worked on the personal staging site no longer work. Diffs - _docs/001-arch.md 0905ad3 _docs/002-tutorial.md 14cae80 _docs/003-yelp.md b65359e _docs/006-interfaces.md ce068a6 _docs/009-datasources.md 3f3d431 _docs/012-rn.md f369335 _docs/013-contribute.md 33db231 _docs/013-rn.md f369335 _docs/014-contribute.md 33db231 _docs/014-sample-ds.md 7212ea0 _docs/015-design.md 00b17e5 _docs/015-sample-ds.md 7212ea0 _docs/016-design.md 00b17e5 _docs/016-progress.md bf19a29 _docs/018-progress.md bf19a29 _docs/arch/arch-hilite/001-flexibility.md 0b5c5e3 _docs/connect/006-reg-hive.md c3d2b1d _docs/connect/007-default-frmt.md 3ab52db _docs/connect/008-mongo-plugin.md 5c5b33d _docs/contribute/001-guidelines.md 686d972 _docs/data-sources/002-hive-udf.md 7c7a48c _docs/data-sources/003-parquet-ref.md f9b5924 _docs/data-sources/004-json-ref.md db9e671 _docs/dev-custom-fcn/001-dev-simple.md ebf3831 _docs/dev-custom-fcn/002-dev-aggregate.md 4fd14d7 _docs/develop/001-compile.md 2cf6ac9 _docs/develop/003-patch-tool.md 3ef3fe5 _docs/install/001-drill-in-10.md 3f74859 _docs/install/002-deploy.md eecd3bc _docs/install/004-install-distributed.md d0f07aa _docs/install/install-embedded/001-install-linux.md b7a0c85 _docs/install/install-embedded/002-install-mac.md a288b94 _docs/install/install-embedded/003-install-win.md 6680019 _docs/interfaces/001-odbc-win.md 86a4167 _docs/interfaces/003-jdbc-squirrel.md 99eba80 _docs/interfaces/odbc-linux/001-install-odbc-linux.md 3ae1930 _docs/interfaces/odbc-linux/002-install-odbc-mac.md 65a35f3 _docs/interfaces/odbc-linux/003-odbc-connections-linux.md 11b660d _docs/interfaces/odbc-linux/005-odbc-connect-str.md 595432b _docs/interfaces/odbc-win/001-install-odbc-win.md 5bb6c8d _docs/interfaces/odbc-win/002-conf-odbc-win.md 636bd9f _docs/interfaces/odbc-win/003-connect-odbc-win.md d60b294 _docs/interfaces/odbc-win/004-tableau-examples.md f25f50d _docs/manage/conf/001-mem-alloc.md 8f98cfc _docs/manage/conf/002-startup-opt.md 9db8b45 _docs/manage/conf/004-persist-conf.md 12439a5 _docs/query/001-get-started.md 92e924d _docs/query/002-query-fs.md ca488fb _docs/query/003-query-hbase.md d2a33d5 _docs/query/005-query-hive.md 01be576 _docs/query/007-query-sys-tbl.md 9b853ec _docs/query/get-started/001-lesson1-connect.md c4619c3 _docs/query/query-fs/002-query-parquet.md cf19fcf _docs/rn/004-0.6.0-rn.md f121ebe _docs/sql-ref/001-data-types.md e425033 _docs/sql-ref/002-operators.md 79afc7d _docs/sql-ref/003-functions.md bc57a39 _docs/sql-ref/004-nest-functions.md 09fe91e _docs/sql-ref/005-cmd-summary.md 8082f88 _docs/sql-ref/nested/001-flatten.md 2769000 _docs/sql-ref/nested/002-kvgen.md f619864 _docs/sql-ref/nested/003-repeated-cnt.md 2b332b3 _docs/tutorial/002-get2kno-sb.md 3ed1929 _docs/tutorial/003-lesson1.md 577ede3 _docs/tutorial/004-lesson2.md 73c4329 _docs/tutorial/005-lesson3.md b3bb83d _docs/tutorial/006-summary.md 552d72f _docs/tutorial/install-sandbox/001-install-mapr-vm.md da8ddd9 _docs/tutorial/install-sandbox/002-install-mapr-vb.md 9d4fb3b Diff: https://reviews.apache.org/r/31521/diff/ Testing --- Thanks, Kristine Hahn
[jira] [Created] (DRILL-2367) drill.exec.storage.file.partition.column.label appears as an option for both BOOT and SYSTEM
Zhiyong Liu created DRILL-2367: -- Summary: drill.exec.storage.file.partition.column.label appears as an option for both BOOT and SYSTEM Key: DRILL-2367 URL: https://issues.apache.org/jira/browse/DRILL-2367 Project: Apache Drill Issue Type: Bug Components: Metadata Reporter: Zhiyong Liu Assignee: Steven Phillips git.commit.id.abbrev=a84f7b9 drill.exec.storage.file.partition.column.label, comes up as both an option for BOOT and SYSTEM: 0: jdbc:drill: select * from sys.options where name = 'drill.exec.storage.file.partition.column.label'; ++++++++ |name|kind|type| num_val | string_val | bool_val | float_val | ++++++++ | drill.exec.storage.file.partition.column.label | null | BOOT | null | dir | null | null | | drill.exec.storage.file.partition.column.label | STRING | SYSTEM | null | dir| null | null | ++++++++ When trying to set system/session options, the values are set but they never take effect: 0: jdbc:drill: alter session set `drill.exec.storage.file.partition.column.label` = 'sessiondir'; +++ | ok | summary | +++ | true | drill.exec.storage.file.partition.column.label updated. | +++ 0: jdbc:drill: alter system set `drill.exec.storage.file.partition.column.label` = 'systemdir'; +++ | ok | summary | +++ | true | drill.exec.storage.file.partition.column.label updated. | +++ 0: jdbc:drill: select * from sys.options where name = 'drill.exec.storage.file.partition.column.label' order by name; ++++++++ |name|kind|type| num_val | string_val | bool_val | float_val | ++++++++ | drill.exec.storage.file.partition.column.label | null | BOOT | null | dir | null | null | | drill.exec.storage.file.partition.column.label | STRING | SYSTEM | null | systemdir | null | null | | drill.exec.storage.file.partition.column.label | STRING | SESSION| null | sessiondir | null | null | ++++++++ 0: jdbc:drill: select * from dfs.`/drill/testdata/json_storage` limit 1; ++++++++++ |dir0|dir1|dir2| first|last|age | sex | salary | registered | ++++++++++ | testDirEmployee | null | null | Jimmy | James | 29 | M | 6300.1 | false | ++++++++++ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 31707: DRILL-2207: New Union-All Implementation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31707/ --- (Updated March 3, 2015, 10:37 p.m.) Review request for drill, Aman Sinha and Jinfeng Ni. Bugs: DRILL-2207 https://issues.apache.org/jira/browse/DRILL-2207 Repository: drill-git Description --- DRILL-2207: New Union-All Implementation Diffs - exec/java-exec/src/main/java/org/apache/drill/exec/expr/ExpressionTreeMaterializer.java 3565bf4 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAll.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java 99aec92 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllTemplate.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/PreProcessLogicalRel.java 4c9d301 exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/FinalColumnReorderer.java 60a9e4b exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/parser/UnsupportedOperatorsVisitor.java dcd5ebf exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java 7b772cc exec/java-exec/src/main/java/org/apache/drill/exec/resolver/TypeCastRules.java f5b0de4 exec/java-exec/src/test/java/org/apache/drill/TestExampleQueries.java 225b21e exec/java-exec/src/test/java/org/apache/drill/TestUnionAll.java 36b062b exec/java-exec/src/test/resources/store/text/data/t.json PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q1.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q10.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q11.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q12.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q13.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q14.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q15.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q2.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q3.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q4.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q5.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q6.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q6_1.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q7.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q8.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q9.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/testExampleQueries/testAggregationOnUnionAllOperator/q1.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/testExampleQueries/testAggregationOnUnionAllOperator/q2.tsv PRE-CREATION Diff: https://reviews.apache.org/r/31707/diff/ Testing (updated) --- Design Doc can be found from: https://issues.apache.org/jira/browse/DRILL-2207 Unit, Customers, TPCH passed waiting for Functional... Thanks, Sean Hsuan-Yi Chu
Re: Review Request 31707: DRILL-2207: New Union-All Implementation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31707/ --- (Updated March 3, 2015, 10:37 p.m.) Review request for drill, Aman Sinha and Jinfeng Ni. Changes --- DRILL-2207 Bugs: DRILL-2207 https://issues.apache.org/jira/browse/DRILL-2207 Repository: drill-git Description --- DRILL-2207: New Union-All Implementation Diffs - exec/java-exec/src/main/java/org/apache/drill/exec/expr/ExpressionTreeMaterializer.java 3565bf4 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAll.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java 99aec92 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllTemplate.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/PreProcessLogicalRel.java 4c9d301 exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/FinalColumnReorderer.java 60a9e4b exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/parser/UnsupportedOperatorsVisitor.java dcd5ebf exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java 7b772cc exec/java-exec/src/main/java/org/apache/drill/exec/resolver/TypeCastRules.java f5b0de4 exec/java-exec/src/test/java/org/apache/drill/TestExampleQueries.java 225b21e exec/java-exec/src/test/java/org/apache/drill/TestUnionAll.java 36b062b exec/java-exec/src/test/resources/store/text/data/t.json PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q1.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q10.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q11.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q12.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q13.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q14.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q15.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q2.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q3.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q4.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q5.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q6.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q6_1.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q7.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q8.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q9.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/testExampleQueries/testAggregationOnUnionAllOperator/q1.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/testExampleQueries/testAggregationOnUnionAllOperator/q2.tsv PRE-CREATION Diff: https://reviews.apache.org/r/31707/diff/ Testing (updated) --- Design Doc can be found from: https://issues.apache.org/jira/browse/DRILL-2207 Unit, Customers, TPCH, Functional (still waiting ...) Thanks, Sean Hsuan-Yi Chu
[jira] [Created] (DRILL-2365) populate values for 'kind' for sys.options with type == 'BOOT'
Zhiyong Liu created DRILL-2365: -- Summary: populate values for 'kind' for sys.options with type == 'BOOT' Key: DRILL-2365 URL: https://issues.apache.org/jira/browse/DRILL-2365 Project: Apache Drill Issue Type: Bug Components: Metadata Reporter: Zhiyong Liu Assignee: Steven Phillips git.commit.id.abbrev=a84f7b9 When querying for BOOT options, we get null for 'kind' for many of these options: ++++++++ |name|kind|type| num_val | string_val | bool_val | float_val | ++++++++ | drill.exec.buffer.impl | null | BOOT | null | org.apache.drill.exec.work.batch.UnlimitedRawBatchBuffer | null | null | | drill.exec.buffer.size | null | BOOT | null | 6| null | null | | drill.exec.cluster-id | null | BOOT | null | cluster105-drillbits | null | null | In contrast, for other options, we get meaningful values for 'kind': 0: jdbc:drill: select * from sys.options where type = 'BOOT' and kind is not null order by name; ++++++++ |name|kind|type| num_val | string_val | bool_val | float_val | ++++++++ | drill.client.supports-complex-types | BOOLEAN| BOOT | null | null | true | null | It appears to make sense to have the right values for the 'kind' column. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 31435: DRILL-2245-core: core changes for improved drillbit stability
On Feb. 26, 2015, 9:56 p.m., Jacques Nadeau wrote: exec/java-exec/src/test/java/org/apache/drill/exec/TestWithZookeeper.java, line 31 https://reviews.apache.org/r/31435/diff/1/?file=876411#file876411line31 why commented? I see no downside to having an not-yet used logger in a class. It produces a warning. I'm trying to eliminate all warnings. It's easy enough to uncomment it the first time someone wants to use a logger in this file. On Feb. 26, 2015, 9:56 p.m., Jacques Nadeau wrote: exec/java-exec/src/test/java/org/apache/drill/BaseTestQuery.java, line 61 https://reviews.apache.org/r/31435/diff/1/?file=876404#file876404line61 This is a stylistic change. I think you need to propose this on the list and we come to a group consensus. As it is, this is inconsistent with the whole rest of the code base. Please discuss on the dev list before changing this everywhere. Done. It looks like the ayes have it. On Feb. 26, 2015, 9:56 p.m., Jacques Nadeau wrote: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryManager.java, line 313 https://reviews.apache.org/r/31435/diff/1/?file=876399#file876399line313 No, but that is on purpose. We keep them forever for log analysis Once this code is put into production, and a lot of queries are issued constantly, this is going to grow quickly. We need to have a plan for the future to manage this. Submitted https://issues.apache.org/jira/browse/DRILL-2362 . On Feb. 26, 2015, 9:56 p.m., Jacques Nadeau wrote: exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java, line 215 https://reviews.apache.org/r/31435/diff/1/?file=876384#file876384line215 Did you reorder for a sepcific reason? Ideally we should unregister from the coordination service before shutting things down. In fact, we should probably also pause before shutting things down after unregistering I checked master, and the order has not changed. And we do unregister first, at the top of close(). We also pause after unregistering and before shutting things down. I see no change from master, other than the introduction of the new AutoCloseables.close() instead of try-catch blocks. On Feb. 26, 2015, 9:56 p.m., Jacques Nadeau wrote: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java, line 867 https://reviews.apache.org/r/31435/diff/1/?file=876396#file876396line867 Shouldn't this be protected? FragmentSubmitFailures doesn't have any derived classes, so I don't see how that helps. It is used to communicate between the various FragmentSubmitListeners that are created, and setupNonRootFragments(), so they all need to see it. We could make it private, but then we just have to add a getter and an incrementer, so protection doesn't really add anything. On Feb. 26, 2015, 9:56 p.m., Jacques Nadeau wrote: exec/java-exec/src/main/java/org/apache/drill/exec/work/WorkManager.java, line 343 https://reviews.apache.org/r/31435/diff/1/?file=876394#file876394line343 Do you make sure that Foremans no longer leak exceptions? This was here to ensure that we get an error rather than the thread terminating and the error showing up in standard err. Since we catch Exception in Foreman.run(), it looks like you can't get out without reporting the error (as the Foreman moves to the FAILURE state). On Feb. 26, 2015, 9:56 p.m., Jacques Nadeau wrote: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java, line 223 https://reviews.apache.org/r/31435/diff/1/?file=876396#file876396line223 is this guaranteed to be called or can something above prematurely end this method via exception? The listener deregistrations are fine, because they're just list removals. But to be safe, I added a try-finally around the response send, just in case that takes some kind of IO or RPC exception. On Feb. 26, 2015, 9:56 p.m., Jacques Nadeau wrote: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java, line 659 https://reviews.apache.org/r/31435/diff/1/?file=876396#file876396line659 Canceled should be a terminal state from the user's perspective. Not sure that happens here or if you manage this some other way. Any notion of cancelled that the user sees is independent of this. And that doesn't appear to be available at on the client at this time -- this was why I was unable to complete any automated testing. I discovered when I tried to use it that QueryState never gets back to the client, and none of our tests seem to check it. They only look for last chunk, or some expected number of rows. I expect to do some work on this in DRILL-1802, and have added a TODO(DRILL-1802) to the code in the state machine. Meanwhile, the code wasn't consistent wrt the principle you're proposing, namely, that CANCELLED is a terminal state. Some code treated it that way, and some didn't. The
Re: Review Request 31435: DRILL-2245-core: core changes for improved drillbit stability
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31435/ --- (Updated March 3, 2015, 4:21 p.m.) Review request for drill and Jacques Nadeau. Changes --- Some small changes including comments, TODO(DRILL-) entries, and some try-catch and try-finally blocks. Bugs: DRILL-2245 https://issues.apache.org/jira/browse/DRILL-2245 Repository: drill-git Description --- DRILL-2245-core: Clean up query setup and execution kickoff in Foreman/WorkManager in order to ensure consistent handling, and avoid hangs and races, with the goal of improving Drillbit robustness. I did my best to keep these clean when I split them up, but this core commit may depend on some minor changes in the hygiene commit that is also associated with this bug, so either both should be applied, or neither. The core commit should be applied first. AutoCloseables - created org.apache.drill.common.AutoCloseables to handle closing these quietly BaseTestQuery, and derivatives - factored out pieces into QueryTestUtil so they can be reused Drillbit - uses AutoCloseables for the WorkManager and for the storeProvider - allow start() to take a RemoteServiceSet - private, final, formatting Foreman - does not need to implement Comparable - does not need to implement Closeable - thread blocking fixes - add resultSent flag - add code to log plan fragments with endpoint assignments - added finals, cleaned up formatting - do queue management in acquireQuerySemaphore; local tests pass - rename getContext() to getQueryContext() - retain DrillbitContext - a couple of exception injections for testing - minor formatting - TODOs FragmentExecutor - eliminated CancelableQuery - common subexpression elimination - cleaned up QueryContext - removed unnecessary functions (with some outside classes tweaked for this) - finals, formatting QueryManager - merge in QueryStatus - affects Foreman, ../batch/ControlHandlerImpl, and ../../server/rest/ProfileResources - made some methods private - removed unused imports - add finals and formatting - variable renaming to improve readability - formatting - comments - TODOs QueryStatus - getAsInfo() private - member renaming - member access changes - formatting - TODOs QueryTestUtil, BaseTestQuery, TestDrillbitResilience - make maxWidth a parameter to server startup SelfCleaningRunnable - created org.apache.drill.common.SelfCleaningRunnable SingleRowListener - created org.apache.drill.SingleRowListener results listener - use in TestDrillbitResilience TestDrillbitResilience - created org.apache.drill.exec.server.TestDrillbitResilience to test drillb resilience in the face of exceptions and failures during queries TestWithZookeeper - factor out work into ZookeeperHelper so that it can be reused by TestDrillbitResilience UserBitShared - get rid of unused UNKNOWN_QUERY WorkEventBus - rename methods, affects Foreman and ControlHandlerImpl - remove unused WorkerBee reference - most members final - formatting WorkManager - Closeable to AutoCloseable - removed unused incomingFragments Set - eliminated unnecessary eventThread and pendingTasks by posting Runnables directly to executor - use SelfCleaningRunnable for Foreman management - FragmentExecutor management uses SelfCleaningRunnable - runningFragments to be a ConcurrentHashMap; TestTpchDistributed passes - other improvements due to bee no longer needed in various places - most members final - minor formatting - comments - TODOs (*) Created exception injection classes to simulate exceptions for testing - ExceptionInjection - ExceptionInjector - ExceptionInjectionUtil - TestExceptionInjection Diffs (updated) - common/src/main/java/org/apache/drill/common/AutoCloseables.java PRE-CREATION common/src/main/java/org/apache/drill/common/SelfCleaningRunnable.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 5efcce8 exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java 04b955b exec/java-exec/src/main/java/org/apache/drill/exec/coord/local/LocalClusterCoordinator.java 035c1aa exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/WorkEventBus.java d6b8637 exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java 67342c4 exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java 83a89df exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java aa0a5ad
Review Request 31719: fold in tech review changes to json doc on gh-pages-master
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31719/ --- Review request for drill and Bridget Bridget. Bugs: DRILL-2364 https://issues.apache.org/jira/browse/DRILL-2364 Repository: drill-git Description --- Patch includes primarily Jason and Andries changes, plus structural fixes to top level docs, data sources titles fixes, and place holder for .gz json file handling in plugin formats doc. Diffs - _docs/009-datasources.md 77348a7 _docs/009-dev-custom-func.md f8a6445 _docs/010-manage.md ec6663b _docs/011-develop.md 2b9ce67 _docs/012-rn.md 25ec29e _docs/013-contribute.md 42108b9 _docs/014-sample-ds.md c6f51e1 _docs/015-design.md 474052e _docs/016-progress.md 680290e _docs/018-bylaws.md 2c35042 _docs/connect/007-default-frmt.md 31cfe29 _docs/data-sources/001-hive-types.md 34d5bb6 _docs/data-sources/002-hive-udf.md ba82145 _docs/data-sources/003-parquet-ref.md 6fee4a6 _docs/data-sources/004-json-ref.md d60db06 Diff: https://reviews.apache.org/r/31719/diff/ Testing --- Thanks, Kristine Hahn
[jira] [Created] (DRILL-2373) JSON query request returns all types as strings
Adam Gilmore created DRILL-2373:
---
Summary: JSON query request returns all types as strings
Key: DRILL-2373
URL: https://issues.apache.org/jira/browse/DRILL-2373
Project: Apache Drill
Issue Type: Bug
Components: Client - HTTP
Affects Versions: 0.8.0
Reporter: Adam Gilmore
Assignee: Jason Altekruse
Priority: Minor
I'm not sure if the query JSON endpoint is a valid, documented feature of
Drill, but at the moment it returns all results as strings.
So, for example:
{code}select count(*) from cp.`employee.json`{code}
Will return 1155 (as a string) as opposed to 1155 (as a numeric JSON value).
It really should be returning numeric values as proper JSON numeric values.
Same should apply for boolean values.
How we handle other data types (i.e. dates, intervals, timestamps, etc.) needs
discussion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 31707: DRILL-2207: New Union-All Implementation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31707/#review75077 --- exec/java-exec/src/test/java/org/apache/drill/TestUnionAll.java https://reviews.apache.org/r/31707/#comment122013 Change all of these to unordered comparisions, there is no guarentee about the ordering in the case of a union all (also even though reads on a single file will produce an ordered result currently we shouldn't rely on it when writing tests) - Jason Altekruse On March 3, 2015, 10:37 p.m., Sean Hsuan-Yi Chu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31707/ --- (Updated March 3, 2015, 10:37 p.m.) Review request for drill, Aman Sinha and Jinfeng Ni. Bugs: DRILL-2207 https://issues.apache.org/jira/browse/DRILL-2207 Repository: drill-git Description --- DRILL-2207: New Union-All Implementation Diffs - exec/java-exec/src/main/java/org/apache/drill/exec/expr/ExpressionTreeMaterializer.java 3565bf4 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAll.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java 99aec92 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllTemplate.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/PreProcessLogicalRel.java 4c9d301 exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/visitor/FinalColumnReorderer.java 60a9e4b exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/parser/UnsupportedOperatorsVisitor.java dcd5ebf exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorContainer.java 7b772cc exec/java-exec/src/main/java/org/apache/drill/exec/resolver/TypeCastRules.java f5b0de4 exec/java-exec/src/test/java/org/apache/drill/TestExampleQueries.java 225b21e exec/java-exec/src/test/java/org/apache/drill/TestUnionAll.java 36b062b exec/java-exec/src/test/resources/store/text/data/t.json PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q1.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q10.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q11.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q12.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q13.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q14.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q15.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q2.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q3.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q4.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q5.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q6.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q6_1.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q7.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q8.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/TestUnionAll/q9.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/testExampleQueries/testAggregationOnUnionAllOperator/q1.tsv PRE-CREATION exec/java-exec/src/test/resources/testframework/testExampleQueries/testAggregationOnUnionAllOperator/q2.tsv PRE-CREATION Diff: https://reviews.apache.org/r/31707/diff/ Testing --- Design Doc can be found from: https://issues.apache.org/jira/browse/DRILL-2207 Unit, Customers, TPCH passed waiting for Functional... Thanks, Sean Hsuan-Yi Chu
[jira] [Created] (DRILL-2372) current_date() should convert input to UTC before storing in vector
Mehant Baid created DRILL-2372: -- Summary: current_date() should convert input to UTC before storing in vector Key: DRILL-2372 URL: https://issues.apache.org/jira/browse/DRILL-2372 Project: Apache Drill Issue Type: Bug Components: Functions - Drill Reporter: Mehant Baid Assignee: Mehant Baid Fix For: 0.8.0 We expect the value set in the vectors to be in UTC, however since we don't perform the transformation we may end up with incorrect date results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2371) quoted and unquoted boolean values in BOOT options have strange behavior
Zhiyong Liu created DRILL-2371:
--
Summary: quoted and unquoted boolean values in BOOT options have
strange behavior
Key: DRILL-2371
URL: https://issues.apache.org/jira/browse/DRILL-2371
Project: Apache Drill
Issue Type: Bug
Components: Metadata
Reporter: Zhiyong Liu
Assignee: Steven Phillips
git.commit.id.abbrev=a84f7b9
In drill-override.conf, specify
http : {
port : 8047,
enabled : true
}
and don't quote the boolean value.
0: jdbc:drill: select * from sys.options where name =
'drill.exec.http.enabled' order by name;
++++++++
|name|kind|type| num_val | string_val | bool_val |
float_val |
++++++++
| drill.exec.http.enabled | BOOLEAN| BOOT | null | null |
true | null |
++++++++
The query correctly returns the boolean value as 'bool_val' and the type set to
'BOOLEAN'.
Check http://hostname:8047 to make sure that it works.
Now, change the block to the following:
http : {
port : 8047,
enabled : true
}
restart drill and query:
0: jdbc:drill: select * from sys.options where name =
'drill.exec.http.enabled' order by name;
++++++++
|name|kind|type| num_val | string_val | bool_val |
float_val |
++++++++
| drill.exec.http.enabled | null | BOOT | null | true |
null | null |
++++++++
Note that type is now null and the boolean value is now string_val.
Check http://hostname:8047 to observe that it works.
It may be more consistent if we either handle the second case the same way as
the first one in query result set, or fail in the second case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2374) Sqlline prompt history needs improvements
Abhishek Girish created DRILL-2374:
--
Summary: Sqlline prompt history needs improvements
Key: DRILL-2374
URL: https://issues.apache.org/jira/browse/DRILL-2374
Project: Apache Drill
Issue Type: Improvement
Components: Client - CLI
Affects Versions: 0.8.0
Reporter: Abhishek Girish
Assignee: Daniel Barclay (Drill)
*On Sqlline:*
{code:sql}
select id from customer
. . . . . . . . . . . . . . . . . limit 1;
+---+
| id |
+---+
| 1 |
+---+
1 row selected (0.095 seconds)
[Up arrow]
limit 1;
{code}
Using the arrow keys, traversing through previously run queries is currently
not useful, as it displays them line by line and not as a whole.
*Compare it with Postgres:*
{code:sql}
=# select c_customer_sk from customer
-# limit 1;
c_customer_sk
---
1
(1 row)
[Up arrow]
=# select c_customer_sk from customer
limit 1;
{code}
*Or with shell prompt:*
{code}
# for i in `seq 1 5`
do
mkdir $i
done
[Up arrow]
# for i in `seq 1 5` ; do mkdir $i; done
{code}
Sqlline should also support other such enhancements such as printing entire
history.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2375) project more than one column from nested array causes indexoutofbounds exception
Chun Chang created DRILL-2375:
-
Summary: project more than one column from nested array causes
indexoutofbounds exception
Key: DRILL-2375
URL: https://issues.apache.org/jira/browse/DRILL-2375
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Affects Versions: 0.8.0
Reporter: Chun Chang
Assignee: Daniel Barclay (Drill)
#Wed Feb 25 17:07:31 EST 2015
git.commit.id.abbrev=f7ef5ec
I have nested array in a json file looks like this:
{code}
aaa:[[[aa0 1], [ab0 1]], [[ba0 1], [bb0 1]],[[ca0 1, ca1
1],[cb0 1, cb1 1, cb2 1]]]
{code}
Following query causes index out of bound exception:
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select t.id, t.aaa[0], t.aaa[1]
from `complex.json` t limit 5;
Query failed: RemoteRpcException: Failure while running fragment., index: -4,
length: 4 (expected: range(0, 16384)) [ cc383967-6db8-459d-86fe-564d57f7c016 on
qa-node120.qa.lab:31010 ]
[ cc383967-6db8-459d-86fe-564d57f7c016 on qa-node120.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
drillbit.log
{code}
2015-03-03 18:37:17,650 [2b099022-67e8-74b5-f68a-950fe3fe9375:frag:0:0] WARN
o.a.d.e.w.fragment.FragmentExecutor - Error while initializing or executing
fragment
java.lang.IndexOutOfBoundsException: index: -4, length: 4 (expected: range(0,
16384))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at
org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.RepeatedListVector$RepeatedListAccessor.get(RepeatedListVector.java:195)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.impl.RepeatedListReaderImpl.setPosition(RepeatedListReaderImpl.java:79)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.impl.RepeatedListReaderImpl.setPosition(RepeatedListReaderImpl.java:86)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.test.generated.ProjectorGen80550.doEval(ProjectorTemplate.java:106)
~[na:na]
at
org.apache.drill.exec.test.generated.ProjectorGen80550.projectRecords(ProjectorTemplate.java:62)
~[na:na]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:174)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext(LimitRecordBatch.java:113)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
[jira] [Created] (DRILL-2377) UNION ALL on Aggregates with GROUP BY returns null when column aliases don't match
Abhishek Girish created DRILL-2377:
--
Summary: UNION ALL on Aggregates with GROUP BY returns null when
column aliases don't match
Key: DRILL-2377
URL: https://issues.apache.org/jira/browse/DRILL-2377
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.8.0
Reporter: Abhishek Girish
Assignee: Jinfeng Ni
The following query returns null :
{code:sql}
select x
from
(SELECT Sum(ss_ext_sales_price) x
FROM store_sales
UNION ALL
SELECT Sum(cs_ext_sales_price) y
FROM catalog_sales) tmp
GROUP BY x;
Results from Drill:
++
| x |
++
| null |
++
1 row selected (3.458 seconds)
Results from Postgres:
x
---
3658019159.35
5265207074.51
(2 rows)
{code}
Removing the GROUP BY returns correct results:
{code:sql}
select x
from
(SELECT Sum(ss_ext_sales_price) x
FROM store_sales
UNION ALL
SELECT Sum(cs_ext_sales_price) y
FROM catalog_sales) tmp
Results from Drill:
++
| x |
++
| 5265207074.51 |
| 3658019159.35 |
++
2 rows selected (3.46 seconds)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-2211) Define realloc(), allocationMonitor behavior when using fixed size allocation methods in ValueVectors
[ https://issues.apache.org/jira/browse/DRILL-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mehant Baid resolved DRILL-2211. Resolution: Fixed Fixed in 9c0738d94f246443554a1b6d03c7a8d4d21d29a8 Define realloc(), allocationMonitor behavior when using fixed size allocation methods in ValueVectors - Key: DRILL-2211 URL: https://issues.apache.org/jira/browse/DRILL-2211 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Reporter: Mehant Baid Assignee: Mehant Baid Fix For: 0.9.0 In our value vectors implementation we have two major schemes to perform allocation. allocateNew(int) - Allocate for exact number of records allocateNew() - Allocate based on allocation monitors. Currently we don't have a paradigm for when a physical operator wants to exercise allocation using both the above schemes causing a few issues for such operators. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Build failed in Jenkins: drill-scm #246
See https://builds.apache.org/job/drill-scm/246/changes
Changes:
[pchandra] DRILL-1875: getTransferClone should ignore given wrappers
[pchandra] DRILL-2118: inform user with a user friendly error message if kvgen
fails due to heterogenous types
--
[...truncated 1248 lines...]
Warning: org.apache.xerces.parsers.SAXParser: Property
'http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized.
Warning: org.apache.xerces.parsers.SAXParser: Property
'http://www.oracle.com/xml/jaxp/properties/entityExpansionLimit' is not
recognized.
[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0
approved: 1306 licence.
[INFO]
[INFO] --- maven-checkstyle-plugin:2.12.1:check (checkstyle-validation) @
drill-java-exec ---
[INFO] Starting audit...
Audit done.
[INFO]
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (avoid_bad_dependencies) @
drill-java-exec ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:copy-resources (copy-fmpp-resources) @
drill-java-exec ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 103 resources
[INFO]
[INFO] --- git-commit-id-plugin:2.1.9:revision (for-jars) @ drill-java-exec ---
[info] dotGitDirectory https://builds.apache.org/job/drill-scm/ws/.git
[info] git.build.user.name Unknown
[info] git.build.user.email Unknown
[info] Using environment variable based branch name. GIT_BRANCH = origin/master
[info] git.branch origin/master
[info] --always = false
[info] --dirty = -dirty
[info] --abbrev = 7
[info] --long = %s true
[info] --match =
[info] Tag refs [
[Ref[refs/tags/0.6.0-incubating=3daef7f553431c5aa76ff3074a8e00943de6dedd],
Ref[refs/tags/drill-1.0.0-m1=04020a8fca8b287874528d86dc7b8be0269ad788],
Ref[refs/tags/drill-root-1.0.0-m1=ad638d9e41aa9efdb1e877cfe7e0a4b910f539fc],
Ref[refs/tags/oscon_workshop=eaf95ed3c30d7bb147afe337e0e0477be6518d90],
Ref[refs/tags/pre_exec_merge=a97a22b0a9547f8639e92258c0a3475b01742f15]] ]
[info] Resolved tag [ 0.6.0-incubating ] [ PersonIdent[Steven Phillips,
sphill...@maprtech.com, Thu Oct 2 02:27:36 2014 -0700] ], points at [ commit
5fa257b38df77dedf5609952e364ce0cfc7383d2 0 -- ]
[info] Resolved tag [ drill-1.0.0-m1 ] [ PersonIdent[Jacques Nadeau,
jacq...@apache.org, Fri Sep 6 13:05:42 2013 -0700] ], points at [ commit
a0d3c6977820516983142c96d7f9374681529968 0 -- ]
[info] Resolved tag [ drill-root-1.0.0-m1 ] [ PersonIdent[Jacques Nadeau,
jacq...@apache.org, Wed Sep 4 04:23:47 2013 -0700] ], points at [ commit
41c18197e3b8ae3c42d55089d641e9a0b68c6f29 0 -- ]
[info] Resolved tag [ pre_exec_merge ] [ PersonIdent[Jacques Nadeau,
jacq...@apache.org, Fri Jul 19 18:33:56 2013 -0700] ], points at [ commit
5052b64d9953857575f8f40995b8da05160e5457 0 -- ]
[info] key [ commit 41c18197e3b8ae3c42d55089d641e9a0b68c6f29 0 -- ], tags
= [ [DatedRevTag{id=ad638d9e41aa9efdb1e877cfe7e0a4b910f539fc,
tagName='drill-root-1.0.0-m1', date=September 4, 2013 11:23:47 AM UTC}] ]
[info] key [ commit 5fa257b38df77dedf5609952e364ce0cfc7383d2 0 -- ], tags
= [ [DatedRevTag{id=3daef7f553431c5aa76ff3074a8e00943de6dedd,
tagName='0.6.0-incubating', date=October 2, 2014 9:27:36 AM UTC}] ]
[info] key [ commit 5052b64d9953857575f8f40995b8da05160e5457 0 -- ], tags
= [ [DatedRevTag{id=a97a22b0a9547f8639e92258c0a3475b01742f15,
tagName='pre_exec_merge', date=July 20, 2013 1:33:56 AM UTC}] ]
[info] key [ commit a0d3c6977820516983142c96d7f9374681529968 0 -- ], tags
= [ [DatedRevTag{id=04020a8fca8b287874528d86dc7b8be0269ad788,
tagName='drill-1.0.0-m1', date=September 6, 2013 8:05:42 PM UTC}] ]
[info] Created map: [ {commit 41c18197e3b8ae3c42d55089d641e9a0b68c6f29 0
--=[drill-root-1.0.0-m1], commit 5fa257b38df77dedf5609952e364ce0cfc7383d2 0
--=[0.6.0-incubating], commit 5052b64d9953857575f8f40995b8da05160e5457 0
--=[pre_exec_merge], commit a0d3c6977820516983142c96d7f9374681529968 0
--=[drill-1.0.0-m1]} ]
[info] HEAD is [ a66b6cc0d7d3de0fbe8aa0715233235aee18f71b ]
[info] Repo is in dirty state [ false ]
[info] git.commit.id.describe drill-1.0.0-m1-1162-ga66b6cc
[info] git.commit.id a66b6cc0d7d3de0fbe8aa0715233235aee18f71b
[info] git.commit.id.abbrev a66b6cc
[info] git.commit.user.name Hanifi Gunes
[info] git.commit.user.email hgu...@maprtech.com
[info] git.commit.message.full DRILL-2118: inform user with a user friendly
error message if kvgen fails due to heterogenous types
[info] git.commit.message.short DRILL-2118: inform user with a user friendly
error message if kvgen fails due to heterogenous types
[info] git.commit.time 04.03.2015 @ 01:43:01 UTC
[info] git.remote.origin.url https://git-wip-us.apache.org/repos/asf/drill.git
[info] git.build.time 04.03.2015 @ 02:01:21 UTC
[info] found property git.commit.id.abbrev
[info] found property git.commit.user.email
[info] found property git.commit.message.full
[info] found property git.commit.id
[info] found property git.commit.message.short
[info] found property
[jira] [Resolved] (DRILL-2112) Throw helpful error message with unsupported KVGen functionality
[
https://issues.apache.org/jira/browse/DRILL-2112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mehant Baid resolved DRILL-2112.
Resolution: Duplicate
Throw helpful error message with unsupported KVGen functionality
-
Key: DRILL-2112
URL: https://issues.apache.org/jira/browse/DRILL-2112
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Reporter: Neeraja
Assignee: Mehant Baid
Fix For: 0.9.0
KVGen function currently doesn't support maps with changing data types on
values. We should throw helpful error message indicating that this is not an
unsupported functionality rather than execution exception.
0: jdbc:drill:zk=local select attributes from
dfs.`/users/nrentachintala/Downloads/yelp/yelp_academic_dataset_business.json`
limit 2;
++
| attributes |
++
| {By Appointment Only:true,Good
For:{},Ambience:{},Parking:{},Music:{},Hair Types Specialized
In:{},Payment Types:{},Dietary Restrictions:{}} |
| {Take-out:true,Good
For:{dessert:false,latenight:false,lunch:true,dinner:false,breakfast:false,brunch:false},Caters:false,Noise
Level:average,Takes
Reservations:false,Delivery:false,Ambience:{romantic:false,intimate:false,touristy:false,hipster:false,divey:false,classy:false,trendy:false,upscale:false,casual:false},Parking:{garage:false,street:false,validated:false,lot:true,valet:false},Has
TV:true,Outdoor Seating:false,Attire:casual,Alcohol:none,Waiter
Service:true,Accepts Credit Cards:true,Good for Kids:true,Good For
Groups:true,Price Range:1,Music:{},Hair Types Specialized
In:{},Payment Types:{},Dietary Restrictions:{}} |
++
2 rows selected (0.274 seconds)
0: jdbc:drill:zk=local select kvgen(attributes) from
dfs.`/users/nrentachintala/Downloads/yelp/yelp_academic_dataset_business.json`
limit 2;
Query failed: Query failed: Failure while running fragment.,
org.apache.drill.exec.vector.complex.impl.NullableBitWriterImpl cannot be
cast to org.apache.drill.exec.vector.complex.impl.SingleMapWriter [
24bcf2c3-c995-4451-8e11-4e804050388f on 10.0.0.6:31010 ]
[ 24bcf2c3-c995-4451-8e11-4e804050388f on 10.0.0.6:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 30313: Fixing Mongo join issue when * is selected
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30313/ --- (Updated March 4, 2015, 2:11 a.m.) Review request for drill, Jacques Nadeau and Jinfeng Ni. Changes --- Attaching patch generated using git format-patch Repository: drill-git Description --- Fixing Mongo join issue when * is selected. Diffs (updated) - contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java da96701 Diff: https://reviews.apache.org/r/30313/diff/ Testing --- Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits. Thanks, Kamesh B
[jira] [Created] (DRILL-2378) UNION ALL on Aggregates with GROUP BY and ORDER BY fails when column aliases don't match
Abhishek Girish created DRILL-2378:
--
Summary: UNION ALL on Aggregates with GROUP BY and ORDER BY fails
when column aliases don't match
Key: DRILL-2378
URL: https://issues.apache.org/jira/browse/DRILL-2378
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.8.0
Reporter: Abhishek Girish
Assignee: Jinfeng Ni
*The following query fails to execute:*
{code:sql}
select x
from
(SELECT Sum(ss_ext_sales_price) x
FROM store_sales
UNION ALL
SELECT Sum(cs_ext_sales_price) y
FROM catalog_sales) tmp
GROUP BY x
ORDER BY x;
Error from Drill:
Query failed: RemoteRpcException: Failure while running fragment., Failure
while reading vector. Expected vector class of
org.apache.drill.exec.vector.Decimal38SparseVector but was holding vector class
org.apache.drill.exec.vector.NullableIntVector. [
7cc962d6-569b-44f6-9118-d5cc5a9632d9 on abhi7.qa.lab:31010 ]
[ 7cc962d6-569b-44f6-9118-d5cc5a9632d9 on abhi7.qa.lab:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
...
{code}
Log snippet attached.
*The following query executes fine (although returns incomplete results - see
DRILL-2376:*
{code:sql}
select x
from
(SELECT Sum(ss_ext_sales_price) x
FROM store_sales
UNION ALL
SELECT Sum(cs_ext_sales_price) x
FROM catalog_sales) tmp
GROUP BY x
ORDER BY x;
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2379) UNION ALL with ORDER BY fails when column aliases don't match
Abhishek Girish created DRILL-2379:
--
Summary: UNION ALL with ORDER BY fails when column aliases don't
match
Key: DRILL-2379
URL: https://issues.apache.org/jira/browse/DRILL-2379
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.8.0
Reporter: Abhishek Girish
Assignee: Jinfeng Ni
The following query fails to execute:
{code:sql}
SELECT x
FROM
(SELECT Sum(ss_ext_sales_price) x
FROM store_sales
UNION ALL
SELECT Sum(cs_ext_sales_price) y
FROM catalog_sales) tmp
ORDER BY x;
SELECT x
FROM
(SELECT ss_ext_sales_price x
FROM store_sales
UNION ALL
SELECT cs_ext_sales_price y
FROM catalog_sales) tmp
ORDER BY x;
SELECT x
FROM
(SELECT ss_ext_sales_price x
FROM store_sales
UNION ALL
SELECT ss_ext_sales_price y
FROM store_sales) tmp
ORDER BY x;
Error from Drill:
Query failed: RemoteRpcException: Failure while running fragment., Sort doesn't
currently support sorts with changing schemas.
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
...
{code}
Log snippet attached.
Changing the alias from 'y' to 'x' in the second sub-query will solve the
issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 31553: DRILL-2262: selecting columns of certain datatypes from a dictionary encoded parquet file created by drill fails
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31553/ --- (Updated March 3, 2015, 5:39 p.m.) Review request for drill and Steven Phillips. Bugs: DRILL-2262 https://issues.apache.org/jira/browse/DRILL-2262 Repository: drill-git Description --- Added proper reader for DECIMAL9, DECIMAL18, TIME and TIMESTAMP for OPTIONAL/REQUIRED columns. Updated ColumnReaderFactory to handle these cases Added unit test to make sure they are read correctly Diffs - exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ColumnReaderFactory.java cbdc0b0 exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/NullableFixedByteAlignedReaders.java 707bc9c exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetFixedWidthDictionaryReaders.java 02b583a exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/columnreaders/TestColumnReaderFactory.java PRE-CREATION exec/java-exec/src/test/resources/parquet/decimal_dictionary.parquet PRE-CREATION exec/java-exec/src/test/resources/parquet/decimal_nodictionary.parquet PRE-CREATION exec/java-exec/src/test/resources/parquet/time_dictionary.parquet PRE-CREATION exec/java-exec/src/test/resources/parquet/time_nodictionary.parquet PRE-CREATION Diff: https://reviews.apache.org/r/31553/diff/ Testing --- added unit tests to make sure DECIMAL9, DECIMAL18, TIME and TIMESTAMP are read correctly with/without DICTIONARY encoding. all unit tests pass along with functional, customer and tpch100 Thanks, abdelhakim deneche
