[jira] [Created] (DRILL-2310) Drill fails to start in embedded mode on windows
Krystal created DRILL-2310: -- Summary: Drill fails to start in embedded mode on windows Key: DRILL-2310 URL: https://issues.apache.org/jira/browse/DRILL-2310 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Affects Versions: 0.8.0 Reporter: Krystal Assignee: Steven Phillips git.commit.id.abbrev=c8d2fe1 I installed it on windows 7 I invoked sqlline in embedded mode via: C:\drill\apache-drill-0.8.0-SNAPSHOT\binsqlline.bat -u jdbc:drill:zk=local I got the following error: Error: Failure while attempting to start Drillbit in embedded mode. (state=,code=0) With debug turned on, the following error is displayed: 15:36:53.608 [main] WARN org.apache.hadoop.fs.FSInputChecker - Problem opening checksum file: /tmp/drill/sys.storage_ ugins/hbase.sys.drill. Ignoring exception: java.io.EOFException: null at java.io.DataInputStream.readFully(Unknown Source) ~[na:1.7.0_10] at java.io.DataInputStream.readFully(Unknown Source) ~[na:1.7.0_10] at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.init(ChecksumFileSystem.java:146) ~[hadoop ommon-2.4.1.jar:na] at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:339) [hadoop-common-2.4.1.jar:na] at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764) [hadoop-common-2.4.1.jar:na] at org.apache.drill.exec.store.dfs.DrillFileSystem.open(DrillFileSystem.java:145) [drill-java-exec-0.8.0-SNAPS T-rebuffed.jar:0.8.0-SNAPSHOT] at org.apache.drill.exec.store.sys.local.FilePStore.get(FilePStore.java:136) [drill-java-exec-0.8.0-SNAPSHOT-r uffed.jar:0.8.0-SNAPSHOT] at org.apache.drill.exec.store.sys.local.FilePStore$Iter$DeferredEntry.getValue(FilePStore.java:218) [drill-ja -exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] at org.apache.drill.exec.store.StoragePluginRegistry.createPlugins(StoragePluginRegistry.java:166) [drill-java xec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] at org.apache.drill.exec.store.StoragePluginRegistry.init(StoragePluginRegistry.java:130) [drill-java-exec-0.8 -SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:155) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0 .0-SNAPSHOT] at org.apache.drill.jdbc.DrillConnectionImpl.init(DrillConnectionImpl.java:79) [drill-jdbc-0.8.0-SNAPSHOT.ja 0.8.0-SNAPSHOT] at org.apache.drill.jdbc.DrillJdbc41Factory$DrillJdbc41Connection.init(DrillJdbc41Factory.java:94) [drill-jd -0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT] at org.apache.drill.jdbc.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:57) [drill-jdbc-0.8.0-S PSHOT.jar:0.8.0-SNAPSHOT] at org.apache.drill.jdbc.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:43) [drill-jdbc-0.8.0-S PSHOT.jar:0.8.0-SNAPSHOT] at org.apache.drill.jdbc.DrillFactory.newConnection(DrillFactory.java:54) [drill-jdbc-0.8.0-SNAPSHOT.jar:0.8.0 NAPSHOT] at net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126) [optiq-avatica-0.9-drill-r20 ar:na] at sqlline.SqlLine$DatabaseConnection.connect(SqlLine.java:4732) [sqlline-1.1.6.jar:na] at sqlline.SqlLine$DatabaseConnection.getConnection(SqlLine.java:4786) [sqlline-1.1.6.jar:na] at sqlline.SqlLine$Commands.connect(SqlLine.java:4026) [sqlline-1.1.6.jar:na] at sqlline.SqlLine$Commands.connect(SqlLine.java:3935) [sqlline-1.1.6.jar:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_10] at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.7.0_10] at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.7.0_10] at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.7.0_10] at sqlline.SqlLine$ReflectiveCommandHandler.execute(SqlLine.java:2884) [sqlline-1.1.6.jar:na] at sqlline.SqlLine.dispatch(SqlLine.java:885) [sqlline-1.1.6.jar:na] at sqlline.SqlLine.initArgs(SqlLine.java:693) [sqlline-1.1.6.jar:na] at sqlline.SqlLine.begin(SqlLine.java:745) [sqlline-1.1.6.jar:na] at sqlline.SqlLine.start(SqlLine.java:498) [sqlline-1.1.6.jar:na] at sqlline.SqlLine.main(SqlLine.java:460) [sqlline-1.1.6.jar:na] Drill successfully created the /tmp/drill/sys.storage_ ugins/hbase.sys.drill file so not sure why it can't access it. The file/directory has appropriate permissions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2311) Create table with same columns of different case results in a java.lang.IllegalStateException
Ramana Inukonda Nagaraj created DRILL-2311: -- Summary: Create table with same columns of different case results in a java.lang.IllegalStateException Key: DRILL-2311 URL: https://issues.apache.org/jira/browse/DRILL-2311 Project: Apache Drill Issue Type: Bug Components: SQL Parser Affects Versions: 0.8.0 Reporter: Ramana Inukonda Nagaraj Assignee: Aman Sinha Priority: Minor Doing a create table with same column in different case results in a runtime exception. This query should fail at planning or parsing. CREATE TABLE drill_parquet_mulCaseColumns3 as select cast( BIGINT_col as bigint) BIGINT_col,cast( DECIMAL9_col as decimal) bigint_col FROM dfs.`/user/root/alltypes.json`; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2313) Query fails when one of the operands is a DATE literal without an explicit cast
Abhishek Girish created DRILL-2313: -- Summary: Query fails when one of the operands is a DATE literal without an explicit cast Key: DRILL-2313 URL: https://issues.apache.org/jira/browse/DRILL-2313 Project: Apache Drill Issue Type: Bug Components: Query Planning Optimization Affects Versions: 0.8.0 Reporter: Abhishek Girish Assignee: Jinfeng Ni For operations involving the date datatype, when one of the operands is a DATE literal without a cast, query fails. *The following query fails to validate:* {code:sql} SELECT * FROM date_dim WHEREd_date BETWEEN '2002-3-01' AND cast('2002-3-01' AS DATE) LIMIT 1; {code} Query failed: SqlValidatorException: Cannot apply 'BETWEEN' to arguments of type 'ANY BETWEEN CHAR(9) AND DATE'. Supported form(s): 'COMPARABLE_TYPE BETWEEN COMPARABLE_TYPE AND COMPARABLE_TYPE' *The following query executes fine:* {code:sql} SELECT * FROM date_dim WHEREd_date BETWEEN '2002-3-01' AND '2002-3-01' LIMIT 1; {code} Both the queries execute fine on postgres -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 31419: DRILL-1378: Ctrl-C to cancel a query that has not returned with the first result set
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31419/#review74081 --- exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillResultSet.java https://reviews.apache.org/r/31419/#comment120588 Yeah, I just noticed that NullPointerException case a couple of days ago. exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillResultSet.java https://reviews.apache.org/r/31419/#comment120589 Should/can this be a call to releaseIfFirst() rather than directly calling latch.countDown()? - Daniel Barclay On Feb. 25, 2015, 6:38 p.m., Parth Chandra wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31419/ --- (Updated Feb. 25, 2015, 6:38 p.m.) Review request for drill, Daniel Barclay, Hanifi Gunes, and Mehant Baid. Repository: drill-git Description --- Sqlline hangs when you try to cancel the query before the first result set has been returned. Diffs - exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillResultSet.java 88a6c6d Diff: https://reviews.apache.org/r/31419/diff/ Testing --- Tested with a query that takes a long time to return the first result set. Ctrl-C to exit the query. This fix simply addresses the hang. It does not fix Avatica to display cancellation in a nicer manner. Also, have not verified that the query cancellation occurs on the drillbit in all cases (waiting for DRILL-2245) Thanks, Parth Chandra
[jira] [Created] (DRILL-2312) JDBC driver returning incorrect data after extended usage
Norris Lee created DRILL-2312: - Summary: JDBC driver returning incorrect data after extended usage Key: DRILL-2312 URL: https://issues.apache.org/jira/browse/DRILL-2312 Project: Apache Drill Issue Type: Bug Components: Client - JDBC Reporter: Norris Lee Assignee: Daniel Barclay (Drill) After executing ~20-30 queries with the JDBC driver, the data returned from a Show files query is incorrect, particularly the isFile and isDirectory columns. The first item in the schema/directory will be correct, but subsequent items will report false for isFile and isDirectory. This was tested with just a simple program that just loops through executeQuery and prints out the values for isFile and isDirectory. The JDBC driver used was the Drill 0.7 snapshot. {code} isFile: true isDirectory: false isFile: false isDirectory: false isFile: false isDirectory: false isFile: false isDirectory: false isFile: false isDirectory: false {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Review Request 31419: DRILL-1378: Ctrl-C to cancel a query that has not returned with the first result set
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31419/ --- Review request for drill, Daniel Barclay, Hanifi Gunes, and Mehant Baid. Repository: drill-git Description --- Sqlline hangs when you try to cancel the query before the first result set has been returned. Diffs - exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillResultSet.java 88a6c6d Diff: https://reviews.apache.org/r/31419/diff/ Testing --- Tested with a query that takes a long time to return the first result set. Ctrl-C to exit the query. This fix simply addresses the hang. It does not fix Avatica to display cancellation in a nicer manner. Also, have not verified that the query cancellation occurs on the drillbit in all cases (waiting for DRILL-2245) Thanks, Parth Chandra
Re: Review Request 31399: DRILL-2130: Fixed JUnit/Hamcrest/Mockito/Paranamer class path problem.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31399/#review74059 --- Ship it! LGTM. - Aditya Kishore On Feb. 25, 2015, 10:28 a.m., Daniel Barclay wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31399/ --- (Updated Feb. 25, 2015, 10:28 a.m.) Review request for drill, Aditya Kishore and Jason Altekruse. Repository: drill-git Description --- Main change: In exec/java-exec/pom.xml, suppressed Paranamer's transitive dependency on Mockito and the Hamcrest files bundled in it, and put JUnit and thereby Hamcrest on classpath before Mockito. NOTE: Please review placement (and details) of the added dependency.../ elements. The current placement seems to work, but probably doesn't follow our intended Maven patterns. Other change: Added tests to detect regression (or confirm other solutions to problem). Diffs - common/src/test/java/org/apache/drill/test/Drill2130CommonHamcrestConfigurationTest.java PRE-CREATION contrib/storage-hbase/src/test/java/org/apache/drill/hbase/test/Drill2130StorageHBaseHamcrestConfigurationTest.java PRE-CREATION contrib/storage-hive/core/src/test/java/org/apache/drill/exec/test/Drill2130StorageHiveCoreHamcrestConfigurationTest.java PRE-CREATION exec/interpreter/src/test/java/org/apache/drill/exec/expr/test/Drill2130InterpreterHamcrestConfigurationTest.java PRE-CREATION exec/java-exec/pom.xml 06f60fb exec/java-exec/src/test/java/org/apache/drill/exec/test/Drill2130JavaExecHamcrestConfigurationTest.java PRE-CREATION exec/jdbc/src/test/java/org/apache/drill/jdbc/test/Drill2130JavaJdbcHamcrestConfigurationTest.java PRE-CREATION pom.xml dc1448c Diff: https://reviews.apache.org/r/31399/diff/ Testing --- Ran new and existing tests. Thanks, Daniel Barclay
[jira] [Resolved] (DRILL-2202) Star is no expanded correctly during group by over named subquery
[ https://issues.apache.org/jira/browse/DRILL-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victoria Markman resolved DRILL-2202. - Resolution: Fixed Fix Version/s: (was: 1.0.0) 0.8.0 Star is no expanded correctly during group by over named subquery - Key: DRILL-2202 URL: https://issues.apache.org/jira/browse/DRILL-2202 Project: Apache Drill Issue Type: Bug Components: Query Planning Optimization Affects Versions: 0.8.0 Reporter: Victoria Markman Assignee: Sean Hsuan-Yi Chu Fix For: 0.8.0 Attachments: drillbit.log I gave subquery name and named all the columns. Group by condition is the same as expanded star ... Should work. {code} 0: jdbc:drill:schema=dfs select * from ( select a1, b1, c1 from t1 ) as dt(x1, x2, x3) group by dt.x1, dt.x2, dt.x3; Query failed: AssertionError: Internal error: invariant violated: conversion result not null Error: exception while executing query: Failure while executing query. (state=,code=0) {code} Workaround: replace star with column names manually: {code} 0: jdbc:drill:schema=dfs select dt.x1, dt.x2, dt.x3 from ( select a1, b1, c1 from t1 ) as dt(x1, x2, x3) group by dt.x1, dt.x2, dt.x3; ++++ | x1 | x2 | x3 | ++++ | 1 | a | 2015-01-01 | | 2 | b | 2015-01-02 | | 3 | c | 2015-01-03 | | 4 | null | 2015-01-04 | | 5 | e | 2015-01-05 | | 6 | f | 2015-01-06 | | 7 | g | 2015-01-07 | | null | h | 2015-01-08 | | 9 | i | null | | 10 | j | 2015-01-10 | ++++ 10 rows selected (0.16 seconds) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-2084) Order by on date, time, timestamp and boolean columns produces wrong results if column contains null values
[ https://issues.apache.org/jira/browse/DRILL-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victoria Markman resolved DRILL-2084. - Resolution: Fixed Fix Version/s: (was: 0.9.0) 0.8.0 Assignee: Daniel Barclay (Drill) (was: Hanifi Gunes) Order by on date, time, timestamp and boolean columns produces wrong results if column contains null values --- Key: DRILL-2084 URL: https://issues.apache.org/jira/browse/DRILL-2084 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Affects Versions: 0.8.0 Reporter: Victoria Markman Assignee: Daniel Barclay (Drill) Priority: Critical Fix For: 0.8.0 Attachments: 0_0_0.parquet, boolean.out, date.out, t1.csv, time.out, timestamp.out Nulls are neither sorted at the end or beginning of the table, they appear in the middle. I created table using parquet format from t1.csv file. Here is create table statement I used to create table that is used in the queries: {code} create table t1(c_varchar, c_integer, c_bigint, c_smalldecimal, c_bigdecimal, c_float, c_date, c_time, c_timestamp, c_boolean) as select case when columns[0] = '' then cast(null as varchar(255)) else cast(columns[0] as varchar(255)) end, case when columns[1] = '' then cast(null as integer) else cast(columns[1] as integer) end, case when columns[2] = '' then cast(null as bigint) else cast(columns[2] as bigint) end, case when columns[3] = '' then cast(null as decimal(18,4)) else cast(columns[3] as decimal(18, 4)) end, case when columns[4] = '' then cast(null as decimal(38,4)) else cast(columns[4] as decimal(38, 4)) end, case when columns[5] = '' then cast(null as float) else cast(columns[5] as float) end, case when columns[6] = '' then cast(null as date) else cast(columns[6] as date) end, case when columns[7] = '' then cast(null as time) else cast(columns[7] as time) end, case when columns[8] = '' then cast(null as timestamp) else cast(columns[8] as timestamp) end, case when columns[9] = '' then cast(null as boolean) else cast(columns[9] as boolean) end from `t1.csv`; {code} Failing queries are: {code} select c_date from t1 order by c_date; select c_time from t1 order by c_time; select c_timestamp from t1 order by c_timestamp; select c_boolean from t1 order by c_boolean; {code} If you add 'IS NULL' or 'IS NOT NULL' predicates to the queries above, you will get correct results. I will attach all the relevant files to this bug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2314) Drill on windows fail to make spill diretory for queries against large data
Krystal created DRILL-2314: -- Summary: Drill on windows fail to make spill diretory for queries against large data Key: DRILL-2314 URL: https://issues.apache.org/jira/browse/DRILL-2314 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Affects Versions: 0.8.0 Reporter: Krystal Assignee: Steven Phillips git.commit.id.abbrev=c8d2fe1 I installed it on windows 7 I connected from sqlline to a remote drill cluster as: sqlline.bat --maxWidth=1 -u jdbc:drill:zk=ip:port I ran the followng query against a large data file with 2880404 records which failed: 0: jdbc:drill:zk=10.10.100.113:5181 select ss_ticket_number, count(*) counts from `dfs.drillTestDirTpcdsImpalaSF1`.stor e_sales group by ss_ticket_number,counts order by ss_ticket_number; +--++ | ss_ticket_number | counts | +--++ Query failed: RemoteRpcException: Failure while running fragment., Mkdirs failed to create /tmp/drill/spill [ 94caf71e-9 a86-4046-8bea-b5c46e6ddd80 on qa-node114.qa.lab:31010 ] [ 94caf71e-9a86-4046-8bea-b5c46e6ddd80 on qa-node114.qa.lab:31010 ] java.lang.RuntimeException: java.sql.SQLException: Failure while executing query. at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514) at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148) at sqlline.SqlLine.print(SqlLine.java:1809) at sqlline.SqlLine$Commands.execute(SqlLine.java:3766) at sqlline.SqlLine$Commands.sql(SqlLine.java:3663) at sqlline.SqlLine.dispatch(SqlLine.java:889) at sqlline.SqlLine.begin(SqlLine.java:763) at sqlline.SqlLine.start(SqlLine.java:498) at sqlline.SqlLine.main(SqlLine.java:460) The /tmp/drill directory has appropriate permissions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 31419: DRILL-1378: Ctrl-C to cancel a query that has not returned with the first result set
On Feb. 25, 2015, 8:05 p.m., Daniel Barclay wrote: exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillResultSet.java, line 219 https://reviews.apache.org/r/31419/diff/1/?file=875900#file875900line219 Should/can this be a call to releaseIfFirst() rather than directly calling latch.countDown()? releaseIfFirst will set receivedMessage to true which may not be correct. - Parth --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31419/#review74081 --- On Feb. 25, 2015, 6:38 p.m., Parth Chandra wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31419/ --- (Updated Feb. 25, 2015, 6:38 p.m.) Review request for drill, Daniel Barclay, Hanifi Gunes, and Mehant Baid. Repository: drill-git Description --- Sqlline hangs when you try to cancel the query before the first result set has been returned. Diffs - exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillResultSet.java 88a6c6d Diff: https://reviews.apache.org/r/31419/diff/ Testing --- Tested with a query that takes a long time to return the first result set. Ctrl-C to exit the query. This fix simply addresses the hang. It does not fix Avatica to display cancellation in a nicer manner. Also, have not verified that the query cancellation occurs on the drillbit in all cases (waiting for DRILL-2245) Thanks, Parth Chandra
Review Request 31435: DRILL-2245-core: core changes for improved drillbit stability
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31435/ --- Review request for drill and Jacques Nadeau. Bugs: DRILL-2245 https://issues.apache.org/jira/browse/DRILL-2245 Repository: drill-git Description --- DRILL-2245-core: Clean up query setup and execution kickoff in Foreman/WorkManager in order to ensure consistent handling, and avoid hangs and races, with the goal of improving Drillbit robustness. I did my best to keep these clean when I split them up, but this core commit may depend on some minor changes in the hygiene commit that is also associated with this bug, so either both should be applied, or neither. The core commit should be applied first. AutoCloseables - created org.apache.drill.common.AutoCloseables to handle closing these quietly BaseTestQuery, and derivatives - factored out pieces into QueryTestUtil so they can be reused Drillbit - uses AutoCloseables for the WorkManager and for the storeProvider - allow start() to take a RemoteServiceSet - private, final, formatting Foreman - does not need to implement Comparable - does not need to implement Closeable - thread blocking fixes - add resultSent flag - add code to log plan fragments with endpoint assignments - added finals, cleaned up formatting - do queue management in acquireQuerySemaphore; local tests pass - rename getContext() to getQueryContext() - retain DrillbitContext - a couple of exception injections for testing - minor formatting - TODOs FragmentExecutor - eliminated CancelableQuery - common subexpression elimination - cleaned up QueryContext - removed unnecessary functions (with some outside classes tweaked for this) - finals, formatting QueryManager - merge in QueryStatus - affects Foreman, ../batch/ControlHandlerImpl, and ../../server/rest/ProfileResources - made some methods private - removed unused imports - add finals and formatting - variable renaming to improve readability - formatting - comments - TODOs QueryStatus - getAsInfo() private - member renaming - member access changes - formatting - TODOs QueryTestUtil, BaseTestQuery, TestDrillbitResilience - make maxWidth a parameter to server startup SelfCleaningRunnable - created org.apache.drill.common.SelfCleaningRunnable SingleRowListener - created org.apache.drill.SingleRowListener results listener - use in TestDrillbitResilience TestDrillbitResilience - created org.apache.drill.exec.server.TestDrillbitResilience to test drillb resilience in the face of exceptions and failures during queries TestWithZookeeper - factor out work into ZookeeperHelper so that it can be reused by TestDrillbitResilience UserBitShared - get rid of unused UNKNOWN_QUERY WorkEventBus - rename methods, affects Foreman and ControlHandlerImpl - remove unused WorkerBee reference - most members final - formatting WorkManager - Closeable to AutoCloseable - removed unused incomingFragments Set - eliminated unnecessary eventThread and pendingTasks by posting Runnables directly to executor - use SelfCleaningRunnable for Foreman management - FragmentExecutor management uses SelfCleaningRunnable - runningFragments to be a ConcurrentHashMap; TestTpchDistributed passes - other improvements due to bee no longer needed in various places - most members final - minor formatting - comments - TODOs (*) Created exception injection classes to simulate exceptions for testing - ExceptionInjection - ExceptionInjector - ExceptionInjectionUtil - TestExceptionInjection Diffs - common/src/main/java/org/apache/drill/common/AutoCloseables.java PRE-CREATION common/src/main/java/org/apache/drill/common/SelfCleaningRunnable.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 5efcce8 exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java 04b955b exec/java-exec/src/main/java/org/apache/drill/exec/coord/local/LocalClusterCoordinator.java 035c1aa exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/WorkEventBus.java d6b8637 exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java 67342c4 exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java 83a89df exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java aa0a5ad exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/profile/ProfileResources.java ae04bad
[jira] [Resolved] (DRILL-2298) Selecting same column multiple times with different case from a parquet file results in just one column returned
[ https://issues.apache.org/jira/browse/DRILL-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deneche A. Hakim resolved DRILL-2298. - Resolution: Duplicate Selecting same column multiple times with different case from a parquet file results in just one column returned Key: DRILL-2298 URL: https://issues.apache.org/jira/browse/DRILL-2298 Project: Apache Drill Issue Type: Bug Components: Storage - Parquet Affects Versions: 0.8.0 Reporter: Ramana Inukonda Nagaraj Assignee: Steven Phillips When selecting the same column twice with the same case everything works as expected. {code} 0: jdbc:drill: select BIGINT_col,BIGINT_col from `parquet_storage/parquet_all_types.parquet` limit 1; ++-+ | BIGINT_col | BIGINT_col0 | ++-+ | 1 | 1 | ++-+ 1 row selected (0.086 seconds) {code} However, when different case is used for the same column only one column is returned. {code} 0: jdbc:drill: select BIGINT_col,bigint_col from `parquet_storage/parquet_all_types.parquet` limit 1; ++ | BIGINT_col | ++ | 1 | ++ 1 row selected (0.079 seconds) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 28686: For DRILL-1735, hook up JDBC connection closing, and fix follow-on resource bugs.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28686/ --- (Updated Feb. 25, 2015, 9:37 p.m.) Review request for drill, Mehant Baid and Parth Chandra. Changes --- Rebased on current, not-yet-split patch for 2245 on today's master and re-made patch. Bugs: DRILL-1735 https://issues.apache.org/jira/browse/DRILL-1735 Repository: drill-git Description --- This change hooks in the previously written but unused DrillHandler so that a local-mode JDBC connection shuts down its embedded DrillBit. It also fixes a chain of follow-on bugs (mostly resource-deallocation omissions) exposed by the original bug fixes. (See the commit message for the list of bugs/fixes and other changes.) Diffs (updated) - exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java 04b955b exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScreenCreator.java d884200 exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java b079428 exec/java-exec/src/main/java/org/apache/drill/exec/server/BootStrapContext.java 3da2ea9 exec/java-exec/src/main/java/org/apache/drill/exec/vector/accessor/SqlAccessor.java b8480b4 exec/java-exec/src/main/java/org/apache/drill/exec/work/WorkManager.java 99c6ab8 exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 378e81a exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java 7ccb64e exec/java-exec/src/test/java/org/apache/drill/exec/ExecTest.java 0272b23 exec/jdbc/src/main/java/org/apache/drill/jdbc/AvaticaDrillSqlAccessor.java cf5829a exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillAccessorList.java 82d51f1 exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionConfig.java 54e31b1 exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionImpl.java 2c51ec0 exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillCursor.java fbe611f exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillResultSet.java 88a6c6d exec/jdbc/src/main/java/org/apache/drill/jdbc/Driver.java 974e786 exec/jdbc/src/main/java/org/apache/drill/jdbc/InvalidCursorStateSqlException.java PRE-CREATION exec/jdbc/src/main/java/org/apache/drill/jdbc/JdbcApiSqlException.java PRE-CREATION exec/jdbc/src/test/java/org/apache/drill/jdbc/DrillResultSetTest.java PRE-CREATION exec/jdbc/src/test/java/org/apache/drill/jdbc/DriverTest.java PRE-CREATION exec/jdbc/src/test/java/org/apache/drill/jdbc/test/Bug1735ConnectionCloseTest.java PRE-CREATION exec/jdbc/src/test/java/org/apache/drill/jdbc/test/Bug1735ResultSetCloseReleasesBuffersTest.java PRE-CREATION exec/jdbc/src/test/java/org/apache/drill/jdbc/test/TestJdbcDistQuery.java b2f86ea exec/jdbc/src/test/java/org/apache/drill/jdbc/test/TestJdbcQuery.java b627c38 exec/jdbc/src/test/java/org/apache/drill/jdbc/test/TestMetadataDDL.java 93925fe exec/jdbc/src/test/java/org/apache/drill/jdbc/test/TestViews.java 0f9e25e Diff: https://reviews.apache.org/r/28686/diff/ Testing --- Ran tests. Thanks, Daniel Barclay
Review Request 31436: DRILL-2245-hygiene: cleaned up a number of files seen while working on DRILL-2245-core.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31436/ --- Review request for drill and Jacques Nadeau. Bugs: DRILL-2245 https://issues.apache.org/jira/browse/DRILL-2245 Repository: drill-git Description --- DRILL-2245-hygiene: General code cleanup encountered while working on the core of this commit. This includes - making members final whenever possible - making members private whenever possible - making loggers private - removing unused imports - removing unused private functions - removing unused public functions - removing unused local variables - removing unused private members - deleting unused files - cleaning up formatting - adding spaces before braces in conditionals and loop bodies - breaking up overly long lines - removing extra blank lines While I tried to keep this clean, this commit may have minor dependencies on DRILL-2245-core that I missed. The intention is just to break this up for review purposes. Either both commits should be applied, or neither. Diffs - exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java e413921 exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java c881432 exec/java-exec/src/main/java/org/apache/drill/exec/opt/BasicOptimizer.java 5288f5d exec/java-exec/src/main/java/org/apache/drill/exec/opt/IdentityOptimizer.java 979c5e2 exec/java-exec/src/main/java/org/apache/drill/exec/opt/Optimizer.java 34d0622 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/BaseRootExec.java 412da85 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/RootExec.java a644c34 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SendingAccountor.java 7af7b65 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/broadcastsender/BroadcastSenderRootExec.java 22fa047 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionRecordBatch.java 352e7ae exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/producer/ProducerConsumerBatch.java 4c9b33b exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Fragment.java ac63bde exec/java-exec/src/main/java/org/apache/drill/exec/record/MaterializedField.java ef53d2a exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicClient.java f358097 exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicServer.java c00df4e exec/java-exec/src/main/java/org/apache/drill/exec/rpc/NamedThreadFactory.java 2b49579 exec/java-exec/src/main/java/org/apache/drill/exec/rpc/ReconnectingConnection.java f214c4d exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RemoteConnection.java 3a139f8 exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RpcBus.java 96c9911 exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/Controller.java 7f84a2b exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/ControllerImpl.java f8f6fd7 exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataConnectionCreator.java 197996d exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataResponseHandlerImpl.java 1fcb3e9 exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataTunnel.java 5aa4aa6 exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserClient.java 4e7fc92 exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserServer.java dffb9a1 exec/java-exec/src/main/java/org/apache/drill/exec/service/ServiceEngine.java ff6e13c exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/local/FilePStore.java baa998d exec/java-exec/src/main/java/org/apache/drill/exec/work/QueryWorkUnit.java 9743d6e exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java 6ee93ab exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java 895918c exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/DrillbitStatusListener.java ca52f0c exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/SimpleRootExec.java 0277876 exec/jdbc/src/test/java/org/apache/drill/jdbc/test/JdbcAssert.java b88d880 Diff: https://reviews.apache.org/r/31436/diff/ Testing --- mvn install Functional - Passing - New Advanced - TPCH SF100 - Parquet Thanks, Chris Westin
[jira] [Resolved] (DRILL-2130) old version Hamcrest on classpath before new causes test failures to fail with NoSuchMethodError
[ https://issues.apache.org/jira/browse/DRILL-2130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Kishore resolved DRILL-2130. --- Resolution: Fixed Fix Version/s: (was: 1.0.0) 0.8.0 Resolved by [b0faf70|https://fisheye6.atlassian.com/changelog/incubator-drill?cs=b0faf708bdbeb53bc3a446d3782554640bdfd6df]. old version Hamcrest on classpath before new causes test failures to fail with NoSuchMethodError Key: DRILL-2130 URL: https://issues.apache.org/jira/browse/DRILL-2130 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Reporter: Daniel Barclay (Drill) Assignee: Daniel Barclay (Drill) Priority: Minor Fix For: 0.8.0 Attachments: DRILL-2130.1.patch.txt, DRILL-2130.2.patch.txt, DRILL-2130.3.patch.txt Apparently we have included multiple versions of JUnit and/or Hamcrest, and the new versions are not first on the class path. This means that when an assertThat call tries to report that an actual value mismatches an expected value, it dies with a NoSuchMethod error. That in turn means that what would have been an informative test-failure message turns into a confusing crash. It seems that something like this is happening: The unit test is calling the new version of some JUnit or Hamcrest class. That class is calling a method in another class (in Hamcrest), but the old version of the class gets loaded, so the call fails with NoSuchMethodError. Here's a sample stacktrace: java.lang.NoSuchMethodError: org.hamcrest.Matcher.describeMismatch(Ljava/lang/Object;Lorg/hamcrest/Description;)V at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18) at org.apache.drill.jdbc.test.DatabaseMetaDataTest.test_getColumns_resultSetMetadataIsCorrect(DatabaseMetaDataTest.java:179) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.lang.reflect.Method.invoke(Method.java:606) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.lang.reflect.Method.invoke(Method.java:606) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-1690) Issue with using HBase plugin to access row_key only
[ https://issues.apache.org/jira/browse/DRILL-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Kishore resolved DRILL-1690. --- Resolution: Fixed Fix Version/s: (was: 0.9.0) 0.8.0 Resolved by [f7ef5ec|https://fisheye6.atlassian.com/changelog/incubator-drill?cs=f7ef5ec784844a99b8b39fe10ab14f001ae149f2]. Issue with using HBase plugin to access row_key only Key: DRILL-1690 URL: https://issues.apache.org/jira/browse/DRILL-1690 Project: Apache Drill Issue Type: Bug Components: Storage - HBase Reporter: Chun Chang Assignee: Aditya Kishore Priority: Critical Fix For: 0.8.0 Attachments: DRILL-1690-Issue-with-using-HBase-plugin-to-access-r.patch #Thu Nov 06 11:38:17 PST 2014 git.commit.id.abbrev=eeb7160 Configure an HBase plugin to access MapRDB table. When, select the row key only, we return two columns: 0: jdbc:drill:schema=M7 select row_key, fivecf from m7student limit 2; +++ | row_key | fivecf | +++ | [B@31966bd8 | {create_date:MjAxNC0wNS0yNyAwMDoyNjowNw==} | | [B@a824b57 | {create_date:MjAxNC0wMy0yNCAyMTo1NzozMw==} | +++ 2 rows selected (0.128 seconds) 0: jdbc:drill:schema=M7 select row_key from m7student limit 2; +++ | row_key | fivecf | +++ | [B@2560bab1 | {create_date:MjAxNC0wNS0yNyAwMDoyNjowNw==} | | [B@7180cda2 | {create_date:MjAxNC0wMy0yNCAyMTo1NzozMw==} | +++ 2 rows selected (0.107 seconds) Here is the plan: 0: jdbc:drill:schema=M7 explain plan for select row_key from m7student limit 2; +++ |text|json| +++ | 00-00Screen 00-01 SelectionVectorRemover 00-02Limit(fetch=[2]) 00-03 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=m7student, startRow=null, stopRow=null, filter=null], columns=[SchemaPath [`row_key`) | { head : { version : 1, generator : { type : ExplainHandler, info : }, type : APACHE_DRILL_PHYSICAL, options : [ ], queue : 0, resultMode : EXEC }, graph : [ { pop : hbase-scan, @id : 3, hbaseScanSpec : { tableName : m7student, startRow : , stopRow : , serializedFilter : null }, storage : { type : hbase, config : { hbase.table.namespace.mappings : *:/test/tables }, size.calculator.enabled : false, enabled : true }, columns : [ `row_key` ], cost : 1048576.0 }, { pop : limit, @id : 2, child : 3, first : 0, last : 2, initialAllocation : 100, maxAllocation : 100, cost : 1048576.0 }, { pop : selection-vector-remover, @id : 1, child : 2, initialAllocation : 100, maxAllocation : 100, cost : 1048576.0 }, { pop : screen, @id : 0, child : 1, initialAllocation : 100, maxAllocation : 100, cost : 1048576.0 } ] } | +++ Drill should strip out the second column. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 31400: corrections to confluence conversions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31400/#review74161 --- Ship it! Looks good. - Bridget Bridget On Feb. 26, 2015, 12:11 a.m., Kristine Hahn wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31400/ --- (Updated Feb. 26, 2015, 12:11 a.m.) Review request for drill and Bridget Bridget. Bugs: DRILL-2315 https://issues.apache.org/jira/browse/DRILL-2315 Repository: drill-git Description --- corrections to confluence conversions Diffs - .gitignore 1520ec3 _docs/001-arch.md PRE-CREATION _docs/001-drill-docs.md 382e2e1 _docs/002-tutorial.md PRE-CREATION _docs/003-yelp.md PRE-CREATION _docs/004-install.md PRE-CREATION _docs/005-connect.md PRE-CREATION _docs/006-interfaces.md PRE-CREATION _docs/007-query.md PRE-CREATION _docs/008-sql-ref.md PRE-CREATION _docs/009-dev-custom-func.md PRE-CREATION _docs/010-manage.md PRE-CREATION _docs/011-develop.md PRE-CREATION _docs/012-rn.md PRE-CREATION _docs/013-contribute.md PRE-CREATION _docs/014-sample-ds.md PRE-CREATION _docs/015-design.md PRE-CREATION _docs/016-progress.md PRE-CREATION _docs/017-archived-pages.md PRE-CREATION _docs/018-bylaws.md PRE-CREATION _docs/arch/001-core-mod.md PRE-CREATION _docs/arch/002-arch-hilite.md PRE-CREATION _docs/arch/arch-hilite/001-flexibility.md PRE-CREATION _docs/arch/arch-hilite/002-performance.md PRE-CREATION _docs/archive/001-how-to-demo.md PRE-CREATION _docs/archive/002-meet-drill.md PRE-CREATION _docs/connect/001-plugin-reg.md PRE-CREATION _docs/connect/002-workspaces.md PRE-CREATION _docs/connect/003-reg-fs.md PRE-CREATION _docs/connect/004-reg-hbase.md PRE-CREATION _docs/connect/005-reg-hive.md PRE-CREATION _docs/connect/006-default-frmt.md PRE-CREATION _docs/connect/007-mongo-plugin.md PRE-CREATION _docs/connect/008-mapr-db-plugin.md PRE-CREATION _docs/contribute/001-guidelines.md PRE-CREATION _docs/contribute/002-ideas.md PRE-CREATION _docs/datasets/001-aol.md PRE-CREATION _docs/datasets/002-enron.md PRE-CREATION _docs/datasets/003-wikipedia.md PRE-CREATION _docs/design/001-plan.md PRE-CREATION _docs/design/002-rpc.md PRE-CREATION _docs/design/003-query-stages.md PRE-CREATION _docs/design/004-research.md PRE-CREATION _docs/design/005-value.md PRE-CREATION _docs/dev-custom-fcn/001-dev-simple.md PRE-CREATION _docs/dev-custom-fcn/002-dev-aggregate.md PRE-CREATION _docs/dev-custom-fcn/003-add-custom.md PRE-CREATION _docs/dev-custom-fcn/004-use-custom.md PRE-CREATION _docs/dev-custom-fcn/005-cust-interface.md PRE-CREATION _docs/develop/001-compile.md PRE-CREATION _docs/develop/002-setup.md PRE-CREATION _docs/develop/003-patch-tool.md PRE-CREATION _docs/drill-docs/001-arch.md e4b26fc _docs/drill-docs/002-tutorial.md 597f994 _docs/drill-docs/003-yelp.md b9339ed _docs/drill-docs/004-install.md fe7578c _docs/drill-docs/005-connect.md 039fc78 _docs/drill-docs/006-query.md 4b4fda0 _docs/drill-docs/006-sql-ref.md 8818ca3 _docs/drill-docs/007-dev-custom-func.md 9bc8e65 _docs/drill-docs/008-manage.md e629b20 _docs/drill-docs/009-develop.md d95f986 _docs/drill-docs/010-rn.md f196714 _docs/drill-docs/011-contribute.md 282ab8a _docs/drill-docs/012-sample-ds.md fe63f6b _docs/drill-docs/013-design.md 57d73c1 _docs/drill-docs/014-progress.md 2a1538c _docs/drill-docs/015-archived-pages.md b2a29c3 _docs/drill-docs/016-bylaws.md 6f2604f _docs/drill-docs/arch/001-core-mod.md 81a8107 _docs/drill-docs/arch/002-arch-hilite.md 8761d62 _docs/drill-docs/arch/arch-hilite/001-flexibility.md a36bb9b _docs/drill-docs/arch/arch-hilite/002-performance.md e18a5b4 _docs/drill-docs/archive/001-how-to-demo.md 1b46e88 _docs/drill-docs/archive/002-meet-drill.md 32d0ba1 _docs/drill-docs/connect/001-plugin-reg.md 88c3c0b _docs/drill-docs/connect/002-mongo-plugin.md 665c79b _docs/drill-docs/connect/003-mapr-db-plugin.md 26d6a9c _docs/drill-docs/connect/workspaces/001-workspaces.md 4ec1f1b _docs/drill-docs/connect/workspaces/002-reg-fs.md 4abb134 _docs/drill-docs/connect/workspaces/003-reg-hbase.md b97de4c _docs/drill-docs/connect/workspaces/004-reg-hive.md c226fd6 _docs/drill-docs/connect/workspaces/005-default-frmt.md 335f415 _docs/drill-docs/contribute/001-guidelines.md 0b24541 _docs/drill-docs/contribute/002-ideas.md ef42adf _docs/drill-docs/datasets/001-aol.md 472f52f _docs/drill-docs/datasets/002-enron.md 2ddbef6
[jira] [Created] (DRILL-2318) Query fails when an ORDER BY clause is used with CTEs
Abhishek Girish created DRILL-2318: -- Summary: Query fails when an ORDER BY clause is used with CTEs Key: DRILL-2318 URL: https://issues.apache.org/jira/browse/DRILL-2318 Project: Apache Drill Issue Type: Bug Components: Query Planning Optimization Affects Versions: 0.8.0 Reporter: Abhishek Girish Assignee: Jinfeng Ni Adding a WITH clause with a simple CTE causes a query with an ORDER BY to fail. This happens even when the CTE is unrelated to the main query. *The following query fails to execute:* {code:sql} WITH x AS (SELECT ss_sold_date_sk a1 FROM store_sales) SELECT x.a1 FROM x ORDER BY x.a1; {code} The following query executes fine: {code:sql} WITH x AS (SELECT ss_sold_date_sk a1 FROM store_sales) SELECT x.a1 FROM x {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2316) Docs Enhancement: Data Sources and File Formats, Basics Tutorial
Kristine Hahn created DRILL-2316: Summary: Docs Enhancement: Data Sources and File Formats, Basics Tutorial Key: DRILL-2316 URL: https://issues.apache.org/jira/browse/DRILL-2316 Project: Apache Drill Issue Type: Task Affects Versions: 0.7.0 Reporter: Kristine Hahn Assignee: Kristine Hahn Fix For: 1.0.0 Hive types, Hive UDF, Parquet, JSON. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2317) UNION ALL on sub-queries with single column projection fails on GROUP BY
Abhishek Girish created DRILL-2317: -- Summary: UNION ALL on sub-queries with single column projection fails on GROUP BY Key: DRILL-2317 URL: https://issues.apache.org/jira/browse/DRILL-2317 Project: Apache Drill Issue Type: Bug Components: Query Planning Optimization Affects Versions: 0.8.0 Reporter: Abhishek Girish Assignee: Jinfeng Ni A UNION ALL on two sub-queries with only a single column projected, and GROUP BY on the whole query fails with an IndexOutOfBoundsException *The following query fails to execute:* {code:sql} SELECT c1 FROM ((SELECT ss_quantity AS c1 FROM store_sales limit 1) UNION ALL (SELECT ws_quantity AS c2 FROM web_sales limit 1) ) foo GROUP BY c1; {code} The following query also fails to execute: {code:sql} SELECT distinct(c) FROM ((SELECT ss_quantity AS c FROM store_sales limit 1) UNION ALL (SELECT ws_quantity AS c FROM web_sales limit 1) ) ; {code} Error: Query failed: RemoteRpcException: Failure while running fragment., index: 0, length: 1 (expected: range(0, 0)) LOG message indicates java.lang.IndexOutOfBoundsException *The following query executes fine:* {code:sql} SELECT c1 FROM ((SELECT ss_quantity AS c1, ss_item_sk c3 FROM store_sales limit 1) UNION ALL (SELECT ws_quantity AS c2, ws_item_sk c4 FROM web_sales limit 1) ) foo GROUP BY c1; {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 31400: corrections to confluence conversions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31400/ --- (Updated Feb. 26, 2015, 12:11 a.m.) Review request for drill and Bridget Bridget. Bugs: DRILL-2315 https://issues.apache.org/jira/browse/DRILL-2315 Repository: drill-git Description --- corrections to confluence conversions Diffs (updated) - .gitignore 1520ec3 _docs/001-arch.md PRE-CREATION _docs/001-drill-docs.md 382e2e1 _docs/002-tutorial.md PRE-CREATION _docs/003-yelp.md PRE-CREATION _docs/004-install.md PRE-CREATION _docs/005-connect.md PRE-CREATION _docs/006-interfaces.md PRE-CREATION _docs/007-query.md PRE-CREATION _docs/008-sql-ref.md PRE-CREATION _docs/009-dev-custom-func.md PRE-CREATION _docs/010-manage.md PRE-CREATION _docs/011-develop.md PRE-CREATION _docs/012-rn.md PRE-CREATION _docs/013-contribute.md PRE-CREATION _docs/014-sample-ds.md PRE-CREATION _docs/015-design.md PRE-CREATION _docs/016-progress.md PRE-CREATION _docs/017-archived-pages.md PRE-CREATION _docs/018-bylaws.md PRE-CREATION _docs/arch/001-core-mod.md PRE-CREATION _docs/arch/002-arch-hilite.md PRE-CREATION _docs/arch/arch-hilite/001-flexibility.md PRE-CREATION _docs/arch/arch-hilite/002-performance.md PRE-CREATION _docs/archive/001-how-to-demo.md PRE-CREATION _docs/archive/002-meet-drill.md PRE-CREATION _docs/connect/001-plugin-reg.md PRE-CREATION _docs/connect/002-workspaces.md PRE-CREATION _docs/connect/003-reg-fs.md PRE-CREATION _docs/connect/004-reg-hbase.md PRE-CREATION _docs/connect/005-reg-hive.md PRE-CREATION _docs/connect/006-default-frmt.md PRE-CREATION _docs/connect/007-mongo-plugin.md PRE-CREATION _docs/connect/008-mapr-db-plugin.md PRE-CREATION _docs/contribute/001-guidelines.md PRE-CREATION _docs/contribute/002-ideas.md PRE-CREATION _docs/datasets/001-aol.md PRE-CREATION _docs/datasets/002-enron.md PRE-CREATION _docs/datasets/003-wikipedia.md PRE-CREATION _docs/design/001-plan.md PRE-CREATION _docs/design/002-rpc.md PRE-CREATION _docs/design/003-query-stages.md PRE-CREATION _docs/design/004-research.md PRE-CREATION _docs/design/005-value.md PRE-CREATION _docs/dev-custom-fcn/001-dev-simple.md PRE-CREATION _docs/dev-custom-fcn/002-dev-aggregate.md PRE-CREATION _docs/dev-custom-fcn/003-add-custom.md PRE-CREATION _docs/dev-custom-fcn/004-use-custom.md PRE-CREATION _docs/dev-custom-fcn/005-cust-interface.md PRE-CREATION _docs/develop/001-compile.md PRE-CREATION _docs/develop/002-setup.md PRE-CREATION _docs/develop/003-patch-tool.md PRE-CREATION _docs/drill-docs/001-arch.md e4b26fc _docs/drill-docs/002-tutorial.md 597f994 _docs/drill-docs/003-yelp.md b9339ed _docs/drill-docs/004-install.md fe7578c _docs/drill-docs/005-connect.md 039fc78 _docs/drill-docs/006-query.md 4b4fda0 _docs/drill-docs/006-sql-ref.md 8818ca3 _docs/drill-docs/007-dev-custom-func.md 9bc8e65 _docs/drill-docs/008-manage.md e629b20 _docs/drill-docs/009-develop.md d95f986 _docs/drill-docs/010-rn.md f196714 _docs/drill-docs/011-contribute.md 282ab8a _docs/drill-docs/012-sample-ds.md fe63f6b _docs/drill-docs/013-design.md 57d73c1 _docs/drill-docs/014-progress.md 2a1538c _docs/drill-docs/015-archived-pages.md b2a29c3 _docs/drill-docs/016-bylaws.md 6f2604f _docs/drill-docs/arch/001-core-mod.md 81a8107 _docs/drill-docs/arch/002-arch-hilite.md 8761d62 _docs/drill-docs/arch/arch-hilite/001-flexibility.md a36bb9b _docs/drill-docs/arch/arch-hilite/002-performance.md e18a5b4 _docs/drill-docs/archive/001-how-to-demo.md 1b46e88 _docs/drill-docs/archive/002-meet-drill.md 32d0ba1 _docs/drill-docs/connect/001-plugin-reg.md 88c3c0b _docs/drill-docs/connect/002-mongo-plugin.md 665c79b _docs/drill-docs/connect/003-mapr-db-plugin.md 26d6a9c _docs/drill-docs/connect/workspaces/001-workspaces.md 4ec1f1b _docs/drill-docs/connect/workspaces/002-reg-fs.md 4abb134 _docs/drill-docs/connect/workspaces/003-reg-hbase.md b97de4c _docs/drill-docs/connect/workspaces/004-reg-hive.md c226fd6 _docs/drill-docs/connect/workspaces/005-default-frmt.md 335f415 _docs/drill-docs/contribute/001-guidelines.md 0b24541 _docs/drill-docs/contribute/002-ideas.md ef42adf _docs/drill-docs/datasets/001-aol.md 472f52f _docs/drill-docs/datasets/002-enron.md 2ddbef6 _docs/drill-docs/datasets/003-wikipedia.md 99e6e24 _docs/drill-docs/design/001-plan.md 66147cb _docs/drill-docs/design/002-rpc.md 05cb1d6 _docs/drill-docs/design/003-query-stages.md 5c54249 _docs/drill-docs/design/004-research.md 77be828 _docs/drill-docs/design/005-value.md 0d19a96 _docs/drill-docs/dev-custom-fcn/001-dev-simple.md 47be7d9 _docs/drill-docs/dev-custom-fcn/002-dev-aggregate.md fe6f406
[jira] [Created] (DRILL-2315) Documentation update
Aditya Kishore created DRILL-2315: - Summary: Documentation update Key: DRILL-2315 URL: https://issues.apache.org/jira/browse/DRILL-2315 Project: Apache Drill Issue Type: Task Components: Documentation Reporter: Aditya Kishore Assignee: Kristine Hahn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-2148) Wrong result with grouping on a column of date type with streaming aggregation
[ https://issues.apache.org/jira/browse/DRILL-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victoria Markman resolved DRILL-2148. - Resolution: Fixed Fix Version/s: (was: 0.9.0) 0.8.0 Assignee: Daniel Barclay (Drill) (was: Chris Westin) Wrong result with grouping on a column of date type with streaming aggregation -- Key: DRILL-2148 URL: https://issues.apache.org/jira/browse/DRILL-2148 Project: Apache Drill Issue Type: Bug Components: Execution - Relational Operators Affects Versions: 0.8.0 Reporter: Victoria Markman Assignee: Daniel Barclay (Drill) Priority: Critical Fix For: 0.8.0 Attachments: 0_0_0.parquet, t1.out, t1.res Disable hash aggregation and run query below: {code} alter system set `planner.enable_hashagg` = false; select c_date, COUNT(*) fromt1 group by c_date order by c_date; {code} You will get wrong result. Because NULLs are sorted in the middle ( see DRILL-2084 ) they are folded in one of the non related groups. We might have the same problem with the merge join on date, time and timestamp columns. Attached is a parquet file that was used in this query. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 31419: DRILL-1378: Ctrl-C to cancel a query that has not returned with the first result set
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31419/#review74178 --- Ship it! Ship It! - Mehant Baid On Feb. 25, 2015, 6:38 p.m., Parth Chandra wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31419/ --- (Updated Feb. 25, 2015, 6:38 p.m.) Review request for drill, Daniel Barclay, Hanifi Gunes, and Mehant Baid. Repository: drill-git Description --- Sqlline hangs when you try to cancel the query before the first result set has been returned. Diffs - exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillResultSet.java 88a6c6d Diff: https://reviews.apache.org/r/31419/diff/ Testing --- Tested with a query that takes a long time to return the first result set. Ctrl-C to exit the query. This fix simply addresses the hang. It does not fix Avatica to display cancellation in a nicer manner. Also, have not verified that the query cancellation occurs on the drillbit in all cases (waiting for DRILL-2245) Thanks, Parth Chandra
Build failed in Jenkins: drill-scm #239
See https://builds.apache.org/job/drill-scm/239/changes Changes: [hsuanyi] DRILL-1325: Throw UnsupportedRelOperatorException for unequal joins, implicit cross joins -- [...truncated 1215 lines...] [INFO] Exclude: **/*.linux [INFO] Exclude: **/client/build/** [INFO] Exclude: **/*.tbl [INFO] 1305 resources included (use -debug for more details) Warning: org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser: Property 'http://www.oracle.com/xml/jaxp/properties/entityExpansionLimit' is not recognized. Compiler warnings: WARNING: 'org.apache.xerces.jaxp.SAXParserImpl: Property 'http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized.' Warning: org.apache.xerces.parsers.SAXParser: Feature 'http://javax.xml.XMLConstants/feature/secure-processing' is not recognized. Warning: org.apache.xerces.parsers.SAXParser: Property 'http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized. Warning: org.apache.xerces.parsers.SAXParser: Property 'http://www.oracle.com/xml/jaxp/properties/entityExpansionLimit' is not recognized. [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 1304 licence. [INFO] [INFO] --- maven-checkstyle-plugin:2.12.1:check (checkstyle-validation) @ drill-java-exec --- [INFO] Starting audit... Audit done. [INFO] [INFO] --- maven-enforcer-plugin:1.3.1:enforce (avoid_bad_dependencies) @ drill-java-exec --- [INFO] [INFO] --- maven-resources-plugin:2.6:copy-resources (copy-fmpp-resources) @ drill-java-exec --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 103 resources [INFO] [INFO] --- git-commit-id-plugin:2.1.9:revision (for-jars) @ drill-java-exec --- [info] dotGitDirectory https://builds.apache.org/job/drill-scm/ws/.git [info] git.build.user.name Unknown [info] git.build.user.email Unknown [info] Using environment variable based branch name. GIT_BRANCH = origin/master [info] git.branch origin/master [info] --always = false [info] --dirty = -dirty [info] --abbrev = 7 [info] --long = %s true [info] --match = [info] Tag refs [ [Ref[refs/tags/0.6.0-incubating=3daef7f553431c5aa76ff3074a8e00943de6dedd], Ref[refs/tags/drill-1.0.0-m1=04020a8fca8b287874528d86dc7b8be0269ad788], Ref[refs/tags/drill-root-1.0.0-m1=ad638d9e41aa9efdb1e877cfe7e0a4b910f539fc], Ref[refs/tags/oscon_workshop=eaf95ed3c30d7bb147afe337e0e0477be6518d90], Ref[refs/tags/pre_exec_merge=a97a22b0a9547f8639e92258c0a3475b01742f15]] ] [info] Resolved tag [ 0.6.0-incubating ] [ PersonIdent[Steven Phillips, sphill...@maprtech.com, Thu Oct 2 02:27:36 2014 -0700] ], points at [ commit 5fa257b38df77dedf5609952e364ce0cfc7383d2 0 -- ] [info] Resolved tag [ drill-1.0.0-m1 ] [ PersonIdent[Jacques Nadeau, jacq...@apache.org, Fri Sep 6 13:05:42 2013 -0700] ], points at [ commit a0d3c6977820516983142c96d7f9374681529968 0 -- ] [info] Resolved tag [ drill-root-1.0.0-m1 ] [ PersonIdent[Jacques Nadeau, jacq...@apache.org, Wed Sep 4 04:23:47 2013 -0700] ], points at [ commit 41c18197e3b8ae3c42d55089d641e9a0b68c6f29 0 -- ] [info] Resolved tag [ pre_exec_merge ] [ PersonIdent[Jacques Nadeau, jacq...@apache.org, Fri Jul 19 18:33:56 2013 -0700] ], points at [ commit 5052b64d9953857575f8f40995b8da05160e5457 0 -- ] [info] key [ commit 41c18197e3b8ae3c42d55089d641e9a0b68c6f29 0 -- ], tags = [ [DatedRevTag{id=ad638d9e41aa9efdb1e877cfe7e0a4b910f539fc, tagName='drill-root-1.0.0-m1', date=September 4, 2013 11:23:47 AM UTC}] ] [info] key [ commit 5fa257b38df77dedf5609952e364ce0cfc7383d2 0 -- ], tags = [ [DatedRevTag{id=3daef7f553431c5aa76ff3074a8e00943de6dedd, tagName='0.6.0-incubating', date=October 2, 2014 9:27:36 AM UTC}] ] [info] key [ commit 5052b64d9953857575f8f40995b8da05160e5457 0 -- ], tags = [ [DatedRevTag{id=a97a22b0a9547f8639e92258c0a3475b01742f15, tagName='pre_exec_merge', date=July 20, 2013 1:33:56 AM UTC}] ] [info] key [ commit a0d3c6977820516983142c96d7f9374681529968 0 -- ], tags = [ [DatedRevTag{id=04020a8fca8b287874528d86dc7b8be0269ad788, tagName='drill-1.0.0-m1', date=September 6, 2013 8:05:42 PM UTC}] ] [info] Created map: [ {commit 41c18197e3b8ae3c42d55089d641e9a0b68c6f29 0 --=[drill-root-1.0.0-m1], commit 5fa257b38df77dedf5609952e364ce0cfc7383d2 0 --=[0.6.0-incubating], commit 5052b64d9953857575f8f40995b8da05160e5457 0 --=[pre_exec_merge], commit a0d3c6977820516983142c96d7f9374681529968 0 --=[drill-1.0.0-m1]} ] [info] HEAD is [ d72d6030ed3961a5e4fa8839b4be5ec1065f4059 ] [info] Repo is in dirty state [ false ] [info] git.commit.id.describe drill-1.0.0-m1-1145-gd72d603 [info] git.commit.id d72d6030ed3961a5e4fa8839b4be5ec1065f4059 [info] git.commit.id.abbrev d72d603 [info] git.commit.user.name Hsuan-Yi Chu [info] git.commit.user.email hsua...@usc.edu [info] git.commit.message.full DRILL-1325: Throw UnsupportedRelOperatorException for unequal joins, implicit cross joins [info] git.commit.message.short DRILL-1325: Throw
[jira] [Created] (DRILL-2319) JDBC DrillStatementRegistry unimplemented; Statements leak, throw errors
Daniel Barclay (Drill) created DRILL-2319: - Summary: JDBC DrillStatementRegistry unimplemented; Statements leak, throw errors Key: DRILL-2319 URL: https://issues.apache.org/jira/browse/DRILL-2319 Project: Apache Drill Issue Type: Bug Components: Client - JDBC Reporter: Daniel Barclay (Drill) Assignee: Daniel Barclay (Drill) The JDBC driver's DrillStatementRegistry's methods don't do anything, so open statements are not tracked, so closing the connection doesn't close any still-open Statements and ResultSets, so resources are leaked and closing the connection results in an exception. (Client-side workaround: Close each created statement.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2320) ResultSet.next() after ResultSet.close() throws NullPointerException (not SQLException)
Daniel Barclay (Drill) created DRILL-2320: - Summary: ResultSet.next() after ResultSet.close() throws NullPointerException (not SQLException) Key: DRILL-2320 URL: https://issues.apache.org/jira/browse/DRILL-2320 Project: Apache Drill Issue Type: Bug Components: Client - JDBC Reporter: Daniel Barclay (Drill) Assignee: Daniel Barclay (Drill) After close() is called on a ResultSet, calling next() on that ResultSet throws a NullPointerException (rather than properly throwing a SQLException). (This pattern seems likely to apply to many siblings of next() (e.g., getXxx(...) methods) also.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 31107: Ability to make PartitionSender multithreaded - useful in case of LocalExchange being enabled, as it allows to deal with high volume of incoming data
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31107/ --- (Updated Feb. 25, 2015, 11:24 p.m.) Review request for drill, Chris Westin, Jacques Nadeau, Steven Phillips, and Venki Korukanti. Changes --- Changes based on latest code review with Jacques and Venki 1. Reuse of ExecutorService from WorkManager 2. Not using anonymous objects 3. Not using callables in favor of runnables 4. other small corrections Bugs: DRILL-2210 https://issues.apache.org/jira/browse/DRILL-2210 Repository: drill-git Description --- In addition to description Fixed few classes that did not handle multithreading well Added/Changed some Stats behavior to allow stats merge from multiple threads, since again this class is not suitable to be used in multithreaded environment Introduced new decorator class to handle multi thrteading (or not) to minimize changes to ParitionSenderRootExec class Diffs (updated) - exec/java-exec/src/main/java/org/apache/drill/exec/compile/CodeCompiler.java 7cc350e exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java e413921 exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorStats.java 0e9da0e exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SendingAccountor.java 7af7b65 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java f09acaa exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/Partitioner.java 5ed9c39 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerDecorator.java PRE-CREATION exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerTemplate.java 4292c09 exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PlannerSettings.java faa8546 exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java 83a89df exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java aa0a5ad exec/java-exec/src/main/java/org/apache/drill/exec/work/WorkManager.java 99c6ab8 exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java 478 Diff: https://reviews.apache.org/r/31107/diff/ Testing --- Still need to provide Unit Tests. Functional tests are passing Performance tests were run and look promising for some queries Thanks, Yuliya Feldman
Re: Review Request 31399: DRILL-2130: Fixed JUnit/Hamcrest/Mockito/Paranamer class path problem.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31399/#review73993 --- exec/java-exec/pom.xml https://reviews.apache.org/r/31399/#comment120430 Why not put this in `drill-root` pom? `jmockit` and `junit` are already there. You'll only need to add `mockito-core`. - Aditya Kishore On Feb. 24, 2015, 10:52 p.m., Daniel Barclay wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31399/ --- (Updated Feb. 24, 2015, 10:52 p.m.) Review request for drill, Aditya Kishore and Jason Altekruse. Repository: drill-git Description --- Main change: In exec/java-exec/pom.xml, suppressed Paranamer's transitive dependency on Mockito and the Hamcrest files bundled in it, and put JUnit and thereby Hamcrest on classpath before Mockito. NOTE: Please review placement (and details) of the added dependency.../ elements. The current placement seems to work, but probably doesn't follow our intendent Maven patterns. Other change: Added tests to detect regression (or confirm other solutions to problem). Diffs - common/src/test/java/org/apache/drill/test/Drill2130CommonHamcrestConfigurationTest.java PRE-CREATION contrib/storage-hbase/src/test/java/org/apache/drill/hbase/test/Drill2130StorageHBaseHamcrestConfigurationTest.java PRE-CREATION contrib/storage-hive/core/src/test/java/org/apache/drill/exec/test/Drill2130StorageHiveCoreHamcrestConfigurationTest.java PRE-CREATION exec/interpreter/src/test/java/org/apache/drill/exec/expr/test/Drill2130InterpreterHamcrestConfigurationTest.java PRE-CREATION exec/java-exec/pom.xml 06f60fb exec/java-exec/src/test/java/org/apache/drill/exec/test/Drill2130JavaExecHamcrestConfigurationTest.java PRE-CREATION exec/jdbc/src/test/java/org/apache/drill/jdbc/test/Drill2130JavaJdbcHamcrestConfigurationTest.java PRE-CREATION Diff: https://reviews.apache.org/r/31399/diff/ Testing --- Ran new and existing tests. Thanks, Daniel Barclay
Review Request 31409: DRILL-1690: Issue with using HBase plugin to access row_key only
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/31409/ --- Review request for drill and Steven Phillips. Bugs: DRILL-1690 https://issues.apache.org/jira/browse/DRILL-1690 Repository: drill-git Description --- Fixed by not projecting other columns if only row_key was requeted. Diffs - contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java da38707 Diff: https://reviews.apache.org/r/31409/diff/ Testing --- All HBase unit tests. Thanks, Aditya Kishore