Review Request 30863: DRILL-2184: Refactor TestFunctionQueries
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30863/ --- Review request for drill and Jason Altekruse. Bugs: DRILL-2184 https://issues.apache.org/jira/browse/DRILL-2184 Repository: drill-git Description --- Patch removes TestFunctionsQuery from being under JDBC to execution and switches to using the new unit test framework. Diffs - exec/java-exec/src/test/java/org/apache/drill/TestFunctionsQuery.java PRE-CREATION exec/jdbc/src/test/java/org/apache/drill/jdbc/test/TestFunctionsQuery.java 881834b Diff: https://reviews.apache.org/r/30863/diff/ Testing --- Thanks, Mehant Baid
Re: Could not find artifact net.hydromatic:optiq-core:jar:0.9-drill-r18
Hey Michael, Is this still happening? Repository where optiq-core resides could be down at the time. http://repository.mapr.com/nexus/content/repositories/drill/net/hydromatic/optiq-core/0.9-drill-r4/ seems working now. Thanks. -Hanifi On Mon, Feb 9, 2015 at 11:34 AM, Michael Spiegel spie...@addthis.com wrote: Hello, I'm trying to build drill from the git repository but I'm running into this error: [ERROR] Failed to execute goal on project drill-common: Could not resolve dependencies for project org.apache.drill:drill-common:jar:0.8.0-SNAPSHOT: Could not find artifact net.hydromatic:optiq-core:jar:0.9-drill-r18 in nexus. Is there something I need to do? Thanks, --Michael
[jira] [Created] (DRILL-2208) Error message must be updated when query contains operations on a flattened column
Abhishek Girish created DRILL-2208:
--
Summary: Error message must be updated when query contains
operations on a flattened column
Key: DRILL-2208
URL: https://issues.apache.org/jira/browse/DRILL-2208
Project: Apache Drill
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Abhishek Girish
Assignee: Jason Altekruse
Currently i observe that if there is a flatten/kvgen operation applied on a
column, no further operations can be performed on the said column unless it is
wrapped inside a nested query.
Consider a simple flatten/kvgen operation on a complex JSON file :
select flatten(kvgen(f.`people`)) as p from `factbook/world.json` f limit 1;
++
| p |
++
| {key:languages,value:{text:Mandarin Chinese 12.44%, Spanish 4.85%,
English 4.83%, Arabic 3.25%, Hindi 2.68%, Bengali 2.66%, Portuguese 2.62%,
Russian 2.12%, Japanese 1.8%, Standard German 1.33%, Javanese 1.25% (2009
est.),note_1:percents are for \first language\ speakers only; the six UN
languages - Arabic, Chinese (Mandarin), English, French, Russian, and Spanish
(Castilian) - are the mother tongue or second language of about half of the
world's population, and are the official languages in more than half the states
in the world; some 150 to 200 languages have more than a million
speakers,note_2:all told, there are an estimated 7,100 languages spoken in
the world; aproximately 80% of these languages are spoken by less than 100,000
people; about 50 languages are spoken by only 1 person; communities that are
isolated from each other in mountainous regions often develop multiple
languages; Papua New Guinea, for example, boasts about 836 separate
languages,note_3:approximately 2,300 languages are spoken in Asia, 2,150,
in Africa, 1,311 in the Pacific, 1,060 in the Americas, and 280 in Europe}} |
| {key:religions,value:{text:Christian 33.39% (of which Roman Catholic
16.85%, Protestant 6.15%, Orthodox 3.96%, Anglican 1.26%), Muslim 22.74%, Hindu
13.8%, Buddhist 6.77%, Sikh 0.35%, Jewish 0.22%, Baha'i 0.11%, other religions
10.95%, non-religious 9.66%, atheists 2.01% (2010 est.)}} |
| {key:population,value:{text:7,095,217,980 (July 2013
est.),top_ten_most_populous_countries_in_millions:China 1,349.59; India
1,220.80; United States 316.67; Indonesia 251.16; Brazil 201.01; Pakistan
193.24; Nigeria 174.51; Bangladesh 163.65; Russia 142.50; Japan 127.25}} |
| {key:age_structure,value:{0_14_years:26% (male 953,496,513/female
890,372,474),15_24_years:16.8% (male 614,574,389/female
579,810,490),25_54_years:40.6% (male 1,454,831,900/female
1,426,721,773),55_64_years:8.4% (male 291,435,881/female
305,185,398),65_years_and_over:8.2% (male 257,035,416/female 321,753,746)
(2013 est.)}} |
| {key:dependency_ratios,value:{total_dependency_ratio:52
%,youth_dependency_ratio:39.9 %,elderly_dependency_ratio:12.1
%,potential_support_ratio:8.3 (2013)}} |
++
*Adding a WHERE clause with conditions on this column fails:*
select flatten(kvgen(f.`people`)) as p from `factbook/world.json` f where
f.p.`key` = 'languages';
Query failed: RemoteRpcException: Failure while running fragment., languages [
686bcd40-c23b-448c-93d8-b98a3b092657 on abhi5.qa.lab:31010 ]
[ 686bcd40-c23b-448c-93d8-b98a3b092657 on abhi5.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
Logs indicate a NumberFormat Exception in the above case.
*And query fails to parse in the below case*
select flatten(kvgen(f.`people`)).`value` as p from `factbook/world.json` f
limit 5;
Query failed: ParseException: Encountered . at line 1, column 34.
Was expecting one of:
FROM ...
, ...
AS ...
OVER ...
Error: exception while executing query: Failure while executing query.
(state=,code=0)
Rewriting using an inner query succeeds:
select g.p.`value`.`note_3` from (select flatten(kvgen(f.`people`)) as p from
`factbook/world.json` f) g where g.p.`key`='languages';
++
| EXPR$0 |
++
| approximately 2,300 languages are spoken in Asia, 2,150, in Africa, 1,311 in
the Pacific, 1,060 in the Americas, and 280 in Europe |
++
In both the failure cases the error message needs to be updated to indicate
that the operation is not supported. The current error message and logs are not
clear for an end user.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 30106: DRILL-1970: Hive views must not be listed with the show tables command
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30106/ --- (Updated Feb. 10, 2015, 6:04 p.m.) Review request for drill. Changes --- updated unit tests to make sure Hive views are not visible when running *SHOW TABLES* or *SELECT * FROM INFORMATION_SCHEMA.VIEWS* Bugs: DRILL-1970 https://issues.apache.org/jira/browse/DRILL-1970 Repository: drill-git Description --- Until Drill can support querying of Hive Views, hive views metadata must not be visible upon issuing the show tables command. Diffs (updated) - contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/schema/HiveSchemaFactory.java 023517b contrib/storage-hive/core/src/test/java/org/apache/drill/exec/store/hive/HiveTestDataGenerator.java 0758791 exec/jdbc/src/test/java/org/apache/drill/jdbc/test/TestMetadataDDL.java c52eafd exec/jdbc/src/test/java/org/apache/drill/jdbc/test/TestViews.java 28fa4a7 Diff: https://reviews.apache.org/r/30106/diff/ Testing --- local and functional tests are running fine, except for some functional tests that are not related to this patch. I also manually tested show tables and confirmed Hive Views are no longer visible. Thanks, abdelhakim deneche
Cancelling hangout this week
Sorry about the last minute notice, but unfortunately I'm going to have to cancel the community hangout today. Yash, I saw both of your messages, which I assume were in anticipation of a meeting today. I will find someone today to review the Cassandra storage plugin. I have some thoughts on the Python UDFs as well, I will put a response on the thread you started. Thanks, Jason
Re: Review Request 27711: To fix DRILL-1062, honoring NULLS FIRST and NULLS LAST
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27711/#review71802 --- exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java https://reviews.apache.org/r/27711/#comment117659 Here, MergeJoin always usesnull_High policy. What if the SORT operator uses null_low? Will it cause incorrect result? Can you try the following query: select X1.C1, X2.C2 from (select C1 from T1 ORDER BY C1 ASC NULLS FIRST) X1 JOIN (select C2 FROM T2 ORDER BY C2 ASC NULLS FIRST) X2 ON X1.C1 = X2.C2; In the above query, SORT will use null_low for Null Direction for both sides of merge join. If merge_join uses null_high, will it return correct query result? - Jinfeng Ni On Feb. 6, 2015, 3:22 p.m., Daniel Barclay wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27711/ --- (Updated Feb. 6, 2015, 3:22 p.m.) Review request for drill, Jinfeng Ni and Mehant Baid. Repository: drill-git Description --- Main change: Augmented *ordering* comparison function templates and calls to to order NULL values correctly (per NULLS FIRST, NULLS LAST, or correct default (depending on whether ordering is ascending or descending) * Cloned each compare_to function template into compare_to_nulls_high and compare_to_nulls_low versions and adjusted to handle NULL correctly. * Added corresponding new version of getComparator(...). * Updated code around calls to getComparator(...) re NULL ordering. * Added test class and test data files. Diffs - .gitignore 838ea6b common/src/main/java/org/apache/drill/common/logical/data/Order.java dada606 exec/java-exec/src/main/codegen/data/CompareTypes.tdd f384d52 exec/java-exec/src/main/codegen/templates/ComparisonFunctions.java 628277c exec/java-exec/src/main/codegen/templates/DateIntervalFunctions.java 8fe13bb exec/java-exec/src/main/codegen/templates/Decimal/CastVarCharDecimal.java 960368a exec/java-exec/src/main/codegen/templates/Decimal/DecimalFunctions.java 0c4af01 exec/java-exec/src/main/java/org/apache/drill/exec/expr/annotations/FunctionTemplate.java 1f732a3 exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/FunctionGenerationHelper.java d007d7c exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/ComparisonFunctions.java bf42ce6 exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/ComparisonFunctionsNullable.java 570aaeb exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java 9829fc6 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggBatch.java 860627d exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java 257b93e exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java d78ba8e exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionRecordBatch.java a062074 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortBatch.java 19f5423 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/StreamingWindowFrameRecordBatch.java 26d23f2 exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java 9026661 exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java 9015a16 exec/jdbc/src/test/java/org/apache/drill/jdbc/test/JdbcNullOrderingAndGroupingTest.java PRE-CREATION exec/jdbc/src/test/java/org/apache/drill/jdbc/test/TestJdbcQuery.java b627c38 exec/jdbc/src/test/resources/donuts.json PRE-CREATION exec/jdbc/src/test/resources/null_ordering_and_grouping_data.json PRE-CREATION pom.xml 17f0e09 Diff: https://reviews.apache.org/r/27711/diff/ Testing --- Ran new fix-specific unit tests. Thanks, Daniel Barclay
[jira] [Resolved] (DRILL-2186) select * is not working with flatten
[
https://issues.apache.org/jira/browse/DRILL-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Hsuan-Yi Chu resolved DRILL-2186.
--
Resolution: Duplicate
select * is not working with flatten
Key: DRILL-2186
URL: https://issues.apache.org/jira/browse/DRILL-2186
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Reporter: Rahul Challapalli
Assignee: Daniel Barclay (Drill/MapR)
git.commit.id.abbrev=c54bd6a
Data Set :
{code}
{
id : 1,
lst : [1,2,3]
}
{code}
The below queries incorrectly return null
{code}
0: jdbc:drill:schema=dfs select *, flatten(lst) from `temp.json`;
+++
| * | EXPR$1 |
+++
| null | 1 |
| null | 2 |
| null | 3 |
+++
{code}
{code}
0: jdbc:drill:schema=dfs select * from `temp.json` order by flatten(lst);
++
| * |
++
| null |
| null |
| null |
++
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2200) Obscure error message from kvgen when the field does not exist in a parquet file
Rahul Challapalli created DRILL-2200:
Summary: Obscure error message from kvgen when the field does not
exist in a parquet file
Key: DRILL-2200
URL: https://issues.apache.org/jira/browse/DRILL-2200
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Reporter: Rahul Challapalli
Assignee: Mehant Baid
Priority: Minor
git.commit.id.abbrev=3d863b5
Json Data Set :
{code}
{
id:1,
map: {
map1:{col1:1, col2:2}
}
}
{code}
Equivalent Parquet File is attached.
The below response properly indicates the reason for failure
{code}
select kvgen(t.map.abcd) from dfs.flatten_operators.`data.json` t;
Query failed: RemoteRpcException: Failure while running fragment., kvgen
function only supports Simple maps as input [
9b441359-9f04-4199-8cca-cdc23aaab517 on qa-node191.qa.lab:31010 ]
[ 9b441359-9f04-4199-8cca-cdc23aaab517 on qa-node191.qa.lab:31010 ]
{code}
However the same query on an equivalent parquet file returns an obscure error
message
{code}
select kvgen(t.map.abcd) from `data.parquet` t;
Query failed: RemoteRpcException: Failure while running fragment., Unable to
find holder type for minorType: LATE [ 9697f4aa-1e8d-4221-8e7f-4d374ffbd916 on
qa-node191.qa.lab:31010 ]
[ 9697f4aa-1e8d-4221-8e7f-4d374ffbd916 on qa-node191.qa.lab:31010 ]
{code}
However if we apply kvgen on a first level non-existent field we get back a
proper error message even for a parquet file
{code}
select kvgen(t.map1) from `data.parquet` t;
Query failed: RemoteRpcException: Failure while running fragment., kvgen
function only supports Simple maps as input [
1d1c4ca3-ceda-4320-8ff8-fe08afe32090 on qa-node191.qa.lab:31010 ]
[ 1d1c4ca3-ceda-4320-8ff8-fe08afe32090 on qa-node191.qa.lab:31010 ]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 29666: DRILL-1757 Add support for wildcards within REPEATED_CONTAINS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29666/#review71766 --- Code changes look ok. Could you please add a couple of unit tests. - Mehant Baid On Jan. 7, 2015, 7:43 p.m., Venkata krishnan Sowrirajan wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29666/ --- (Updated Jan. 7, 2015, 7:43 p.m.) Review request for drill and Mehant Baid. Repository: drill-git Description --- Support for wildcards with in repeated_contains() is implemented similar to MS SQL contains(). MS SQL contains() documentation http://msdn.microsoft.com/en-us/library/ms187787.aspx Diffs - exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/SimpleRepeatedFunctions.java f7f0e91 Diff: https://reviews.apache.org/r/29666/diff/ Testing --- Thanks, Venkata krishnan Sowrirajan
Re: Review Request 30636: DRILL-2143 - part 1 - remove record batch interface from UDF interface
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30636/#review71769 --- Ship it! Changes look ok. There are a few unused imports of RecordBatch class that might need to be removed. - Mehant Baid On Feb. 7, 2015, 12:25 a.m., Jason Altekruse wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30636/ --- (Updated Feb. 7, 2015, 12:25 a.m.) Review request for drill, Jacques Nadeau, Jinfeng Ni, and Mehant Baid. Bugs: DRILL-2143 https://issues.apache.org/jira/browse/DRILL-2143 Repository: drill-git Description --- Remove incoming record batch from setup method of DrillFuncs. Date functions that need the incoming batch have been disbled here, see the other reviewboard link posted on the JIRA for this issue. This patch only has the find/replace in freemarker templates and intellij refactorings of the non-build-time-generated functions. The next patch adds an injectable type to bring back the date functions and make the interpreted expression evaluation work with the new interface. Diffs - exec/java-exec/src/main/codegen/templates/AggrBitwiseLogicalTypeFunctions.java e659d32 exec/java-exec/src/main/codegen/templates/AggrTypeFunctions1.java e4386b2 exec/java-exec/src/main/codegen/templates/AggrTypeFunctions2.java fda1457 exec/java-exec/src/main/codegen/templates/AggrTypeFunctions3.java acf877a exec/java-exec/src/main/codegen/templates/CastDateDate.java b849c76 exec/java-exec/src/main/codegen/templates/CastDateVarChar.java c261e29 exec/java-exec/src/main/codegen/templates/CastFunctions.java 41c1557 exec/java-exec/src/main/codegen/templates/CastFunctionsSrcVarLen.java 57740c9 exec/java-exec/src/main/codegen/templates/CastFunctionsSrcVarLenTargetVarLen.java cd8f7bd exec/java-exec/src/main/codegen/templates/CastFunctionsTargetVarLen.java 455ab13 exec/java-exec/src/main/codegen/templates/CastHigh.java 934b60b exec/java-exec/src/main/codegen/templates/CastIntervalInterval.java e85b724 exec/java-exec/src/main/codegen/templates/CastIntervalVarChar.java 829fe47 exec/java-exec/src/main/codegen/templates/CastVarCharDate.java 41c7855 exec/java-exec/src/main/codegen/templates/CastVarCharInterval.java eaebf02 exec/java-exec/src/main/codegen/templates/CastVarCharToNullableNumeric.java e417f26 exec/java-exec/src/main/codegen/templates/ComparisonFunctions.java 628277c exec/java-exec/src/main/codegen/templates/ConvertToNullableHolder.java 89a400d exec/java-exec/src/main/codegen/templates/CorrelationTypeFunctions.java 19f9c59 exec/java-exec/src/main/codegen/templates/CovarTypeFunctions.java b8131c2 exec/java-exec/src/main/codegen/templates/DateIntervalAggrFunctions1.java 1c9b76f exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/DateDateArithmeticFunctions.java 6c03f3b exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/DateIntervalArithmeticFunctions.java 0b0da52 exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/DateToCharFunctions.java 695cfcf exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/DateTruncFunctions.java b9cd0c8 exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/Extract.java 0858c3f exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/IntervalIntervalArithmetic.java 484c8f4 exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/IntervalNumericArithmetic.java ce73f74 exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/ToDateTypeFunctions.java fffaef4 exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/ToTimeStampFunction.java 4fd619f exec/java-exec/src/main/codegen/templates/DateIntervalFunctions.java 8fe13bb exec/java-exec/src/main/codegen/templates/Decimal/CastDecimalDenseDecimalSparse.java cb0dc58 exec/java-exec/src/main/codegen/templates/Decimal/CastDecimalFloat.java 4d8d1a5 exec/java-exec/src/main/codegen/templates/Decimal/CastDecimalInt.java c435f6d exec/java-exec/src/main/codegen/templates/Decimal/CastDecimalSimilar.java cd2c390 exec/java-exec/src/main/codegen/templates/Decimal/CastDecimalSparseDecimalDense.java 0f8790f exec/java-exec/src/main/codegen/templates/Decimal/CastDecimalVarchar.java ed62444 exec/java-exec/src/main/codegen/templates/Decimal/CastDownwardDecimal.java bd86819 exec/java-exec/src/main/codegen/templates/Decimal/CastFloatDecimal.java c05b2e5
Re: Review Request 30636: DRILL-2143 - part 1 - remove record batch interface from UDF interface
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30636/#review71768 --- - Mehant Baid On Feb. 7, 2015, 12:25 a.m., Jason Altekruse wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30636/ --- (Updated Feb. 7, 2015, 12:25 a.m.) Review request for drill, Jacques Nadeau, Jinfeng Ni, and Mehant Baid. Bugs: DRILL-2143 https://issues.apache.org/jira/browse/DRILL-2143 Repository: drill-git Description --- Remove incoming record batch from setup method of DrillFuncs. Date functions that need the incoming batch have been disbled here, see the other reviewboard link posted on the JIRA for this issue. This patch only has the find/replace in freemarker templates and intellij refactorings of the non-build-time-generated functions. The next patch adds an injectable type to bring back the date functions and make the interpreted expression evaluation work with the new interface. Diffs - exec/java-exec/src/main/codegen/templates/AggrBitwiseLogicalTypeFunctions.java e659d32 exec/java-exec/src/main/codegen/templates/AggrTypeFunctions1.java e4386b2 exec/java-exec/src/main/codegen/templates/AggrTypeFunctions2.java fda1457 exec/java-exec/src/main/codegen/templates/AggrTypeFunctions3.java acf877a exec/java-exec/src/main/codegen/templates/CastDateDate.java b849c76 exec/java-exec/src/main/codegen/templates/CastDateVarChar.java c261e29 exec/java-exec/src/main/codegen/templates/CastFunctions.java 41c1557 exec/java-exec/src/main/codegen/templates/CastFunctionsSrcVarLen.java 57740c9 exec/java-exec/src/main/codegen/templates/CastFunctionsSrcVarLenTargetVarLen.java cd8f7bd exec/java-exec/src/main/codegen/templates/CastFunctionsTargetVarLen.java 455ab13 exec/java-exec/src/main/codegen/templates/CastHigh.java 934b60b exec/java-exec/src/main/codegen/templates/CastIntervalInterval.java e85b724 exec/java-exec/src/main/codegen/templates/CastIntervalVarChar.java 829fe47 exec/java-exec/src/main/codegen/templates/CastVarCharDate.java 41c7855 exec/java-exec/src/main/codegen/templates/CastVarCharInterval.java eaebf02 exec/java-exec/src/main/codegen/templates/CastVarCharToNullableNumeric.java e417f26 exec/java-exec/src/main/codegen/templates/ComparisonFunctions.java 628277c exec/java-exec/src/main/codegen/templates/ConvertToNullableHolder.java 89a400d exec/java-exec/src/main/codegen/templates/CorrelationTypeFunctions.java 19f9c59 exec/java-exec/src/main/codegen/templates/CovarTypeFunctions.java b8131c2 exec/java-exec/src/main/codegen/templates/DateIntervalAggrFunctions1.java 1c9b76f exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/DateDateArithmeticFunctions.java 6c03f3b exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/DateIntervalArithmeticFunctions.java 0b0da52 exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/DateToCharFunctions.java 695cfcf exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/DateTruncFunctions.java b9cd0c8 exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/Extract.java 0858c3f exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/IntervalIntervalArithmetic.java 484c8f4 exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/IntervalNumericArithmetic.java ce73f74 exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/ToDateTypeFunctions.java fffaef4 exec/java-exec/src/main/codegen/templates/DateIntervalFunctionTemplates/ToTimeStampFunction.java 4fd619f exec/java-exec/src/main/codegen/templates/DateIntervalFunctions.java 8fe13bb exec/java-exec/src/main/codegen/templates/Decimal/CastDecimalDenseDecimalSparse.java cb0dc58 exec/java-exec/src/main/codegen/templates/Decimal/CastDecimalFloat.java 4d8d1a5 exec/java-exec/src/main/codegen/templates/Decimal/CastDecimalInt.java c435f6d exec/java-exec/src/main/codegen/templates/Decimal/CastDecimalSimilar.java cd2c390 exec/java-exec/src/main/codegen/templates/Decimal/CastDecimalSparseDecimalDense.java 0f8790f exec/java-exec/src/main/codegen/templates/Decimal/CastDecimalVarchar.java ed62444 exec/java-exec/src/main/codegen/templates/Decimal/CastDownwardDecimal.java bd86819 exec/java-exec/src/main/codegen/templates/Decimal/CastFloatDecimal.java c05b2e5 exec/java-exec/src/main/codegen/templates/Decimal/CastIntDecimal.java 139fff8 exec/java-exec/src/main/codegen/templates/Decimal/CastSrcDecimalSimple.java 7750c62
[jira] [Created] (DRILL-2201) clear error message on join on complex type
Chun Chang created DRILL-2201:
-
Summary: clear error message on join on complex type
Key: DRILL-2201
URL: https://issues.apache.org/jira/browse/DRILL-2201
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Affects Versions: 0.8.0
Reporter: Chun Chang
Assignee: Daniel Barclay (Drill/MapR)
Priority: Minor
#Mon Feb 09 15:58:57 EST 2015
git.commit.id.abbrev=3d863b5
Dataset can be downloaded from
https://s3.amazonaws.com/apache-drill/files/complex.json.gz
We do not support join condition on complex type. But the error message is not
clear to end user.
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select a.id from `complex.json`
a left outer join `complex.json` b on a.oooa=b.oooa;
Query failed: RemoteRpcException: Failure while running fragment., Failure
while trying to materialize incoming schema. Errors:
Error in expression at index 0. Error: Missing function implementation:
[hash(MAP-REQUIRED)]. Full expression: null.. [
6a61d61f-670f-4ddc-bb1d-09a47f49f38e on qa-node120.qa.lab:31010 ]
[ 6a61d61f-670f-4ddc-bb1d-09a47f49f38e on qa-node120.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
Here oooa is a complex type:
{code}
{
oooa: {
oa: {
oab: {
oabc: [
{
rowId: 1
},
{
rowValue1: 1,
rowValue2: 1
}
]
}
}
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2202) Star is no expanded correctly during group by over named subquery
Victoria Markman created DRILL-2202:
---
Summary: Star is no expanded correctly during group by over named
subquery
Key: DRILL-2202
URL: https://issues.apache.org/jira/browse/DRILL-2202
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.8.0
Reporter: Victoria Markman
Assignee: Jinfeng Ni
Attachments: drillbit.log
I gave subquery name and named all the columns.
Group by condition is the same as expanded star ...
Should work.
{code}
0: jdbc:drill:schema=dfs select * from ( select a1, b1, c1 from t1 ) as dt(x1,
x2, x3) group by dt.x1, dt.x2, dt.x3;
Query failed: AssertionError: Internal error: invariant violated: conversion
result not null
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
Workaround: replace star with column names manually:
{code}
0: jdbc:drill:schema=dfs select dt.x1, dt.x2, dt.x3 from ( select a1, b1, c1
from t1 ) as dt(x1, x2, x3) group by dt.x1, dt.x2, dt.x3;
++++
| x1 | x2 | x3 |
++++
| 1 | a | 2015-01-01 |
| 2 | b | 2015-01-02 |
| 3 | c | 2015-01-03 |
| 4 | null | 2015-01-04 |
| 5 | e | 2015-01-05 |
| 6 | f | 2015-01-06 |
| 7 | g | 2015-01-07 |
| null | h | 2015-01-08 |
| 9 | i | null |
| 10 | j | 2015-01-10 |
++++
10 rows selected (0.16 seconds)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2209) Save on CPU cycles by adding Project with column that has hash calculated
Yuliya Feldman created DRILL-2209: - Summary: Save on CPU cycles by adding Project with column that has hash calculated Key: DRILL-2209 URL: https://issues.apache.org/jira/browse/DRILL-2209 Project: Apache Drill Issue Type: Improvement Components: Execution - Flow, Query Planning Optimization Reporter: Yuliya Feldman Assignee: Yuliya Feldman Related to DRILL-133. Wrapping HashToRandomExhcnage and/or LocalExchange with Project operator with additional column that represents hash function of column(s) we are hashing on. This is to save CPU cycles and not recalculate hash every time -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2210) Allow multithreaded copy and/or flush in ParittionSender
Yuliya Feldman created DRILL-2210: - Summary: Allow multithreaded copy and/or flush in ParittionSender Key: DRILL-2210 URL: https://issues.apache.org/jira/browse/DRILL-2210 Project: Apache Drill Issue Type: Improvement Components: Execution - Flow Reporter: Yuliya Feldman Assignee: Yuliya Feldman Related to DRILL-133. As in LocalExchange we merge data from multiple receivers into LocalExchange to fan it out later to multiple Senders, amount of data that needs to be sent out increases. Add ability to copy/flush data in multiple threads -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2203) DISTINCT over UNION ALL subquery with fully qualified column names returns wrong result
Victoria Markman created DRILL-2203:
---
Summary: DISTINCT over UNION ALL subquery with fully qualified
column names returns wrong result
Key: DRILL-2203
URL: https://issues.apache.org/jira/browse/DRILL-2203
Project: Apache Drill
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Victoria Markman
Priority: Critical
{code}
0: jdbc:drill:schema=dfs select a1, b1, c1 from t1 union all select a2, b2, c2
from t2;
++++
| a1 | b1 | c1 |
++++
| 1 | a | 2015-01-01 |
| 2 | b | 2015-01-02 |
| 3 | c | 2015-01-03 |
| 4 | null | 2015-01-04 |
| 5 | e | 2015-01-05 |
| 6 | f | 2015-01-06 |
| 7 | g | 2015-01-07 |
| null | h | 2015-01-08 |
| 9 | i | null |
| 10 | j | 2015-01-10 |
| 0 | zzz| 2014-12-31 |
| 1 | a | 2015-01-01 |
| 2 | b | 2015-01-02 |
| 2 | b | 2015-01-02 |
| 2 | b | 2015-01-02 |
| 3 | c | 2015-01-03 |
| 4 | d | 2015-01-04 |
| 5 | e | 2015-01-05 |
| 6 | f | 2015-01-06 |
| 7 | g | 2015-01-07 |
| 7 | g | 2015-01-07 |
| 8 | h | 2015-01-08 |
| 9 | i | 2015-01-09 |
++++
{code}
Wrong result:
{code}
0: jdbc:drill:schema=dfs select distinct sq.x1, sq.x2, sq.x3 from ( select a1,
b1, c1 from t1 union all select a2, b2, c2 from t2 ) as sq(x1,x2,x3);
++++
| x1 | x2 | x3 |
++++
| null | null | null |
++++
1 row selected (0.127 seconds)
{code}
Query plan:
{code}
00-01 Project(x1=[$0], x2=[$1], x3=[$2])
00-02HashAgg(group=[{0, 1, 2}])
00-03 Project(x1=[$0], x2=[$1], x3=[$2])
00-04UnionAll(all=[true])
00-06 Project(a1=[$2], b1=[$1], c1=[$0])
00-08Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:/aggregation/sanity/t1]],
selectionRoot=/aggregation/sanity/t1, numFiles=1, columns=[`a1`, `b1`, `c1`]]])
00-05 Project(a2=[$1], b2=[$0], c2=[$2])
00-07Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:/aggregation/sanity/t2]],
selectionRoot=/aggregation/sanity/t2, numFiles=1, columns=[`a2`, `b2`, `c2`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2205) REST API should have a consistent way of reporting failure in case a request fails
Hanifi Gunes created DRILL-2205:
---
Summary: REST API should have a consistent way of reporting
failure in case a request fails
Key: DRILL-2205
URL: https://issues.apache.org/jira/browse/DRILL-2205
Project: Apache Drill
Issue Type: Improvement
Components: Client - HTTP
Affects Versions: 0.8.0
Reporter: Hanifi Gunes
Assignee: Jason Altekruse
In general, REST API does not do a good job reporting failures. Making a bogus
HTTP call to create a new storage plugin such like
{code}
POST [drill-http-server-url]/storage/[storage-plugin-name]
{name:bogus}
{code}
returns HTTP OK with no evidence that the former query failed except the string
payload. This is quite a bit of problem for clients that must safely create or
update storage plugin before attempting to execute queries as currently there
is no unified definitive way to tell if a call succeeds w/o checking the
response body. I presume test frameworks heavily rely on this functionality
uploading plugin before each test run.
As part of DRILL-1357, our greater effort to re-design the REST API, we should
make sure to define a consistent error message skeleton that is used across
resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2204) DISTINCT statement over UNION ALL subquery asserts during execution with streaming aggregation
Victoria Markman created DRILL-2204:
---
Summary: DISTINCT statement over UNION ALL subquery asserts during
execution with streaming aggregation
Key: DRILL-2204
URL: https://issues.apache.org/jira/browse/DRILL-2204
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Affects Versions: 0.8.0
Reporter: Victoria Markman
Assignee: Daniel Barclay (Drill/MapR)
{code}
0: jdbc:drill:schema=dfs select distinct sq.x1, sq.x2, sq.x3 from ( select a1,
b1, c1 from t1 union all select a2, b2, c2 from t2 ) as sq(x1,x2,x3);
++++
| x1 | x2 | x3 |
++++
Query failed: RemoteRpcException: Failure while running fragment., Failure
while reading vector. Expected vector class of
org.apache.drill.exec.vector.NullableVarCharVector but was holding vector class
org.apache.drill.exec.vector.NullableIntVector. [
dd2fedd7-bbee-40a4-8a26-9ca86fa774f6 on atsqa4-134.qa.lab:31010 ]
[ dd2fedd7-bbee-40a4-8a26-9ca86fa774f6 on atsqa4-134.qa.lab:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
Plan:
{code}
00-01 Project(x1=[$0], x2=[$1], x3=[$2])
00-02StreamAgg(group=[{0, 1, 2}])
00-03 Sort(sort0=[$0], sort1=[$1], sort2=[$2], dir0=[ASC], dir1=[ASC],
dir2=[ASC])
00-04Project(x1=[$0], x2=[$1], x3=[$2])
00-05 UnionAll(all=[true])
00-07Project(a1=[$2], b1=[$1], c1=[$0])
00-09 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:/aggregation/sanity/t1]],
selectionRoot=/aggregation/sanity/t1, numFiles=1, columns=[`a1`, `b1`, `c1`]]])
00-06Project(a2=[$1], b2=[$0], c2=[$2])
00-08 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:/aggregation/sanity/t2]],
selectionRoot=/aggregation/sanity/t2, numFiles=1, columns=[`a2`, `b2`, `c2`]]])
{code}
Query works if columns in the union query have the same name.
{code}
0: jdbc:drill:schema=dfs select distinct sq.x1, sq.x2, sq.x3 from ( select a1,
b1, c1 from t1 union all select a1, b1, c1 from t4 ) as sq(x1,x2,x3);
++++
| x1 | x2 | x3 |
++++
| 1 | a | 2015-01-01 |
| 2 | b | 2015-01-02 |
| 3 | c | 2015-01-03 |
| 4 | null | 2015-01-04 |
| 5 | e | 2015-01-05 |
| 6 | f | 2015-01-06 |
| 7 | g | 2015-01-07 |
| 9 | i | null |
| 10 | j | 2015-01-10 |
| null | h | 2015-01-08 |
++++
10 rows selected (0.131 seconds)
{code}
It's possible, that bug is caused by drill-2203, but I'm filing it anyway,
because the way it fails it is different and will need to be verified after it
is fixed. Tables for the query are attached in drill-2203
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2206) Error message must be updated when querying a JSON file with arrays as first level
Abhishek Girish created DRILL-2206:
--
Summary: Error message must be updated when querying a JSON file
with arrays as first level
Key: DRILL-2206
URL: https://issues.apache.org/jira/browse/DRILL-2206
Project: Apache Drill
Issue Type: Bug
Components: Storage - JSON
Reporter: Abhishek Girish
Assignee: Steven Phillips
Structure of the file:
[
{
key: value,
...
},
{
...
}
]
*Valid Error Message:*
select * from `file.json`;
Query failed: Query stopped., Drill doesn't support objects whose first level
is a scalar or array. Objects must start as maps. [
1dc6dfc2-77ed-4a22-8ef0-e5a31c24f6fb on abhi8.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
*Error message to be updated:*
select count(*) from `file.json`;
++
| EXPR$0 |
++
Query failed: RemoteRpcException: Failure while running fragment., You tried to
do a batch data read operation when you were in a state of STOP. You can only
do this type of operation when you are in a state of OK or OK_NEW_SCHEMA. [
08135c32-fbb8-445e-8c0e-05292e6613e2 on abhi8.qa.lab:31010 ]
[ 08135c32-fbb8-445e-8c0e-05292e6613e2 on abhi8.qa.lab:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
