[jira] [Closed] (DRILL-8270) Delete obsolete zookeeper patch (tech debt)
[ https://issues.apache.org/jira/browse/DRILL-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8270. --- > Delete obsolete zookeeper patch (tech debt) > --- > > Key: DRILL-8270 > URL: https://issues.apache.org/jira/browse/DRILL-8270 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.20.1 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Minor > Fix For: 2.0.0 > > > Patch files are in the `.gitignore` and yet a .patch file > ([contrib/native/client/patches/zookeeper-3.4.6-x64.patch|https://github.com/apache/drill/pull/2585/files/06625708f0419442d823d0025afa6e043fffcc4e#diff-0b6d0330fc567658b83263c83e902ec72dc0e95bb0ad0830736dc5cae8449168]) > somehow has been included in the Drill build. This PR removes it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (DRILL-8270) Delete obsolete zookeeper patch (tech debt)
[ https://issues.apache.org/jira/browse/DRILL-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton resolved DRILL-8270. - Resolution: Fixed > Delete obsolete zookeeper patch (tech debt) > --- > > Key: DRILL-8270 > URL: https://issues.apache.org/jira/browse/DRILL-8270 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.20.1 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Minor > Fix For: 2.0.0 > > > Patch files are in the `.gitignore` and yet a .patch file > ([contrib/native/client/patches/zookeeper-3.4.6-x64.patch|https://github.com/apache/drill/pull/2585/files/06625708f0419442d823d0025afa6e043fffcc4e#diff-0b6d0330fc567658b83263c83e902ec72dc0e95bb0ad0830736dc5cae8449168]) > somehow has been included in the Drill build. This PR removes it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8270) Delete obsolete zookeeper patch (tech debt)
[ https://issues.apache.org/jira/browse/DRILL-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8270: Summary: Delete obsolete zookeeper patch (tech debt) (was: Delete absolete zookeeper patch (tech debt)) > Delete obsolete zookeeper patch (tech debt) > --- > > Key: DRILL-8270 > URL: https://issues.apache.org/jira/browse/DRILL-8270 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.20.1 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Minor > Fix For: 2.0.0 > > > Patch files are in the `.gitignore` and yet a .patch file > ([contrib/native/client/patches/zookeeper-3.4.6-x64.patch|https://github.com/apache/drill/pull/2585/files/06625708f0419442d823d0025afa6e043fffcc4e#diff-0b6d0330fc567658b83263c83e902ec72dc0e95bb0ad0830736dc5cae8449168]) > somehow has been included in the Drill build. This PR removes it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8269) Classified metrics for easier maintenance
[ https://issues.apache.org/jira/browse/DRILL-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8269: Affects Version/s: 1.20.1 > Classified metrics for easier maintenance > - > > Key: DRILL-8269 > URL: https://issues.apache.org/jira/browse/DRILL-8269 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Priority: Major > > 1. Add classified prefix for metrics parameters like thread, heap and etc, > for easier Classified metrics parameters for easier maintenance. > 2. Optimized UI. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8269) Classified metrics for easier maintenance
[ https://issues.apache.org/jira/browse/DRILL-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8269. --- Resolution: Fixed > Classified metrics for easier maintenance > - > > Key: DRILL-8269 > URL: https://issues.apache.org/jira/browse/DRILL-8269 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Priority: Major > Fix For: 2.0.0 > > > 1. Add classified prefix for metrics parameters like thread, heap and etc, > for easier Classified metrics parameters for easier maintenance. > 2. Optimized UI. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8269) Classified metrics for easier maintenance
[ https://issues.apache.org/jira/browse/DRILL-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8269: Fix Version/s: 2.0.0 > Classified metrics for easier maintenance > - > > Key: DRILL-8269 > URL: https://issues.apache.org/jira/browse/DRILL-8269 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: Jingchuan Hu >Priority: Major > Fix For: 2.0.0 > > > 1. Add classified prefix for metrics parameters like thread, heap and etc, > for easier Classified metrics parameters for easier maintenance. > 2. Optimized UI. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8266) Fix LGTM implicit narrowing conversion warnings
[ https://issues.apache.org/jira/browse/DRILL-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8266: Affects Version/s: 1.20.1 > Fix LGTM implicit narrowing conversion warnings > --- > > Key: DRILL-8266 > URL: https://issues.apache.org/jira/browse/DRILL-8266 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > > https://github.com/apache/drill/security/code-scanning -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8265) Upgrade aws-java-sdk-s3 to 1.12.261
[ https://issues.apache.org/jira/browse/DRILL-8265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8265: Fix Version/s: 1.20.2 > Upgrade aws-java-sdk-s3 to 1.12.261 > --- > > Key: DRILL-8265 > URL: https://issues.apache.org/jira/browse/DRILL-8265 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.20.2 > > > https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3/1.12.260 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8266) Fix LGTM implicit narrowing conversion warnings
[ https://issues.apache.org/jira/browse/DRILL-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8266. --- Resolution: Fixed > Fix LGTM implicit narrowing conversion warnings > --- > > Key: DRILL-8266 > URL: https://issues.apache.org/jira/browse/DRILL-8266 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.20.2 > > > https://github.com/apache/drill/security/code-scanning -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8265) Upgrade aws-java-sdk-s3 to 1.12.261
[ https://issues.apache.org/jira/browse/DRILL-8265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8265: Affects Version/s: 1.20.1 > Upgrade aws-java-sdk-s3 to 1.12.261 > --- > > Key: DRILL-8265 > URL: https://issues.apache.org/jira/browse/DRILL-8265 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > > https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3/1.12.260 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8266) Fix LGTM implicit narrowing conversion warnings
[ https://issues.apache.org/jira/browse/DRILL-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8266: Fix Version/s: 1.20.2 > Fix LGTM implicit narrowing conversion warnings > --- > > Key: DRILL-8266 > URL: https://issues.apache.org/jira/browse/DRILL-8266 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.20.2 > > > https://github.com/apache/drill/security/code-scanning -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8265) Upgrade aws-java-sdk-s3 to 1.12.261
[ https://issues.apache.org/jira/browse/DRILL-8265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8265. --- Resolution: Fixed > Upgrade aws-java-sdk-s3 to 1.12.261 > --- > > Key: DRILL-8265 > URL: https://issues.apache.org/jira/browse/DRILL-8265 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.20.2 > > > https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3/1.12.260 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8264) Upgrade joda-time to 2.10.14
[ https://issues.apache.org/jira/browse/DRILL-8264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8264: Fix Version/s: 1.20.2 > Upgrade joda-time to 2.10.14 > > > Key: DRILL-8264 > URL: https://issues.apache.org/jira/browse/DRILL-8264 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.20.2 > > > A bug in joda-time pom causes this: > https://github.com/apache/drill/security/code-scanning/27 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8264) Upgrade joda-time to 2.10.14
[ https://issues.apache.org/jira/browse/DRILL-8264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8264. --- Resolution: Fixed > Upgrade joda-time to 2.10.14 > > > Key: DRILL-8264 > URL: https://issues.apache.org/jira/browse/DRILL-8264 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.20.2 > > > A bug in joda-time pom causes this: > https://github.com/apache/drill/security/code-scanning/27 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8264) Upgrade joda-time to 2.10.14
[ https://issues.apache.org/jira/browse/DRILL-8264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8264: Affects Version/s: 1.20.1 > Upgrade joda-time to 2.10.14 > > > Key: DRILL-8264 > URL: https://issues.apache.org/jira/browse/DRILL-8264 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > > A bug in joda-time pom causes this: > https://github.com/apache/drill/security/code-scanning/27 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8263) Upgrade libpam4j to 1.11
[ https://issues.apache.org/jira/browse/DRILL-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8263: Affects Version/s: 1.20.1 > Upgrade libpam4j to 1.11 > > > Key: DRILL-8263 > URL: https://issues.apache.org/jira/browse/DRILL-8263 > Project: Apache Drill > Issue Type: Improvement > Components: Execution - Data Types >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > > https://github.com/apache/drill/blob/master/exec/java-exec/pom.xml#L32 > See dependency with CVE in: > https://mvnrepository.com/artifact/org.apache.drill.exec/drill-java-exec/1.20.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8263) Upgrade libpam4j to 1.11
[ https://issues.apache.org/jira/browse/DRILL-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8263: Fix Version/s: 1.20.2 > Upgrade libpam4j to 1.11 > > > Key: DRILL-8263 > URL: https://issues.apache.org/jira/browse/DRILL-8263 > Project: Apache Drill > Issue Type: Improvement > Components: Execution - Data Types >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.20.2 > > > https://github.com/apache/drill/blob/master/exec/java-exec/pom.xml#L32 > See dependency with CVE in: > https://mvnrepository.com/artifact/org.apache.drill.exec/drill-java-exec/1.20.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8262) Drop dependency on Xalan which is EOL
[ https://issues.apache.org/jira/browse/DRILL-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8262. --- Resolution: Fixed > Drop dependency on Xalan which is EOL > - > > Key: DRILL-8262 > URL: https://issues.apache.org/jira/browse/DRILL-8262 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.20.2 > > > Xalan is no longer supported. > https://lists.apache.org/thread/s8kjny5270ssfcp46v0fl39lk98987w7 > It is better to use JAXP TransformerFactory than using xalan directly. If you > add xalan dependency just to ensure that you have a JAXP compliant > transformer on the classpath, this is unnecessary - the Java runtime has a > built-in implementation. > Drill dependency: > https://mvnrepository.com/artifact/org.apache.drill.exec/drill-java-exec/1.20.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8262) Drop dependency on Xalan which is EOL
[ https://issues.apache.org/jira/browse/DRILL-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8262: Affects Version/s: 1.20.1 > Drop dependency on Xalan which is EOL > - > > Key: DRILL-8262 > URL: https://issues.apache.org/jira/browse/DRILL-8262 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > > Xalan is no longer supported. > https://lists.apache.org/thread/s8kjny5270ssfcp46v0fl39lk98987w7 > It is better to use JAXP TransformerFactory than using xalan directly. If you > add xalan dependency just to ensure that you have a JAXP compliant > transformer on the classpath, this is unnecessary - the Java runtime has a > built-in implementation. > Drill dependency: > https://mvnrepository.com/artifact/org.apache.drill.exec/drill-java-exec/1.20.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8262) Drop dependency on Xalan which is EOL
[ https://issues.apache.org/jira/browse/DRILL-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8262: Fix Version/s: 1.20.2 > Drop dependency on Xalan which is EOL > - > > Key: DRILL-8262 > URL: https://issues.apache.org/jira/browse/DRILL-8262 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.20.2 > > > Xalan is no longer supported. > https://lists.apache.org/thread/s8kjny5270ssfcp46v0fl39lk98987w7 > It is better to use JAXP TransformerFactory than using xalan directly. If you > add xalan dependency just to ensure that you have a JAXP compliant > transformer on the classpath, this is unnecessary - the Java runtime has a > built-in implementation. > Drill dependency: > https://mvnrepository.com/artifact/org.apache.drill.exec/drill-java-exec/1.20.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8263) Upgrade libpam4j to 1.11
[ https://issues.apache.org/jira/browse/DRILL-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8263. --- Resolution: Fixed > Upgrade libpam4j to 1.11 > > > Key: DRILL-8263 > URL: https://issues.apache.org/jira/browse/DRILL-8263 > Project: Apache Drill > Issue Type: Improvement > Components: Execution - Data Types >Affects Versions: 1.20.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.20.2 > > > https://github.com/apache/drill/blob/master/exec/java-exec/pom.xml#L32 > See dependency with CVE in: > https://mvnrepository.com/artifact/org.apache.drill.exec/drill-java-exec/1.20.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-7862) Upgrade excel-streaming-reader to 3.0.3
[ https://issues.apache.org/jira/browse/DRILL-7862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-7862. --- Resolution: Fixed > Upgrade excel-streaming-reader to 3.0.3 > --- > > Key: DRILL-7862 > URL: https://issues.apache.org/jira/browse/DRILL-7862 > Project: Apache Drill > Issue Type: Improvement > Components: Execution - Data Types >Affects Versions: 1.18.0 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.20.0 > > > [excel-streaming-reader](https://github.com/pjfanning/excel-streaming-reader) > is used by Drill. > > v3.0.3 builds on the version added in DRILL-7846 - with some fixes and also > support for xlsx files in OOXML Strict format. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-7862) Upgrade excel-streaming-reader to 3.0.3
[ https://issues.apache.org/jira/browse/DRILL-7862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-7862: Fix Version/s: 1.20.0 > Upgrade excel-streaming-reader to 3.0.3 > --- > > Key: DRILL-7862 > URL: https://issues.apache.org/jira/browse/DRILL-7862 > Project: Apache Drill > Issue Type: Improvement > Components: Execution - Data Types >Affects Versions: 1.18.0 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.20.0 > > > [excel-streaming-reader](https://github.com/pjfanning/excel-streaming-reader) > is used by Drill. > > v3.0.3 builds on the version added in DRILL-7846 - with some fixes and also > support for xlsx files in OOXML Strict format. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-7862) Upgrade excel-streaming-reader to 3.0.3
[ https://issues.apache.org/jira/browse/DRILL-7862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-7862: Summary: Upgrade excel-streaming-reader to 3.0.3 (was: uptake excel-streaming-reader 3.0.3) > Upgrade excel-streaming-reader to 3.0.3 > --- > > Key: DRILL-7862 > URL: https://issues.apache.org/jira/browse/DRILL-7862 > Project: Apache Drill > Issue Type: Improvement > Components: Execution - Data Types >Affects Versions: 1.18.0 >Reporter: PJ Fanning >Priority: Major > > [excel-streaming-reader](https://github.com/pjfanning/excel-streaming-reader) > is used by Drill. > > v3.0.3 builds on the version added in DRILL-7846 - with some fixes and also > support for xlsx files in OOXML Strict format. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-7775) drill won't load. The last thing it shows is the tagline.
[ https://issues.apache.org/jira/browse/DRILL-7775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-7775. --- Resolution: Not A Problem > drill won't load. The last thing it shows is the tagline. > > > Key: DRILL-7775 > URL: https://issues.apache.org/jira/browse/DRILL-7775 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill >Affects Versions: 1.17.0 > Environment: Mac catalina > >Reporter: Kun Deng >Priority: Major > > Hi, > New to drill but I can't get it to run on my mac. > It seems to start correctly when I run drill-embedded but it won't show the > prompt for me. > I can press and type but there is no response and the only way to quit is > typing ctrl-C 3 times. > 1.17.0 xxx$ bin/drill-embedded > Apache Drill 1.17.0 > "Data is the new oil. Ready to Drill some?" > > Why? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-7770) Query on Parquet SNAPPY and GZIP failed but succeeded using UNCOMPRESSED data
[ https://issues.apache.org/jira/browse/DRILL-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572399#comment-17572399 ] James Turton commented on DRILL-7770: - Hi [~ruffacto] are you able to test this again using Drill 1.20? There have been some changes to the Parquet de/compression code. > Query on Parquet SNAPPY and GZIP failed but succeeded using UNCOMPRESSED data > - > > Key: DRILL-7770 > URL: https://issues.apache.org/jira/browse/DRILL-7770 > Project: Apache Drill > Issue Type: Bug > Components: Storage - Parquet >Affects Versions: 1.17.0 > Environment: Replicated test in OSX 10.15.4 and Debian 10 >Reporter: Ruben Hernandez >Priority: Major > Attachments: sqlline_JIRA.log > > > I have a deeply nested Parquet file with ~1000 columns created using > avro-parquet java library. > > Data with *CompressionCodec.UNCOMPRESSED*: queries work great!! > Data with *CompressionCodec.SNAPPY*: fails > Data with *CompressionCodec.GZIP*: fails > > Attached log for SNAPPY > > [^sqlline_JIRA.log] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-5982) CTAS creates parquet files with inconsistent nullable column
[ https://issues.apache.org/jira/browse/DRILL-5982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572416#comment-17572416 ] James Turton commented on DRILL-5982: - Running this test with the UNION type enabled also fails but with a different error. CC [~vitalii]. {code:java} Caused by: java.lang.ClassCastException: class org.apache.drill.exec.vector.IntVector cannot be cast to class org.apache.drill.exec.vector.NullableIntVector (org.apache.drill.exec.vector.IntVector and org.apache.drill.exec.vector.NullableIntVector are in unnamed module of loader 'app') at org.apache.drill.exec.vector.complex.UnionVector.getIntVector(UnionVector.java:313) at org.apache.drill.exec.vector.complex.UnionVector.getMember(UnionVector.java:472) at org.apache.drill.exec.vector.complex.UnionVector$Mutator.setValueCount(UnionVector.java:885) at org.apache.drill.exec.vector.complex.UnionVector.setFirstType(UnionVector.java:656) at org.apache.drill.exec.record.SchemaUtil.coerceVector(SchemaUtil.java:122) at org.apache.drill.exec.record.SchemaUtil.coerceContainer(SchemaUtil.java:178) at org.apache.drill.exec.physical.impl.xsort.BufferedBatches.convertBatch(BufferedBatches.java:121) at org.apache.drill.exec.physical.impl.xsort.BufferedBatches.add(BufferedBatches.java:88) at org.apache.drill.exec.physical.impl.xsort.SortImpl.addBatch(SortImpl.java:309) at org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.loadBatch(ExternalSortBatch.java:455) at org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.load(ExternalSortBatch.java:400) at org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:355) at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:160) {code} > CTAS creates parquet files with inconsistent nullable column > > > Key: DRILL-5982 > URL: https://issues.apache.org/jira/browse/DRILL-5982 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.11.0 > Environment: windows 10 >Reporter: Raymond Wong >Priority: Major > > Create two CTAS parquet files. One CTAS statement uses a MySQL as data > source. The other one uses {{(Values(1))}} as data source. Both files have > the same schema - same column names and data type. > The Parquet file created with MySQL data source has nullable columns and the > file created with {{(Values(1))}} has non-nullable columns. > {quote} > DROP TABLE dfs.tmp.table1; > CREATE TABLE dfs.tmp.table1 AS > SELECT 'CA' AS state, CAST(1 AS BIGINT) AS id > FROM `mysql_dw_reporting.datawarehouse1`.DW_Qualbe_Cust_And_CustPay > LIMIT 1; > DROP TABLE dfs.tmp.table2; > CREATE TABLE dfs.tmp.table2 AS > SELECT 'NY' AS state, CAST(2 AS BIGINT) AS id > FROM (Values(1)) > ; > {quote} > The result of this inconsistency impacts the ability to apply SQL window > function across parquet tables. Querying table1 and table2 with a SQL window > function generates an error message as follows > {quote} > SELECT id, FIRST_VALUE(state) OVER( PARTITION BY id ) AS state > FROM dfs.tmp.`table*` > SQL Error: UNSUPPORTED_OPERATION ERROR: Sort doesn't currently support sorts > with changing schemas > {quote} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-7080) Wrong result schema for wildcard and partition columns
[ https://issues.apache.org/jira/browse/DRILL-7080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572419#comment-17572419 ] James Turton commented on DRILL-7080: - This bug is not limited to the implicit dirN columns. Running {{{}select *, foobar from `myTable`{}}}, where foo does not exist in myTable, will give you a schema of (a, b, c, foobar, foobar0). > Wrong result schema for wildcard and partition columns > -- > > Key: DRILL-7080 > URL: https://issues.apache.org/jira/browse/DRILL-7080 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.15.0 >Reporter: Paul Rogers >Priority: Minor > > Drill supports queries of the form: > {code:sql} > SELECT *, dir0 FROM `myTable` > {code} > Where `myTable` is, say, a set of CSV files with columns "a", "b" and "c". As > shown in the (soon to be submitted) {{TestCsvWithHeaders}} test, behavior of > partition columns is wildly inconsistent and nearly unusable. This ticket > focus on one specific issue: the query above results in a schema like (dir0, > a, b, c, dir00). That is: > * The wildcard generates "dir0", "dir1" columns. > * The Project operator inserts a second column, "dir00" as type Nullable Int. > This behavior is surprising as the following query produces the expected > result: > {code:sql} > SELECT *, filename from `myTable` > {code} > That is, the above produces a schema of the form (a, b, c, filename) with > "filename" of the expected type: VARCHAR. > This appears to be a bug somewhere in the project operator and/or the > planner, but I've not tracked down the root cause. > The workaround is to either: > 1. Not include the "dir0" column explicitly with the wildcard, or > 2. Don't use the wildcard: list columns explicitly, including the partition > columns. > Given how late in the game that this bug is filed, I would guess that few > people actually use this feature. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (DRILL-8273) Complex typed columns cannot be made implicit
[ https://issues.apache.org/jira/browse/DRILL-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton reassigned DRILL-8273: --- Assignee: James Turton > Complex typed columns cannot be made implicit > - > > Key: DRILL-8273 > URL: https://issues.apache.org/jira/browse/DRILL-8273 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.20.1 >Reporter: James Turton >Assignee: James Turton >Priority: Minor > Fix For: 1.20.2 > > > When a plugin returns an implicit column of a complex type, e.g. MAP, and > sets it to be implicit then that column still appears in wildcard queries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (DRILL-8273) Complex typed columns cannot be made implicit
James Turton created DRILL-8273: --- Summary: Complex typed columns cannot be made implicit Key: DRILL-8273 URL: https://issues.apache.org/jira/browse/DRILL-8273 Project: Apache Drill Issue Type: Bug Affects Versions: 1.20.1 Reporter: James Turton Fix For: 1.20.2 When a plugin returns an implicit column of a complex type, e.g. MAP, and sets it to be implicit then that column still appears in wildcard queries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8273) Complex typed columns cannot be made implicit
[ https://issues.apache.org/jira/browse/DRILL-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572507#comment-17572507 ] ASF GitHub Bot commented on DRILL-8273: --- jnturton opened a new pull request, #2615: URL: https://github.com/apache/drill/pull/2615 # [DRILL-8273](https://issues.apache.org/jira/browse/DRILL-8273): Complex typed columns cannot be made implicit ## Description A number of column metadata implementations drop column properties, including the wildcard exclusion property, when cloned with cloneEmpty. This PR makes those cloneEmpty methods copy the column properties too. ## Documentation N/A ## Testing New additions in TestTupleProjection. > Complex typed columns cannot be made implicit > - > > Key: DRILL-8273 > URL: https://issues.apache.org/jira/browse/DRILL-8273 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.20.1 >Reporter: James Turton >Assignee: James Turton >Priority: Minor > Fix For: 1.20.2 > > > When a plugin returns an implicit column of a complex type, e.g. MAP, and > sets it to be implicit then that column still appears in wildcard queries. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8268) Fix Hadoop 2 and Netty lib exclusions, REST mem limiter disabled by default
[ https://issues.apache.org/jira/browse/DRILL-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572574#comment-17572574 ] ASF GitHub Bot commented on DRILL-8268: --- vvysotskyi commented on code in PR #2610: URL: https://github.com/apache/drill/pull/2610#discussion_r932494153 ## exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/sequencefile/SequenceFileBatchReader.java: ## @@ -154,6 +153,19 @@ public boolean next() { @Override public void close() { -AutoCloseables.closeSilently(reader); +try { + // Hadoop 2 compat: {@link org.apache.hadoop.mapred.RecordReader} does not + // support AutoCloseable and must be closed manually. + if (reader != null) { +reader.close(); +reader = null; + } +} catch (IOException e) { + throw UserException Review Comment: `AutoCloseables.closeSilently` doesn't throw any errors. ## exec/jdbc-all/pom.xml: ## @@ -1230,7 +1230,7 @@ hadoop-2 -5040 +5280 Review Comment: Can we exclude some unnecessary dependencies from JDBC Driver to avoid its inflation? > Fix Hadoop 2 and Netty lib exclusions, REST mem limiter disabled by default > --- > > Key: DRILL-8268 > URL: https://issues.apache.org/jira/browse/DRILL-8268 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.20.1 >Reporter: James Turton >Assignee: James Turton >Priority: Major > Fix For: 1.20.2 > > > # New exclusions of reload4j, slf4j-reload4j are required in the Hadoop 2 > profile, probably due to the upgrade of Hadoop from 2.10.1 to 2.10.2. > # We remove the netty-all metapackage which entered the dependency tree with > the change introducing the Netty bom bringing many uneeded libs with it. > # The heap memory usage limiting logic in the REST server becomes disabled > by default since REST query results are streamed these days. This change aims > to let the Java GC now do its job without interference and if that results in > OOM under a constant load then there is good evidence for a heap leak which > must be tracked down and completely resolved anyway, not mitigated or "swept > under a rug". -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8268) Fix Hadoop 2 and Netty lib exclusions, REST mem limiter disabled by default
[ https://issues.apache.org/jira/browse/DRILL-8268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572752#comment-17572752 ] ASF GitHub Bot commented on DRILL-8268: --- jnturton commented on code in PR #2610: URL: https://github.com/apache/drill/pull/2610#discussion_r932878062 ## exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/sequencefile/SequenceFileBatchReader.java: ## @@ -154,6 +153,19 @@ public boolean next() { @Override public void close() { -AutoCloseables.closeSilently(reader); +try { + // Hadoop 2 compat: {@link org.apache.hadoop.mapred.RecordReader} does not + // support AutoCloseable and must be closed manually. + if (reader != null) { +reader.close(); +reader = null; + } +} catch (IOException e) { + throw UserException Review Comment: @vvysotskyi the implementation of close() in this PR restores what was there up until a few months ago when AutoCloseables came in (#2498). So we have only been swallowing errors here since March, we never used to (and the stable branch of Drill still does not swallow errors here since #2498 was not backported). Should I nevertheless change the exception handling logic to swallow errors? > Fix Hadoop 2 and Netty lib exclusions, REST mem limiter disabled by default > --- > > Key: DRILL-8268 > URL: https://issues.apache.org/jira/browse/DRILL-8268 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.20.1 >Reporter: James Turton >Assignee: James Turton >Priority: Major > Fix For: 1.20.2 > > > # New exclusions of reload4j, slf4j-reload4j are required in the Hadoop 2 > profile, probably due to the upgrade of Hadoop from 2.10.1 to 2.10.2. > # We remove the netty-all metapackage which entered the dependency tree with > the change introducing the Netty bom bringing many uneeded libs with it. > # The heap memory usage limiting logic in the REST server becomes disabled > by default since REST query results are streamed these days. This change aims > to let the Java GC now do its job without interference and if that results in > OOM under a constant load then there is good evidence for a heap leak which > must be tracked down and completely resolved anyway, not mitigated or "swept > under a rug". -- This message was sent by Atlassian Jira (v8.20.10#820010)