[jira] [Resolved] (DRILL-7854) When writing to S3 "WriteOperationHelper.newUploadPartRequest" throws "NoSuchMethodError"
[
https://issues.apache.org/jira/browse/DRILL-7854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laurent Goujon resolved DRILL-7854.
---
Resolution: Fixed
Should have been resolved when Guava version was updated to 30.0
> When writing to S3 "WriteOperationHelper.newUploadPartRequest" throws
> "NoSuchMethodError"
> -
>
> Key: DRILL-7854
> URL: https://issues.apache.org/jira/browse/DRILL-7854
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Other
>Affects Versions: 1.18.0
> Environment: Standalone drill running in a docker environment on
> x86_64 (centos7, alpine 3, debian buster)
>
> Drill 1.18.0 was downloaded from an official mirror
>Reporter: Joshua Pedrick
>Priority: Major
> Fix For: 1.19.0
>
>
> When writing to S3(hosted on local Minio cluster) I am getting a
> "NoSuchMethod" error. It appears to be related to the guava version included
> with hadoop-3.2.1.
>
> {code:java}
> 2021-02-01 21:39:17,753 [1fe78bd6-0525-196e-b3d2-25c294a604e2:frag:1:13]
> ERROR o.a.d.e.p.i.p.ProjectRecordBatch -
> ProjectRecordBatch[projector=Projector[vector2=null,
> selectionVectorMode=NONE], hasRemainder=false, remainderIndex=0,
> recordCount=0,
> container=org.apache.drill.exec.record.VectorContainer@3efe0be4[recordCount =
> 27884, schemaChanged = false, schema = BatchSchema [, ...]]2021-02-01
> 21:39:17,754 [1fe78bd6-0525-196e-b3d2-25c294a604e2:frag:1:13] ERROR
> o.a.d.e.physical.impl.BaseRootExec - Batch dump completed.2021-02-01
> 21:39:17,754 [1fe78bd6-0525-196e-b3d2-25c294a604e2:frag:1:13] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 1fe78bd6-0525-196e-b3d2-25c294a604e2:1:13: State change requested
> CANCELLATION_REQUESTED --> FAILED2021-02-01 21:39:25,199
> [1fe78bd6-0525-196e-b3d2-25c294a604e2:frag:1:13] ERROR
> o.a.d.exec.server.BootStrapContext -
> org.apache.drill.exec.work.WorkManager$WorkerBee$1.run() leaked an
> exception.java.lang.NoSuchMethodError:
> com/google/common/base/Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;J)V
> (loaded from by sun.misc.Launcher$AppClassLoader@3156fd60) called
> from class org.apache.hadoop.fs.s3a.WriteOperationHelper (loaded from
> file:/opt/drill/jars/3rdparty/hadoop-aws-3.2.1.jar by
> sun.misc.Launcher$AppClassLoader@3156fd60).at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.newUploadPartRequest(WriteOperationHelper.java:397)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.uploadBlockAsync(S3ABlockOutputStream.java:584)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream$MultiPartUpload.access$000(S3ABlockOutputStream.java:521)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream.uploadCurrentBlock(S3ABlockOutputStream.java:314)
> at
> org.apache.hadoop.fs.s3a.S3ABlockOutputStream.write(S3ABlockOutputStream.java:292)
>at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:57)
> at java.io.DataOutputStream.write(DataOutputStream.java:107)at
> java.io.FilterOutputStream.write(FilterOutputStream.java:97) at
> org.apache.parquet.hadoop.util.HadoopPositionOutputStream.write(HadoopPositionOutputStream.java:45)
> at
> org.apache.parquet.bytes.CapacityByteArrayOutputStream.writeToOutput(CapacityByteArrayOutputStream.java:234)
> at
> org.apache.parquet.bytes.CapacityByteArrayOutputStream.writeTo(CapacityByteArrayOutputStream.java:247)
>at
> org.apache.parquet.bytes.BytesInput$CapacityBAOSBytesInput.writeAllTo(BytesInput.java:421)
>at
> org.apache.parquet.hadoop.ParquetFileWriter.writeColumnChunk(ParquetFileWriter.java:620)
> at
> org.apache.parquet.hadoop.ParquetColumnChunkPageWriteStore$ColumnChunkPageWriter.writeToFileWriter(ParquetColumnChunkPageWriteStore.java:268)
> at
> org.apache.parquet.hadoop.ParquetColumnChunkPageWriteStore.flushToFileWriter(ParquetColumnChunkPageWriteStore.java:89)
>at
> org.apache.drill.exec.store.parquet.ParquetRecordWriter.flushParquetFileWriter(ParquetRecordWriter.java:737)
> at
> org.apache.drill.exec.store.parquet.ParquetRecordWriter.flush(ParquetRecordWriter.java:435)
> at
> org.apache.drill.exec.store.parquet.ParquetRecordWriter.cleanup(ParquetRecordWriter.java:703)
> at
> org.apache.drill.exec.physical.impl.WriterRecordBatch.closeWriter(WriterRecordBatch.java:203)
> at
> org.apache.drill.exec.physical.impl.WriterRecordBatch.close(WriterRecordBatch.java:221)
> at
> org.apache.drill.common.DeferredException.suppressingClose(DeferredException.java:159)
>at
> org.apache.drill.exec.physical.impl.BaseRootExec.close(BaseRootExec.java:169)
>at
> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentEx
[jira] [Resolved] (DRILL-7914) Apache Drill 1.19.0 Release Activities
[ https://issues.apache.org/jira/browse/DRILL-7914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent Goujon resolved DRILL-7914. --- Resolution: Fixed > Apache Drill 1.19.0 Release Activities > -- > > Key: DRILL-7914 > URL: https://issues.apache.org/jira/browse/DRILL-7914 > Project: Apache Drill > Issue Type: Task >Reporter: Laurent Goujon >Assignee: Laurent Goujon >Priority: Major > Fix For: 1.19.0 > > > Source - https://github.com/apache/drill/blob/master/docs/dev/Release.md > 1.19.0 Dashboard: > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336127 > Kanban Board - > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=185 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7921) Support the Linux ARM64 based system
[ https://issues.apache.org/jira/browse/DRILL-7921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent Goujon resolved DRILL-7921. --- Resolution: Fixed > Support the Linux ARM64 based system > > > Key: DRILL-7921 > URL: https://issues.apache.org/jira/browse/DRILL-7921 > Project: Apache Drill > Issue Type: New Feature >Reporter: Cong Luo >Assignee: Martin Tzvetanov Grigorov >Priority: Major > Fix For: 1.19.0 > > > This is an umbrella Jira for support to running on Linux ARM64 based system. > # Build must be passed. > # All the JUnit test must be passed. > # Running stable on the Linux aarch64 machines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7946) Bump HttpClient from 4.5.12 to 4.5.13 for CVE-2020-13956
[ https://issues.apache.org/jira/browse/DRILL-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent Goujon resolved DRILL-7946. --- Resolution: Fixed > Bump HttpClient from 4.5.12 to 4.5.13 for CVE-2020-13956 > > > Key: DRILL-7946 > URL: https://issues.apache.org/jira/browse/DRILL-7946 > Project: Apache Drill > Issue Type: Improvement >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > Fix For: 1.19.0 > > > Apache HttpClient versions prior to version 4.5.13 and 5.0.3 can misinterpret > malformed authority component in request URIs passed to the library as > java.net.URI object and pick the wrong target host for request execution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7945) Unable to patch Guava classes
[
https://issues.apache.org/jira/browse/DRILL-7945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laurent Goujon resolved DRILL-7945.
---
Assignee: Laurent Goujon (was: Vitalii Diravka)
Resolution: Fixed
> Unable to patch Guava classes
> -
>
> Key: DRILL-7945
> URL: https://issues.apache.org/jira/browse/DRILL-7945
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Vova Vysotskyi
>Assignee: Laurent Goujon
>Priority: Blocker
> Fix For: 1.19.0
>
>
> When starting Drill in embedded mode, logs have the following line:
> {noformat}
> 2021-06-03 21:59:07,711 [main] WARN o.a.drill.common.util.GuavaPatcher -
> Unable to patch Guava classes: [source error]
> format(java.lang.String,java.lang.Object[]) not found in
> com.google.common.base.Preconditions
> {noformat}
> Most likely it is a regression after DRILL-7904.
> The importance of this issue is that all 3rd party libraries that initially
> relied on patched methods from {{Preconditions}} will stop working because of
> method not found errors during the runtime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (DRILL-7795) Add option to native Drill Client to set TLS SNI property
[ https://issues.apache.org/jira/browse/DRILL-7795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent Goujon resolved DRILL-7795. --- Fix Version/s: 1.19.0 Resolution: Fixed > Add option to native Drill Client to set TLS SNI property > - > > Key: DRILL-7795 > URL: https://issues.apache.org/jira/browse/DRILL-7795 > Project: Apache Drill > Issue Type: Improvement > Components: Client - C++ >Reporter: James Duong >Priority: Major > Fix For: 1.19.0 > > > Add an option to the C++ DrillClient library that sets the TLS SNI extension > to let you set the Server Name Indication TLS property. If no option for this > value is specified, choose a default based on the target host. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-6547) IllegalStateException: Tried to remove unmanaged buffer.
[
https://issues.apache.org/jira/browse/DRILL-6547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laurent Goujon resolved DRILL-6547.
---
Fix Version/s: 1.19.0
Resolution: Fixed
> IllegalStateException: Tried to remove unmanaged buffer.
>
>
> Key: DRILL-6547
> URL: https://issues.apache.org/jira/browse/DRILL-6547
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.14.0
>Reporter: Robert Hou
>Assignee: Pritesh Maker
>Priority: Major
> Fix For: 1.19.0
>
>
> This is the query:
> select * from (
> select Index, concat(BinaryValue, 'aaa') NewVarcharValue from (select * from
> dfs.`/drill/testdata/batch_memory/alltypes_large_1MB.parquet`)) d where
> d.Index = 1;
> This is the plan:
> {noformat}
> | 00-00Screen
> 00-01 Project(Index=[$0], NewVarcharValue=[$1])
> 00-02SelectionVectorRemover
> 00-03 Filter(condition=[=($0, 1)])
> 00-04Project(Index=[$0], NewVarcharValue=[CONCAT($1, 'aaa')])
> 00-05 Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath
> [path=maprfs:///drill/testdata/batch_memory/alltypes_large_1MB.parquet]],
> selectionRoot=maprfs:/drill/testdata/batch_memory/alltypes_large_1MB.parquet,
> numFiles=1, numRowGroups=1, usedMetadataFile=false, columns=[`Index`,
> `BinaryValue`]]])
> {noformat}
> Here is the stack trace from drillbit.log:
> {noformat}
> 2018-06-27 13:55:03,291 [24cc0659-30b7-b290-7fae-ecb1c1f15c05:frag:0:0] ERROR
> o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IllegalStateException:
> Tried to remove unmanaged buffer.
> Fragment 0:0
> [Error Id: bc1f2f72-c31b-4b9a-964f-96dec9e0f388 on qa-node186.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalStateException: Tried to remove unmanaged buffer.
> Fragment 0:0
> [Error Id: bc1f2f72-c31b-4b9a-964f-96dec9e0f388 on qa-node186.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> ~[drill-common-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:361)
> [drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:216)
> [drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:327)
> [drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_161]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> Caused by: java.lang.IllegalStateException: Tried to remove unmanaged buffer.
> at
> org.apache.drill.exec.ops.BufferManagerImpl.replace(BufferManagerImpl.java:50)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at io.netty.buffer.DrillBuf.reallocIfNeeded(DrillBuf.java:97)
> ~[drill-memory-base-1.14.0-SNAPSHOT.jar:4.0.48.Final]
> at
> org.apache.drill.exec.test.generated.ProjectorGen4046.doEval(ProjectorTemplate.java:77)
> ~[na:na]
> at
> org.apache.drill.exec.test.generated.ProjectorGen4046.projectRecords(ProjectorTemplate.java:67)
> ~[na:na]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:236)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:117)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:147)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63)
> ~[drill-java-exec-1.14.0-SNAPSHOT.jar:1.14.0-SNAPSHOT]
> at
> org.apache.d
[jira] [Resolved] (DRILL-7936) Remove Guava Files#createTempDir usage
[ https://issues.apache.org/jira/browse/DRILL-7936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent Goujon resolved DRILL-7936. --- Fix Version/s: 1.19.0 Reviewer: Charles Givre Resolution: Fixed > Remove Guava Files#createTempDir usage > -- > > Key: DRILL-7936 > URL: https://issues.apache.org/jira/browse/DRILL-7936 > Project: Apache Drill > Issue Type: Bug >Reporter: Laurent Goujon >Assignee: Laurent Goujon >Priority: Major > Fix For: 1.19.0 > > > According to CVE-2020-8908, method Files#createTempDir has some security > issues, and usage has been deprecated in Guava 30.0. > Usage of this function in Drill should be replaced with the native Java > version as recommended by the Guava team. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7162) Apache Drill uses 3rd Party with Highest CVEs
[ https://issues.apache.org/jira/browse/DRILL-7162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent Goujon resolved DRILL-7162. --- Fix Version/s: 1.19.0 Resolution: Fixed Apache Drill has been updated to the latest Jetty 9.4 version available. This should address all the CVEs on the list. > Apache Drill uses 3rd Party with Highest CVEs > -- > > Key: DRILL-7162 > URL: https://issues.apache.org/jira/browse/DRILL-7162 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.13.0, 1.14.0, 1.15.0 >Reporter: Ayush Sharma >Priority: Major > Fix For: 1.19.0 > > Attachments: Jars.xlsx > > > Apache Drill uses 3rd party libraries with almost 250+ CVEs. > Most of the CVEs are in the older version of Jetty (9.1.x) whereas the > current version of Jetty is 9.4.x > Also many of the other libraries are in EOF versions and the are not patched > even in the latest release. > This creates an issue of security when we use it in production. > We are able to replace many older version of libraries with the latest > versions with no CVEs , however many of them are not replaceable as it is and > would require some changes in the source code. > The jetty version is of the highest priority and needs migration to 9.4.x > version immediately. > > Please look into this issue at immediate priority as it compromises with the > security of the application utilizing Apache Drill. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7135) Upgrade to Jetty 9.4
[ https://issues.apache.org/jira/browse/DRILL-7135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent Goujon resolved DRILL-7135. --- Fix Version/s: 1.19.0 Reviewer: Paul Rogers Assignee: Laurent Goujon Resolution: Fixed > Upgrade to Jetty 9.4 > > > Key: DRILL-7135 > URL: https://issues.apache.org/jira/browse/DRILL-7135 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.15.0 >Reporter: Vitalii Diravka >Assignee: Laurent Goujon >Priority: Minor > Fix For: 1.19.0 > > > Initially DRILL-7051 updated Jetty to 9.4 version and DRILL-7081 updated > Jersey version to 2.28 version. These versions work fine for Drill with > Hadoop version below 3.0. > Starting from Hadoop 3.0 it uses > [org.eclipse.jetty|https://github.com/apache/hadoop/blob/branch-3.0/hadoop-project/pom.xml#L38] > 9.3 version. > That's why it conflicts with newer Jetty&Jersey versions. > Drill can update Jetty and Jersey versions after resolution HADOOP-14930 and > HBASE-19256. > Or alternatively these libs can be shaded in Drill, but there is no real > reason to do it nowadays. > See details in > [#1681|https://github.com/apache/drill/pull/1681#discussion_r265904521] PR. > _Notes_: > * For Jersey update it is necessary to add > org.glassfish.jersey.inject:jersey-hk2 in Drill to solve all compilation > failures. > * See doc for Jetty update: > https://www.eclipse.org/jetty/documentation/9.4.x/upgrading-jetty.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7932) Update Hadoop version to 3.2.2.
[ https://issues.apache.org/jira/browse/DRILL-7932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent Goujon resolved DRILL-7932. --- Fix Version/s: 1.19.0 Reviewer: Cong Luo Resolution: Fixed Update done as part of the fix for DRILL-7135 > Update Hadoop version to 3.2.2. > --- > > Key: DRILL-7932 > URL: https://issues.apache.org/jira/browse/DRILL-7932 > Project: Apache Drill > Issue Type: Improvement >Reporter: Laurent Goujon >Assignee: Laurent Goujon >Priority: Major > Fix For: 1.19.0 > > > Update Hadoop version to the latest patch version for the 3.2.x branch. > This should address possible security vulnerability > [https://nvd.nist.gov/vuln/detail/CVE-2020-9492|CVE-2020-9492] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7936) Remove Guava Files#createTempDir usage
Laurent Goujon created DRILL-7936: - Summary: Remove Guava Files#createTempDir usage Key: DRILL-7936 URL: https://issues.apache.org/jira/browse/DRILL-7936 Project: Apache Drill Issue Type: Bug Reporter: Laurent Goujon Assignee: Laurent Goujon According to CVE-2020-8908, method Files#createTempDir has some security issues, and usage has been deprecated in Guava 30.0. Usage of this function in Drill should be replaced with the native Java version as recommended by the Guava team. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7932) Update Hadoop version to 3.2.2.
Laurent Goujon created DRILL-7932: - Summary: Update Hadoop version to 3.2.2. Key: DRILL-7932 URL: https://issues.apache.org/jira/browse/DRILL-7932 Project: Apache Drill Issue Type: Improvement Reporter: Laurent Goujon Assignee: Laurent Goujon Update Hadoop version to the latest patch version for the 3.2.x branch. This should address possible security vulnerability [https://nvd.nist.gov/vuln/detail/CVE-2020-9492|CVE-2020-9492] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7914) Apache Drill 1.19.0 Release Activities
Laurent Goujon created DRILL-7914: - Summary: Apache Drill 1.19.0 Release Activities Key: DRILL-7914 URL: https://issues.apache.org/jira/browse/DRILL-7914 Project: Apache Drill Issue Type: Task Reporter: Laurent Goujon Assignee: Laurent Goujon Source - https://github.com/apache/drill/blob/master/docs/dev/Release.md 1.19.0 Dashboard: https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336127 Kanban Board - https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=185 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7726) Boost requirement is incorrect
[ https://issues.apache.org/jira/browse/DRILL-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laurent Goujon resolved DRILL-7726. --- Fix Version/s: 1.18.0 Resolution: Fixed > Boost requirement is incorrect > -- > > Key: DRILL-7726 > URL: https://issues.apache.org/jira/browse/DRILL-7726 > Project: Apache Drill > Issue Type: Bug > Components: Client - C++ >Reporter: Laurent Goujon >Assignee: Laurent Goujon >Priority: Major > Fix For: 1.18.0 > > > Drill C++ connector documentation states that Boost 1.53 is required, but as > support for tlsv12 is present, Boost 1.54 is actually required. Trying to > compile with Boost 1.53 actually result in a compilation error for undefined > constant -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7727) Address Protobuf C++ warnings
Laurent Goujon created DRILL-7727:
-
Summary: Address Protobuf C++ warnings
Key: DRILL-7727
URL: https://issues.apache.org/jira/browse/DRILL-7727
Project: Apache Drill
Issue Type: Test
Components: Client - C++
Reporter: Laurent Goujon
Assignee: Laurent Goujon
Compiling C++ client with recent versions of protobuf (3.4.0 or higher)
produces compilation warnings:
{noformat}
.../drill/contrib/native/client/src/clientlib/rpcMessage.cpp:208:32: warning:
'ByteSize' is deprecated: Please use ByteSizeLong() instead
[-Wdeprecated-declarations]
int header_length = header.ByteSize();
^
/usr/local/include/google/protobuf/message_lite.h:401:3: note: 'ByteSize' has
been explicitly marked deprecated here
PROTOBUF_DEPRECATED_MSG("Please use ByteSizeLong() instead")
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (DRILL-7726) Boost requirement is incorrect
Laurent Goujon created DRILL-7726: - Summary: Boost requirement is incorrect Key: DRILL-7726 URL: https://issues.apache.org/jira/browse/DRILL-7726 Project: Apache Drill Issue Type: Bug Components: Client - C++ Reporter: Laurent Goujon Assignee: Laurent Goujon Drill C++ connector documentation states that Boost 1.53 is required, but as support for tlsv12 is present, Boost 1.54 is actually required. Trying to compile with Boost 1.53 actually result in a compilation error for undefined constant -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-5668) C++ connector crash when query error message is too long
Laurent Goujon created DRILL-5668:
-
Summary: C++ connector crash when query error message is too long
Key: DRILL-5668
URL: https://issues.apache.org/jira/browse/DRILL-5668
Project: Apache Drill
Issue Type: Bug
Components: Client - C++
Affects Versions: 1.10.0
Reporter: Laurent Goujon
Assignee: Laurent Goujon
Fix For: 1.11.0
The C++ connector crashes if the query sent to the server is invalid and the
error message returned by the server is too long.
The cause of the crash is a call to a {{vsprintf}} function which takes a fixed
sized char buffer, but doesn't take a size argument.
https://github.com/apache/drill/blob/master/contrib/native/client/src/clientlib/errmsgs.cpp#L80
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (DRILL-5369) Missing initialization for ServerMetaContext
Laurent Goujon created DRILL-5369:
-
Summary: Missing initialization for ServerMetaContext
Key: DRILL-5369
URL: https://issues.apache.org/jira/browse/DRILL-5369
Project: Apache Drill
Issue Type: Bug
Components: Client - C++
Reporter: Laurent Goujon
Assignee: Laurent Goujon
{{ServerMetaContext}} is not initialized properly which might cause some
unexpected issues (like getMetadata() returning before receiving the answer) in
some cases.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (DRILL-5368) Memory leak in C++ server metadata handler
Laurent Goujon created DRILL-5368: - Summary: Memory leak in C++ server metadata handler Key: DRILL-5368 URL: https://issues.apache.org/jira/browse/DRILL-5368 Project: Apache Drill Issue Type: Bug Components: Client - C++ Affects Versions: 1.10.0 Reporter: Laurent Goujon Assignee: Laurent Goujon Priority: Minor When receiving server metadata response, a protobuf ServerMetaResp object is dynamically allocated but never freed. Since for this handler, there's no need to keep the instance attached to the handler (content is copied over by the MetaData class), a reference is enough and allocation can be done on the stack. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (DRILL-5311) C++ connector connect doesn't wait for handshake to complete
Laurent Goujon created DRILL-5311: - Summary: C++ connector connect doesn't wait for handshake to complete Key: DRILL-5311 URL: https://issues.apache.org/jira/browse/DRILL-5311 Project: Apache Drill Issue Type: Bug Components: Client - C++ Reporter: Laurent Goujon The C++ connector connect methods returns okay as soon as the tcp connection is succesfully established between client and server, and the handshake message is sent. However it doesn't wait for handshake to have completed. The consequence is that if handshake failed, the error is deferred to the first query, which might be unexpected by the application. I believe that validateHanshake method in drillClientImpl should wait for the handshake to complete, as it seems a bit more saner... -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (DRILL-5301) Add server metadata API
Laurent Goujon created DRILL-5301: - Summary: Add server metadata API Key: DRILL-5301 URL: https://issues.apache.org/jira/browse/DRILL-5301 Project: Apache Drill Issue Type: Improvement Components: Server, Client - C++, Client - Java, Client - JDBC, Client - ODBC Reporter: Laurent Goujon JDBC and ODBC clients exposes lots of metadata regarding server version and support of various parts of the SQL standard. Currently the returned information is hardcoded in both clients/drivers which means that the infomation returned is support as of the client version, not the server version. Instead, a new method should be provided to the clients to query the actual server support. Support on the client or the server should be optional (for example a client should not use this API if the server doesn't support it and fallback to default values). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (DRILL-5221) cancel message is delayed until queryid or data is received
Laurent Goujon created DRILL-5221: - Summary: cancel message is delayed until queryid or data is received Key: DRILL-5221 URL: https://issues.apache.org/jira/browse/DRILL-5221 Project: Apache Drill Issue Type: Improvement Components: Client - C++ Affects Versions: 1.9.0 Reporter: Laurent Goujon When user is calling the cancel method of the C++ client, the client wait for a message from the server to reply back with a cancellation message. In case of queries taking a long time to return their first batch, it means cancellation is taking the same amount of time to be effective, instead of cancelling right away the query (assuming the query id has already been received, which is generally the case). It seems this was foreseen by [~vkorukanti] in his initial patch (https://github.com/vkorukanti/drill/commit/e0ef6349aac48de5828b6d725c2cf013905d18eb) but was omitted when I backported it post metadata changes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-5220) Add api to set application name in C++ connector
Laurent Goujon created DRILL-5220: - Summary: Add api to set application name in C++ connector Key: DRILL-5220 URL: https://issues.apache.org/jira/browse/DRILL-5220 Project: Apache Drill Issue Type: Improvement Components: Client - C++ Affects Versions: 1.8.0 Reporter: Laurent Goujon Priority: Minor There's no API for a C++ connector user to specify the name of the application, and to provide it to the server (optional field added in DRILL-4369) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-5219) Remove DrillUserProperties filtering in C++ driver
Laurent Goujon created DRILL-5219: - Summary: Remove DrillUserProperties filtering in C++ driver Key: DRILL-5219 URL: https://issues.apache.org/jira/browse/DRILL-5219 Project: Apache Drill Issue Type: Bug Components: Client - C++ Reporter: Laurent Goujon Priority: Minor Unlike the Java client, the C++ connector filter out unknown Drill user properties: https://github.com/apache/drill/blob/master/contrib/native/client/src/clientlib/drillClientImpl.cpp#L374 This prevents a client (like the ODBC driver) to pass extra properties to the server (like extra metainformation, or some specific behavior for a given software) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-5167) C++ connector does not set escape string for metadata search pattern
Laurent Goujon created DRILL-5167: - Summary: C++ connector does not set escape string for metadata search pattern Key: DRILL-5167 URL: https://issues.apache.org/jira/browse/DRILL-5167 Project: Apache Drill Issue Type: Bug Reporter: Laurent Goujon Priority: Minor C++ connector does not set the escape string for search pattern when doing metadata operation (getCatalogs/getSchema/getTables/getColumns). It is assumed to be '\\' as returned by DrillMetadata::getSearchEscapeString(), but because this is not sent over the wire, the server will actually consider that there's no escape character, and might return different or no result compared to what has been requested. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4994) Prepared statement stopped working between 1.8.0 client and < 1.7.0 server
Laurent Goujon created DRILL-4994: - Summary: Prepared statement stopped working between 1.8.0 client and < 1.7.0 server Key: DRILL-4994 URL: https://issues.apache.org/jira/browse/DRILL-4994 Project: Apache Drill Issue Type: Bug Reporter: Laurent Goujon Older servers (pre-1.8.0) don't support the prepared statement rpc method, but the JDBC client doesn't check if it is available or not. The end result is that the statement is stuck as the server is not responding back. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4969) Have basic implementation for displaySize
Laurent Goujon created DRILL-4969: - Summary: Have basic implementation for displaySize Key: DRILL-4969 URL: https://issues.apache.org/jira/browse/DRILL-4969 Project: Apache Drill Issue Type: Sub-task Components: Metadata Reporter: Laurent Goujon display size is fixed to 10, but for most types display size is well defined as shown in the ODBC table: https://msdn.microsoft.com/en-us/library/ms713974(v=vs.85).aspx -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4968) Add column size information to ColumnMetadata
Laurent Goujon created DRILL-4968: - Summary: Add column size information to ColumnMetadata Key: DRILL-4968 URL: https://issues.apache.org/jira/browse/DRILL-4968 Project: Apache Drill Issue Type: Bug Components: Metadata Reporter: Laurent Goujon Assignee: Laurent Goujon Both ODBC and JDBC needs column size information for the column metadata. Instead of duplicating the logic between C++ and Java (and having to keep in them sync), column size should be computed on the server so that value is kept consistent across clients. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4945) Missing subtype information in metadata returned by prepared statement
Laurent Goujon created DRILL-4945: - Summary: Missing subtype information in metadata returned by prepared statement Key: DRILL-4945 URL: https://issues.apache.org/jira/browse/DRILL-4945 Project: Apache Drill Issue Type: Bug Reporter: Laurent Goujon Priority: Minor Column metadata returned by prepared statement contains partial type information, especially for interval types. Currently it only returns "INTERVAL" instead of a more precise type like "INTERVAL MONTH" for example. There's also no minor type, so the client can adjust the type itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4930) Metadata results are not sorted
Laurent Goujon created DRILL-4930: - Summary: Metadata results are not sorted Key: DRILL-4930 URL: https://issues.apache.org/jira/browse/DRILL-4930 Project: Apache Drill Issue Type: Bug Components: Metadata Reporter: Laurent Goujon Priority: Minor According to JDBC and ODBC specs, metadata results should be ordered. Currently, results are unordered. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4925) Add types filter to getTables metadata API
Laurent Goujon created DRILL-4925: - Summary: Add types filter to getTables metadata API Key: DRILL-4925 URL: https://issues.apache.org/jira/browse/DRILL-4925 Project: Apache Drill Issue Type: Bug Components: Metadata Reporter: Laurent Goujon Priority: Minor Both ODBC and JDBC API have a types parameter to filter tables based on their types. Metadata API should support it too to avoid connectors to have to do extra filtering on client-side. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4853) Update C++ protobuf source files
Laurent Goujon created DRILL-4853: - Summary: Update C++ protobuf source files Key: DRILL-4853 URL: https://issues.apache.org/jira/browse/DRILL-4853 Project: Apache Drill Issue Type: Task Components: Client - C++ Reporter: Laurent Goujon Assignee: Laurent Goujon C++ protobuf files have not been updated since january, missing changes for prepared statement and metadata querying support -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4576) Add StoragePlugin API to register materialization into planner
Laurent Goujon created DRILL-4576:
-
Summary: Add StoragePlugin API to register materialization into
planner
Key: DRILL-4576
URL: https://issues.apache.org/jira/browse/DRILL-4576
Project: Apache Drill
Issue Type: Bug
Reporter: Laurent Goujon
Assignee: Jacques Nadeau
There's no currently a good way to register materializations into Drill
planner. Calcite's MaterializationService.instance() would be the way to go,
but the registration happens in
{{org.apache.calcite.prepare.Prepare.PreparedResult#prepareSql()}}, which is
not called by Drill.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4547) Javadoc fails with Java8
Laurent Goujon created DRILL-4547:
-
Summary: Javadoc fails with Java8
Key: DRILL-4547
URL: https://issues.apache.org/jira/browse/DRILL-4547
Project: Apache Drill
Issue Type: Bug
Components: Tools, Build & Test
Affects Versions: 1.6.0
Reporter: Laurent Goujon
Javadoc cannot be generated when using Java8 (likely because the parser is now
more strict).
Here's an example of issues when trying to generate javadocs in module
{{drill-fmpp-maven-plugin}}
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-javadoc-plugin:2.9.1:jar (attach-javadocs) on
project drill-fmpp-maven-plugin: MavenReportException: Error while creating
archive:
[ERROR] Exit code: 1 -
/Users/laurent/devel/drill/tools/fmpp/src/main/java/org/apache/drill/fmpp/mojo/FMPPMojo.java:44:
error: unknown tag: goal
[ERROR] * @goal generate
[ERROR] ^
[ERROR]
/Users/laurent/devel/drill/tools/fmpp/src/main/java/org/apache/drill/fmpp/mojo/FMPPMojo.java:45:
error: unknown tag: phase
[ERROR] * @phase generate-sources
[ERROR] ^
[ERROR]
/Users/laurent/devel/drill/tools/fmpp/target/generated-sources/plugin/org/apache/drill/fmpp/mojo/HelpMojo.java:25:
error: unknown tag: goal
[ERROR] * @goal help
[ERROR] ^
[ERROR]
/Users/laurent/devel/drill/tools/fmpp/target/generated-sources/plugin/org/apache/drill/fmpp/mojo/HelpMojo.java:26:
error: unknown tag: requiresProject
[ERROR] * @requiresProject false
[ERROR] ^
[ERROR]
/Users/laurent/devel/drill/tools/fmpp/target/generated-sources/plugin/org/apache/drill/fmpp/mojo/HelpMojo.java:27:
error: unknown tag: threadSafe
[ERROR] * @threadSafe
[ERROR] ^
[ERROR]
[ERROR] Command line was:
/Library/Java/JavaVirtualMachines/jdk1.8.0_72.jdk/Contents/Home/bin/javadoc
@options @packages
[ERROR]
[ERROR] Refer to the generated Javadoc files in
'/Users/laurent/devel/drill/tools/fmpp/target/apidocs' dir.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :drill-fmpp-maven-plugin
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4546) mvn deploy pushes the same zip artifact twice
Laurent Goujon created DRILL-4546: - Summary: mvn deploy pushes the same zip artifact twice Key: DRILL-4546 URL: https://issues.apache.org/jira/browse/DRILL-4546 Project: Apache Drill Issue Type: Bug Components: Tools, Build & Test Reporter: Laurent Goujon When using the apache-release profile, both Apache and Drill assembly descriptors are used. This caused the zip artifact to be generated twice, and pushed twice to the remote repository. Because some repositories are configured to not allow the same artifact to be pushed several times, this might cause the build to fail. Ideally, only one zip file should be built and push. Also, the Apache Parent pom provides a descriptor to build both the tar and the zip archives. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4468) Aggregates over empty input might fail
Laurent Goujon created DRILL-4468:
-
Summary: Aggregates over empty input might fail
Key: DRILL-4468
URL: https://issues.apache.org/jira/browse/DRILL-4468
Project: Apache Drill
Issue Type: Bug
Environment: Linux/OpenJDK 7
Reporter: Laurent Goujon
Some aggregation queries over empty input might fail, depending of the column
ordering.
This query for example would fail:
{noformat}
select sum(int_col) col1, sum(bigint_col) col2 from cp.`employee.json` where 1
= 0
org.apache.drill.common.exceptions.UserRemoteException: UNSUPPORTED_OPERATION
ERROR: Only COUNT, MIN and MAX aggregate functions supported for VarChar type
Fragment 0:0 [Error Id: dcef042c-1c53-40df-88b0-816d3cb109a7 on xxx:31010]
{noformat}
But this one would succeed:
{noformat}
select sum(bigint_col) col2, sum(int_col) col1 from cp.`employee.json` where 1
= 0
nullnull
{noformat}
The reason for why only one query fails is because of DRILL-4467. The
consequence is that the plans are significantly different, and don't behave
quite the same way.
Here's the Physical plan for the first query:
{noformat}
00-00Screen : rowType = RecordType(ANY col1, ANY col2): rowcount = 1.0,
cumulative cost = {464.1 rows, 950.1 cpu, 0.0 io, 0.0 network, 0.0 memory}, id
= 339
00-01 Project(col1=[$0], col2=[$1]) : rowType = RecordType(ANY col1, ANY
col2): rowcount = 1.0, cumulative cost = {464.0 rows, 950.0 cpu, 0.0 io, 0.0
network, 0.0 memory}, id = 338
00-02StreamAgg(group=[{}], col1=[SUM($0)], col2=[SUM($1)]) : rowType =
RecordType(ANY col1, ANY col2): rowcount = 1.0, cumulative cost = {464.0 rows,
950.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 337
00-03 Limit(offset=[0], fetch=[0]) : rowType = RecordType(ANY int_col,
ANY bigint_col): rowcount = 1.0, cumulative cost = {463.0 rows, 926.0 cpu, 0.0
io, 0.0 network, 0.0 memory}, id = 336
00-04Scan(groupscan=[EasyGroupScan
[selectionRoot=classpath:/employee.json, numFiles=1, columns=[`int_col`,
`bigint_col`], files=[classpath:/employee.json]]]) : rowType = RecordType(ANY
int_col, ANY bigint_col): rowcount = 463.0, cumulative cost = {463.0 rows,
926.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 335
{noformat}
and the physical plan for the second query:
{noformat}
00-00Screen : rowType = RecordType(ANY col2, ANY col1): rowcount = 1.0,
cumulative cost = {464.1 rows, 950.1 cpu, 0.0 io, 0.0 network, 0.0 memory}, id
= 775
00-01 Project(col2=[$0], col1=[$1]) : rowType = RecordType(ANY col2, ANY
col1): rowcount = 1.0, cumulative cost = {464.0 rows, 950.0 cpu, 0.0 io, 0.0
network, 0.0 memory}, id = 774
00-02StreamAgg(group=[{}], col2=[SUM($0)], col1=[SUM($1)]) : rowType =
RecordType(ANY col2, ANY col1): rowcount = 1.0, cumulative cost = {464.0 rows,
950.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 773
00-03 Limit(offset=[0], fetch=[0]) : rowType = RecordType(ANY
bigint_col, ANY int_col): rowcount = 1.0, cumulative cost = {463.0 rows, 926.0
cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 772
00-04Project(bigint_col=[$1], int_col=[$0]) : rowType =
RecordType(ANY bigint_col, ANY int_col): rowcount = 463.0, cumulative cost =
{463.0 rows, 926.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 771
00-05 Scan(groupscan=[EasyGroupScan
[selectionRoot=classpath:/employee.json, numFiles=1, columns=[`bigint_col`,
`int_col`], files=[classpath:/employee.json]]]) : rowType = RecordType(ANY
int_col, ANY bigint_col): rowcount = 463.0, cumulative cost = {463.0 rows,
926.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 770
{noformat}
The extra projection just before the scan seems to hide the VARCHAR type of the
columns, and allow for aggregation to succeed. On the other hand, the storage
plugin allows for column push down, so the projection is theoretically
unnecessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4467) Invalid projection created using PrelUtil.getColumns
Laurent Goujon created DRILL-4467:
-
Summary: Invalid projection created using PrelUtil.getColumns
Key: DRILL-4467
URL: https://issues.apache.org/jira/browse/DRILL-4467
Project: Apache Drill
Issue Type: Bug
Reporter: Laurent Goujon
Assignee: Laurent Goujon
In {{DrillPushProjIntoScan}}, a new scan and a new projection are created using
{{PrelUtil#getColumn(RelDataType, List)}}.
The returned {{ProjectPushInfo}} instance has several fields, one of them is
{{desiredFields}} which is the list of projected fields. There's one instance
per {{RexNode}} but because instances were initially added to a set, they might
not be in the same order as the order they were created.
The issue happens in the following code:
{code:java}
List newProjects = Lists.newArrayList();
for (RexNode n : proj.getChildExps()) {
newProjects.add(n.accept(columnInfo.getInputRewriter()));
}
{code}
This code creates a new list of projects out of the initial ones, by mapping
the indices from the old projects to the new projects, but the indices of the
new RexNode instances might be out of order (because of the ordering of
desiredFields). And if indices are out of order, the check
{{ProjectRemoveRule.isTrivial(newProj)}} will fail.
My guess is that desiredFields ordering should be preserved when instances are
added, to satisfy the condition above.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4452) Update avatica version for Drill jdbc
Laurent Goujon created DRILL-4452: - Summary: Update avatica version for Drill jdbc Key: DRILL-4452 URL: https://issues.apache.org/jira/browse/DRILL-4452 Project: Apache Drill Issue Type: Improvement Components: Client - JDBC Affects Versions: 1.5.0 Reporter: Laurent Goujon Assignee: Laurent Goujon Priority: Minor Drill depends on a very old version of Avatica (0.9.0/pre-calcite), which makes integrating changes harder and harder. Although Avatica has evolved to support a custom protocol, with a server stub, I believe it is still possible for Drill to use the client part as the JDBC facade, with small adjustments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4390) Drill webserver cannot get static assets if another jar contains a rest/static directory
Laurent Goujon created DRILL-4390: - Summary: Drill webserver cannot get static assets if another jar contains a rest/static directory Key: DRILL-4390 URL: https://issues.apache.org/jira/browse/DRILL-4390 Project: Apache Drill Issue Type: Bug Components: Web Server Reporter: Laurent Goujon Assignee: Laurent Goujon Priority: Minor WebServer is configured to serve static/ URL from a resource containing "rest/static" but if another jar than drill java-exec jar contains the same directory, all queries to static/ will result most likely in 404s. I propose to be more precise regarding the resource to use, and find the one containing the Drill favicon under rest/static. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4361) Allow for FileSystemPlugin subclasses to override FormatCreator
Laurent Goujon created DRILL-4361: - Summary: Allow for FileSystemPlugin subclasses to override FormatCreator Key: DRILL-4361 URL: https://issues.apache.org/jira/browse/DRILL-4361 Project: Apache Drill Issue Type: Improvement Reporter: Laurent Goujon Assignee: Laurent Goujon Priority: Minor FileSystemPlugin subclasses are not able to customize plugins, as FormatCreator in created in FileSystemPlugin constructor and immediately used to create SchemaFactory instance. FormatCreator instantiation should be moved to a protected method so that subclass can choose to implement it differently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4359) EndpointAffinity missing equals method
Laurent Goujon created DRILL-4359: - Summary: EndpointAffinity missing equals method Key: DRILL-4359 URL: https://issues.apache.org/jira/browse/DRILL-4359 Project: Apache Drill Issue Type: Improvement Reporter: Laurent Goujon Assignee: Laurent Goujon Priority: Trivial EndpointAffinity is a placeholder class, but has no equals method to allow comparison. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4327) Fix rawtypes warning emitted by compiler
Laurent Goujon created DRILL-4327: - Summary: Fix rawtypes warning emitted by compiler Key: DRILL-4327 URL: https://issues.apache.org/jira/browse/DRILL-4327 Project: Apache Drill Issue Type: Improvement Reporter: Laurent Goujon Assignee: Laurent Goujon Priority: Minor The Drill codebase references lots of rawtypes, which generates lots of warning from the compiler. Since Drill is now compiled with Java 1.7, it should use generic types as much as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-4295) Obsolete protobuf generated files under protocol/
Laurent Goujon created DRILL-4295:
-
Summary: Obsolete protobuf generated files under protocol/
Key: DRILL-4295
URL: https://issues.apache.org/jira/browse/DRILL-4295
Project: Apache Drill
Issue Type: Task
Components: Tools, Build & Test
Reporter: Laurent Goujon
Priority: Trivial
The following two files don't have a protobuf definition anymore, and are not
generated when running {{mvn process-sources -P proto-compile}} under
{{protocol/}}:
{noformat}
src/main/java/org/apache/drill/exec/proto/beans/RpcFailure.java
src/main/java/org/apache/drill/exec/proto/beans/ViewPointer.java
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4292) Avoid multiple new Configuration/FileSystem instantiation
Laurent Goujon created DRILL-4292:
-
Summary: Avoid multiple new Configuration/FileSystem instantiation
Key: DRILL-4292
URL: https://issues.apache.org/jira/browse/DRILL-4292
Project: Apache Drill
Issue Type: Task
Reporter: Laurent Goujon
There are lots of places where Drill code has the following pattern:
{noformat}
conf = new Configuration();
{noformat}
or
{noformat}
Configuration conf = new Configuration();
[...]
fs = FileSystem.get(conf);
{noformat}
Creating Configuration instances is a pretty expensive operation as it triggers
a classpath scan to find resources (this lazily happens when accessing a key).
Also, these extra instances use more memory than expected (because the default
resources are not shared between Configuration instances.
FileSystem instances should not be expensive to create, because by default,
instances are cached (by scheme/authority/ugi), but it also means that the
Configuration instance has little value when getting the FileSystem instnce
(except when getting the default filesystem (can be replaced by
FileSystem#get(URI, Configuration)) or creating the FileSystem for the given
scheme, IF not already in the cache).
If possible, all these examples should be refactored to inject the right
FileSystem instance, instead of letting each class trying to create its own.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4288) Obsolete generated files under protocol/
Laurent Goujon created DRILL-4288:
-
Summary: Obsolete generated files under protocol/
Key: DRILL-4288
URL: https://issues.apache.org/jira/browse/DRILL-4288
Project: Apache Drill
Issue Type: Task
Components: Tools, Build & Test
Reporter: Laurent Goujon
Priority: Trivial
The protocol module contains the following files directly at the root of the
module:
{noformat}
protocol/org/apache/drill/exec/proto/BitControlHandshake.java
protocol/org/apache/drill/exec/proto/BitStatus.java
protocol/org/apache/drill/exec/proto/FragmentStatus.java
protocol/org/apache/drill/exec/proto/PlanFragment.java
protocol/org/apache/drill/exec/proto/RpcType.java
protocol/org/apache/drill/exec/proto/WorkQueueStatus.java
{noformat}
These files are not generated anymore when running {{mvn process-sources -P
proto-compile}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4285) maven cannot find Could not find artifact org.freemarker:freemarker:jar:2.3.24-SNAPSHOT in apache.snapshots
Laurent Goujon created DRILL-4285:
-
Summary: maven cannot find Could not find artifact
org.freemarker:freemarker:jar:2.3.24-SNAPSHOT in apache.snapshots
Key: DRILL-4285
URL: https://issues.apache.org/jira/browse/DRILL-4285
Project: Apache Drill
Issue Type: Bug
Components: Tools, Build & Test
Environment: MacOS 10.11, Java 7 (1.7.0_80), maven 3.2.x and 3.3.x
Reporter: Laurent Goujon
Priority: Minor
When building from master, build fails in component drill-fmpp-maven-plugin,
with the following error message:
{noformat}
[ERROR] Failed to execute goal on project drill-fmpp-maven-plugin: Could not
resolve dependencies for project
org.apache.drill.tools:drill-fmpp-maven-plugin:maven-plugin:1.5.0-SNAPSHOT:
Could not find artifact org.freemarker:freemarker:jar:2.3.24-SNAPSHOT in
apache.snapshots (http://repository.apache.org/snapshots) -> [Help 1]
{noformat}
The project itself doesn't depend directly on the SNAPSHOT version, but FMPP
dependency specifies a version range for FreeMarker.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
