[jira] [Created] (DRILL-8275) Prevent the JDBC Client to create error path in Zookeeper
Cong Luo created DRILL-8275: --- Summary: Prevent the JDBC Client to create error path in Zookeeper Key: DRILL-8275 URL: https://issues.apache.org/jira/browse/DRILL-8275 Project: Apache Drill Issue Type: Improvement Components: Client - JDBC Reporter: Cong Luo Assignee: Cong Luo Fix For: 2.0.0 Use the ZK style on the connection string and the zkRoot does not match the actual path of the cluster, then the client always creates the error path (as a permanent) in the Zookeeper. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (DRILL-8259) Supports advanced HBase persistence storage options
Cong Luo created DRILL-8259:
---
Summary: Supports advanced HBase persistence storage options
Key: DRILL-8259
URL: https://issues.apache.org/jira/browse/DRILL-8259
Project: Apache Drill
Issue Type: New Feature
Components: Storage - HBase
Reporter: Cong Luo
Assignee: Cong Luo
Fix For: 2.0.0
Its contents are as follows
{code:java}
sys.store.provider: {
class: "org.apache.drill.exec.store.hbase.config.HBasePStoreProvider",
hbase: {
table : "drill_store",
config: {
"hbase.zookeeper.quorum": "zk_host3,zk_host2,zk_host1",
"hbase.zookeeper.property.clientPort": "2181",
"zookeeper.znode.parent": "/hbase-test"
},
table_config : {
"durability": "ASYNC_WAL",
"compaction_enabled": false,
"split_enabled": false,
"max_filesize": 10737418240,
"memstore_flushsize": 536870912
},
column_config : {
"versions": 1,
"ttl": 2626560,
"compression": "SNAPPY",
"blockcache": true,
"blocksize": 131072,
"data_block_encoding": "FAST_DIFF",
"in_memory": true,
"dfs_replication": 3
}
}
}{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (DRILL-8253) Support the limit results in kafka scan
Title: Message Title Cong Luo created an issue Apache Drill / DRILL-8253 Support the limit results in kafka scan Issue Type: Improvement Assignee: Cong Luo Components: Storage - Kafka Created: 28/Jun/22 10:42 Fix Versions: 1.20.2 Priority: Major Reporter: Cong Luo In the current implementation of the kafka storage, although we use the limit detection method in the result loader, but the actual maxRecords is always -1. Add Comment
[jira] [Created] (DRILL-8189) Convert Image format to EVF2
Cong Luo created DRILL-8189: --- Summary: Convert Image format to EVF2 Key: DRILL-8189 URL: https://issues.apache.org/jira/browse/DRILL-8189 Project: Apache Drill Issue Type: Improvement Affects Versions: 1.20.0 Reporter: Cong Luo Assignee: Cong Luo Second refactor of the Image format. In addition, continue to deal with minor bugs in the V2 framework. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (DRILL-8188) Convert HDF5 format to EVF2
Cong Luo created DRILL-8188: --- Summary: Convert HDF5 format to EVF2 Key: DRILL-8188 URL: https://issues.apache.org/jira/browse/DRILL-8188 Project: Apache Drill Issue Type: Improvement Affects Versions: 1.20.0 Reporter: Cong Luo Assignee: Cong Luo Use EVF V2 instead of old V1. Also, fixed a few bugs in V2 framework. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (DRILL-8186) Convert XML format to EVF2
Cong Luo created DRILL-8186: --- Summary: Convert XML format to EVF2 Key: DRILL-8186 URL: https://issues.apache.org/jira/browse/DRILL-8186 Project: Apache Drill Issue Type: Improvement Affects Versions: 1.20.0 Reporter: Cong Luo Assignee: Cong Luo Use EVF V2 instead of old V1. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (DRILL-8174) Convert Avro format to EVF2
Cong Luo created DRILL-8174: --- Summary: Convert Avro format to EVF2 Key: DRILL-8174 URL: https://issues.apache.org/jira/browse/DRILL-8174 Project: Apache Drill Issue Type: Improvement Reporter: Cong Luo Assignee: Cong Luo Second refactor of the Avro format. We may need to resolve multiple bugs related to the Map Vector in this pull request. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (DRILL-8172) Use the specified memory usage for Travis CI
Cong Luo created DRILL-8172: --- Summary: Use the specified memory usage for Travis CI Key: DRILL-8172 URL: https://issues.apache.org/jira/browse/DRILL-8172 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Reporter: Cong Luo Assignee: Cong Luo Unlike GitHub CI, Travis CI may have more memory. If so, we can run unit tests using various policies to prevent failure. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (DRILL-8171) Convert SequenceFile to EVF2
Cong Luo created DRILL-8171: --- Summary: Convert SequenceFile to EVF2 Key: DRILL-8171 URL: https://issues.apache.org/jira/browse/DRILL-8171 Project: Apache Drill Issue Type: Improvement Affects Versions: 1.20.0 Reporter: Cong Luo Assignee: Cong Luo -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (DRILL-8170) Use the new Liquibase feature to override the obsolete feature
Cong Luo created DRILL-8170:
---
Summary: Use the new Liquibase feature to override the obsolete
feature
Key: DRILL-8170
URL: https://issues.apache.org/jira/browse/DRILL-8170
Project: Apache Drill
Issue Type: Improvement
Reporter: Cong Luo
*[Very minor, but can be good for newcomers]*
It is recommended to use the new function if the following issue is resolved.
[https://github.com/liquibase/liquibase/issues/2349]
{code:java}
// Deprecated
DatabaseFactory.getInstance()
// New
Scope.getCurrentScope().getSingleton(DatabaseFactory.class){code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (DRILL-8165) Upgrade liquibase because of CVE-2022-0839
Cong Luo created DRILL-8165: --- Summary: Upgrade liquibase because of CVE-2022-0839 Key: DRILL-8165 URL: https://issues.apache.org/jira/browse/DRILL-8165 Project: Apache Drill Issue Type: Task Affects Versions: 1.20.0 Reporter: Cong Luo Before upgrading to the recommended version 4.8, should note that : What is the difference between 3.8 and 4.8? [https://github.com/liquibase/liquibase/releases] Which code files should be updated? *_org.apache.drill.metastore.rdbms.RdbmsMetastore_* -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (DRILL-8164) Upgrade metadata-extractor because of CVE-2022-24613
Cong Luo created DRILL-8164: --- Summary: Upgrade metadata-extractor because of CVE-2022-24613 Key: DRILL-8164 URL: https://issues.apache.org/jira/browse/DRILL-8164 Project: Apache Drill Issue Type: Task Affects Versions: 1.20.0 Reporter: Cong Luo Assignee: Cong Luo Please note that the *metadata-extractor* is only used for format-image, so we should not define the version in the root *pom.xml* file. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (DRILL-8125) jdbc postgresql & null JSONB fields
[
https://issues.apache.org/jira/browse/DRILL-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489580#comment-17489580
]
Cong Luo commented on DRILL-8125:
-
[~matteoredaelli] Thanks for the information. Looks like the code is limited,
but it should be supported.
> jdbc postgresql & null JSONB fields
> ---
>
> Key: DRILL-8125
> URL: https://issues.apache.org/jira/browse/DRILL-8125
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - JDBC
>Reporter: Matteo Redaelli
>Priority: Minor
>
> Using streamline, psql, sqlline via jdbc/postgres I can query a postgres table
> {noformat}
> select instance_id, tags from aws_prd.aws_ec2_instance limit 1;
> instance_id i-0xxx
> tags {"Name": "talend-engine-01", ...}{noformat}
> {noformat}
> select instance_id, tags -> 'Name' from aws_dm_prd.aws_ec2_instance limit 1;
>
> instance_id i-0xxx
> ?column? "talend-engine-01"
> {noformat}
>
> With drill command line (drill-embeeded) I get null value for "tags" fields
> {noformat}
> apache drill> select instance_id, tags from
> steampipe.aws_dm_prd.aws_ec2_instance limit 1;
> instance_id i-0xxx
> tags null{noformat}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (DRILL-8125) jdbc postgresql & null JSONB fields
[
https://issues.apache.org/jira/browse/DRILL-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487620#comment-17487620
]
Cong Luo commented on DRILL-8125:
-
[~matteoredaelli] Hi. Are there any error messages in the log?
> jdbc postgresql & null JSONB fields
> ---
>
> Key: DRILL-8125
> URL: https://issues.apache.org/jira/browse/DRILL-8125
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - JDBC
>Reporter: Matteo Redaelli
>Priority: Minor
>
> Using streamline, psql, sqlline via jdbc/postgres I can query a postgres table
> {noformat}
> select instance_id, tags from aws_prd.aws_ec2_instance limit 1;
> instance_id i-0xxx
> tags {"Name": "talend-engine-01", ...}{noformat}
> {noformat}
> select instance_id, tags -> 'Name' from aws_dm_prd.aws_ec2_instance limit 1;
>
> instance_id i-0xxx
> ?column? "talend-engine-01"
> {noformat}
>
> With drill command line (drill-embeeded) I get null value for "tags" fields
> {noformat}
> apache drill> select instance_id, tags from
> steampipe.aws_dm_prd.aws_ec2_instance limit 1;
> instance_id i-0xxx
> tags null{noformat}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (DRILL-8122) Change kafka metadata obtaining due to KAFKA-5697
[ https://issues.apache.org/jira/browse/DRILL-8122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485802#comment-17485802 ] Cong Luo commented on DRILL-8122: - [~mrymar] Thank you for the report and assign this task. Do you already use Kafka 2.x in the production environment? > Change kafka metadata obtaining due to KAFKA-5697 > - > > Key: DRILL-8122 > URL: https://issues.apache.org/jira/browse/DRILL-8122 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.17.0, 1.18.0, 1.19.0 >Reporter: Maksym Rymar >Assignee: Maksym Rymar >Priority: Major > Fix For: 1.20.0 > > > Starting from kafka 2.0 was changed behavior of the > [KafkaConsumer#poll|https://kafka.apache.org/11/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#poll-long-] > . In Drill was used one hack, where poll() is used only to update consumer > metadata. But after the next change: > [https://github.com/apache/kafka/pull/4855] this hack is not working now as > poll() is not blocked more until metadata update yet more. > Unfortunately, Kafka doesn't have public method to only update assignment > metadata in consumer and it's on the discussion now whether to add it or not: > [KIP-505|https://cwiki.apache.org/confluence/display/KAFKA/KIP-505%3A+Add+new+public+method+to+only+update+assignment+metadata+in+consumer] > . Once it will be introduced we can use it but until that it needs to > implement a workaround. > Code to change: > https://github.com/apache/drill/blob/15b2f52260e4f0026f2dfafa23c5d32e0fb66502/contrib/storage-kafka/src/main/java/org/apache/drill/exec/store/kafka/KafkaGroupScan.java#L185 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (DRILL-8120) Make Drill functional tests viable again.
[ https://issues.apache.org/jira/browse/DRILL-8120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17485794#comment-17485794 ] Cong Luo commented on DRILL-8120: - [~angozhiy] This is a great invention. I'm thinking about how to start the integration test. Drill now includes a lot of unit tests, but there's no simulation test framework: verifying the availability, stability, and high performance in a distributed environment. We can move this test framework under _*apache*_ : # github.com/apache/drill # codebase # github.com/apache/drill-site # website # github.com/apache/drill-it-framework # integration test Have we started working on that framework now? > Make Drill functional tests viable again. > - > > Key: DRILL-8120 > URL: https://issues.apache.org/jira/browse/DRILL-8120 > Project: Apache Drill > Issue Type: Task >Reporter: Anton Gozhiy >Assignee: Anton Gozhiy >Priority: Major > > There is an external test framework that was used for Drill regression > testing before: > [https://github.com/mapr/drill-test-framework] > Although it is under mapr domain, it is public and licensed under the Apache > License 2.0 so it can be used again. > *Problems need to be solved to make it work:* > # Environment. It used to run on a quite powerful physical cluster with HDFS > and Drill, with static configuration and reusable test data. This makes the > framework inflexible, end even if you have suitable env it may still require > some amount of manual tuning. Possible solution: wrap it up into a docker > container to make it platform independent and minimize the effort to set it > up. > # Tests were not updated for 2 years so they need to be brought up to date. > It can be done step by step, fixing some test suites, removing or disabling > those that are not needed anymore. > # Test pipeline. After first two paragraphs are resolved, a CI tool can be > used to run tests regularly. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (DRILL-8098) Error starting Drill (drill-embedded) 1.20.0 on ppc64le
[
https://issues.apache.org/jira/browse/DRILL-8098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17467067#comment-17467067
]
Cong Luo commented on DRILL-8098:
-
[~makino] Hello, could you please try to start the drill on distributed mode?
> Error starting Drill (drill-embedded) 1.20.0 on ppc64le
> ---
>
> Key: DRILL-8098
> URL: https://issues.apache.org/jira/browse/DRILL-8098
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - CLI
>Affects Versions: 1.20.0
> Environment: Apache Drill 1.20.0 on Debian Bullseye ppc64le with
> Apache Maven 3.6.3 and OpenJDK 11.0.13
>Reporter: makino
>Priority: Minor
>
> Building Drill as described in [Compiling Drill from
> Source|https://drill.apache.org/docs/compiling-drill-from-source/] via
> {{mvn clean install -DskipTests}}
> I am able to start drill-embedded, however the CLI (apache drill>) is not
> responding, nor showing any errors. In order to exit drill-embedded I have to
> forcefully stop the session. There is no matching description on
> [Troubleshooting|https://drill.apache.org/docs/troubleshooting/]. Is there
> anything I can do to further debug this or provide more information to
> analyze the error?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (DRILL-8091) Godrej Ananda Residential Apartments For Sale In Bangalore
[
https://issues.apache.org/jira/browse/DRILL-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cong Luo closed DRILL-8091.
---
Resolution: Not A Bug
> Godrej Ananda Residential Apartments For Sale In Bangalore
> ---
>
> Key: DRILL-8091
> URL: https://issues.apache.org/jira/browse/DRILL-8091
> Project: Apache Drill
> Issue Type: New Feature
> Components: Client - Java
>Affects Versions: 1.17.0
>Reporter: Godrej Ananda
>Priority: Major
> Labels: Realestate
> Fix For: 1.20.0
>
> Attachments: owen-lystrup-boLgiM0qwkg-unsplash.jpg
>
> Original Estimate: 0.05h
> Remaining Estimate: 0.05h
>
> {*}{*}[Godrej Ananda|https://www.godrej-ananda.net.in/] Woods is among the
> milestone manifestations of Godrej Properties. Close to Devanahalli, North
> Bangalore. It offers 1 BHK, 2 BHK and 3 BHK extravagance homes. It comprises
> of 1000+ trees, eight sorts of Orchards, ten sorts of themed gardens, a
> timberland themed clubhouse and condos with a treetop view. The task is a
> continuous lodging spread across 13 sections of land of land region.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (DRILL-8086) Convert the CSV (AKA "compliant text") reader to EVF V2
[ https://issues.apache.org/jira/browse/DRILL-8086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17462549#comment-17462549 ] Cong Luo commented on DRILL-8086: - [~Paul.Rogers] I remember Charles saying he was going to make improvements to the CSV reader, this PR is timely. > Convert the CSV (AKA "compliant text") reader to EVF V2 > --- > > Key: DRILL-8086 > URL: https://issues.apache.org/jira/browse/DRILL-8086 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.19.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > > Work was done some time ago to convert the CSV reader to use EVF V3. Merge > that work into the master branch. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (DRILL-8085) EVF V2 support in the "Easy" format plugin
[
https://issues.apache.org/jira/browse/DRILL-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17462547#comment-17462547
]
Cong Luo commented on DRILL-8085:
-
[~Paul.Rogers] Thanks for your work. Please let me know if you need an real
distributed cluster to test these.
> EVF V2 support in the "Easy" format plugin
> --
>
> Key: DRILL-8085
> URL: https://issues.apache.org/jira/browse/DRILL-8085
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Paul Rogers
>Assignee: Paul Rogers
>Priority: Major
>
> Add support for EVF V2 to the {{EasyFormatPlugin}} similar to how EVF V1
> support already exists. Provide examples for others to follow.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (DRILL-8083) HttpdLogBatchReader creates unnecessary empty maps
[
https://issues.apache.org/jira/browse/DRILL-8083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17462540#comment-17462540
]
Cong Luo commented on DRILL-8083:
-
[~Paul.Rogers] Thanks for the report. Do you think this issue suits the
newcomers? If so, I will mark it as the `newcomers` for the active contributors.
> HttpdLogBatchReader creates unnecessary empty maps
> --
>
> Key: DRILL-8083
> URL: https://issues.apache.org/jira/browse/DRILL-8083
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.19.0
>Reporter: Paul Rogers
>Priority: Minor
>
> Run the {{TestHTTPDLogReader.testStarRowSet}} test. Set a breakpoint in
> {{MapWriter.SingleMapWriter.endWrite}}. Step into the {{super.endWrite()}}
> method which will walk the set of child fields. Notice that there are none
> for any of the several map fields.
> One can see that empty maps are expected in the
> {{TestHTTPDLogReader.expectedAllFieldsSchema()}} method.
> Maps (i.e. tuples) are not well defined in SQL. Although Drill makes great
> efforts to support them, an empty tuple is not well defined even in Drill:
> there is nothing one can do with such fields.
> Suggestion: don't create a map field if there are to be no members of the
> map.
> Affected maps:
> * {{request_firstline_original_uri_query_$}}
> * {{request_firstline_uri_query_$}}
> * {{request_referer_last_query_$}}
> * {{request_referer_query_$}}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (DRILL-8081) Maven Connection timed out
Cong Luo created DRILL-8081:
---
Summary: Maven Connection timed out
Key: DRILL-8081
URL: https://issues.apache.org/jira/browse/DRILL-8081
Project: Apache Drill
Issue Type: Improvement
Reporter: Cong Luo
Assignee: Cong Luo
Fix For: 1.20.0
{code:java}
Error: Failed to execute goal on project drill-format-excel: Could not resolve
dependencies for project
org.apache.drill.contrib:drill-format-excel:jar:1.20.0-SNAPSHOT: Failed to
collect dependencies at com.github.pjfanning:excel-streaming-reader:jar:3.2.6:
Failed to read artifact descriptor for
com.github.pjfanning:excel-streaming-reader:jar:3.2.6: Could not transfer
artifact com.github.pjfanning:excel-streaming-reader:pom:3.2.6 from/to central
(https://repo.maven.apache.org/maven2): transfer failed for
https://repo.maven.apache.org/maven2/com/github/pjfanning/excel-streaming-reader/3.2.6/excel-streaming-reader-3.2.6.pom:
Connection timed out (Read failed) -> [Help 1] {code}
We can learn from this note [Maven Connection timed
out|https://github.com/actions/virtual-environments/issues/1499]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (DRILL-8080) Enable the describe Case-Insensitive for Phoenix
Cong Luo created DRILL-8080: --- Summary: Enable the describe Case-Insensitive for Phoenix Key: DRILL-8080 URL: https://issues.apache.org/jira/browse/DRILL-8080 Project: Apache Drill Issue Type: Improvement Reporter: Cong Luo Assignee: Cong Luo Fix For: 1.20.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (DRILL-8061) Add Impersonation Support for Phoenix
[ https://issues.apache.org/jira/browse/DRILL-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453084#comment-17453084 ] Cong Luo commented on DRILL-8061: - [~vitalii] Hi, thank you for doing this. > Add Impersonation Support for Phoenix > - > > Key: DRILL-8061 > URL: https://issues.apache.org/jira/browse/DRILL-8061 > Project: Apache Drill > Issue Type: New Feature > Components: Storage - Other >Reporter: Cong Luo >Assignee: Vitalii Diravka >Priority: Major > Fix For: 1.20.0 > > > * https://phoenix.apache.org/server.html#Impersonation > * https://drill.apache.org/docs/configuring-user-impersonation -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (DRILL-8062) Add testcontainers Support for Phoenix
Cong Luo created DRILL-8062: --- Summary: Add testcontainers Support for Phoenix Key: DRILL-8062 URL: https://issues.apache.org/jira/browse/DRILL-8062 Project: Apache Drill Issue Type: New Feature Reporter: Cong Luo Assignee: Cong Luo Fix For: Future The phoenix storage use the mini queryserver as test framework, it is required to bundle the Hadoop 3 (rebuild HBase with Hadoop 3). If you found that exists a phoenix docker images (based on Phoenix 5 and PQS 6), it is best to use it as the default test framework. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (DRILL-8061) Add Impersonation Support for Phoenix
Cong Luo created DRILL-8061: --- Summary: Add Impersonation Support for Phoenix Key: DRILL-8061 URL: https://issues.apache.org/jira/browse/DRILL-8061 Project: Apache Drill Issue Type: New Feature Reporter: Cong Luo Assignee: Cong Luo Fix For: Future * https://phoenix.apache.org/server.html#Impersonation * https://drill.apache.org/docs/configuring-user-impersonation -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (DRILL-8060) Avoid the NPE in the Filter of Iceberg Implementor
Cong Luo created DRILL-8060:
---
Summary: Avoid the NPE in the Filter of Iceberg Implementor
Key: DRILL-8060
URL: https://issues.apache.org/jira/browse/DRILL-8060
Project: Apache Drill
Issue Type: Improvement
Reporter: Cong Luo
Assignee: Cong Luo
Fix For: 1.20.0
Query the phoenix data using the filter got the NPE (stack error at the Iceberg
format module), because the _#canImplement(Filter filter)_ might accept a
_*JDBCTableScan*_ (or an object that extend the _*TableScan*_ of Calcite), then
_#getDrillTable(TableScan)_ will get the null.
{code:java}
Caused by: java.lang.NullPointerException: null
at
org.apache.drill.exec.store.iceberg.plan.IcebergPluginImplementor.canImplement(IcebergPluginImplementor.java:102)
at
org.apache.drill.exec.store.plan.rule.PluginConverterRule.matches(PluginConverterRule.java:64)
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.matchRecurse(VolcanoRuleCall.java:263)
at
org.apache.calcite.plan.volcano.VolcanoRuleCall.match(VolcanoRuleCall.java:247)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.fireRules(VolcanoPlanner.java:1566)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1840)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:848)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:864)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:92)
at
org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:329)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1701)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:848)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:864)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.changeTraits(VolcanoPlanner.java:531)
at
org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:324)
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:405)
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:351)
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel(DefaultSqlHandler.java:245)
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:308)
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:173)
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:283)
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan(DrillSqlWorker.java:163)
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan(DrillSqlWorker.java:128)
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:93)
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:593)
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:274)
... 3 common frames omitted {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (DRILL-8025) Bump HBase client version to 2.4.8
[ https://issues.apache.org/jira/browse/DRILL-8025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo reassigned DRILL-8025: --- Assignee: (was: Cong Luo) > Bump HBase client version to 2.4.8 > -- > > Key: DRILL-8025 > URL: https://issues.apache.org/jira/browse/DRILL-8025 > Project: Apache Drill > Issue Type: Task >Reporter: Cong Luo >Priority: Major > Fix For: 1.20.0 > > > *[Very minor, but can be good for newcomers]* > Notice that, the HBase community have changed the stable point to 2.4.x (no > longer the 2.3). In the next release, Drill use the 2.5 or 2.6 series of > client and server (bundled the mini cluster). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (DRILL-8025) Bump HBase client version to 2.4.8
[ https://issues.apache.org/jira/browse/DRILL-8025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo reassigned DRILL-8025: --- Assignee: Cong Luo > Bump HBase client version to 2.4.8 > -- > > Key: DRILL-8025 > URL: https://issues.apache.org/jira/browse/DRILL-8025 > Project: Apache Drill > Issue Type: Task >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > Fix For: 1.20.0 > > > *[Very minor, but can be good for newcomers]* > Notice that, the HBase community have changed the stable point to 2.4.x (no > longer the 2.3). In the next release, Drill use the 2.5 or 2.6 series of > client and server (bundled the mini cluster). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (DRILL-8025) Bump HBase client version to 2.4.8
[ https://issues.apache.org/jira/browse/DRILL-8025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440175#comment-17440175 ] Cong Luo commented on DRILL-8025: - [~volodymyr] Hi, could you please add [~z0ltrix] to this JIRA list? Thanks. > Bump HBase client version to 2.4.8 > -- > > Key: DRILL-8025 > URL: https://issues.apache.org/jira/browse/DRILL-8025 > Project: Apache Drill > Issue Type: Task >Reporter: Cong Luo >Priority: Major > Fix For: 1.20.0 > > > *[Very minor, but can be good for newcomers]* > Notice that, the HBase community have changed the stable point to 2.4.x (no > longer the 2.3). In the next release, Drill use the 2.5 or 2.6 series of > client and server (bundled the mini cluster). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (DRILL-8025) Bump HBase client version to 2.4.8
[ https://issues.apache.org/jira/browse/DRILL-8025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo reassigned DRILL-8025: --- Assignee: (was: Cong Luo) > Bump HBase client version to 2.4.8 > -- > > Key: DRILL-8025 > URL: https://issues.apache.org/jira/browse/DRILL-8025 > Project: Apache Drill > Issue Type: Task >Reporter: Cong Luo >Priority: Major > Fix For: 1.20.0 > > > *[Very minor, but can be good for newcomers]* > Notice that, the HBase community have changed the stable point to 2.4.x (no > longer the 2.3). In the next release, Drill use the 2.5 or 2.6 series of > client and server (bundled the mini cluster). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (DRILL-8025) Bump HBase client version to 2.4.8
[ https://issues.apache.org/jira/browse/DRILL-8025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438792#comment-17438792 ] Cong Luo edited comment on DRILL-8025 at 11/4/21, 3:52 PM: --- [~z0ltrix] Hello. Please see the [HBase releases|http://archive.apache.org/dist/hbase/]. Actually, there is an email is related to the [[DISCUSS] move stable pointer to 2.4.6+|https://lists.apache.org/thread/nqhh0gxx51n2ny6vx9bkq5qpp74pw0m7]. In addition, are you interested in this contribution? was (Author: luoc): [~z0ltrix] Hello. Please see the [HBase releases|http://archive.apache.org/dist/hbase/]. Actually, there is an email is related to the [[DISCUSS] move stable pointer to 2.4.6+ |https://lists.apache.org/thread/nqhh0gxx51n2ny6vx9bkq5qpp74pw0m7]. > Bump HBase client version to 2.4.8 > -- > > Key: DRILL-8025 > URL: https://issues.apache.org/jira/browse/DRILL-8025 > Project: Apache Drill > Issue Type: Task >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > Fix For: 1.20.0 > > > *[Very minor, but can be good for newcomers]* > Notice that, the HBase community have changed the stable point to 2.4.x (no > longer the 2.3). In the next release, Drill use the 2.5 or 2.6 series of > client and server (bundled the mini cluster). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (DRILL-8025) Bump HBase client version to 2.4.8
[ https://issues.apache.org/jira/browse/DRILL-8025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438792#comment-17438792 ] Cong Luo edited comment on DRILL-8025 at 11/4/21, 3:45 PM: --- [~z0ltrix] Hello. Please see the [HBase releases|http://archive.apache.org/dist/hbase/]. Actually, there is an email is related to the [[DISCUSS] move stable pointer to 2.4.6+ |https://lists.apache.org/thread/nqhh0gxx51n2ny6vx9bkq5qpp74pw0m7]. was (Author: luoc): [~z0ltrix] Hello. Please see the [HBase releases|http://archive.apache.org/dist/hbase/]. Actually, there is an email is related to the [DISCUSS] move stable pointer to 2.4.6+ |https://lists.apache.org/thread/nqhh0gxx51n2ny6vx9bkq5qpp74pw0m7]. > Bump HBase client version to 2.4.8 > -- > > Key: DRILL-8025 > URL: https://issues.apache.org/jira/browse/DRILL-8025 > Project: Apache Drill > Issue Type: Task >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > Fix For: 1.20.0 > > > *[Very minor, but can be good for newcomers]* > Notice that, the HBase community have changed the stable point to 2.4.x (no > longer the 2.3). In the next release, Drill use the 2.5 or 2.6 series of > client and server (bundled the mini cluster). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-8025) Bump HBase client version to 2.4.8
[ https://issues.apache.org/jira/browse/DRILL-8025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438792#comment-17438792 ] Cong Luo commented on DRILL-8025: - [~z0ltrix] Hello. Please see the [HBase releases|http://archive.apache.org/dist/hbase/]. Actually, there is an email is related to the [DISCUSS] move stable pointer to 2.4.6+ |https://lists.apache.org/thread/nqhh0gxx51n2ny6vx9bkq5qpp74pw0m7]. > Bump HBase client version to 2.4.8 > -- > > Key: DRILL-8025 > URL: https://issues.apache.org/jira/browse/DRILL-8025 > Project: Apache Drill > Issue Type: Task >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > Fix For: 1.20.0 > > > *[Very minor, but can be good for newcomers]* > Notice that, the HBase community have changed the stable point to 2.4.x (no > longer the 2.3). In the next release, Drill use the 2.5 or 2.6 series of > client and server (bundled the mini cluster). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-8026) Sqlline 1.12 upgrade
Cong Luo created DRILL-8026: --- Summary: Sqlline 1.12 upgrade Key: DRILL-8026 URL: https://issues.apache.org/jira/browse/DRILL-8026 Project: Apache Drill Issue Type: Task Reporter: Cong Luo Fix For: 1.20.0 Upgrade to SqlLine 1.12 once it is released (https://github.com/julianhyde/sqlline/issues/451). The goal of the update is to better support Apple M1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-8025) Bump HBase client version to 2.4.8
Cong Luo created DRILL-8025: --- Summary: Bump HBase client version to 2.4.8 Key: DRILL-8025 URL: https://issues.apache.org/jira/browse/DRILL-8025 Project: Apache Drill Issue Type: Task Reporter: Cong Luo Assignee: Cong Luo Fix For: 1.20.0 *[Very minor, but can be good for newcomers]* Notice that, the HBase community have changed the stable point to 2.4.x (no longer the 2.3). In the next release, Drill use the 2.5 or 2.6 series of client and server (bundled the mini cluster). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-8024) Bump Mongo client and Testcontainers version to latest
Cong Luo created DRILL-8024: --- Summary: Bump Mongo client and Testcontainers version to latest Key: DRILL-8024 URL: https://issues.apache.org/jira/browse/DRILL-8024 Project: Apache Drill Issue Type: Task Reporter: Cong Luo Assignee: Cong Luo Fix For: 1.20.0 Update for the mongo 4 series. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-8021) Add the srv protocol support for the mongo storage
Cong Luo created DRILL-8021:
---
Summary: Add the srv protocol support for the mongo storage
Key: DRILL-8021
URL: https://issues.apache.org/jira/browse/DRILL-8021
Project: Apache Drill
Issue Type: Improvement
Reporter: Cong Luo
Assignee: Cong Luo
Fix For: 1.20.0
Add the srv protocol support for the mongo storage :
{code:java}
"connection": "mongodb+srv://"
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (DRILL-7997) Fix the type mismatch of mongo module in eclipse
Cong Luo created DRILL-7997:
---
Summary: Fix the type mismatch of mongo module in eclipse
Key: DRILL-7997
URL: https://issues.apache.org/jira/browse/DRILL-7997
Project: Apache Drill
Issue Type: Bug
Reporter: Cong Luo
Assignee: Cong Luo
Fix For: 1.20.0
Import the source code into the Eclipse (newer version), got the error message
below :
{code:java}
Type mismatch: cannot convert from Z#0-SELF'#8 extends
GenericContainer & ... to Z#0-SELF'#4 extends
GenericContainer & ...
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (DRILL-7989) Use the UTC formatter in the JSON reader
Cong Luo created DRILL-7989: --- Summary: Use the UTC formatter in the JSON reader Key: DRILL-7989 URL: https://issues.apache.org/jira/browse/DRILL-7989 Project: Apache Drill Issue Type: Improvement Reporter: Cong Luo Assignee: Cong Luo Fix For: 1.20.0 MongoDB use the UTC format to specify the date value by default. But the JSON reader (old version) use the fixed date formatter : "-MM-dd'T'HH:mm:ss.SSSXX". Need to change to the "-MM-dd'T'HH:mm:ss.SSS'Z'" format. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (DRILL-7987) Missing Javadoc
[ https://issues.apache.org/jira/browse/DRILL-7987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402122#comment-17402122 ] Cong Luo edited comment on DRILL-7987 at 8/20/21, 9:14 AM: --- [~CoolKangaroo2020] Sure. Thanks for your interest in the Apache Drill. there is a link resource about the [Drill : 1.19.0 API|https://drill.apache.org/apidocs/overview-summary.html]. Then, Welcome to join our [Slack Channel|https://bit.ly/3t4rozO]. was (Author: luoc): [~CoolKangaroo2020] Sure. Thanks for your interest in the Apache Drill. there is a link resource about the [Drill : 1.19.0 API|[https://drill.apache.org/apidocs/overview-summary.html].]. Then, Welcome to join our [Slack Channel|https://bit.ly/3t4rozO]. > Missing Javadoc > --- > > Key: DRILL-7987 > URL: https://issues.apache.org/jira/browse/DRILL-7987 > Project: Apache Drill > Issue Type: Wish >Reporter: CoolGuy >Priority: Minor > > Hello! I find in your [Contribution > Guidelines|http://drill.apache.org/docs/apache-drill-contribution-guidelines/] > that > ??All public classes and methods should have informative Javadoc comments.?? > However, I still witness some public classes without Javadocs, e.g., > * protocol/src/main/java/org/apache/drill/exec/proto/BitData.java > * > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/MergingReceiverCreator.java > * > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTable.java > .. > I also find likely missing Javadoc for public methods in the following files: > * drill-yarn/src/main/java/org/apache/drill/yarn/core/YarnRMClient.java > * protocol/src/main/java/org/apache/drill/exec/proto/SchemaBitControl.java > .. > Would you like to add Javadoc for these methods or classes, or they are just > not that important? > If there is anything I can help with? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (DRILL-7987) Missing Javadoc
[ https://issues.apache.org/jira/browse/DRILL-7987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402122#comment-17402122 ] Cong Luo edited comment on DRILL-7987 at 8/20/21, 9:13 AM: --- [~CoolKangaroo2020] Sure. Thanks for your interest in the Apache Drill. there is a link resource about the [Drill : 1.19.0 API|[https://drill.apache.org/apidocs/overview-summary.html].]. Then, Welcome to join our [Slack Channel|https://bit.ly/3t4rozO]. was (Author: luoc): [~CoolKangaroo2020] Sure. Thanks for your interest in the Apache Drill. there is a link resource about the [Drill : 1.19.0 API|[https://drill.apache.org/apidocs/overview-summary.html|https://drill.apache.org/apidocs/overview-summary.html].]]. Then, Welcome to join our [Slack Channel|https://bit.ly/3t4rozO]. > Missing Javadoc > --- > > Key: DRILL-7987 > URL: https://issues.apache.org/jira/browse/DRILL-7987 > Project: Apache Drill > Issue Type: Wish >Reporter: CoolGuy >Priority: Minor > > Hello! I find in your [Contribution > Guidelines|http://drill.apache.org/docs/apache-drill-contribution-guidelines/] > that > ??All public classes and methods should have informative Javadoc comments.?? > However, I still witness some public classes without Javadocs, e.g., > * protocol/src/main/java/org/apache/drill/exec/proto/BitData.java > * > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/MergingReceiverCreator.java > * > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTable.java > .. > I also find likely missing Javadoc for public methods in the following files: > * drill-yarn/src/main/java/org/apache/drill/yarn/core/YarnRMClient.java > * protocol/src/main/java/org/apache/drill/exec/proto/SchemaBitControl.java > .. > Would you like to add Javadoc for these methods or classes, or they are just > not that important? > If there is anything I can help with? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7987) Missing Javadoc
[ https://issues.apache.org/jira/browse/DRILL-7987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402122#comment-17402122 ] Cong Luo commented on DRILL-7987: - [~CoolKangaroo2020] Sure. Thanks for your interest in the Apache Drill. there is a link resource about the [Drill : 1.19.0 API|[https://drill.apache.org/apidocs/overview-summary.html|https://drill.apache.org/apidocs/overview-summary.html].]]. Then, Welcome to join our [Slack Channel|https://bit.ly/3t4rozO]. > Missing Javadoc > --- > > Key: DRILL-7987 > URL: https://issues.apache.org/jira/browse/DRILL-7987 > Project: Apache Drill > Issue Type: Wish >Reporter: CoolGuy >Priority: Minor > > Hello! I find in your [Contribution > Guidelines|http://drill.apache.org/docs/apache-drill-contribution-guidelines/] > that > ??All public classes and methods should have informative Javadoc comments.?? > However, I still witness some public classes without Javadocs, e.g., > * protocol/src/main/java/org/apache/drill/exec/proto/BitData.java > * > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/MergingReceiverCreator.java > * > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTable.java > .. > I also find likely missing Javadoc for public methods in the following files: > * drill-yarn/src/main/java/org/apache/drill/yarn/core/YarnRMClient.java > * protocol/src/main/java/org/apache/drill/exec/proto/SchemaBitControl.java > .. > Would you like to add Javadoc for these methods or classes, or they are just > not that important? > If there is anything I can help with? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7983) Add a REST API to support the get running or completed profiles
Cong Luo created DRILL-7983:
---
Summary: Add a REST API to support the get running or completed
profiles
Key: DRILL-7983
URL: https://issues.apache.org/jira/browse/DRILL-7983
Project: Apache Drill
Issue Type: New Feature
Reporter: Cong Luo
Fix For: 1.20.0
Drill have a REST API to get the profiles of running and completed queries.
this goal of ticket is extended from the `/profiles.json` to support get the
running or completed profiles (not all of them). the client does not need to
filter with the state.
old URI
{code:java}
/profiles.json{code}
new URI
{code:java}
/profiles.json
/profiles/running
/profiles/completed{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (DRILL-7982) Timed out waiting for container port to open
Cong Luo created DRILL-7982:
---
Summary: Timed out waiting for container port to open
Key: DRILL-7982
URL: https://issues.apache.org/jira/browse/DRILL-7982
Project: Apache Drill
Issue Type: Task
Components: Tools, Build Test
Reporter: Cong Luo
Assignee: Cong Luo
Fix For: 1.20.0
If the default 60s timeout is not sufficient, it can be altered with the
{{withStartupTimeout()}} method.
See also [Waiting for containers to start or be
ready|https://www.testcontainers.org/features/startup_and_waits/]
The java stack trace logs :
{code:java}
Error: org.apache.drill.exec.store.cassandra.TestCassandraSuit Time elapsed:
190.343 s <<< ERROR!
16634org.testcontainers.containers.ContainerLaunchException: Container startup
failed
16635 at
org.apache.drill.exec.store.cassandra.TestCassandraSuit.startCassandra(TestCassandraSuit.java:69)
16636 at
org.apache.drill.exec.store.cassandra.TestCassandraSuit.initCassandra(TestCassandraSuit.java:46)
16637Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit
hit with exception
16638 at
org.apache.drill.exec.store.cassandra.TestCassandraSuit.startCassandra(TestCassandraSuit.java:69)
16639 at
org.apache.drill.exec.store.cassandra.TestCassandraSuit.initCassandra(TestCassandraSuit.java:46)
16640Caused by: org.testcontainers.containers.ContainerLaunchException: Could
not create/start container
16641 at
org.apache.drill.exec.store.cassandra.TestCassandraSuit.startCassandra(TestCassandraSuit.java:69)
16642 at
org.apache.drill.exec.store.cassandra.TestCassandraSuit.initCassandra(TestCassandraSuit.java:46)
16643Caused by: org.testcontainers.containers.ContainerLaunchException: Timed
out waiting for container port to open (localhost ports: [49174] should be
listening)
16644 at
org.apache.drill.exec.store.cassandra.TestCassandraSuit.startCassandra(TestCassandraSuit.java:69)
16645 at
org.apache.drill.exec.store.cassandra.TestCassandraSuit.initCassandra(TestCassandraSuit.java:46)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (DRILL-7981) Bump commons-compress from 1.20 to 1.21 for CVE-2021-36090
Cong Luo created DRILL-7981: --- Summary: Bump commons-compress from 1.20 to 1.21 for CVE-2021-36090 Key: DRILL-7981 URL: https://issues.apache.org/jira/browse/DRILL-7981 Project: Apache Drill Issue Type: Improvement Reporter: Cong Luo Assignee: Cong Luo Fix For: 1.20.0 When reading a specially crafted ZIP archive, Compress can be made to allocate large amounts of memory that finally leads to an out of memory error even for very small inputs. This could be used to mount a denial of service attack against services that use Compress' zip package. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7962) Stop query hbase region when other fragment have received more records than LIMIT records
[ https://issues.apache.org/jira/browse/DRILL-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17374156#comment-17374156 ] Cong Luo commented on DRILL-7962: - [~Paul.Rogers] Thank you for the information. I think the key point is to reproduce the issue at first. [~Homaebic] Hi. Would you mind posting the profile of query (A json file on the drill directory) and all the log files (drillbit.log)? thanks. > Stop query hbase region when other fragment have received more records than > LIMIT records > - > > Key: DRILL-7962 > URL: https://issues.apache.org/jira/browse/DRILL-7962 > Project: Apache Drill > Issue Type: Wish > Components: Storage - HBase >Affects Versions: 1.19.0 >Reporter: Hanoch Yang >Priority: Blocker > > I find if one fragment have received records more than limit records, drill > will not let other fragment cancel, but work unit received message, it will > slow the query. > case: what data i want locate in 2M region, but there is a 100M region and 2 > left 2M region. i use limit 10 to query. 4 fragement query 4 region, when one > fragment get 10 record from 2M region, another 100M scanning fragment will > not work. so all fragment must wait that fragment to finish scan, but it > could have been unnecessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7963) Unexpected results when casting expression as decimal from parquet file
[ https://issues.apache.org/jira/browse/DRILL-7963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373939#comment-17373939 ] Cong Luo commented on DRILL-7963: - [~mpn] Hi. Are you interested in the hot fix (contribute the pull request)? Please let me know if you are going to do that. > Unexpected results when casting expression as decimal from parquet file > --- > > Key: DRILL-7963 > URL: https://issues.apache.org/jira/browse/DRILL-7963 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill, Storage - Parquet >Affects Versions: 1.19.0 >Reporter: Martin Puge Nielsen >Priority: Major > Attachments: TestFile_1.parquet, TestFile_1_script.txt > > > See attached parquet file & test script. > I would expect all "calculatedHrsNorm" columns to return "1.00" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7962) Stop query hbase region when other fragment have received more records than LIMIT records
[ https://issues.apache.org/jira/browse/DRILL-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373933#comment-17373933 ] Cong Luo commented on DRILL-7962: - [~paul-rogers] Hi. Is that the right way to process that (All the fragments must be end)? I think the only reason might be that Drill wants to make per query (included the LIMIT) return the same results (If the data sets are unchanged). > Stop query hbase region when other fragment have received more records than > LIMIT records > - > > Key: DRILL-7962 > URL: https://issues.apache.org/jira/browse/DRILL-7962 > Project: Apache Drill > Issue Type: Wish > Components: Storage - HBase >Affects Versions: 1.19.0 >Reporter: Hanoch Yang >Priority: Blocker > > I find if one fragment have received records more than limit records, drill > will not let other fragment cancel, but work unit received message, it will > slow the query. > case: what data i want locate in 2M region, but there is a 100M region and 2 > left 2M region. i use limit 10 to query. 4 fragement query 4 region, when one > fragment get 10 record from 2M region, another 100M scanning fragment will > not work. so all fragment must wait that fragment to finish scan, but it > could have been unnecessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7958) Build fails due to splunk repo is unavailable
[
https://issues.apache.org/jira/browse/DRILL-7958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373925#comment-17373925
]
Cong Luo commented on DRILL-7958:
-
[~code_kr_dev_s] Hi, You can also pay attention to the
[Slack|https://bit.ly/3t4rozO] channel if you are interested in the
contribution.
> Build fails due to splunk repo is unavailable
> -
>
> Key: DRILL-7958
> URL: https://issues.apache.org/jira/browse/DRILL-7958
> Project: Apache Drill
> Issue Type: Bug
> Components: Tools, Build Test
>Affects Versions: 1.19.0
>Reporter: Vitalii Diravka
>Priority: Blocker
> Labels: maven, splunk
> Fix For: 1.19.0
>
>
> _mvn clean install_ fails due to
> [https://splunk.jfrog.io/artifactory/ext-releases-local] repo is unavailable
> now
> {code:java}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
> (process-resource-bundles) on project distribution: Error resolving project
> artifact: Could not transfer artifact io.confluent:common-utils:pom:6.1.1
> from/to splunk-artifactory
> (https://splunk.jfrog.io/artifactory/ext-releases-local): Authorization
> failed for
> https://splunk.jfrog.io/artifactory/ext-releases-local/io/confluent/common-utils/6.1.1/common-utils-6.1.1.pom
> 403 Forbidden for project io.confluent:common-utils:jar:6.1.1 -> [Help 1]
> {code}
> The "Dev Splunk" claims to use:
> {code:java}
>
> splunk-artifactory
> Splunk Releases
> https://splunk.jfrog.io/artifactory/ext-releases-local
>
> {code}
> as we're using now.
>
> [https://dev.splunk.com/enterprise/docs/devtools/java/sdk-java/gettingstartedsdkjava/installsdkjava/#Use-Maven]
> But it gives the following error:
> {code:java}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
> (process-resource-bundles) on project distribution: Error resolving project
> artifact: Could not transfer artifact io.confluent:common-utils:pom:6.1.1
> from/to splunk-artifactory
> (https://splunk.jfrog.io/artifactory/ext-releases-local): Authorization
> failed for
> https://splunk.jfrog.io/artifactory/ext-releases-local/io/confluent/common-utils/6.1.1/common-utils-6.1.1.pom
> 403 Forbidden for project io.confluent:common-utils:jar:6.1.1 -> [Help 1]
> {code}
> In case of removing this repo, the build fails:
> {code:java}
> [ERROR] Failed to execute goal on project drill-storage-splunk: Could not
> resolve dependencies for project
> org.apache.drill.contrib:drill-storage-splunk:jar:1.19.0-SNAPSHOT: Failed to
> collect dependencies at com.splunk:splunk:jar:1.6.5.0: Failed to read
> artifact descriptor for com.splunk:splunk:jar:1.6.5.0: Could not transfer
> artifact com.splunk:splunk:pom:1.6.5.0 from/to splunk-artifactory
> (https://splunk.jfrog.io/artifactory/ext-releases-local): Authorization
> failed for
> https://splunk.jfrog.io/artifactory/ext-releases-local/com/splunk/splunk/1.6.5.0/splunk-1.6.5.0.pom
> 403 Forbidden -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> {code}
> We can wait for a while until they respond with a solution. In case they will
> not respond soon, we might want to disable Splunk related staff from the
> build.
> We can consider to upload slunk libs to some other repo, to make sure they
> will be available always.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (DRILL-7956) Web interface hangs afte execute too many queries
[
https://issues.apache.org/jira/browse/DRILL-7956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373430#comment-17373430
]
Cong Luo edited comment on DRILL-7956 at 7/3/21, 6:17 AM:
--
[~Homaebic] Hi, Have you tried the release 1.19? The 1.19 included an enhanced
feature for the RESTful queries (support the streaming read)
was (Author: luoc):
[~Homaebic] Hi, Have you tried the release 1.19? The 1.19 included an enhanced
feature for the RESTful querise (support the streaming read)
> Web interface hangs afte execute too many queries
> -
>
> Key: DRILL-7956
> URL: https://issues.apache.org/jira/browse/DRILL-7956
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - HTTP
>Affects Versions: 1.18.0
>Reporter: Hanoch Yang
>Priority: Major
> Attachments: 1561.jstack
>
>
> I used 1.18.0 to install the drill locally and installed an hbase storage
> plugin. When I executed the query continuously, when the fifth query was
> submitted, the REST API started to become unresponsive and the web page could
> not be accessed. After a few minutes, everything returned to normal. However,
> during this period of time, the REST API was unavailable, and the logs and
> system monitoring did not find any abnormalities. Why?
> Every sigle query will spent 7min to execute
> start command :
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre/bin/java
> -Xms4G -Xmx4G -XX:MaxDirectMemorySize=8G -XX:ReservedCodeCacheSize=1G
> -Ddrill.exec.enable-epoll=false
> -Dlog.path=/usr/local/apache-drill-1.18.0/log/sqlline.log
> -Dlog.query.path=/usr/local/apache-drill-1.18.0/log/sqlline_queries.json
> -Dorg.jline.terminal.dumb=true -cp
> /usr/local/apache-drill-1.18.0/conf:/usr/local/apache-drill-1.18.0/jars/*:/usr/local/apache-drill-1.18.0/jars/ext/*:/usr/local/apache-drill-1.18.0/jars/3rdparty/*:/usr/local/apache-drill-1.18.0/jars/classb/*:/usr/local/apache-drill-1.18.0/jars/3rdparty/linux/*
> sqlline.SqlLine -ac org.apache.drill.exec.client.DrillSqlLineApplication
> --color=true -u jdbc:drill:zk=local
>
> This is hbase config:
>
> {
> "type": "hbase",
> "config":
> { "hbase.zookeeper.quorum": "ip,ip,ip,ip",
> "hbase.zookeeper.property.clientPort": "2181" }
> ,
> "size.calculator.enabled": false,
> "enabled": true
> }
> This is the query statement:
> select * from `hbase`.`qqq` where CONVERT_FROM(`qqq`.c1.ip,'UTF8') =
> '10.38.12.94' and CONVERT_FROM(`qqq`.c1.`time`,'UTF8') >= '162178560' and
> CONVERT_FROM(`qqq`.c1.`time`,'UTF8') <= '162446400'
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (DRILL-7952) Fix Scan Stats for Mongo Plugin
[ https://issues.apache.org/jira/browse/DRILL-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7952. - Reviewer: Jason Altekruse Resolution: Fixed > Fix Scan Stats for Mongo Plugin > --- > > Key: DRILL-7952 > URL: https://issues.apache.org/jira/browse/DRILL-7952 > Project: Apache Drill > Issue Type: Bug > Components: Storage - MongoDB >Affects Versions: 1.19.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.20.0 > > > Minor fix. Updates the scan stats to use Estimated Row Count instead of > exact. No user facing changes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7955) Extend XOR bitwise functions to match AND and OR
[ https://issues.apache.org/jira/browse/DRILL-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7955. - Reviewer: Charles Givre Resolution: Fixed > Extend XOR bitwise functions to match AND and OR > > > Key: DRILL-7955 > URL: https://issues.apache.org/jira/browse/DRILL-7955 > Project: Apache Drill > Issue Type: Improvement > Components: Functions - Drill >Affects Versions: 1.19.0 >Reporter: James Turton >Assignee: James Turton >Priority: Minor > Fix For: 1.20.0 > > Original Estimate: 24h > Remaining Estimate: 24h > > The bitwise XOR function (^) is only implemented for INT while AND (&) and OR > (|) are implemented for all widths of integer. Also there is no BIT_XOR > aggregate while there are both BIT_AND and BIT_OR. This improvement proposes > to bring XOR in line with AND and OR in both of these respects. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (DRILL-7951) Add Response Metadata Fields to HTTP Storage Plugin
[ https://issues.apache.org/jira/browse/DRILL-7951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo closed DRILL-7951. --- Fix Version/s: (was: Future) 1.20.0 Reviewer: Cong Luo Resolution: Fixed > Add Response Metadata Fields to HTTP Storage Plugin > --- > > Key: DRILL-7951 > URL: https://issues.apache.org/jira/browse/DRILL-7951 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Other >Affects Versions: 1.19.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.20.0 > > > In many http calls, the response code can contain meaningful information, > however these fields are not available unless the call results in an error. > This PR adds four implicit metadata fields to the HTTP storage plugin which > are: > * _response_code: The HTTP response code. This field is returned as an INT. > > * _resposne_message: The response message. > * _response_protocol: The response protocol. > * _response_url: The actual URL that is sent to the API. > As these are implicit fields, they will not appear in star queries. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7949) documentation error - missing link
[ https://issues.apache.org/jira/browse/DRILL-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7949. - Fix Version/s: 1.20.0 Reviewer: James Turton Resolution: Fixed > documentation error - missing link > -- > > Key: DRILL-7949 > URL: https://issues.apache.org/jira/browse/DRILL-7949 > Project: Apache Drill > Issue Type: Task >Reporter: Ted Dunning >Priority: Major > Fix For: 1.20.0 > > > In checking rc1 for 1.19, I noted that this page: > [https://drill.apache.org/docs/configuring-storage-plugins/] > has a link to "Start the web UI" to > [https://drill.apache.org/docs/starting-the-web-console/] > and that page does not exist. > I think that link should go to > [https://drill.apache.org/docs/starting-the-web-ui/] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7956) Web interface hangs afte execute too many queries
[
https://issues.apache.org/jira/browse/DRILL-7956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373430#comment-17373430
]
Cong Luo commented on DRILL-7956:
-
[~Homaebic] Hi, Have you tried the release 1.19? The 1.19 included an enhanced
feature for the RESTful querise (support the streaming read)
> Web interface hangs afte execute too many queries
> -
>
> Key: DRILL-7956
> URL: https://issues.apache.org/jira/browse/DRILL-7956
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - HTTP
>Affects Versions: 1.18.0
>Reporter: Hanoch Yang
>Priority: Major
> Attachments: 1561.jstack
>
>
> I used 1.18.0 to install the drill locally and installed an hbase storage
> plugin. When I executed the query continuously, when the fifth query was
> submitted, the REST API started to become unresponsive and the web page could
> not be accessed. After a few minutes, everything returned to normal. However,
> during this period of time, the REST API was unavailable, and the logs and
> system monitoring did not find any abnormalities. Why?
> Every sigle query will spent 7min to execute
> start command :
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jre/bin/java
> -Xms4G -Xmx4G -XX:MaxDirectMemorySize=8G -XX:ReservedCodeCacheSize=1G
> -Ddrill.exec.enable-epoll=false
> -Dlog.path=/usr/local/apache-drill-1.18.0/log/sqlline.log
> -Dlog.query.path=/usr/local/apache-drill-1.18.0/log/sqlline_queries.json
> -Dorg.jline.terminal.dumb=true -cp
> /usr/local/apache-drill-1.18.0/conf:/usr/local/apache-drill-1.18.0/jars/*:/usr/local/apache-drill-1.18.0/jars/ext/*:/usr/local/apache-drill-1.18.0/jars/3rdparty/*:/usr/local/apache-drill-1.18.0/jars/classb/*:/usr/local/apache-drill-1.18.0/jars/3rdparty/linux/*
> sqlline.SqlLine -ac org.apache.drill.exec.client.DrillSqlLineApplication
> --color=true -u jdbc:drill:zk=local
>
> This is hbase config:
>
> {
> "type": "hbase",
> "config":
> { "hbase.zookeeper.quorum": "ip,ip,ip,ip",
> "hbase.zookeeper.property.clientPort": "2181" }
> ,
> "size.calculator.enabled": false,
> "enabled": true
> }
> This is the query statement:
> select * from `hbase`.`qqq` where CONVERT_FROM(`qqq`.c1.ip,'UTF8') =
> '10.38.12.94' and CONVERT_FROM(`qqq`.c1.`time`,'UTF8') >= '162178560' and
> CONVERT_FROM(`qqq`.c1.`time`,'UTF8') <= '162446400'
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (DRILL-7953) Query failed with (Too many open files)
[ https://issues.apache.org/jira/browse/DRILL-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17373420#comment-17373420 ] Cong Luo commented on DRILL-7953: - [~dony.dong] Hi, I think the issues has already been resolved after the release 1.17. > Query failed with (Too many open files) > --- > > Key: DRILL-7953 > URL: https://issues.apache.org/jira/browse/DRILL-7953 > Project: Apache Drill > Issue Type: Bug > Components: Server >Affects Versions: 1.15.0 >Reporter: Dony Dong >Priority: Major > > Hi Support, > > When we query a complex view that will access a lot of parquet files, the > query failed with error, > Caused by: org.apache.drill.common.exceptions.ExecutionSetupException: Error > opening or reading metadata for parquet file at location: > part-6-df5fe7db-6086-43a3-9575-1b18c140b5e6-c000.snappy.parquet > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.(PageReader.java:151) > ~[drill-java-exec-1.15.0.jar:1.15.0] > at > org.apache.drill.exec.store.parquet.columnreaders.AsyncPageReader.(AsyncPageReader.java:97) > ~[drill-java-exec-1.15.0.jar:1.15.0] > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReader.(ColumnReader.java:100) > ~[drill-java-exec-1.15.0.jar:1.15.0] > at > org.apache.drill.exec.store.parquet.columnreaders.NullableColumnReader.(NullableColumnReader.java:43) > ~[drill-java-exec-1.15.0.jar:1.15.0] > at > org.apache.drill.exec.store.parquet.columnreaders.NullableFixedByteAlignedReaders$NullableFixedByteAlignedReader.(NullableFixedByteAlignedReaders.java:54) > ~[drill-java-exec-1.15.0.jar:1.15.0] > at > org.apache.drill.exec.store.parquet.columnreaders.NullableFixedByteAlignedReaders$NullableConvertedReader.(NullableFixedByteAlignedReaders.java:328) > ~[drill-java-exec-1.15.0.jar:1.15.0] > at > org.apache.drill.exec.store.parquet.columnreaders.NullableFixedByteAlignedReaders$NullableDateReader.(NullableFixedByteAlignedReaders.java:348) > ~[drill-java-exec-1.15.0.jar:1.15.0] > at > org.apache.drill.exec.store.parquet.columnreaders.ColumnReaderFactory.createFixedColumnReader(ColumnReaderFactory.java:185) > ~[drill-java-exec-1.15.0.jar:1.15.0] > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetColumnMetadata.makeFixedWidthReader(ParquetColumnMetadata.java:141) > ~[drill-java-exec-1.15.0.jar:1.15.0] > at > org.apache.drill.exec.store.parquet.columnreaders.ReadState.buildReader(ReadState.java:123) > ~[drill-java-exec-1.15.0.jar:1.15.0] > at > org.apache.drill.exec.store.parquet.columnreaders.ParquetRecordReader.setup(ParquetRecordReader.java:253) > ~[drill-java-exec-1.15.0.jar:1.15.0] > ... 29 common frames omitted > Caused by: java.io.FileNotFoundException: > /data/testing/CH/part-6-df5fe7db-6086-43a3-9575-1b18c140b5e6-c000.snappy.parquet > (Too many open files) > at java.io.FileInputStream.open0(Native Method) ~[na:1.8.0_181] > at java.io.FileInputStream.open(FileInputStream.java:195) ~[na:1.8.0_181] > at java.io.FileInputStream.(FileInputStream.java:138) ~[na:1.8.0_181] > at > org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.(RawLocalFileSystem.java:106) > ~[hadoop-common-2.7.4.jar:na] > at org.apache.hadoop.fs.RawLocalFileSystem.open(RawLocalFileSystem.java:202) > ~[hadoop-common-2.7.4.jar:na] > at > org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.(ChecksumFileSystem.java:143) > ~[hadoop-common-2.7.4.jar:na] > at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:346) > ~[hadoop-common-2.7.4.jar:na] > at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:769) > ~[hadoop-common-2.7.4.jar:na] > at > org.apache.drill.exec.store.dfs.DrillFileSystem.open(DrillFileSystem.java:151) > ~[drill-java-exec-1.15.0.jar:1.15.0] > at > org.apache.drill.exec.store.parquet.columnreaders.PageReader.(PageReader.java:133) > ~[drill-java-exec-1.15.0.jar:1.15.0] > ... 39 common frames omitted > > We add below in the /etc/security/limits.conf, but Drill still uses the > default setting 1024 when startup. > * hard nofile 65536 > * soft nofile 65536 > > Fri Jun 18 19:40:33 AEST 2021 Starting drillbit on drill-testing > core file size (blocks, -c) 0 > data seg size (kbytes, -d) unlimited > scheduling priority (-e) 0 > file size (blocks, -f) unlimited > pending signals (-i) 740731 > max locked memory (kbytes, -l) 64 > max memory size (kbytes, -m) unlimited > open files (-n) 1024 > pipe size (512 bytes, -p) 8 > POSIX message queues (bytes, -q) 819200 > real-time priority (-r) 0 > stack size (kbytes, -s) 8192 > cpu time (seconds, -t) unlimited > max user processes (-u) 740731 > virtual memory (kbytes, -v) unlimited > file locks (-x) unlimited > > is there some place we can set this parameter? > > > -- This message was sent by Atlassian Jira
[jira] [Resolved] (DRILL-7920) Documentation missing for "defaultSchema" in REST Api
[ https://issues.apache.org/jira/browse/DRILL-7920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7920. - Reviewer: Cong Luo Resolution: Fixed > Documentation missing for "defaultSchema" in REST Api > - > > Key: DRILL-7920 > URL: https://issues.apache.org/jira/browse/DRILL-7920 > Project: Apache Drill > Issue Type: Task > Components: Documentation >Affects Versions: 1.18.0 >Reporter: Nick Stenroos-Dam >Assignee: Charles Givre >Priority: Trivial > Fix For: 1.19.0 > > > Looks like "defaultSchema" was added as an parameter to the REST API in 1.18, > however documentation is missing for this issue. > See https://issues.apache.org/jira/browse/DRILL-7603 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7946) Bump HttpClient from 4.5.12 to 4.5.13 for CVE-2020-13956
Cong Luo created DRILL-7946: --- Summary: Bump HttpClient from 4.5.12 to 4.5.13 for CVE-2020-13956 Key: DRILL-7946 URL: https://issues.apache.org/jira/browse/DRILL-7946 Project: Apache Drill Issue Type: Improvement Reporter: Cong Luo Assignee: Cong Luo Fix For: 1.19.0 Apache HttpClient versions prior to version 4.5.13 and 5.0.3 can misinterpret malformed authority component in request URIs passed to the library as java.net.URI object and pick the wrong target host for request execution. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7901) Bump junit from 4.12 to 4.13.1
[ https://issues.apache.org/jira/browse/DRILL-7901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7901. - Reviewer: Vova Vysotskyi Resolution: Fixed > Bump junit from 4.12 to 4.13.1 > -- > > Key: DRILL-7901 > URL: https://issues.apache.org/jira/browse/DRILL-7901 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.18.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.19.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7872) Heap memory within docker container does not get recycled
[
https://issues.apache.org/jira/browse/DRILL-7872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cong Luo resolved DRILL-7872.
-
Resolution: Not A Problem
> Heap memory within docker container does not get recycled
> -
>
> Key: DRILL-7872
> URL: https://issues.apache.org/jira/browse/DRILL-7872
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Christian
>Priority: Major
> Fix For: 1.19.0
>
>
> Hi - I'm running the official Docker container apache/drill:1.18.0. The Heap
> memory usage keeps increasing after each query until I get a
> "\{\n \"errorMessage\" : \"RESOURCE ERROR: There is not enough heap memory to
> run this query using the web interface. \\n\\nPlease try a query with fewer
> columns or with a filter or limit condition to limit the data returned.
> \\nYou can also try an ODBC/JDBC client. \\n\\n[Error Id:
> d5fef16a-cca7-4ee2-8316-98d510e0f41b ]\"\n}" error.
> I tested it by literally sending the same query over and over and it always
> does the same - the first few times the queries succeed but at the same time
> the heap increases until the max heap size is reached and then I just get
> above error.
> I'm querying parquet files through the REST api.
> I changed the config of the drill-env.sh file but no luck with these either.
> export DRILLBIT_MAX_PROC_MEM=${DRILLBIT_MAX_PROC_MEM:-"20G"}
> export DRILL_HEAP=${DRILL_HEAP:-"8G"}
> export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"8G"}
>
> export DRILLBIT_CODE_CACHE_SIZE=${DRILLBIT_CODE_CACHE_SIZE:-"2G"}
> Any guidance would be appreciated.
> Regards,
> Christian
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (DRILL-7860) Suppress the transfer progress when downloading in CI maven build
[
https://issues.apache.org/jira/browse/DRILL-7860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cong Luo resolved DRILL-7860.
-
Reviewer: Charles Givre
Resolution: Fixed
> Suppress the transfer progress when downloading in CI maven build
> -
>
> Key: DRILL-7860
> URL: https://issues.apache.org/jira/browse/DRILL-7860
> Project: Apache Drill
> Issue Type: Improvement
> Components: Tools, Build Test
>Reporter: Cong Luo
>Assignee: Cong Luo
>Priority: Trivial
> Fix For: 1.19.0
>
>
> For example:
> {code:java}
> Downloading from jhdf5-repo:
> https://maven.scijava.org/content/repositories/public/org/apache/hbase/hbase-annotations/2.2.2/hbase-annotations-2.2.2.jar
> Downloading from jhdf5-repo:
> https://maven.scijava.org/content/repositories/public/org/apache/htrace/htrace-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar
> Downloading from jhdf5-repo:
> https://maven.scijava.org/content/repositories/public/org/jruby/jcodings/jcodings/1.0.8/jcodings-1.0.8.jar
> Downloading from jhdf5-repo:
> https://maven.scijava.org/content/repositories/public/org/jruby/joni/joni/2.1.2/joni-2.1.2.jar
> Progress (1): 4.1/187 kB
> Progress (1): 7.7/187 kB
> Progress (2): 7.7/187 kB | 0/1.3 MB
> Progress (3): 7.7/187 kB | 0/1.3 MB | 4.1/6.8 kB
> Progress (3): 7.7/187 kB | 0/1.3 MB | 6.8 kB
> Progress (4): 7.7/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 12/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 16/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 20/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 24/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 28/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 32/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 36/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 40/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 40/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 40/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 40/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 40/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 40/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 40/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 40/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 40/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 44/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 48/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 52/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 56/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 60/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 64/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 68/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 72/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 74/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 78/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 82/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> Progress (4): 82/187 kB | 0/1.3 MB | 6.8 kB | 0/1.5 MB
> {code}
> Use an option:
> {code:java}
> mvn --no-transfer-progress {code}
> See also [Release Notes – Maven 3.6.1|
> http://maven.apache.org/docs/3.6.1/release-notes.html#user-visible-changes]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (DRILL-7922) Add the profile import function
[ https://issues.apache.org/jira/browse/DRILL-7922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341333#comment-17341333 ] Cong Luo commented on DRILL-7922: - [~volodymyr] Hi, Could you please assign the PR to [~leon_wtf]? thanks > Add the profile import function > --- > > Key: DRILL-7922 > URL: https://issues.apache.org/jira/browse/DRILL-7922 > Project: Apache Drill > Issue Type: New Feature >Reporter: Cong Luo >Priority: Major > Fix For: 1.19.0 > > > We can easy to read the profile metrics on the web console, but hard to > review the profile of other cluster (specified the users. although they can > provide the profile json, but the file is not easy to readable). > Since we can export the profile json via the console, So, We can also add the > import function to quickly review the new cluster metrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7922) Add the profile import function
Cong Luo created DRILL-7922: --- Summary: Add the profile import function Key: DRILL-7922 URL: https://issues.apache.org/jira/browse/DRILL-7922 Project: Apache Drill Issue Type: New Feature Reporter: Cong Luo Fix For: 1.19.0 We can easy to read the profile metrics on the web console, but hard to review the profile of other cluster (specified the users. although they can provide the profile json, but the file is not easy to readable). Since we can export the profile json via the console, So, We can also add the import function to quickly review the new cluster metrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (DRILL-7921) Support the ARM-based system
[ https://issues.apache.org/jira/browse/DRILL-7921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo updated DRILL-7921: Description: This is an umbrella Jira for support to running on ARM-based system. # Build must be passed. # All the JUnit test must be passed. # Running stable on the aarch64 machines. was: This is a umbrella Jira for support to running on ARM-based system. # Build must be passed. # All the JUnit test must be passed. # Running stable on the aarch64 machines. > Support the ARM-based system > > > Key: DRILL-7921 > URL: https://issues.apache.org/jira/browse/DRILL-7921 > Project: Apache Drill > Issue Type: New Feature >Reporter: Cong Luo >Assignee: Martin Tzvetanov Grigorov >Priority: Major > Fix For: 1.19.0 > > > This is an umbrella Jira for support to running on ARM-based system. > # Build must be passed. > # All the JUnit test must be passed. > # Running stable on the aarch64 machines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (DRILL-7921) Support the ARM-based system
[ https://issues.apache.org/jira/browse/DRILL-7921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo reassigned DRILL-7921: --- Assignee: Martin Tzvetanov Grigorov (was: Cong Luo) > Support the ARM-based system > > > Key: DRILL-7921 > URL: https://issues.apache.org/jira/browse/DRILL-7921 > Project: Apache Drill > Issue Type: New Feature >Reporter: Cong Luo >Assignee: Martin Tzvetanov Grigorov >Priority: Major > Fix For: 1.19.0 > > > This is a umbrella Jira for support to running on ARM-based system. > # Build must be passed. > # All the JUnit test must be passed. > # Running stable on the aarch64 machines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (DRILL-7921) Support the ARM-based system
[ https://issues.apache.org/jira/browse/DRILL-7921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo reassigned DRILL-7921: --- Assignee: Cong Luo > Support the ARM-based system > > > Key: DRILL-7921 > URL: https://issues.apache.org/jira/browse/DRILL-7921 > Project: Apache Drill > Issue Type: New Feature >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > Fix For: 1.19.0 > > > This is a umbrella Jira for support to running on ARM-based system. > # Build must be passed. > # All the JUnit test must be passed. > # Running stable on the aarch64 machines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (DRILL-7911) Build faillure on Linux ARM64
[
https://issues.apache.org/jira/browse/DRILL-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cong Luo updated DRILL-7911:
Parent: DRILL-7921
Issue Type: Sub-task (was: Bug)
> Build faillure on Linux ARM64
> -
>
> Key: DRILL-7911
> URL: https://issues.apache.org/jira/browse/DRILL-7911
> Project: Apache Drill
> Issue Type: Sub-task
>Reporter: Martin Tzvetanov Grigorov
>Assignee: Martin Tzvetanov Grigorov
>Priority: Major
> Fix For: 1.19.0
>
>
> There are few problems when trying to build Drill on Linux ARM64 machine:
>
> 1) org.apache.drill.exec.record.vector.TestValueVector fails with
> OutOfMemoryError while trying to allocate native memory.
> {code:java}
> org.apache.drill.exec.record.vector.TestValueVector.testVariableVectorReallocation
> Time elapsed: 0.012 s <<< ERROR!
> java.lang.Exception: Unexpected exception,
> expected but
> was
> at
> org.apache.drill.exec.record.vector.TestValueVector.testVariableVectorReallocation(TestValueVector.java:165)
> Caused by: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate
> 2147483647 byte(s) of direct memory (used: 889192448, max: 262144)
> at
> org.apache.drill.exec.record.vector.TestValueVector.testVariableVectorReallocation(TestValueVector.java:165)
> {code}
> I've tried to debug it by adding io.netty.leakDetection.level=paranoid to
> maven-surefire-plugin's argLine but instead of reporting the leaks this
> actually worked around the issue
>
> 2) Contrib Storage Mongo module tests fail because flapdoodle.mongo
> dependency does not support Linux ARM64
> I've created the following PRs to fix it:
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/119]{color:#00}
> - Add support for AARCH64{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/12|https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/120]+0+{color:#00}
> - Add support for comparing versions{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/34|https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/343]+3+{color:#00}
> - Add support for Linux AARCH64 and newer Mongodb versions{color}
>
> *{color:#00}Update:{color}* {color:#00}Contrib{color}
> {color:#00}Storage{color}{color:#00} Mongo module has been migrated
> to use TestContainers-Mongo{color}
>
> {color:#00}3) Contrib Storage JDBC tests fail because
> {color}com.wix:wix-embedded-mysql does not support Linux ARM64
> I've started adding support for ARM64 to Wix but then I realized that the
> project might be dead. Last commit to
> [https://github.com/wix/wix-embedded-mysql] is at Nov 3rd 2019. So I am not
> sure whether the improvement will be ever accepted and released.
> My new approach here is to use TestContainers-MySQL instead of
> com.wix:wix-embedded-mysql. Apache Drill already uses TestContainers for
> Storage Cassandra and TestContainers Vault for java-exec module.
>
> 4) Module "Contrib : Storage : Splunk" fails because there is no ARM64 Docker
> image for "splunk". For now I cannot find any third party Docker image for
> Splunk on ARM64. The only solutions I see is to disable the Splunk module
> tests on Linux ARM64
> According to
> [https://docs.splunk.com/Documentation/Splunk/latest/Installation/Systemrequirements#Unix_operating_systems]
> Linux ARM64 is not supported.
> I've created [https://github.com/splunk/docker-splunk/issues/479] requesting
> for adding support for ARM64.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (DRILL-7921) Support the ARM-based system
Cong Luo created DRILL-7921: --- Summary: Support the ARM-based system Key: DRILL-7921 URL: https://issues.apache.org/jira/browse/DRILL-7921 Project: Apache Drill Issue Type: New Feature Reporter: Cong Luo Fix For: 1.19.0 This is a umbrella Jira for support to running on ARM-based system. # Build must be passed. # All the JUnit test must be passed. # Running stable on the aarch64 machines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7918) Dependency Updates for CVE
[ https://issues.apache.org/jira/browse/DRILL-7918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7918. - Reviewer: Vova Vysotskyi Resolution: Fixed > Dependency Updates for CVE > -- > > Key: DRILL-7918 > URL: https://issues.apache.org/jira/browse/DRILL-7918 > Project: Apache Drill > Issue Type: Task >Affects Versions: 1.18.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.19.0 > > > This is a blanket PR to bump dependency versions to address CVEs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7913) Improve the tips using the Tooltips
[ https://issues.apache.org/jira/browse/DRILL-7913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7913. - Reviewer: Charles Givre Resolution: Fixed > Improve the tips using the Tooltips > --- > > Key: DRILL-7913 > URL: https://issues.apache.org/jira/browse/DRILL-7913 > Project: Apache Drill > Issue Type: Improvement >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > Fix For: 1.19.0 > > > Actually, the tips of web console is hard to use. > # Too slowly (Delay 2s +) > # No good for the readable (Font size and bg color) > Based on the Bootstrap (existed) and Popper.js (new), Use the new module > Tooltips to improve these. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (DRILL-7535) Convert Ltsv to EVF
[ https://issues.apache.org/jira/browse/DRILL-7535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo closed DRILL-7535. --- Fix Version/s: (was: 1.19.0) Resolution: Duplicate > Convert Ltsv to EVF > --- > > Key: DRILL-7535 > URL: https://issues.apache.org/jira/browse/DRILL-7535 > Project: Apache Drill > Issue Type: Sub-task >Reporter: Arina Ielchiieva >Assignee: Charles Givre >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7920) Documentation missing for "defaultSchema" in REST Api
[ https://issues.apache.org/jira/browse/DRILL-7920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341312#comment-17341312 ] Cong Luo commented on DRILL-7920: - [~Nisd] Nice. We can talk the detail of contribute on [Slack|https://bit.ly/3t4rozO] channel. > Documentation missing for "defaultSchema" in REST Api > - > > Key: DRILL-7920 > URL: https://issues.apache.org/jira/browse/DRILL-7920 > Project: Apache Drill > Issue Type: Task > Components: Documentation >Affects Versions: 1.18.0 >Reporter: Nick Stenroos-Dam >Priority: Trivial > > Looks like "defaultSchema" was added as an parameter to the REST API in 1.18, > however documentation is missing for this issue. > See https://issues.apache.org/jira/browse/DRILL-7603 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (DRILL-7911) Build faillure on Linux ARM64
[
https://issues.apache.org/jira/browse/DRILL-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339574#comment-17339574
]
Cong Luo edited comment on DRILL-7911 at 5/5/21, 10:33 AM:
---
[~mgrigorov] Done. We can talk the detail of progress on Slack channel.
was (Author: luoc):
[~mgrigorov] Done. We can talk about the detail of progress on Slack channel.
> Build faillure on Linux ARM64
> -
>
> Key: DRILL-7911
> URL: https://issues.apache.org/jira/browse/DRILL-7911
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Martin Tzvetanov Grigorov
>Priority: Major
> Fix For: 1.19.0
>
>
> There are few problems when trying to build Drill on Linux ARM64 machine:
>
> 1) org.apache.drill.exec.record.vector.TestValueVector fails with
> OutOfMemoryError while trying to allocate native memory.
> I've tried to debug it by adding io.netty.leakDetection.level=paranoid to
> maven-surefire-plugin's argLine but instead of reporting the leaks this
> actually worked around the issue
>
> 2) Contrib Storage Mongo module tests fail because flapdoodle.mongo
> dependency does not support Linux ARM64
> I've created the following PRs to fix it:
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/119]{color:#00}
> - Add support for AARCH64{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/12|https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/120]+0+{color:#00}
> - Add support for comparing versions{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/34|https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/343]+3+{color:#00}
> - Add support for Linux AARCH64 and newer Mongodb versions{color}
> {color:#00}3) Contrib Storage JDBC tests fail because
> {color}com.wix:wix-embedded-mysql does not support Linux ARM64
> I've started adding support for ARM64 to Wix but then I realized that the
> project might be dead. Last commit to
> [https://github.com/wix/wix-embedded-mysql] is at Nov 3rd 2019. So I am not
> sure whether the improvement will be ever accepted and released.
> My new approach here is to use TestContainers-MySQL instead of
> com.wix:wix-embedded-mysql. Apache Drill already uses TestContainers for
> Storage Cassandra and TestContainers Vault for java-exec module.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (DRILL-7911) Build faillure on Linux ARM64
[
https://issues.apache.org/jira/browse/DRILL-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339574#comment-17339574
]
Cong Luo commented on DRILL-7911:
-
[~mgrigorov] Done. We can talk about the detail of progress on Slack channel.
> Build faillure on Linux ARM64
> -
>
> Key: DRILL-7911
> URL: https://issues.apache.org/jira/browse/DRILL-7911
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Martin Tzvetanov Grigorov
>Priority: Major
> Fix For: 1.19.0
>
>
> There are few problems when trying to build Drill on Linux ARM64 machine:
>
> 1) org.apache.drill.exec.record.vector.TestValueVector fails with
> OutOfMemoryError while trying to allocate native memory.
> I've tried to debug it by adding io.netty.leakDetection.level=paranoid to
> maven-surefire-plugin's argLine but instead of reporting the leaks this
> actually worked around the issue
>
> 2) Contrib Storage Mongo module tests fail because flapdoodle.mongo
> dependency does not support Linux ARM64
> I've created the following PRs to fix it:
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/119]{color:#00}
> - Add support for AARCH64{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/12|https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/120]+0+{color:#00}
> - Add support for comparing versions{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/34|https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/343]+3+{color:#00}
> - Add support for Linux AARCH64 and newer Mongodb versions{color}
> {color:#00}3) Contrib Storage JDBC tests fail because
> {color}com.wix:wix-embedded-mysql does not support Linux ARM64
> I've started adding support for ARM64 to Wix but then I realized that the
> project might be dead. Last commit to
> [https://github.com/wix/wix-embedded-mysql] is at Nov 3rd 2019. So I am not
> sure whether the improvement will be ever accepted and released.
> My new approach here is to use TestContainers-MySQL instead of
> com.wix:wix-embedded-mysql. Apache Drill already uses TestContainers for
> Storage Cassandra and TestContainers Vault for java-exec module.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (DRILL-7911) Build faillure on Linux ARM64
[
https://issues.apache.org/jira/browse/DRILL-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cong Luo updated DRILL-7911:
Fix Version/s: 1.19.0
> Build faillure on Linux ARM64
> -
>
> Key: DRILL-7911
> URL: https://issues.apache.org/jira/browse/DRILL-7911
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Martin Tzvetanov Grigorov
>Priority: Major
> Fix For: 1.19.0
>
>
> There are few problems when trying to build Drill on Linux ARM64 machine:
>
> 1) org.apache.drill.exec.record.vector.TestValueVector fails with
> OutOfMemoryError while trying to allocate native memory.
> I've tried to debug it by adding io.netty.leakDetection.level=paranoid to
> maven-surefire-plugin's argLine but instead of reporting the leaks this
> actually worked around the issue
>
> 2) Contrib Storage Mongo module tests fail because flapdoodle.mongo
> dependency does not support Linux ARM64
> I've created the following PRs to fix it:
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/119]{color:#00}
> - Add support for AARCH64{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/12|https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/120]+0+{color:#00}
> - Add support for comparing versions{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/34|https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/343]+3+{color:#00}
> - Add support for Linux AARCH64 and newer Mongodb versions{color}
> {color:#00}3) Contrib Storage JDBC tests fail because
> {color}com.wix:wix-embedded-mysql does not support Linux ARM64
> I've started adding support for ARM64 to Wix but then I realized that the
> project might be dead. Last commit to
> [https://github.com/wix/wix-embedded-mysql] is at Nov 3rd 2019. So I am not
> sure whether the improvement will be ever accepted and released.
> My new approach here is to use TestContainers-MySQL instead of
> com.wix:wix-embedded-mysql. Apache Drill already uses TestContainers for
> Storage Cassandra and TestContainers Vault for java-exec module.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (DRILL-7911) Build faillure on Linux ARM64
[
https://issues.apache.org/jira/browse/DRILL-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17339569#comment-17339569
]
Cong Luo commented on DRILL-7911:
-
[~mgrigorov] Thanks for doing that. Would you mind letting me change the status
of Jira to *In-Progress*?
> Build faillure on Linux ARM64
> -
>
> Key: DRILL-7911
> URL: https://issues.apache.org/jira/browse/DRILL-7911
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Martin Tzvetanov Grigorov
>Priority: Major
>
> There are few problems when trying to build Drill on Linux ARM64 machine:
>
> 1) org.apache.drill.exec.record.vector.TestValueVector fails with
> OutOfMemoryError while trying to allocate native memory.
> I've tried to debug it by adding io.netty.leakDetection.level=paranoid to
> maven-surefire-plugin's argLine but instead of reporting the leaks this
> actually worked around the issue
>
> 2) Contrib Storage Mongo module tests fail because flapdoodle.mongo
> dependency does not support Linux ARM64
> I've created the following PRs to fix it:
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/119]{color:#00}
> - Add support for AARCH64{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/12|https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/120]+0+{color:#00}
> - Add support for comparing versions{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/34|https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/343]+3+{color:#00}
> - Add support for Linux AARCH64 and newer Mongodb versions{color}
> {color:#00}3) Contrib Storage JDBC tests fail because
> {color}com.wix:wix-embedded-mysql does not support Linux ARM64
> I've started adding support for ARM64 to Wix but then I realized that the
> project might be dead. Last commit to
> [https://github.com/wix/wix-embedded-mysql] is at Nov 3rd 2019. So I am not
> sure whether the improvement will be ever accepted and released.
> My new approach here is to use TestContainers-MySQL instead of
> com.wix:wix-embedded-mysql. Apache Drill already uses TestContainers for
> Storage Cassandra and TestContainers Vault for java-exec module.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (DRILL-7915) Joker123 Slot - Daftar Slot Online Terpercaya - POPSLOT22
[ https://issues.apache.org/jira/browse/DRILL-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo closed DRILL-7915. --- Resolution: Not A Problem > Joker123 Slot - Daftar Slot Online Terpercaya - POPSLOT22 > - > > Key: DRILL-7915 > URL: https://issues.apache.org/jira/browse/DRILL-7915 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.16.0 >Reporter: Popslot22 >Priority: Major > Fix For: 2.0.0 > > Attachments: daftar-situs.gif > > > Popslot22 adalah agen joker123 terpercaya 2021 di Indonesia yang menyediakan > beragam jenis permainan slot dari total 12 pilihan provider dan masih banyak > permainan lain yang bisa anda mainkan untuk mendapatkan uang secara cepat. > Seperti yang kita ketahui bahwa bermain judi slot 2021 sangat menyenangkan > jika anda tidak salah memilih situs. Pilihlah situs yang sudah terbukti > membayar berapa pun kemenangan anda. > !daftar-situs.gif! > h2. Agen Joker123 Terpercaya 2021 > Adanya standar agen di permainan joker123 telah memperjelas semua tujuan > permainan yang selama ini kamu pilih. Sejak pertama kali dimainkan, sudah > tersedia berbagai macam layanan terbaik dari satu agen terpercaya. Hanya > saja, ada konsep baru yang membuat peluang berbeda yang dapat saja diakses > dengan melihat indikator tertentu dalam memilih agen. Maka dari itu, standar > dari satu agen harusnya dapat kamu perhatikan sebagai landasan utama untuk > mendapat nilai komisi yang lebih baik. > h2. Ada Tidaknya Sarana Lengkap Bermain Joker123 > Sebagai satu indikator utama pemain joker123 yang mana sekarang ada metode > tepat untuk mengetahui seperti apa dasar-dasar bermain lebih menarik. Sampai > detik ini ada metode terbaik agar ke depannya skema bermain juga sering > diperhitungkan sebagai langkah baru agar Anda mengetahui seperti apa peluang > besar untuk dijadikan sebuah pengalaman penting bagi semua pemain baru. > Sarana lengkap untuk dapat menghasilkan nilai profit besar semakin menarik > dengan adanya layanan yang lebih pasti. Dalam hal ini, metode bermain harus > dapat diakses secara langsung, sampai akhirnya teknik bermain lebih mudah > diakses dalam satu agen dengan fasilitas lebih lengkap sejak awal dimainkan. > h3. Memiliki Kredibilitas yang Sudah Terbukti > Hampir semua pemain memiliki standar terbaik dari pengembangan semua fitur > lebih lengkap dan akses kredibilitasnya sudah terbukti. Pada akhirnya, ada > metode-metode joker123 gaming terbaik yang selama ini dipakai sebagai langkah > tepat agar pemain benar-benar mengetahui bagaimana cara kerja hingga > memperlihatkan hasil menarik sejak awal diperhitungkan sebagai modal bermain > dari sisi kredibilitas agen. > Mengapa kredibilitas agen menjadi diperhitungkan bagi semua bettor? Sudah > pasti adanya aspek kredibilitas inilah membuat pola bermain menjadi lebih > nyaman saat dipakai. Ditambah lagi ada beberapa poin utama membuat pembuktian > dari semua pola dan beberapa aspek baru dalam bermain diharapkan > memperlihatkan hasil kemenangan yang lebih pasti. > h3. Menghasilkan Banyak Pola Bermain yang Lebih Menguntungkan > Ketika kamu bermain di satu agen terpercaya joker123, kesempatan meraih nilai > komisi yang lebih banyak semakin terbuka. Bentuk peluang yang lebih maksimal > semakin mudah dimanfaatkan dengan mempertimbangkan hasil bermain lebih > menarik. Karena itulah, kamu dapat mengetahui seperti apa dasar-dasar bermain > yang lebih pasti. Sampai akhirnya, ada metode terbaik yang dapat > diperhitungkan untuk tahu seperti apa pola-pola terbaik dan menguntungkan. > Jika dari pola menguntungkan ini dapat diakses secara tepat dan cepat, > peluang besar yang selama ini kamu inginkan dapat diperoleh. Nilai komisi > besar dalam mengelola hasil bermain juga perlu diperhatikan sebagai > langkah-langkah awal mendapatkan hasil kemenangan yang lebih terbuka. > h3. Tersedianya Promo Lengkap atau Tidak > Sebagai satu bagian penting dalam mengembangkan hasil-hasil bermain yang > secara pasti memperlihatkan langkah tepat agar kamu tidak salah langkah. Maka > dari itu, sejak awal ada potensi besar yang seharusnya dapat dipakai sebagai > langkah tepat memanfaatkan promo yang lebih pasti sejak awal. Pada dasarnya, > ada metode tepat yang seharusnya dapat diperhitungkan sebagai langkah > menguntungkan agar semua komponen bermain dapat diandalkan secara pasti. > Target utama dari satu pemain dapat saja mendapatkan promo lebih besar dalam > waktu cepat. Hasil inilah membuat pemain mengerti bagaimana cara-cara menang > lebih sempurna. Bahkan, ada metode bermain joker123 secara cepat untuk > melihat bagaimana perkembangan modal sampai fasilitas terbarunya. > Bentuk peluang besar dari pemilihan agen terpercaya joker123 sampai detik ini > terus
[jira] [Updated] (DRILL-7916) Support new plugin installation on the running system
[ https://issues.apache.org/jira/browse/DRILL-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo updated DRILL-7916: Description: Drill does not support the new plugin installation on the running system : # Boot the Drill. # Load plugins to the persistent storage : `pluginStore`. ## Upgrade the plugin if the override file exist (storage-plugins-override.conf). (Done) ## Check and add new plugin with the new release. (To-do) ## If 1 and 2 are not true, then initial all the plugins via loading bootstrap configuration. (Done) # End the Boot. was: Drill does not support the new plugin installation on the running system : # Boot the Drill. # Load plugins to the persistent storage : `pluginStore`. ## Upgrade the plugin if the override file exist (storage-plugins-override.conf). (Done) ## Check and add new plugin with the new release. (To-do) ## If 1 and 2 are not true, then initial all the plugins via loading bootstrap configuration. (Done) # End the Boot. > Support new plugin installation on the running system > - > > Key: DRILL-7916 > URL: https://issues.apache.org/jira/browse/DRILL-7916 > Project: Apache Drill > Issue Type: New Feature >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > Fix For: 1.19.0 > > > Drill does not support the new plugin installation on the running system : > # Boot the Drill. > # Load plugins to the persistent storage : `pluginStore`. > ## Upgrade the plugin if the override file exist > (storage-plugins-override.conf). (Done) > ## Check and add new plugin with the new release. (To-do) > ## If 1 and 2 are not true, then initial all the plugins via loading > bootstrap configuration. (Done) > # End the Boot. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7916) Support new plugin installation on the running system
Cong Luo created DRILL-7916: --- Summary: Support new plugin installation on the running system Key: DRILL-7916 URL: https://issues.apache.org/jira/browse/DRILL-7916 Project: Apache Drill Issue Type: New Feature Reporter: Cong Luo Assignee: Cong Luo Fix For: 1.19.0 Drill does not support the new plugin installation on the running system : # Boot the Drill. # Load plugins to the persistent storage : `pluginStore`. ## Upgrade the plugin if the override file exist (storage-plugins-override.conf). (Done) ## Check and add new plugin with the new release. (To-do) ## If 1 and 2 are not true, then initial all the plugins via loading bootstrap configuration. (Done) # End the Boot. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7912) Add Sheet Names to Excel Reader
[ https://issues.apache.org/jira/browse/DRILL-7912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7912. - Reviewer: Cong Luo Resolution: Fixed > Add Sheet Names to Excel Reader > --- > > Key: DRILL-7912 > URL: https://issues.apache.org/jira/browse/DRILL-7912 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Other >Affects Versions: 1.18.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.19.0 > > > Currently, there is no way to determine what sheets are available in an Excel > file. This PR adds a metadata field called `_sheets` which a user can query > to determine available sheets in a given file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7826) Merge Pcap and Pcapng format plugin based on EVF
[ https://issues.apache.org/jira/browse/DRILL-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7826. - Reviewer: Vitalii Diravka Resolution: Fixed > Merge Pcap and Pcapng format plugin based on EVF > > > Key: DRILL-7826 > URL: https://issues.apache.org/jira/browse/DRILL-7826 > Project: Apache Drill > Issue Type: Improvement >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > Fix For: 1.19.0 > > > This is umbrella Jira to track down process of merging Pcap and Pcapng format > plugin based on EVF. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7910) Bumps commons-io from 2.4 to 2.7
[ https://issues.apache.org/jira/browse/DRILL-7910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7910. - Reviewer: Cong Luo Resolution: Fixed > Bumps commons-io from 2.4 to 2.7 > > > Key: DRILL-7910 > URL: https://issues.apache.org/jira/browse/DRILL-7910 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.18.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.19.0 > > > This fix addresses CVE-2021-29425. > In Apache Commons IO before 2.7, When invoking the method > FileNameUtils.normalize with an improper input string, like "//../foo", or > "\..\foo", the result would be the same value, thus possibly providing access > to files in the parent directory, but not further above (thus "limited" path > traversal), if the calling code would use the result to construct a path > value. > Simple solution is to update the library. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (DRILL-7914) Apache Drill 1.19.0 Release Activities
[ https://issues.apache.org/jira/browse/DRILL-7914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo updated DRILL-7914: Fix Version/s: 1.19.0 > Apache Drill 1.19.0 Release Activities > -- > > Key: DRILL-7914 > URL: https://issues.apache.org/jira/browse/DRILL-7914 > Project: Apache Drill > Issue Type: Task >Reporter: Laurent Goujon >Assignee: Laurent Goujon >Priority: Major > Fix For: 1.19.0 > > > Source - https://github.com/apache/drill/blob/master/docs/dev/Release.md > 1.19.0 Dashboard: > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336127 > Kanban Board - > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=185 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7914) Apache Drill 1.19.0 Release Activities
[ https://issues.apache.org/jira/browse/DRILL-7914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338734#comment-17338734 ] Cong Luo commented on DRILL-7914: - Great start. We not only use mailing lists, but also [Slack|https://bit.ly/3t4rozO]. > Apache Drill 1.19.0 Release Activities > -- > > Key: DRILL-7914 > URL: https://issues.apache.org/jira/browse/DRILL-7914 > Project: Apache Drill > Issue Type: Task >Reporter: Laurent Goujon >Assignee: Laurent Goujon >Priority: Major > > Source - https://github.com/apache/drill/blob/master/docs/dev/Release.md > 1.19.0 Dashboard: > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336127 > Kanban Board - > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=185 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7913) Improve the tips using the Tooltips
Cong Luo created DRILL-7913: --- Summary: Improve the tips using the Tooltips Key: DRILL-7913 URL: https://issues.apache.org/jira/browse/DRILL-7913 Project: Apache Drill Issue Type: Improvement Reporter: Cong Luo Assignee: Cong Luo Fix For: 1.19.0 Actually, the tips of web console is hard to use. # Too slowly (Delay 2s +) # No good for the readable (Font size and bg color) Based on the Bootstrap (existed) and Popper.js (new), Use the new module Tooltips to improve these. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (DRILL-7911) Build faillure on Linux ARM64
[
https://issues.apache.org/jira/browse/DRILL-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335302#comment-17335302
]
Cong Luo edited comment on DRILL-7911 at 4/29/21, 9:45 AM:
---
[~mgrigorov] It is great to hear that someone had started build the Drill on
ARM machine. Welcome to join our [Slack Channel|https://bit.ly/3t4rozO]
was (Author: luoc):
[~mgrigorov] It is great to heat that someone had started build the Drill on
ARM machine. Welcome to join our [Slack Channel|https://bit.ly/3t4rozO]
> Build faillure on Linux ARM64
> -
>
> Key: DRILL-7911
> URL: https://issues.apache.org/jira/browse/DRILL-7911
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Martin Tzvetanov Grigorov
>Priority: Major
>
> There are few problems when trying to build Drill on Linux ARM64 machine:
>
> 1) org.apache.drill.exec.record.vector.TestValueVector fails with
> OutOfMemoryError while trying to allocate native memory.
> I've tried to debug it by adding io.netty.leakDetection.level=paranoid to
> maven-surefire-plugin's argLine but instead of reporting the leaks this
> actually worked around the issue
>
> 2) Contrib Storage Mongo module tests fail because flapdoodle.mongo
> dependency does not support Linux ARM64
> I've created the following PRs to fix it:
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/119]{color:#00}
> - Add support for AARCH64{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/12|https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/120]+0+{color:#00}
> - Add support for comparing versions{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/34|https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/343]+3+{color:#00}
> - Add support for Linux AARCH64 and newer Mongodb versions{color}
> {color:#00}3) Contrib Storage JDBC tests fail because
> {color}com.wix:wix-embedded-mysql does not support Linux ARM64
> I've started adding support for ARM64 to Wix but then I realized that the
> project might be dead. Last commit to
> [https://github.com/wix/wix-embedded-mysql] is at Nov 3rd 2019. So I am not
> sure whether the improvement will be ever accepted and released.
> My new approach here is to use TestContainers-MySQL instead of
> com.wix:wix-embedded-mysql. Apache Drill already uses TestContainers for
> Storage Cassandra and TestContainers Vault for java-exec module.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (DRILL-7911) Build faillure on Linux ARM64
[
https://issues.apache.org/jira/browse/DRILL-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17335302#comment-17335302
]
Cong Luo commented on DRILL-7911:
-
[~mgrigorov] It is great to heat that someone had started build the Drill on
ARM machine. Welcome to join our [Slack Channel|https://bit.ly/3t4rozO]
> Build faillure on Linux ARM64
> -
>
> Key: DRILL-7911
> URL: https://issues.apache.org/jira/browse/DRILL-7911
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Martin Tzvetanov Grigorov
>Priority: Major
>
> There are few problems when trying to build Drill on Linux ARM64 machine:
>
> 1) org.apache.drill.exec.record.vector.TestValueVector fails with
> OutOfMemoryError while trying to allocate native memory.
> I've tried to debug it by adding io.netty.leakDetection.level=paranoid to
> maven-surefire-plugin's argLine but instead of reporting the leaks this
> actually worked around the issue
>
> 2) Contrib Storage Mongo module tests fail because flapdoodle.mongo
> dependency does not support Linux ARM64
> I've created the following PRs to fix it:
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/119]{color:#00}
> - Add support for AARCH64{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/12|https://github.com/flapdoodle-oss/de.flapdoodle.embed.process/pull/120]+0+{color:#00}
> - Add support for comparing versions{color}
> [https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/34|https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/pull/343]+3+{color:#00}
> - Add support for Linux AARCH64 and newer Mongodb versions{color}
> {color:#00}3) Contrib Storage JDBC tests fail because
> {color}com.wix:wix-embedded-mysql does not support Linux ARM64
> I've started adding support for ARM64 to Wix but then I realized that the
> project might be dead. Last commit to
> [https://github.com/wix/wix-embedded-mysql] is at Nov 3rd 2019. So I am not
> sure whether the improvement will be ever accepted and released.
> My new approach here is to use TestContainers-MySQL instead of
> com.wix:wix-embedded-mysql. Apache Drill already uses TestContainers for
> Storage Cassandra and TestContainers Vault for java-exec module.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (DRILL-7890) mongodb java driver should be updated to latest (4.2) since connect to mongodb >4.0 is not possible
[ https://issues.apache.org/jira/browse/DRILL-7890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7890. - Fix Version/s: 1.19.0 Reviewer: Cong Luo Assignee: Vova Vysotskyi Resolution: Fixed > mongodb java driver should be updated to latest (4.2) since connect to > mongodb >4.0 is not possible > --- > > Key: DRILL-7890 > URL: https://issues.apache.org/jira/browse/DRILL-7890 > Project: Apache Drill > Issue Type: Bug > Components: Storage - MongoDB >Affects Versions: 1.18.0 >Reporter: Ognian Tschakalov >Assignee: Vova Vysotskyi >Priority: Major > Fix For: 1.19.0 > > > Please update the mongodb java driver from the current 3.8 to the latest 4.2 > version. > Trying to connect to a mongodb version higher than 4.0 results in a timeout. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7903) Bump mongo driver from 3.12 to 4.2
[ https://issues.apache.org/jira/browse/DRILL-7903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323844#comment-17323844 ] Cong Luo commented on DRILL-7903: - [~volodymyr] That's great. Looking forward to the PR. > Bump mongo driver from 3.12 to 4.2 > -- > > Key: DRILL-7903 > URL: https://issues.apache.org/jira/browse/DRILL-7903 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - MongoDB >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > Fix For: 1.19.0 > > > There is a good practice for upgrade the mongo driver. because the mongo > compatibility is more friendly for the driver (previous version). Please see > also > [MongoDB Compatibility|https://docs.mongodb.com/drivers/java/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DRILL-7903) Bump mongo driver from 3.12 to 4.2
Cong Luo created DRILL-7903: --- Summary: Bump mongo driver from 3.12 to 4.2 Key: DRILL-7903 URL: https://issues.apache.org/jira/browse/DRILL-7903 Project: Apache Drill Issue Type: Improvement Components: Storage - MongoDB Reporter: Cong Luo Assignee: Cong Luo Fix For: 1.19.0 There is a good practice for upgrade the mongo driver. because the mongo compatibility is more friendly for the driver (previous version). Please see also [MongoDB Compatibility|https://docs.mongodb.com/drivers/java/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7902) You need to describe how drill uses heap memory in the documentation
[ https://issues.apache.org/jira/browse/DRILL-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322755#comment-17322755 ] Cong Luo commented on DRILL-7902: - [~aglukho] Hello. There is a docs related to the `configuring drill memory` [Configuring Drill Memory|http://drill.apache.org/docs/configuring-drill-memory/] > You need to describe how drill uses heap memory in the documentation > > > Key: DRILL-7902 > URL: https://issues.apache.org/jira/browse/DRILL-7902 > Project: Apache Drill > Issue Type: Wish >Affects Versions: 1.17.0, 1.18.0 >Reporter: Александр Глухов >Priority: Major > > You need to describe how drill uses heap memory in the documentation. > What parameters the memory heap is regulated by. How is the memory allocated > between allocated memory and heap memory? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7712) Fix issues after ZK upgrade
[
https://issues.apache.org/jira/browse/DRILL-7712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cong Luo resolved DRILL-7712.
-
Reviewer: Vova Vysotskyi
Assignee: (was: Vova Vysotskyi)
Resolution: Fixed
> Fix issues after ZK upgrade
> ---
>
> Key: DRILL-7712
> URL: https://issues.apache.org/jira/browse/DRILL-7712
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Arina Ielchiieva
>Priority: Major
> Fix For: 1.19.0
>
>
> Warnings during jdbc-all build (absent when building with Mapr profile):
> {noformat}
> netty-transport-native-epoll-4.1.45.Final.jar,
> netty-transport-native-epoll-4.0.48.Final-linux-x86_64.jar define 46
> overlapping classes:
> - io.netty.channel.epoll.AbstractEpollStreamChannel$2
> - io.netty.channel.epoll.AbstractEpollServerChannel$EpollServerSocketUnsafe
> - io.netty.channel.epoll.EpollDatagramChannel
> - io.netty.channel.epoll.AbstractEpollStreamChannel$SpliceInChannelTask
> - io.netty.channel.epoll.NativeDatagramPacketArray
> - io.netty.channel.epoll.EpollSocketChannelConfig
> - io.netty.channel.epoll.EpollTcpInfo
> - io.netty.channel.epoll.EpollEventArray
> - io.netty.channel.epoll.EpollEventLoop
> - io.netty.channel.epoll.EpollSocketChannel
> - 36 more...
> netty-transport-native-unix-common-4.1.45.Final.jar,
> netty-transport-native-epoll-4.0.48.Final-linux-x86_64.jar define 15
> overlapping classes:
> - io.netty.channel.unix.Errors$NativeConnectException
> - io.netty.channel.unix.ServerDomainSocketChannel
> - io.netty.channel.unix.DomainSocketAddress
> - io.netty.channel.unix.Socket
> - io.netty.channel.unix.NativeInetAddress
> - io.netty.channel.unix.DomainSocketChannelConfig
> - io.netty.channel.unix.Errors$NativeIoException
> - io.netty.channel.unix.DomainSocketReadMode
> - io.netty.channel.unix.ErrorsStaticallyReferencedJniMethods
> - io.netty.channel.unix.UnixChannel
> - 5 more...
> maven-shade-plugin has detected that some class files are
> present in two or more JARs. When this happens, only one
> single version of the class is copied to the uber jar.
> Usually this is not harmful and you can skip these warnings,
> otherwise try to manually exclude artifacts based on
> mvn dependency:tree -Ddetail=true and the above output.
> See http://maven.apache.org/plugins/maven-shade-plugin/
> {noformat}
> Additional warning build with Mapr profile:
> {noformat}
> The following patterns were never triggered in this artifact inclusion filter:
> o 'org.apache.zookeeper:zookeeper-jute'
> {noformat}
> NPEs in tests (though tests do not fail):
> {noformat}
> [INFO] Running org.apache.drill.exec.coord.zk.TestZookeeperClient
> 4880
> java.lang.NullPointerException
> 4881
> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.fastForwardFromEdits(FileTxnSnapLog.java:269)
> 4882
> at
> org.apache.zookeeper.server.ZKDatabase.fastForwardDataBase(ZKDatabase.java:251)
> 4883
> at
> org.apache.zookeeper.server.ZooKeeperServer.shutdown(ZooKeeperServer.java:583)
> 4884
> at
> org.apache.zookeeper.server.ZooKeeperServer.shutdown(ZooKeeperServer.java:546)
> 4885
> at
> org.apache.zookeeper.server.NIOServerCnxnFactory.shutdown(NIOServerCnxnFactory.java:
> {noformat}
> {noformat}
> [INFO] Running org.apache.drill.exec.coord.zk.TestEphemeralStore
> 5278
> java.lang.NullPointerException
> 5279
> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.fastForwardFromEdits(FileTxnSnapLog.java:269)
> 5280
> at org.apache.zookeepe
> {noformat}
> {noformat}
> [INFO] Running org.apache.drill.yarn.zk.TestAmRegistration
> 6767
> java.lang.NullPointerException
> 6768
> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.fastForwardFromEdits(FileTxnSnapLog.java:269)
> 6769
> at
> org.apache.zookeeper.server.ZKDatabase.fastForwardDataBase(ZKDatabase.java:251)
> 6770
> at
> org.apache.zookeeper.server.ZooKeeperServer.shutdown(ZooKeeperServer.java:583)
> 6771
> at
> org.apache.zookeeper.server.ZooKeeperServer.shutdown(ZooKeeperServer.java:546)
> 6772
> at
> org.apache.zookeeper.server.NIOServerCnxnFactory.shutdown(NIOServerCnxnFactory.java:929)
> 6773
> at org.apache.curator.t
> {noformat}
> {noformat}
> org.apache.drill.yarn.client.TestCommandLineOptions
> 6823
> java.lang.NullPointerException
> 6824
> at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.fastForwardFromEdits(FileTxnSnapLog.java:269)
> 6825
> at
> org.apache.zookeeper.server.ZKDatabase.fastForwardDataBase(ZKDatabase.java:251)
> 6826
> at
> org.apache.zookeeper.server.ZooKeeperServer.shutdown(ZooKeeperServer.java:583)
> 6827
> at
> org.apache.zookeeper.server.ZooKeeperServer.shutdown(ZooKeeperServer.java:546)
> 6828
> at org.apac
>
[jira] [Resolved] (DRILL-7900) Bump libthrift from 0.13.0 to 0.14.0
[ https://issues.apache.org/jira/browse/DRILL-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7900. - Reviewer: Cong Luo Resolution: Fixed > Bump libthrift from 0.13.0 to 0.14.0 > > > Key: DRILL-7900 > URL: https://issues.apache.org/jira/browse/DRILL-7900 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.18.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.19.0 > > > Bumps [libthrift](https://github.com/apache/thrift) from 0.13.0 to 0.14.0. > > Release notes > Sourced from href="https://github.com/apache/thrift/releases;>libthrift's > releases. > > Version 0.14.0 > For release 0.14.0 head over to the official release download source: > href="http://thrift.apache.org/download;>http://thrift.apache.org/download > The assets below are added by Github based on the release tag and they may > therefore not match the checkums. > > > > Changelog > Sourced from href="https://github.com/apache/thrift/blob/master/CHANGES.md;>libthrift's > changelog. > > 0.14.0 > Deprecated Languages > > href="https://issues.apache.org/jira/browse/THRIFT-5229;>THRIFT-5229 - > Deprecate ActionScript 3 support > > Removed Languages > > href="https://issues.apache.org/jira/browse/THRIFT-4980;>THRIFT-4980 - > Remove deprecated C# and netcore bindings from the code base > href="https://issues.apache.org/jira/browse/THRIFT-4981;>THRIFT-4981 - > Remove deprecated netcore bindings from the code base > https://issues.apache.org/jira/browse/THRIFT-4982;>THRIFT -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7899) Bump checkstyle from 8.16 to 8.29
[ https://issues.apache.org/jira/browse/DRILL-7899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7899. - Reviewer: Cong Luo Resolution: Fixed > Bump checkstyle from 8.16 to 8.29 > -- > > Key: DRILL-7899 > URL: https://issues.apache.org/jira/browse/DRILL-7899 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.18.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.19.0 > > > Drill currently uses checkstyle 8.16 for which there is a CVE. > (https://github.com/advisories/GHSA-763g-fqq7-48wg). This PR updates the > checkstyle dependency and resolves the CVE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (DRILL-7894) Large Attributes Crash HDF5 Reader
[ https://issues.apache.org/jira/browse/DRILL-7894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Luo resolved DRILL-7894. - Reviewer: Cong Luo Resolution: Fixed > Large Attributes Crash HDF5 Reader > -- > > Key: DRILL-7894 > URL: https://issues.apache.org/jira/browse/DRILL-7894 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.18.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.19.0 > > > Certain HDF5 files crash when attempting to read attributes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7891) Drill Cluster Authentication failed. Incorrect credentials? PLAIN PAM authentication on node-2
[
https://issues.apache.org/jira/browse/DRILL-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314761#comment-17314761
]
Cong Luo commented on DRILL-7891:
-
[~noor.ciit...@hotmail.com] Welcome to join our [Slack
Channel|https://bit.ly/3t4rozO]
> Drill Cluster Authentication failed. Incorrect credentials? PLAIN PAM
> authentication on node-2
> --
>
> Key: DRILL-7891
> URL: https://issues.apache.org/jira/browse/DRILL-7891
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - JDBC, Functions - Drill
>Affects Versions: 1.18.0
> Environment: drill.exec: {
> cluster-id: "DrillMFSYS",
> zk.connect: "zk1:2181,zk2:2181,zk3:2181"
> impersonation: {
> enabled: true,
> max_chained_user_hops: 3
> },
> security: {
> auth.mechanisms : ["PLAIN"],
> user.encryption.sasl.enabled: false
> },
> security.user.auth: {
> enabled: true,
> packages += "org.apache.drill.exec.rpc.user.security",
> impl: "pam",
> pam_profiles: ["sudo"]
> }
> }
>Reporter: Noor
>Priority: Major
> Original Estimate: 504h
> Remaining Estimate: 504h
>
> [Client-1] ERROR o.o.a.d.e.r.s.AuthenticationOutcomeListener - Authentication
> with encryption context: Encryption: disabled , MaxWrappedSize: 65536 ,
> WrapSizeLimit: 0 using mechanism PLAIN failed with Authentication failed.
> Incorrect credentials? [Details: Encryption: disabled , MaxWrappedSize: 65536
> , WrapSizeLimit: 0]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (DRILL-7891) Drill Cluster Authentication failed. Incorrect credentials? PLAIN PAM authentication on node-2
[
https://issues.apache.org/jira/browse/DRILL-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17314376#comment-17314376
]
Cong Luo commented on DRILL-7891:
-
[~noor.ciit...@hotmail.com] There are users that have discussed the
troubleshooting (with auth) on the Slack recently, you can take a look on Slack
channel.
> Drill Cluster Authentication failed. Incorrect credentials? PLAIN PAM
> authentication on node-2
> --
>
> Key: DRILL-7891
> URL: https://issues.apache.org/jira/browse/DRILL-7891
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - JDBC, Functions - Drill
>Affects Versions: 1.18.0
> Environment: drill.exec: {
> cluster-id: "DrillMFSYS",
> zk.connect: "zk1:2181,zk2:2181,zk3:2181"
> impersonation: {
> enabled: true,
> max_chained_user_hops: 3
> },
> security: {
> auth.mechanisms : ["PLAIN"],
> user.encryption.sasl.enabled: false
> },
> security.user.auth: {
> enabled: true,
> packages += "org.apache.drill.exec.rpc.user.security",
> impl: "pam",
> pam_profiles: ["sudo"]
> }
> }
>Reporter: Noor
>Priority: Major
> Original Estimate: 504h
> Remaining Estimate: 504h
>
> [Client-1] ERROR o.o.a.d.e.r.s.AuthenticationOutcomeListener - Authentication
> with encryption context: Encryption: disabled , MaxWrappedSize: 65536 ,
> WrapSizeLimit: 0 using mechanism PLAIN failed with Authentication failed.
> Incorrect credentials? [Details: Encryption: disabled , MaxWrappedSize: 65536
> , WrapSizeLimit: 0]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (DRILL-7890) mongodb java driver should be updated to latest (4.2) since connect to mongodb >4.0 is not possible
[ https://issues.apache.org/jira/browse/DRILL-7890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313162#comment-17313162 ] Cong Luo commented on DRILL-7890: - [~Ognian] Thanks for your attention. I think it's easy to upgrade the mongo driver. However, Are you interested in the testing works (both 3.x and 4.x)? > mongodb java driver should be updated to latest (4.2) since connect to > mongodb >4.0 is not possible > --- > > Key: DRILL-7890 > URL: https://issues.apache.org/jira/browse/DRILL-7890 > Project: Apache Drill > Issue Type: Bug > Components: Storage - MongoDB >Affects Versions: 1.18.0 >Reporter: Ognian Tschakalov >Priority: Major > > Please update the mongodb java driver from the current 3.8 to the latest 4.2 > version. > Trying to connect to a mongodb version higher than 4.0 results in a timeout. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DRILL-7889) ODBC driver issues
[ https://issues.apache.org/jira/browse/DRILL-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17310321#comment-17310321 ] Cong Luo commented on DRILL-7889: - [~Klar] Welcome. Glad to see your spirit of research. > ODBC driver issues > -- > > Key: DRILL-7889 > URL: https://issues.apache.org/jira/browse/DRILL-7889 > Project: Apache Drill > Issue Type: Test > Components: Client - ODBC >Affects Versions: 1.18.0 >Reporter: Christian >Priority: Blocker > Attachments: Capture.PNG > > > Hi - after facing the REST API heap memory issue I decided to give ODBC a try > however I can't get it to work even after reading through the full > documentation. > I'm running Drill within the Docker container (1.18.0) and e.g. sending REST > requests from my Windows machine works fine. > I downloaded the 64bit driver version from here: > [http://package.mapr.com/tools/MapR-ODBC/MapR_Drill/MapRDrill_odbc_v1.3.22.1055/] > It installs fine, however whenever I'm trying to Test the ODBC connection I'm > getting the following error: > [MapR][Drill] (20) Failure occurred while trying to connect to > local=10.103.155.103:8047. (Details: [30004]Connection failed with error: > Failed to read entire handshake message.) > I added the screenshot of the DSN setup - however all I really changed was > the IP and the port. > Any help would be appreciated. > > As an update I tried the below instead because 1.3.8 seems to be referenced > in the documentation. Here it does not error out, however indefinitely hangs. > [http://package.mapr.com/tools/MapR-ODBC/MapR_Drill/MapRDrill_odbc_v1.3.8.1030/] > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
