[jira] [Updated] (DRILL-8331) UNION ALL for multiple sheets in excel giving unexpected result when used with a subquery in not in
[
https://issues.apache.org/jira/browse/DRILL-8331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8331:
Priority: Major (was: Critical)
> UNION ALL for multiple sheets in excel giving unexpected result when used
> with a subquery in not in
> ---
>
> Key: DRILL-8331
> URL: https://issues.apache.org/jira/browse/DRILL-8331
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Excel
>Affects Versions: 1.20.2
>Reporter: Anshuman Mishra
>Priority: Major
> Attachments: TestingMerge.xlsx
>
>
> Hi All,
> I am having a query regarding a query I am firing up in drill and not getting
> the expected result set. Basically what I am doing is merging the data from
> three sheets and eliminating only those records present in the 4th sheet.
> The query:
> {quote}{_}select * from (SELECT EmpId,Test FROM
> TABLE(dfs.{_}{{{}_[C://TestingMerge.xlsx|file:///C://ATOP//all_files//XLS_XLSX//TestingMerge.xlsx]_{}}}{_}(type
> => 'excel', sheetName =>'Sheet1')) UNION ALL SELECT EmpId,Test FROM
> TABLE(dfs.{_}{{{}_[C://TestingMerge.xlsx|file:///C://ATOP//all_files//XLS_XLSX//TestingMerge.xlsx]_{}}}{_}(type
> => 'excel', sheetName =>'Sheet2')) UNION ALL SELECT EmpId,Test FROM
> TABLE(dfs.{_}{{{}_[C://TestingMerge.xlsx|file:///C://ATOP//all_files//XLS_XLSX//TestingMerge.xlsx]_{}}}{_}(type
> => 'excel', sheetName =>'Sheet3'))) where EmpID not in (select EmpID from
> TABLE(dfs.{_}{{{}_[C://TestingMerge.xlsx|file:///C://ATOP//all_files//XLS_XLSX//TestingMerge.xlsx]_{}}}{_}(type
> => 'excel', sheetName =>'Sheet4'))){_}
> {quote}
> The query when we execute it, returns no data, when it should just eliminate
> the data present in sheet4 from the excel sheet. Strangely enough when we do
> a union All on two sheets this query works perfectly fine and returns the
> expected result. One more finding is, if we use static values in 'not in'
> instead of 'subquery', the query works fine with all the 3 sheets as union
> all. I even gave a null check in the subquery used in 'not in' as it might
> consider null values as well, still no luck.
> NOTE: Instead of UNION ALL if UNION is applied then it is giving the expected
> result set.
> I have attached a sample file I am using to recreate the issue.
> [^TestingMerge.xlsx]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] (DRILL-7884) Error with list in Mongodb
[ https://issues.apache.org/jira/browse/DRILL-7884 ]
James Turton deleted comment on DRILL-7884:
-
was (Author: dzamo):
Hi, my first thought here is that the ODBC standard does not support complex
types like LIST and MAP. I know that JDBC doesn't and we make use of the
java.lang.Object type for LIST and MAP as a fallback.
> Error with list in Mongodb
> ---
>
> Key: DRILL-7884
> URL: https://issues.apache.org/jira/browse/DRILL-7884
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - ODBC
>Affects Versions: 1.18.0
> Environment: CentOS 7.7
> apache-drill-1.18.0
> drill-override.conf
> drill.exec: {
> cluster-id: "drillbits1",
> zk.connect: "localhost:2181"
> }
> apache-zookeeper-3.6.2
> Mongodb CE 3.6.5
>Reporter: Djamel MALKI
>Priority: Blocker
>
> Hello,
> We try to connect Apache-drill to Mongodb 3.6.5 with the ODBC driver MapR
> Drill 1.3 64-bit.
> The connection works well, We have only one collection inside the database
> with document with this json format :
> {
> "_id" : ObjectId("600fe61e47f2b8c6335515d0"),
> "hostid" : "HOST-07A8EEBACFDFA658",
> "cloudType" : "OnPrem",
> "hostname" : "lemon-dev02",
> "logicalCpuCores" : 4,
> "instanceType" : "NA",
> "host_cpu_usage_avg" : 17.39,
> "host_cpu_usage_max" : 42.96,
> "host_mem_usage_avg" : 46.32,
> "host_mem_usage_max" : 46.78,
> "host_mem_used" : 14.552923838297525,
> "host_avail_hours" : 2,
> "host_unavail_hours" : 0,
> "host_mem_avail_bytes" : 16.868035634358723,
> "host_total_mem" : "NA",
> "host_trafficIn" : {
> "ens160" : 0.16
> },
> "host_trafficOut" : {
> "ens160" : 0.3
> },
> "Disks" : [ ],
> {color:#FF}"tags"{color} : [
> "TECHNO_RABBITMQ",
> "rab",
> "TECHNO_MONGODB",
> "gpa",
> "TECHNO_POSTGRES",
> "nsm",
> "rse",
> "gre",
> "tech-unix",
> "ppe",
> "ide",
> "mpo",
> "l3m",
> "TECHNO_NGINX",
> "tech-dba",
> "sma",
> "TECHNO_JAVA",
> "api",
> "inf"
> ],
> "FirstSeenInMs" : NumberLong("1606298156021")
> }
> the name of the database in mogodb is : nmo_01_db
> the name of the collection is : hosts
> when we do this request with Apache drill:
> SELECT (`DRILL_2mongo.nmo_301_3db_4hosts`.`tags`) AS `Column_1_1` FROM
> `mongo.nmo_01_db`.`hosts` AS `DRILL_2mongo.nmo_301_3db_4hosts` GROUP BY
> (`DRILL_2mongo.nmo_301_3db_4hosts`.`tags`)
> We have this error:
> *UserRemoteException :* SYSTEM ERROR: UnsupportedOperationException: Unable
> to get size for minor type [LIST] and mode [OPTIONAL]
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> UnsupportedOperationException: Unable to get size for minor type [LIST] and
> mode [OPTIONAL]
>
> Regards,
> Djamel
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] (DRILL-8246) Elastic search - DESCRIBE return ** (DYNAMIC STAR)
[ https://issues.apache.org/jira/browse/DRILL-8246 ]
James Turton deleted comment on DRILL-8246:
-
was (Author: dzamo):
Thanks for the report [~DamD]. This is expected behaviour for the
> Elastic search - DESCRIBE return ** (DYNAMIC STAR)
> --
>
> Key: DRILL-8246
> URL: https://issues.apache.org/jira/browse/DRILL-8246
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.20.1
>Reporter: DEOM Damien
>Priority: Major
>
> DRILL Version: 1.20.1
> Describe statement returns ** (DYNAMIC STAR)
>
> Storage plugin configuration:
> {
> "type": "elastic",
> "hosts": [
> "http://10.50.1.135:9200;,
> "http://10.50.1.136:9200;,
> "http://10.50.1.137:9200;
> ],
> "enabled": true
> }
> Elastic search version:
>
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"version"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#dcdcdc}{{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"number"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"7.17.0"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_flavor"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"default"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_type"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"deb"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_hash"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
>
> {color}{color:#ce9178}"bee86328705acaa9a6daede7140defd4d9ec56bd"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_date"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
>
> {color}{color:#ce9178}"2022-01-28T08:36:04.875279988Z"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_snapshot"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}false{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"lucene_version"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"8.11.1"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"minimum_wire_compatibility_version"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"6.8.0"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"minimum_index_compatibility_version"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"6.0.0-beta1"{color}
> {color:#d4d4d4}{color}{color:#dcdcdc}},{color}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (DRILL-8246) Elastic search - DESCRIBE return ** (DYNAMIC STAR)
[
https://issues.apache.org/jira/browse/DRILL-8246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton closed DRILL-8246.
---
Resolution: Not A Bug
Thanks for the report [~DamD]. This is expected behaviour for the
> Elastic search - DESCRIBE return ** (DYNAMIC STAR)
> --
>
> Key: DRILL-8246
> URL: https://issues.apache.org/jira/browse/DRILL-8246
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.20.1
>Reporter: DEOM Damien
>Priority: Major
>
> DRILL Version: 1.20.1
> Describe statement returns ** (DYNAMIC STAR)
>
> Storage plugin configuration:
> {
> "type": "elastic",
> "hosts": [
> "http://10.50.1.135:9200;,
> "http://10.50.1.136:9200;,
> "http://10.50.1.137:9200;
> ],
> "enabled": true
> }
> Elastic search version:
>
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"version"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#dcdcdc}{{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"number"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"7.17.0"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_flavor"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"default"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_type"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"deb"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_hash"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
>
> {color}{color:#ce9178}"bee86328705acaa9a6daede7140defd4d9ec56bd"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_date"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
>
> {color}{color:#ce9178}"2022-01-28T08:36:04.875279988Z"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_snapshot"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}false{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"lucene_version"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"8.11.1"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"minimum_wire_compatibility_version"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"6.8.0"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"minimum_index_compatibility_version"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"6.0.0-beta1"{color}
> {color:#d4d4d4}{color}{color:#dcdcdc}},{color}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Reopened] (DRILL-8246) Elastic search - DESCRIBE return ** (DYNAMIC STAR)
[
https://issues.apache.org/jira/browse/DRILL-8246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton reopened DRILL-8246:
-
> Elastic search - DESCRIBE return ** (DYNAMIC STAR)
> --
>
> Key: DRILL-8246
> URL: https://issues.apache.org/jira/browse/DRILL-8246
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.20.1
>Reporter: DEOM Damien
>Priority: Major
>
> DRILL Version: 1.20.1
> Describe statement returns ** (DYNAMIC STAR)
>
> Storage plugin configuration:
> {
> "type": "elastic",
> "hosts": [
> "http://10.50.1.135:9200;,
> "http://10.50.1.136:9200;,
> "http://10.50.1.137:9200;
> ],
> "enabled": true
> }
> Elastic search version:
>
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"version"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#dcdcdc}{{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"number"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"7.17.0"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_flavor"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"default"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_type"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"deb"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_hash"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
>
> {color}{color:#ce9178}"bee86328705acaa9a6daede7140defd4d9ec56bd"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_date"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
>
> {color}{color:#ce9178}"2022-01-28T08:36:04.875279988Z"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"build_snapshot"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}false{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"lucene_version"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"8.11.1"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"minimum_wire_compatibility_version"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"6.8.0"{color}{color:#dcdcdc},{color}
> {color:#d4d4d4}
> {color}{color:#9cdcfe}"minimum_index_compatibility_version"{color}{color:#dcdcdc}:{color}{color:#d4d4d4}
> {color}{color:#ce9178}"6.0.0-beta1"{color}
> {color:#d4d4d4}{color}{color:#dcdcdc}},{color}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8188) Convert HDF5 format to EVF2
[ https://issues.apache.org/jira/browse/DRILL-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627746#comment-17627746 ] ASF GitHub Bot commented on DRILL-8188: --- cgivre commented on PR #2515: URL: https://github.com/apache/drill/pull/2515#issuecomment-1300548353 Hey @luocooong @paul-rogers I hope all is well. I wanted to check in on this PR to see where we are. At this point, nearly all the other format plugins have been converted to EVF V2. The other outstanding ones are the image format and LTSV. I'd really like to see this merged so that we can remove the EVF V1 code. Do you think we could get this ready to go soon? > Convert HDF5 format to EVF2 > --- > > Key: DRILL-8188 > URL: https://issues.apache.org/jira/browse/DRILL-8188 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.0 >Reporter: Cong Luo >Assignee: Cong Luo >Priority: Major > Fix For: 2.0.0 > > > Use EVF V2 instead of old V1. > Also, fixed a few bugs in V2 framework. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8346) Refactor duplicated JDBC reader and writer tests to use ParameterizedTest
[ https://issues.apache.org/jira/browse/DRILL-8346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8346: Fix Version/s: Future (was: 2.0.0) > Refactor duplicated JDBC reader and writer tests to use ParameterizedTest > - > > Key: DRILL-8346 > URL: https://issues.apache.org/jira/browse/DRILL-8346 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.2 >Reporter: James Turton >Priority: Minor > Fix For: Future > > > The JDBC storage plugin contains heavily duplicated unit tests against > different DBMSes, e.g. PostgreSQL, MySQL, MSSQL and H2. It might be possible > to reduce this duplication using JUnit ParameterizedTests. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8350) Convert PCAP Format Plugin to EVF2
[ https://issues.apache.org/jira/browse/DRILL-8350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627737#comment-17627737 ] ASF GitHub Bot commented on DRILL-8350: --- cgivre merged PR #2698: URL: https://github.com/apache/drill/pull/2698 > Convert PCAP Format Plugin to EVF2 > -- > > Key: DRILL-8350 > URL: https://issues.apache.org/jira/browse/DRILL-8350 > Project: Apache Drill > Issue Type: Task > Components: Format - PCAP >Affects Versions: 1.20.2 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Minor > Fix For: 2.0.0 > > > Convert the PCAP format plugin to EVF2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (DRILL-8347) Complete the migration of the impersonation switch from SYSTEM to storage config
[ https://issues.apache.org/jira/browse/DRILL-8347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton reassigned DRILL-8347: --- Assignee: James Turton > Complete the migration of the impersonation switch from SYSTEM to storage > config > > > Key: DRILL-8347 > URL: https://issues.apache.org/jira/browse/DRILL-8347 > Project: Apache Drill > Issue Type: Improvement > Components: Security >Affects Versions: 1.20.2 >Reporter: James Turton >Assignee: James Turton >Priority: Major > Fix For: 2.0.0 > > > DRILL-8155 introduced the configuration of the auth mode for a storage plugin > in the storage config. The legacy SYSTEM option drill.exec.impersonation was > not removed in that change and continues to control impersonation for > Hadoop-based plugins. All plugins should use the new authMode property of > their storage config to decided with mode of authentication, including > impersonation, at which point the drill.exec.impersonation SYSTEM option can > be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8350) Convert PCAP Format Plugin to EVF2
[ https://issues.apache.org/jira/browse/DRILL-8350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8350. --- > Convert PCAP Format Plugin to EVF2 > -- > > Key: DRILL-8350 > URL: https://issues.apache.org/jira/browse/DRILL-8350 > Project: Apache Drill > Issue Type: Task > Components: Format - PCAP >Affects Versions: 1.20.2 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Minor > Fix For: 2.0.0 > > > Convert the PCAP format plugin to EVF2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8299) HashMap key type mismatch in MetadataContext
[
https://issues.apache.org/jira/browse/DRILL-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627723#comment-17627723
]
ASF GitHub Bot commented on DRILL-8299:
---
jnturton commented on PR #2697:
URL: https://github.com/apache/drill/pull/2697#issuecomment-1300446343
> @jnturton Thanks for submitting this. I approved this but what were the
implications of this bug?
@cgivre the bug had a performance implication. A cache recording which
directories already had their modification times checked for the purpose of
determining where Parquet metadata cache files (created by `REFRESH TABLE
METADATA`) need updating was being accessed using objects of wrong type that,
barring an unexpected override of Path.equals(), meant that the cache had a
100% miss rate.
> HashMap key type mismatch in MetadataContext
>
>
> Key: DRILL-8299
> URL: https://issues.apache.org/jira/browse/DRILL-8299
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.20.2
>Reporter: PJ Fanning
>Assignee: James Turton
>Priority: Minor
> Fix For: 1.20.3
>
>
> The dirModifCheckMap used in this lookup is keyed using a HDFS Path instance,
> not a string, so this code is not going to work:
> {code:java}
> public boolean getStatus(String dir) {
> if (dirModifCheckMap.containsKey(dir)) {
> return dirModifCheckMap.get(dir);
> }
> return false;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8299) HashMap key type mismatch in MetadataContext
[
https://issues.apache.org/jira/browse/DRILL-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627708#comment-17627708
]
ASF GitHub Bot commented on DRILL-8299:
---
cgivre merged PR #2697:
URL: https://github.com/apache/drill/pull/2697
> HashMap key type mismatch in MetadataContext
>
>
> Key: DRILL-8299
> URL: https://issues.apache.org/jira/browse/DRILL-8299
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.20.2
>Reporter: PJ Fanning
>Assignee: James Turton
>Priority: Minor
> Fix For: 1.20.3
>
>
> The dirModifCheckMap used in this lookup is keyed using a HDFS Path instance,
> not a string, so this code is not going to work:
> {code:java}
> public boolean getStatus(String dir) {
> if (dirModifCheckMap.containsKey(dir)) {
> return dirModifCheckMap.get(dir);
> }
> return false;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8283) Add a configurable recursive file listing size limit
[
https://issues.apache.org/jira/browse/DRILL-8283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627699#comment-17627699
]
ASF GitHub Bot commented on DRILL-8283:
---
jnturton commented on code in PR #2699:
URL: https://github.com/apache/drill/pull/2699#discussion_r1011758384
##
contrib/storage-mongo/src/test/java/org/apache/drill/exec/store/mongo/TestMongoProjectPushDown.java:
##
@@ -93,14 +93,27 @@ public void testMultipleColumnsProject() throws Exception {
@Test
public void testStarProject() throws Exception {
-String query = String.format(TEST_QUERY_PROJECT_PUSH_DOWN__TEMPLATE_3,
EMPLOYEE_DB, EMPINFO_COLLECTION);
+String query = String.format(TEST_QUERY_PROJECT_PUSH_DOWN_TEMPLATE_3,
EMPLOYEE_DB, EMPINFO_COLLECTION);
testBuilder()
.sqlQuery(query)
.unOrdered()
.expectsNumRecords(19)
.go();
}
+ // DRILL-8238
+ @Test
+ public void testOperatorsProject() throws Exception {
+String query = String.format(TEST_QUERY_PROJECT_PUSH_DOWN_TEMPLATE_4,
EMPLOYEE_DB, EMPINFO_COLLECTION);
+
+testBuilder()
+.sqlQuery(query)
+.unOrdered()
+.baselineColumns("position_id_or_default")
+.expectsNumRecords(19)
Review Comment:
I guess it should be possible to explicitly test whether the coalescing
operator was applied successfully, not just whether the query now completes
without an error.
> Add a configurable recursive file listing size limit
>
>
> Key: DRILL-8283
> URL: https://issues.apache.org/jira/browse/DRILL-8283
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Other
>Affects Versions: 1.20.2
>Reporter: James Turton
>Assignee: James Turton
>Priority: Minor
> Fix For: 1.20.3
>
>
> Currently a malicious or merely unwitting user can crash their Drill foreman
> by sending
> {code:java}
> select * from dfs.huge_workspace limit 10
> {code}
> causing the query planner to recurse over every file in huge_workspace and
> culminating in
> {code:java}
> 2022-08-09 15:13:22,251 [1d0da29f-e50c-fd51-43d9-8a5086d52c4e:foreman] ERROR
> o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurred,
> exiting. Information message: Unable to handle out of memory condition in
> Foreman.java.lang.OutOfMemoryError: null {code}
> if there are enough files in huge_workspace. A SHOW FILES command can produce
> the same effect. This issue proposes a new BOOT option named
> drill.exec.storage.file.recursive_listing_max_size with a default value of,
> say 10 000. If a file listing task exceeds this limit then the initiating
> operation is terminated with a UserException preventing runaway resource
> usage.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8238) Translation of IS NOT NULL($1) is not supported by MongoProject
[
https://issues.apache.org/jira/browse/DRILL-8238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8238:
Fix Version/s: 1.20.3
> Translation of IS NOT NULL($1) is not supported by MongoProject
> ---
>
> Key: DRILL-8238
> URL: https://issues.apache.org/jira/browse/DRILL-8238
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - MongoDB
>Affects Versions: 1.20.1
> Environment: Affects Drill 1.20 and not Drill 1.19. Projection push
> down is probably enabled when it breaks in 1.20.
>Reporter: James Turton
>Assignee: James Turton
>Priority: Major
> Fix For: 1.20.3
>
>
> Running
> {{}}
> {code:java}
> SELECT COALESCE(`Filename`,'sdfsafdsd')
> FROM dd7819ee-438e-429b-9a3a-a68c7f1cc123
> LIMIT 500
> {code}
>
> produces
> {code:java}
> Caused by:
> org.apache.drill.exec.work.foreman.ForemanException: Unexpected
> exception during fragment initialization: Translation of IS NOT NULL($2)
> is not supported by MongoProject at
> org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:301) at
> ...(:0) Caused by: java.lang.IllegalArgumentException: Translation
> of IS NOT NULL($1) is not supported by MongoProject at
> org.apache.drill.exec.store.mongo.plan.RexToMongoTranslator.visitCall(RexToMongoTranslator.java:183)
> at
> org.apache.drill.exec.store.mongo.plan.RexToMongoTranslator.visitCall(RexToMongoTranslator.java:49)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:191) at
> org.apache.drill.exec.store.mongo.plan.RexToMongoTranslator.lambda$visitCall$0(RexToMongoTranslator.java:136)
> at ...(:0) at
> org.apache.drill.exec.store.mongo.plan.RexToMongoTranslator.visitCall(RexToMongoTranslator.java:137)
> at
> org.apache.drill.exec.store.mongo.plan.RexToMongoTranslator.visitCall(RexToMongoTranslator.java:49)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:191) at
> org.apache.drill.exec.store.mongo.plan.MongoPluginImplementor.implement(MongoPluginImplementor.java:150)
> at
> org.apache.drill.exec.store.plan.rel.PluginProjectRel.implement(PluginProjectRel.java:59)
> at
> org.apache.drill.exec.store.plan.PluginImplementor.visitChild(PluginImplementor.java:84)
> at
> org.apache.drill.exec.store.mongo.plan.MongoPluginImplementor.implement(MongoPluginImplementor.java:93)
> at
> org.apache.drill.exec.store.plan.rel.PluginAggregateRel.implement(PluginAggregateRel.java:59)
> at
> org.apache.drill.exec.store.plan.PluginImplementor.visitChild(PluginImplementor.java:84)
> at
> org.apache.drill.exec.store.mongo.plan.MongoPluginImplementor.implement(MongoPluginImplementor.java:173)
> at
> org.apache.drill.exec.store.plan.rel.PluginSortRel.implement(PluginSortRel.java:57)
> at
> org.apache.drill.exec.store.plan.rel.PluginPrel.(PluginPrel.java:52)
> at
> org.apache.drill.exec.store.plan.rel.PluginIntermediatePrel.finalizeRel(PluginIntermediatePrel.java:71)
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler$PrelFinalizer.visit(DefaultSqlHandler.java:322)
> at
> org.apache.calcite.rel.AbstractRelNode.accept(AbstractRelNode.java:272)
> at
> org.apache.calcite.rel.RelShuttleImpl.visitChild(RelShuttleImpl.java:55)
> at
> org.apache.calcite.rel.RelShuttleImpl.visitChildren(RelShuttleImpl.java:69)
> at org.apache.calcite.rel.RelShuttleImpl.visit(RelShuttleImpl.java:131)
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler$PrelFinalizer.visit(DefaultSqlHandler.java:324)
> at
> org.apache.calcite.rel.AbstractRelNode.accept(AbstractRelNode.java:272)
> at
> org.apache.calcite.rel.RelShuttleImpl.visitChild(RelShuttleImpl.java:55)
> at
> org.apache.calcite.rel.RelShuttleImpl.visitChildren(RelShuttleImpl.java:69)
> at org.apache.calcite.rel.RelShuttleImpl.visit(RelShuttleImpl.java:131)
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler$PrelFinalizer.visit(DefaultSqlHandler.java:324)
> at
> org.apache.calcite.rel.AbstractRelNode.accept(AbstractRelNode.java:272)
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToPrel(DefaultSqlHandler.java:437)
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:174)
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:283)
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan(DrillSqlWorker.java:163)
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan(DrillSqlWorker.java:128)
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:93)
> at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:593)
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:274) ... 1
> common frames omitted{code}
[jira] [Assigned] (DRILL-8238) Translation of IS NOT NULL($1) is not supported by MongoProject
[
https://issues.apache.org/jira/browse/DRILL-8238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton reassigned DRILL-8238:
---
Assignee: James Turton
> Translation of IS NOT NULL($1) is not supported by MongoProject
> ---
>
> Key: DRILL-8238
> URL: https://issues.apache.org/jira/browse/DRILL-8238
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - MongoDB
>Affects Versions: 1.20.1
> Environment: Affects Drill 1.20 and not Drill 1.19. Projection push
> down is probably enabled when it breaks in 1.20.
>Reporter: James Turton
>Assignee: James Turton
>Priority: Major
>
> Running
> {{}}
> {code:java}
> SELECT COALESCE(`Filename`,'sdfsafdsd')
> FROM dd7819ee-438e-429b-9a3a-a68c7f1cc123
> LIMIT 500
> {code}
>
> produces
> {code:java}
> Caused by:
> org.apache.drill.exec.work.foreman.ForemanException: Unexpected
> exception during fragment initialization: Translation of IS NOT NULL($2)
> is not supported by MongoProject at
> org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:301) at
> ...(:0) Caused by: java.lang.IllegalArgumentException: Translation
> of IS NOT NULL($1) is not supported by MongoProject at
> org.apache.drill.exec.store.mongo.plan.RexToMongoTranslator.visitCall(RexToMongoTranslator.java:183)
> at
> org.apache.drill.exec.store.mongo.plan.RexToMongoTranslator.visitCall(RexToMongoTranslator.java:49)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:191) at
> org.apache.drill.exec.store.mongo.plan.RexToMongoTranslator.lambda$visitCall$0(RexToMongoTranslator.java:136)
> at ...(:0) at
> org.apache.drill.exec.store.mongo.plan.RexToMongoTranslator.visitCall(RexToMongoTranslator.java:137)
> at
> org.apache.drill.exec.store.mongo.plan.RexToMongoTranslator.visitCall(RexToMongoTranslator.java:49)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:191) at
> org.apache.drill.exec.store.mongo.plan.MongoPluginImplementor.implement(MongoPluginImplementor.java:150)
> at
> org.apache.drill.exec.store.plan.rel.PluginProjectRel.implement(PluginProjectRel.java:59)
> at
> org.apache.drill.exec.store.plan.PluginImplementor.visitChild(PluginImplementor.java:84)
> at
> org.apache.drill.exec.store.mongo.plan.MongoPluginImplementor.implement(MongoPluginImplementor.java:93)
> at
> org.apache.drill.exec.store.plan.rel.PluginAggregateRel.implement(PluginAggregateRel.java:59)
> at
> org.apache.drill.exec.store.plan.PluginImplementor.visitChild(PluginImplementor.java:84)
> at
> org.apache.drill.exec.store.mongo.plan.MongoPluginImplementor.implement(MongoPluginImplementor.java:173)
> at
> org.apache.drill.exec.store.plan.rel.PluginSortRel.implement(PluginSortRel.java:57)
> at
> org.apache.drill.exec.store.plan.rel.PluginPrel.(PluginPrel.java:52)
> at
> org.apache.drill.exec.store.plan.rel.PluginIntermediatePrel.finalizeRel(PluginIntermediatePrel.java:71)
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler$PrelFinalizer.visit(DefaultSqlHandler.java:322)
> at
> org.apache.calcite.rel.AbstractRelNode.accept(AbstractRelNode.java:272)
> at
> org.apache.calcite.rel.RelShuttleImpl.visitChild(RelShuttleImpl.java:55)
> at
> org.apache.calcite.rel.RelShuttleImpl.visitChildren(RelShuttleImpl.java:69)
> at org.apache.calcite.rel.RelShuttleImpl.visit(RelShuttleImpl.java:131)
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler$PrelFinalizer.visit(DefaultSqlHandler.java:324)
> at
> org.apache.calcite.rel.AbstractRelNode.accept(AbstractRelNode.java:272)
> at
> org.apache.calcite.rel.RelShuttleImpl.visitChild(RelShuttleImpl.java:55)
> at
> org.apache.calcite.rel.RelShuttleImpl.visitChildren(RelShuttleImpl.java:69)
> at org.apache.calcite.rel.RelShuttleImpl.visit(RelShuttleImpl.java:131)
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler$PrelFinalizer.visit(DefaultSqlHandler.java:324)
> at
> org.apache.calcite.rel.AbstractRelNode.accept(AbstractRelNode.java:272)
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToPrel(DefaultSqlHandler.java:437)
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:174)
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:283)
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan(DrillSqlWorker.java:163)
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan(DrillSqlWorker.java:128)
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:93)
> at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:593)
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:274) ... 1
> common frames omitted{code}
--
This message was sent
[jira] [Commented] (DRILL-8283) Add a configurable recursive file listing size limit
[
https://issues.apache.org/jira/browse/DRILL-8283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627694#comment-17627694
]
ASF GitHub Bot commented on DRILL-8283:
---
jnturton opened a new pull request, #2699:
URL: https://github.com/apache/drill/pull/2699
# [DRILL-8283](https://issues.apache.org/jira/browse/DRILL-8283):
Translation of IS NOT NULL($1) is not supported by MongoProject
## Description
Add translations of IS NULL and IS NOT NULL to $eq and $ne respectively.
## Documentation
N/A
## Testing
TestMongoProjectPushdown#testOperatorsProject
> Add a configurable recursive file listing size limit
>
>
> Key: DRILL-8283
> URL: https://issues.apache.org/jira/browse/DRILL-8283
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Other
>Affects Versions: 1.20.2
>Reporter: James Turton
>Assignee: James Turton
>Priority: Minor
> Fix For: 1.20.3
>
>
> Currently a malicious or merely unwitting user can crash their Drill foreman
> by sending
> {code:java}
> select * from dfs.huge_workspace limit 10
> {code}
> causing the query planner to recurse over every file in huge_workspace and
> culminating in
> {code:java}
> 2022-08-09 15:13:22,251 [1d0da29f-e50c-fd51-43d9-8a5086d52c4e:foreman] ERROR
> o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurred,
> exiting. Information message: Unable to handle out of memory condition in
> Foreman.java.lang.OutOfMemoryError: null {code}
> if there are enough files in huge_workspace. A SHOW FILES command can produce
> the same effect. This issue proposes a new BOOT option named
> drill.exec.storage.file.recursive_listing_max_size with a default value of,
> say 10 000. If a file listing task exceeds this limit then the initiating
> operation is terminated with a UserException preventing runaway resource
> usage.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8299) HashMap key type mismatch in MetadataContext
[
https://issues.apache.org/jira/browse/DRILL-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627671#comment-17627671
]
ASF GitHub Bot commented on DRILL-8299:
---
cgivre commented on PR #2697:
URL: https://github.com/apache/drill/pull/2697#issuecomment-1300262791
@jnturton Thanks for submitting this. I approved this but what were the
implications of this bug?
> HashMap key type mismatch in MetadataContext
>
>
> Key: DRILL-8299
> URL: https://issues.apache.org/jira/browse/DRILL-8299
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.20.2
>Reporter: PJ Fanning
>Assignee: James Turton
>Priority: Minor
> Fix For: 1.20.3
>
>
> The dirModifCheckMap used in this lookup is keyed using a HDFS Path instance,
> not a string, so this code is not going to work:
> {code:java}
> public boolean getStatus(String dir) {
> if (dirModifCheckMap.containsKey(dir)) {
> return dirModifCheckMap.get(dir);
> }
> return false;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8350) Convert PCAP Format Plugin to EVF2
[ https://issues.apache.org/jira/browse/DRILL-8350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627666#comment-17627666 ] ASF GitHub Bot commented on DRILL-8350: --- cgivre opened a new pull request, #2698: URL: https://github.com/apache/drill/pull/2698 # [DRILL-8350](https://issues.apache.org/jira/browse/DRILL-8350): Convert PCAP Format Plugin to EVF2 ## Description Convert the PCAP format plugin to EVF2 ## Documentation No user facing changes. ## Testing Ran existing unit tests. > Convert PCAP Format Plugin to EVF2 > -- > > Key: DRILL-8350 > URL: https://issues.apache.org/jira/browse/DRILL-8350 > Project: Apache Drill > Issue Type: Task > Components: Format - PCAP >Affects Versions: 1.20.2 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Minor > Fix For: 2.0.0 > > > Convert the PCAP format plugin to EVF2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (DRILL-8350) Convert PCAP Format Plugin to EVF2
Charles Givre created DRILL-8350: Summary: Convert PCAP Format Plugin to EVF2 Key: DRILL-8350 URL: https://issues.apache.org/jira/browse/DRILL-8350 Project: Apache Drill Issue Type: Task Components: Format - PCAP Affects Versions: 1.20.2 Reporter: Charles Givre Assignee: Charles Givre Fix For: 2.0.0 Convert the PCAP format plugin to EVF2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8297) NPE in the constructor of OrderedPartitionRecordBatch
[ https://issues.apache.org/jira/browse/DRILL-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8297: Affects Version/s: 1.20.2 > NPE in the constructor of OrderedPartitionRecordBatch > - > > Key: DRILL-8297 > URL: https://issues.apache.org/jira/browse/DRILL-8297 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.20.2 >Reporter: PJ Fanning >Priority: Trivial > > The constructor will always throw a NullPointerException because cache is > always null. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8297) NPE in the constructor of OrderedPartitionRecordBatch
[ https://issues.apache.org/jira/browse/DRILL-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627595#comment-17627595 ] James Turton commented on DRILL-8297: - Note that OrderedPartitionRecordBatch is unused code so this needs a clean up rather than a bug fix. > NPE in the constructor of OrderedPartitionRecordBatch > - > > Key: DRILL-8297 > URL: https://issues.apache.org/jira/browse/DRILL-8297 > Project: Apache Drill > Issue Type: Improvement >Reporter: PJ Fanning >Priority: Major > > The constructor will always throw a NullPointerException because cache is > always null. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8297) NPE in the constructor of OrderedPartitionRecordBatch
[ https://issues.apache.org/jira/browse/DRILL-8297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8297: Priority: Trivial (was: Major) > NPE in the constructor of OrderedPartitionRecordBatch > - > > Key: DRILL-8297 > URL: https://issues.apache.org/jira/browse/DRILL-8297 > Project: Apache Drill > Issue Type: Improvement >Reporter: PJ Fanning >Priority: Trivial > > The constructor will always throw a NullPointerException because cache is > always null. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8298) Possible bug in NonCoveringIndexPlanGenerator
[
https://issues.apache.org/jira/browse/DRILL-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton closed DRILL-8298.
---
Assignee: Vova Vysotskyi
Resolution: Fixed
Fixed in 975b7d60a6520e1ff2ba17bbae7347739ed2e628.
> Possible bug in NonCoveringIndexPlanGenerator
> -
>
> Key: DRILL-8298
> URL: https://issues.apache.org/jira/browse/DRILL-8298
> Project: Apache Drill
> Issue Type: Bug
>Reporter: PJ Fanning
>Assignee: Vova Vysotskyi
>Priority: Major
>
> I'm not a Calcite expert by LGTM.com and IntelliJ suggest that this set and
> the type of the instance in the contains check do not type match.
> {code:java}
> (restrictedScanTraitSet.contains(RelCollationTraitDef.INSTANCE))
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (DRILL-8299) HashMap key type mismatch in MetadataContext
[
https://issues.apache.org/jira/browse/DRILL-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton reassigned DRILL-8299:
---
Fix Version/s: 1.20.3
Affects Version/s: 1.20.2
Assignee: James Turton
Priority: Minor (was: Major)
> HashMap key type mismatch in MetadataContext
>
>
> Key: DRILL-8299
> URL: https://issues.apache.org/jira/browse/DRILL-8299
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.20.2
>Reporter: PJ Fanning
>Assignee: James Turton
>Priority: Minor
> Fix For: 1.20.3
>
>
> The dirModifCheckMap used in this lookup is keyed using a HDFS Path instance,
> not a string, so this code is not going to work:
> {code:java}
> public boolean getStatus(String dir) {
> if (dirModifCheckMap.containsKey(dir)) {
> return dirModifCheckMap.get(dir);
> }
> return false;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8299) HashMap key type mismatch in MetadataContext
[
https://issues.apache.org/jira/browse/DRILL-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627563#comment-17627563
]
ASF GitHub Bot commented on DRILL-8299:
---
jnturton opened a new pull request, #2697:
URL: https://github.com/apache/drill/pull/2697
# [DRILL-8299](https://issues.apache.org/jira/browse/DRILL-8299): HashMap
key type mismatch in MetadataContext
## Description
The Map dirModifCheckMap is keyed using a HDFS Path instance, not a string.
This updates accessing code accordingly.
## Documentation
N/A
## Testing
Existing unit tests.
> HashMap key type mismatch in MetadataContext
>
>
> Key: DRILL-8299
> URL: https://issues.apache.org/jira/browse/DRILL-8299
> Project: Apache Drill
> Issue Type: Bug
>Reporter: PJ Fanning
>Priority: Major
>
> The dirModifCheckMap used in this lookup is keyed using a HDFS Path instance,
> not a string, so this code is not going to work:
> {code:java}
> public boolean getStatus(String dir) {
> if (dirModifCheckMap.containsKey(dir)) {
> return dirModifCheckMap.get(dir);
> }
> return false;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8299) HashMap key type mismatch in MetadataContext
[
https://issues.apache.org/jira/browse/DRILL-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8299:
Summary: HashMap key type mismatch in MetadataContext (was: Type mismatch
in MetadataContext)
> HashMap key type mismatch in MetadataContext
>
>
> Key: DRILL-8299
> URL: https://issues.apache.org/jira/browse/DRILL-8299
> Project: Apache Drill
> Issue Type: Bug
>Reporter: PJ Fanning
>Priority: Major
>
> The dirModifCheckMap used in this lookup is keyed using a HDFS Path instance,
> not a string, so this code is not going to work:
> {code:java}
> public boolean getStatus(String dir) {
> if (dirModifCheckMap.containsKey(dir)) {
> return dirModifCheckMap.get(dir);
> }
> return false;
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (DRILL-8329) Close HTTP Caching Resources
[ https://issues.apache.org/jira/browse/DRILL-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8329. --- Resolution: Fixed > Close HTTP Caching Resources > - > > Key: DRILL-8329 > URL: https://issues.apache.org/jira/browse/DRILL-8329 > Project: Apache Drill > Issue Type: Bug > Components: Storage - HTTP >Affects Versions: 1.20.2 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Minor > Fix For: 1.20.3 > > > The HTTP plugin has the ability to cache API responses. However, the storage > plugin was not closing the connection to the file cache. This minor PR fixes > that. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8335) Add Ability to Query GoogleSheets Tabs by Index
[ https://issues.apache.org/jira/browse/DRILL-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8335. --- > Add Ability to Query GoogleSheets Tabs by Index > --- > > Key: DRILL-8335 > URL: https://issues.apache.org/jira/browse/DRILL-8335 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - GoogleSheets >Affects Versions: 1.20.2 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 2.0.0 > > > The GoogleSheets plugin does not provide a way for a user to query data if > they do not know the available tab names. This adds the ability to query by > index of the tabs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8341) Add Scanned Plugin List to Sys Profiles Table
[ https://issues.apache.org/jira/browse/DRILL-8341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8341. --- Resolution: Fixed > Add Scanned Plugin List to Sys Profiles Table > - > > Key: DRILL-8341 > URL: https://issues.apache.org/jira/browse/DRILL-8341 > Project: Apache Drill > Issue Type: Improvement > Components: Execution - Monitoring >Affects Versions: 1.20.2 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Minor > Fix For: 2.0.0 > > > In DRILL-8322, [~dzamo] added the list of scanned plugins to the query > profiles. This information is extremely useful in query analysis. This > minor PR adds this same information to the sys.profiles table. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8337) Upgrade Hive libs to 3.1.3 due to sonatype-2019-0400
[ https://issues.apache.org/jira/browse/DRILL-8337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8337. --- Resolution: Fixed > Upgrade Hive libs to 3.1.3 due to sonatype-2019-0400 > > > Key: DRILL-8337 > URL: https://issues.apache.org/jira/browse/DRILL-8337 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Hive >Affects Versions: 1.20.2 >Reporter: James Turton >Assignee: James Turton >Priority: Minor > Fix For: 1.20.3 > > > https://ds-3p.sonatype.com/assets/index.html#/vulnerabilities/sonatype-2019-0400 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8338) Upgrade jQuery to 3.6.1 and DataTables to 1.12.1 due to sonatype-2020-0988
[ https://issues.apache.org/jira/browse/DRILL-8338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8338. --- Resolution: Fixed > Upgrade jQuery to 3.6.1 and DataTables to 1.12.1 due to sonatype-2020-0988 > -- > > Key: DRILL-8338 > URL: https://issues.apache.org/jira/browse/DRILL-8338 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.20.2 >Reporter: James Turton >Assignee: James Turton >Priority: Minor > Fix For: 1.20.3 > > > https://ds-3p.sonatype.com/assets/index.html#/vulnerabilities/sonatype-2020-0988 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8343) Upgrade Commons Text to 1.10.0
[
https://issues.apache.org/jira/browse/DRILL-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton closed DRILL-8343.
---
Resolution: Duplicate
> Upgrade Commons Text to 1.10.0
> --
>
> Key: DRILL-8343
> URL: https://issues.apache.org/jira/browse/DRILL-8343
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Jason-Morries Adam
>Priority: Critical
>
> Apache Commons Text versions prior to 1.10.0 are vulnerable to
> [CVE-2022-42889|https://nvd.nist.gov/vuln/detail/CVE-2022-42889], which
> involves potential script execution when processing untrusted input using
> {{{}StringLookup{}}}. Direct and transitive references to Apache Commons Text
> prior to 1.10.0 should be upgraded to avoid the default interpolation
> behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (DRILL-8342) Add Automatic Retry for Rate Limited APIs
[ https://issues.apache.org/jira/browse/DRILL-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8342. --- Resolution: Fixed > Add Automatic Retry for Rate Limited APIs > - > > Key: DRILL-8342 > URL: https://issues.apache.org/jira/browse/DRILL-8342 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - HTTP >Affects Versions: 1.20.2 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 2.0.0 > > > Many APIs have a burst limit for number of requests. This PR adds a retry > capability to the HTTP Storage Plugin, whereby if a 429 response code is > received, Drill will wait a configurable amount of time, and retry the > request once. > To prevent runaway pagination, this retry will only happen once per request. > This PR adds a new configuration option called retryDelay which is the number > of milliseconds that Drill should wait between retrys. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8349) GoogleSheets Not Registering Schemas with Non Default Name
[ https://issues.apache.org/jira/browse/DRILL-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8349. --- Resolution: Fixed > GoogleSheets Not Registering Schemas with Non Default Name > -- > > Key: DRILL-8349 > URL: https://issues.apache.org/jira/browse/DRILL-8349 > Project: Apache Drill > Issue Type: Bug > Components: Storage - GoogleSheets >Affects Versions: 2.0.0 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Blocker > Fix For: 2.0.0 > > > GoogleSheets plugin fails to register plugin instances with names other than > `GoogleSheets`. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8348) Cannot delete disabled storage plugins
[ https://issues.apache.org/jira/browse/DRILL-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8348. --- Resolution: Fixed > Cannot delete disabled storage plugins > -- > > Key: DRILL-8348 > URL: https://issues.apache.org/jira/browse/DRILL-8348 > Project: Apache Drill > Issue Type: Improvement > Components: Web Server >Affects Versions: 1.20.2 >Reporter: James Turton >Assignee: James Turton >Priority: Major > Fix For: 1.20.3 > > > Attempting to delete a disabled storage plugin fails due to code that tries > to clean up its OAuth token registries in the process. -- This message was sent by Atlassian Jira (v8.20.10#820010)
