[jira] [Closed] (DRILL-8437) Add Header Index Pagination
[ https://issues.apache.org/jira/browse/DRILL-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8437. --- Resolution: Fixed > Add Header Index Pagination > --- > > Key: DRILL-8437 > URL: https://issues.apache.org/jira/browse/DRILL-8437 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - HTTP >Affects Versions: 1.21.1 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.22.0 > > > Some APIs include pagination fields in the HTTP response headers. This PR > adds a new pagination method called Header Index which supports that. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8460) Upgrade ZooKeeper 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17802046#comment-17802046 ] ASF GitHub Bot commented on DRILL-8460: --- jnturton opened a new pull request, #2865: URL: https://github.com/apache/drill/pull/2865 # [DRILL-8460](https://issues.apache.org/jira/browse/DRILL-8460): Upgrade ZooKeeper 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 ## Description Note that the initial attempt to upgrade ZooKeeper to 3.7.2 for this ticket led to test failures and so to keep this upgrade backportable the upgrade is limited to 3.5.10. ## Documentation N/A ## Testing Unit tests pass. > Upgrade ZooKeeper 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 > -- > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Assignee: James Turton >Priority: Major > Fix For: 1.21.2 > > > Note that the initial attempt to upgrade ZooKeeper to 3.7.2 for this ticket > led to test failures and so to keep this upgrade backportable the upgrade is > limited to 3.5.10. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8460) Upgrade ZooKeeper 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8460: Summary: Upgrade ZooKeeper 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 (was: Upgrade ZooKeeper: 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0) > Upgrade ZooKeeper 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 > -- > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Assignee: James Turton >Priority: Major > Fix For: 1.21.2 > > > Note that the initial attempt to upgrade ZooKeeper to 3.7.2 for this ticket > led to test failures and so to keep this upgrade backportable the upgrade is > limited to 3.5.10. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8460) Upgrade ZooKeeper: 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8460: Description: Note that the initial attempt to upgrade ZooKeeper to 3.7.2 for this ticket led to test failures and so to keep this upgrade backportable the upgrade is limited to 3.5.10. (was: Note that the initial attempt to upgrade ZooKeeper to 3.7.2 for this ticket led to test failures and so is not considered suitable for the stable branch.) > Upgrade ZooKeeper: 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 > --- > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Assignee: James Turton >Priority: Major > Fix For: 1.21.2 > > > Note that the initial attempt to upgrade ZooKeeper to 3.7.2 for this ticket > led to test failures and so to keep this upgrade backportable the upgrade is > limited to 3.5.10. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8460) Upgrade ZooKeeper: 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8460: Summary: Upgrade ZooKeeper: 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 (was: Upgrade ZooKeeper: 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 in Drill 1.21) > Upgrade ZooKeeper: 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 > --- > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Assignee: James Turton >Priority: Major > Fix For: 1.21.2 > > > Note that the initial attempt to upgrade ZooKeeper to 3.7.2 for this ticket > led to test failures and so is not considered suitable for the stable branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8460) Upgrade ZooKeeper: 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 in Drill 1.21
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8460: Description: Note that the initial attempt to upgrade ZooKeeper to 3.7.2 for this ticket led to test failures and so is not considered suitable for the stable branch. > Upgrade ZooKeeper: 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 in Drill 1.21 > - > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Assignee: James Turton >Priority: Major > Fix For: 1.21.2 > > > Note that the initial attempt to upgrade ZooKeeper to 3.7.2 for this ticket > led to test failures and so is not considered suitable for the stable branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8460) Upgrade ZooKeeper: 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 in Drill 1.21
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8460: Summary: Upgrade ZooKeeper: 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 in Drill 1.21 (was: Upgrade ZooKeeper: 3.5.7 → 3.5.10 in Drill 1.21) > Upgrade ZooKeeper: 3.5.7 → 3.5.10 and Curator 5.2.0 → 5.5.0 in Drill 1.21 > - > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Assignee: James Turton >Priority: Major > Fix For: 1.21.2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8460) Upgrade ZooKeeper: 3.5.7 → 3.5.10 in Drill 1.21
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8460: Description: (was: [https://github.com/apache/drill/security/dependabot/51]) > Upgrade ZooKeeper: 3.5.7 → 3.5.10 in Drill 1.21 > --- > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Assignee: James Turton >Priority: Major > Fix For: 1.21.2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (DRILL-8460) Upgrade ZooKeeper: 3.5.7 → 3.5.10 in Drill 1.21
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton reassigned DRILL-8460: --- Assignee: James Turton > Upgrade ZooKeeper: 3.5.7 → 3.5.10 in Drill 1.21 > --- > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Assignee: James Turton >Priority: Major > Fix For: 1.21.2 > > > [https://github.com/apache/drill/security/dependabot/51] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8460) Upgrade ZooKeeper: 3.5.7 → 3.5.10 in Drill 1.21
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8460: Fix Version/s: 1.21.2 (was: 1.22.0) > Upgrade ZooKeeper: 3.5.7 → 3.5.10 in Drill 1.21 > --- > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.21.2 > > > [https://github.com/apache/drill/security/dependabot/51] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8460) Upgrade ZooKeeper: 3.5.7 → 3.5.10 in Drill 1.21
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8460: Summary: Upgrade ZooKeeper: 3.5.7 → 3.5.10 in Drill 1.21 (was: Upgrade ZooKeeper: 3.5.7 → 3.5.10) > Upgrade ZooKeeper: 3.5.7 → 3.5.10 in Drill 1.21 > --- > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.22.0 > > > [https://github.com/apache/drill/security/dependabot/51] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8460) Upgrade ZooKeeper: 3.5.7 → 3.5.10
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8460: Description: Further (was: https://github.com/apache/drill/security/dependabot/51) > Upgrade ZooKeeper: 3.5.7 → 3.5.10 > - > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.22.0 > > > Further -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8460) Upgrade ZooKeeper: 3.5.7 → 3.5.10
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8460: Description: [https://github.com/apache/drill/security/dependabot/51] (was: Further) > Upgrade ZooKeeper: 3.5.7 → 3.5.10 > - > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.22.0 > > > [https://github.com/apache/drill/security/dependabot/51] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8460) Upgrade ZooKeeper: 3.5.7 → 3.5.10
[ https://issues.apache.org/jira/browse/DRILL-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8460: Summary: Upgrade ZooKeeper: 3.5.7 → 3.5.10 (was: Bump zookeeper jar to 3.7.2 due to CVE) > Upgrade ZooKeeper: 3.5.7 → 3.5.10 > - > > Key: DRILL-8460 > URL: https://issues.apache.org/jira/browse/DRILL-8460 > Project: Apache Drill > Issue Type: Sub-task >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Priority: Major > Fix For: 1.22.0 > > > https://github.com/apache/drill/security/dependabot/51 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8464) GitHubActions: checkout action needs to be upgraded to v4 due to node16 deprecation
[ https://issues.apache.org/jira/browse/DRILL-8464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8464: Fix Version/s: 1.21.2 > GitHubActions: checkout action needs to be upgraded to v4 due to node16 > deprecation > > > Key: DRILL-8464 > URL: https://issues.apache.org/jira/browse/DRILL-8464 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Fix For: 1.21.2 > > > The following actions uses node12 which is deprecated and will be forced to > run on node16: actions/checkout@v2. For more info: > https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8464) GitHubActions: checkout action needs to be upgraded to v4 due to node16 deprecation
[ https://issues.apache.org/jira/browse/DRILL-8464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8464. --- Resolution: Fixed > GitHubActions: checkout action needs to be upgraded to v4 due to node16 > deprecation > > > Key: DRILL-8464 > URL: https://issues.apache.org/jira/browse/DRILL-8464 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > Fix For: 1.21.2 > > > The following actions uses node12 which is deprecated and will be forced to > run on node16: actions/checkout@v2. For more info: > https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (DRILL-8464) GitHubActions: checkout action needs to be upgraded to v4 due to node16 deprecation
[ https://issues.apache.org/jira/browse/DRILL-8464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton reassigned DRILL-8464: --- Assignee: PJ Fanning > GitHubActions: checkout action needs to be upgraded to v4 due to node16 > deprecation > > > Key: DRILL-8464 > URL: https://issues.apache.org/jira/browse/DRILL-8464 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > > The following actions uses node12 which is deprecated and will be forced to > run on node16: actions/checkout@v2. For more info: > https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8467) Update netty to 4.1.101
[
https://issues.apache.org/jira/browse/DRILL-8467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8467:
Fix Version/s: 1.21.2
(was: 1.22.0)
> Update netty to 4.1.101
> ---
>
> Key: DRILL-8467
> URL: https://issues.apache.org/jira/browse/DRILL-8467
> Project: Apache Drill
> Issue Type: Task
>Affects Versions: 1.21.1
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Minor
> Fix For: 1.21.2
>
>
> Update Netty to the last available at this moment stable Netty build –
> {{{}4.1.101{}}}.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8464) GitHubActions: checkout action needs to be upgraded to v4 due to node16 deprecation
[ https://issues.apache.org/jira/browse/DRILL-8464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8464: Affects Version/s: 1.21.1 > GitHubActions: checkout action needs to be upgraded to v4 due to node16 > deprecation > > > Key: DRILL-8464 > URL: https://issues.apache.org/jira/browse/DRILL-8464 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.21.1 >Reporter: PJ Fanning >Priority: Major > > The following actions uses node12 which is deprecated and will be forced to > run on node16: actions/checkout@v2. For more info: > https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8467) Update netty to 4.1.101
[
https://issues.apache.org/jira/browse/DRILL-8467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton closed DRILL-8467.
---
Resolution: Fixed
> Update netty to 4.1.101
> ---
>
> Key: DRILL-8467
> URL: https://issues.apache.org/jira/browse/DRILL-8467
> Project: Apache Drill
> Issue Type: Task
>Affects Versions: 1.21.1
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Minor
> Fix For: 1.21.2
>
>
> Update Netty to the last available at this moment stable Netty build –
> {{{}4.1.101{}}}.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8467) Update netty to 4.1.101
[
https://issues.apache.org/jira/browse/DRILL-8467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8467:
Affects Version/s: 1.21.1
> Update netty to 4.1.101
> ---
>
> Key: DRILL-8467
> URL: https://issues.apache.org/jira/browse/DRILL-8467
> Project: Apache Drill
> Issue Type: Task
>Affects Versions: 1.21.1
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Minor
> Fix For: 1.22.0
>
>
> Update Netty to the last available at this moment stable Netty build –
> {{{}4.1.101{}}}.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8468) Drill doesn't perform drill.exec.storage.action_on_plugins_override_file action
[
https://issues.apache.org/jira/browse/DRILL-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8468:
Fix Version/s: 1.21.2
> Drill doesn't perform drill.exec.storage.action_on_plugins_override_file
> action
> ---
>
> Key: DRILL-8468
> URL: https://issues.apache.org/jira/browse/DRILL-8468
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Major
> Fix For: 1.21.2
>
>
> After the Plugin Registry refactoring
> https://issues.apache.org/jira/browse/DRILL-7590, Drill stopped honoring
> {{drill.exec.storage.action_on_plugins_override_file}} property.
> After the refactoring, Drill performs
> {{drill.exec.storage.action_on_plugins_override_file}} action only once, on
> initial storage plugin registry initialization. After the initialization, on
> every next Drill restart, the action will not be performed. It means, that
> after {{storage-plugins-override.conf}} apply, this file is neither renamed
> nor removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (DRILL-8468) Drill doesn't perform drill.exec.storage.action_on_plugins_override_file action
[
https://issues.apache.org/jira/browse/DRILL-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton closed DRILL-8468.
---
Resolution: Fixed
> Drill doesn't perform drill.exec.storage.action_on_plugins_override_file
> action
> ---
>
> Key: DRILL-8468
> URL: https://issues.apache.org/jira/browse/DRILL-8468
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Major
> Fix For: 1.21.2
>
>
> After the Plugin Registry refactoring
> https://issues.apache.org/jira/browse/DRILL-7590, Drill stopped honoring
> {{drill.exec.storage.action_on_plugins_override_file}} property.
> After the refactoring, Drill performs
> {{drill.exec.storage.action_on_plugins_override_file}} action only once, on
> initial storage plugin registry initialization. After the initialization, on
> every next Drill restart, the action will not be performed. It means, that
> after {{storage-plugins-override.conf}} apply, this file is neither renamed
> nor removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8470) Bump MongoDB Driver to Latest Version
[ https://issues.apache.org/jira/browse/DRILL-8470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17802040#comment-17802040 ] ASF GitHub Bot commented on DRILL-8470: --- jnturton commented on code in PR #2862: URL: https://github.com/apache/drill/pull/2862#discussion_r1440145449 ## contrib/format-image/pom.xml: ## @@ -39,7 +39,7 @@ com.drewnoakes metadata-extractor - 2.18.0 + 2.19.0 Review Comment: Now that I've seen DRILL-8472, did you mean to include [0fda54c](https://github.com/apache/drill/pull/2862/commits/0fda54c2432efc91181ed19af5af0122f46d8dbd) in this PR? > Bump MongoDB Driver to Latest Version > - > > Key: DRILL-8470 > URL: https://issues.apache.org/jira/browse/DRILL-8470 > Project: Apache Drill > Issue Type: Task > Components: Storage - MongoDB >Affects Versions: 1.21.1 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.21.2 > > > Bump mongoDB driver to latest version. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8445) Upgrade Janino 3.1.8 → 3.1.11
[ https://issues.apache.org/jira/browse/DRILL-8445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17802036#comment-17802036 ] ASF GitHub Bot commented on DRILL-8445: --- jnturton opened a new pull request, #2864: URL: https://github.com/apache/drill/pull/2864 # [DRILL-8445](https://issues.apache.org/jira/browse/DRILL-8445): Upgrade Janino 3.1.8 → 3.1.11 ## Description Includes some corrections to the 3rdpaty/ and classb/ library separation. ## Documentation N/A ## Testing Unit tests pass. > Upgrade Janino 3.1.8 → 3.1.11 > - > > Key: DRILL-8445 > URL: https://issues.apache.org/jira/browse/DRILL-8445 > Project: Apache Drill > Issue Type: Task > Components: Server >Reporter: PJ Fanning >Assignee: James Turton >Priority: Major > > I'm not familar with exactly how janino is used inside Drill. > There is a new 3.1.10 release today to fix > [https://github.com/janino-compiler/janino/issues/201] > This may be an issue if Janino is used to parse input that may not be > entirely trustworthy. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8385) Add support for disabling SSL certificate verification in the Elasticsearch plugin
[ https://issues.apache.org/jira/browse/DRILL-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8385. --- Resolution: Fixed > Add support for disabling SSL certificate verification in the Elasticsearch > plugin > -- > > Key: DRILL-8385 > URL: https://issues.apache.org/jira/browse/DRILL-8385 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - ElasticSearch >Affects Versions: 1.20.3 >Reporter: James Turton >Assignee: Charles Givre >Priority: Minor > Fix For: 1.21.2 > > > In Calcite, provide a custom TrustManager that trusts every certificate to > the ES RestClient builder in ElasticsearchSchemaFactory if a corresponding > config option has been set by application code. > In Drill, add a config option to the ES plugin allowing certificate > verification to be toggled and pass it through to the Calcite option > mentioned above. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8385) Add support for disabling SSL certificate verification in the Elasticsearch plugin
[ https://issues.apache.org/jira/browse/DRILL-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8385: Fix Version/s: 1.21.2 (was: Future) > Add support for disabling SSL certificate verification in the Elasticsearch > plugin > -- > > Key: DRILL-8385 > URL: https://issues.apache.org/jira/browse/DRILL-8385 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - ElasticSearch >Affects Versions: 1.20.3 >Reporter: James Turton >Assignee: Charles Givre >Priority: Minor > Fix For: 1.21.2 > > > In Calcite, provide a custom TrustManager that trusts every certificate to > the ES RestClient builder in ElasticsearchSchemaFactory if a corresponding > config option has been set by application code. > In Drill, add a config option to the ES plugin allowing certificate > verification to be toggled and pass it through to the Calcite option > mentioned above. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8393) Allow parameters to be passed to headers through SQL in WHERE clause
[
https://issues.apache.org/jira/browse/DRILL-8393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton closed DRILL-8393.
---
Resolution: Fixed
> Allow parameters to be passed to headers through SQL in WHERE clause
>
>
> Key: DRILL-8393
> URL: https://issues.apache.org/jira/browse/DRILL-8393
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - HTTP
>Affects Versions: 1.20.0
>Reporter: Yuchen Liang
>Assignee: Charles Givre
>Priority: Major
> Fix For: 1.22.0
>
>
> Some APIs require parameters (e.g. digital signature) in the headers to be
> generated at access time.So I'm wondering if we can pass it in through filter
> statement.
> Perhaps we could design it like the params field in connections parameter.
> For example:
>
> Config:
> { "url": "https://api.sunrise-sunset.org/json;, "requireTail": false,
> "params": ["body.lat", "body.lng", "body.date", "header.header1"],
> "parameterLocation": "json_body" }
>
> SQL Query:
> SELECT * FROM api.sunrise
> WHERE `body.lat` = 36.7201600
> AND `body.lng` = -4.4203400
> AND `body.date` = '2019-10-02'
> AND `header.header1` = 'value1';
>
> Post body:
> { "lat": 36.7201600, "lng": -4.4203400, "date": "2019-10-02"}
>
> Headers:
> { "header1": "value1", ……}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8393) Allow parameters to be passed to headers through SQL in WHERE clause
[
https://issues.apache.org/jira/browse/DRILL-8393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton updated DRILL-8393:
Fix Version/s: 1.22.0
> Allow parameters to be passed to headers through SQL in WHERE clause
>
>
> Key: DRILL-8393
> URL: https://issues.apache.org/jira/browse/DRILL-8393
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - HTTP
>Affects Versions: 1.20.0
>Reporter: Yuchen Liang
>Priority: Major
> Fix For: 1.22.0
>
>
> Some APIs require parameters (e.g. digital signature) in the headers to be
> generated at access time.So I'm wondering if we can pass it in through filter
> statement.
> Perhaps we could design it like the params field in connections parameter.
> For example:
>
> Config:
> { "url": "https://api.sunrise-sunset.org/json;, "requireTail": false,
> "params": ["body.lat", "body.lng", "body.date", "header.header1"],
> "parameterLocation": "json_body" }
>
> SQL Query:
> SELECT * FROM api.sunrise
> WHERE `body.lat` = 36.7201600
> AND `body.lng` = -4.4203400
> AND `body.date` = '2019-10-02'
> AND `header.header1` = 'value1';
>
> Post body:
> { "lat": 36.7201600, "lng": -4.4203400, "date": "2019-10-02"}
>
> Headers:
> { "header1": "value1", ……}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (DRILL-8393) Allow parameters to be passed to headers through SQL in WHERE clause
[
https://issues.apache.org/jira/browse/DRILL-8393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Turton reassigned DRILL-8393:
---
Assignee: Charles Givre
> Allow parameters to be passed to headers through SQL in WHERE clause
>
>
> Key: DRILL-8393
> URL: https://issues.apache.org/jira/browse/DRILL-8393
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - HTTP
>Affects Versions: 1.20.0
>Reporter: Yuchen Liang
>Assignee: Charles Givre
>Priority: Major
> Fix For: 1.22.0
>
>
> Some APIs require parameters (e.g. digital signature) in the headers to be
> generated at access time.So I'm wondering if we can pass it in through filter
> statement.
> Perhaps we could design it like the params field in connections parameter.
> For example:
>
> Config:
> { "url": "https://api.sunrise-sunset.org/json;, "requireTail": false,
> "params": ["body.lat", "body.lng", "body.date", "header.header1"],
> "parameterLocation": "json_body" }
>
> SQL Query:
> SELECT * FROM api.sunrise
> WHERE `body.lat` = 36.7201600
> AND `body.lng` = -4.4203400
> AND `body.date` = '2019-10-02'
> AND `header.header1` = 'value1';
>
> Post body:
> { "lat": 36.7201600, "lng": -4.4203400, "date": "2019-10-02"}
>
> Headers:
> { "header1": "value1", ……}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (DRILL-8428) ElasticSearch Config Missing Getters
[ https://issues.apache.org/jira/browse/DRILL-8428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8428: Fix Version/s: 1.21.2 > ElasticSearch Config Missing Getters > > > Key: DRILL-8428 > URL: https://issues.apache.org/jira/browse/DRILL-8428 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.21.1 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.21.2 > > > The ElasticSearch config was missing some getters and as a result, prevented > users from setting certain config variables. This PR fixes this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8428) ElasticSearch Config Missing Getters
[ https://issues.apache.org/jira/browse/DRILL-8428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8428. --- Resolution: Fixed > ElasticSearch Config Missing Getters > > > Key: DRILL-8428 > URL: https://issues.apache.org/jira/browse/DRILL-8428 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.21.1 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.21.2 > > > The ElasticSearch config was missing some getters and as a result, prevented > users from setting certain config variables. This PR fixes this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8428) ElasticSearch Config Missing Getters
[ https://issues.apache.org/jira/browse/DRILL-8428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8428: Affects Version/s: 1.21.1 > ElasticSearch Config Missing Getters > > > Key: DRILL-8428 > URL: https://issues.apache.org/jira/browse/DRILL-8428 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.21.1 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > > The ElasticSearch config was missing some getters and as a result, prevented > users from setting certain config variables. This PR fixes this. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (DRILL-8433) Add Percent Change UDF to Drill
[ https://issues.apache.org/jira/browse/DRILL-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8433. --- Resolution: Fixed > Add Percent Change UDF to Drill > --- > > Key: DRILL-8433 > URL: https://issues.apache.org/jira/browse/DRILL-8433 > Project: Apache Drill > Issue Type: Improvement > Components: Functions - Drill >Affects Versions: 1.21.1 >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Fix For: 1.22.0 > > > Adds a function to calculate the percent change between two columns. Doing > this without a custom function is cumbersome because you have to include a > check for division by zero. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8445) Upgrade Janino 3.1.8 → 3.1.11
[ https://issues.apache.org/jira/browse/DRILL-8445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8445: Summary: Upgrade Janino 3.1.8 → 3.1.11 (was: Upgrade Janino) > Upgrade Janino 3.1.8 → 3.1.11 > - > > Key: DRILL-8445 > URL: https://issues.apache.org/jira/browse/DRILL-8445 > Project: Apache Drill > Issue Type: Task > Components: Server >Reporter: PJ Fanning >Priority: Major > > I'm not familar with exactly how janino is used inside Drill. > There is a new 3.1.10 release today to fix > [https://github.com/janino-compiler/janino/issues/201] > This may be an issue if Janino is used to parse input that may not be > entirely trustworthy. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (DRILL-8445) Upgrade Janino 3.1.8 → 3.1.11
[ https://issues.apache.org/jira/browse/DRILL-8445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton reassigned DRILL-8445: --- Assignee: James Turton > Upgrade Janino 3.1.8 → 3.1.11 > - > > Key: DRILL-8445 > URL: https://issues.apache.org/jira/browse/DRILL-8445 > Project: Apache Drill > Issue Type: Task > Components: Server >Reporter: PJ Fanning >Assignee: James Turton >Priority: Major > > I'm not familar with exactly how janino is used inside Drill. > There is a new 3.1.10 release today to fix > [https://github.com/janino-compiler/janino/issues/201] > This may be an issue if Janino is used to parse input that may not be > entirely trustworthy. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (DRILL-8473) Update Agirish/drill-helm-charts and incorporate it into Drill
[ https://issues.apache.org/jira/browse/DRILL-8473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton updated DRILL-8473: Summary: Update Agirish/drill-helm-charts and incorporate it into Drill (was: Update and incorporate Agirish/drill-helm-charts in Drill) > Update Agirish/drill-helm-charts and incorporate it into Drill > -- > > Key: DRILL-8473 > URL: https://issues.apache.org/jira/browse/DRILL-8473 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.21.1 >Reporter: James Turton >Assignee: James Turton >Priority: Minor > Fix For: 1.22.0 > > > Helm charts for deploying Drill on Kubernetes were developed by [~agirish] > and [released under the Apache > License|https://github.com/Agirish/drill-helm-charts]. These charts can be > updated to make use of the container images that are automatically published > to Docker Hub and incorporated in the Drill codebase where they can be > maintained by Drill contributors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-2835) IndexOutOfBoundsException in partition sender when doing streaming aggregate with LIMIT
[ https://issues.apache.org/jira/browse/DRILL-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17802012#comment-17802012 ] ASF GitHub Bot commented on DRILL-2835: --- paul-rogers commented on PR #2836: URL: https://github.com/apache/drill/pull/2836#issuecomment-1874845274 Hi Mike, Just jumping in with a random thought. Drill has accumulated a number of schema systems: Parquet metadata cache, HMS, Drill's own metastore, "provided schema", and now DFDL. All provide ways of defining data: be it Parquet, JSON, CSV or whatever. One can't help but wonder, should some future version try to reduce this variation somewhat? Maybe map all the variations to DFDL? Map DFDL to Drill's own mechanisms? Drill uses two kinds of metadata: schema definitions and file metadata used for scan pruning. Schema information could be used at plan time (to provide column types), but certainly at scan time (to "discover" the defined schema.) File metadata is used primarily at plan time to work out how to distribute work. A bit of background on scan pruning. Back in the day, it was common to have thousands or millions of files in Hadoop to scan: this was why tools like Drill were distributed: divide and conquer. And, of course, the fastest scan is to skip files that we know can't contain the information we want. File metadata captures this information outside of the files themselves. HMS was the standard solution in the Hadoop days. (Amazon Glue, for S3, is evidently based on HMS.) For example, Drill's Parquet metadata cache, the Drill metastore and HMS all provide both schema and file metadata information. The schema information mainly helped with schema evolution: over time, different files have different sets of columns. File metadata provides information *about* the file, such as the data ranges stored in each file. For Parquet, we might track that '2023-01-Boston.parquet' has data from the office='Boston' range. (So, no use scanning the file for office='Austin'.) And so on. With Hadoop HFS, it was customary to use directory structure as a partial primary index: our file above would live in the /sales/2023/01 directory, for example, and logic chooses the proper set of directories to scan. In Drill, it is up to the user to add crufty conditionals on the path name. In Impala, and other HMS-aware tools, the user just says WHERE order_year = 2023 AND order_month = 1, and HMS tells the tool that the order_year and order_month columns translate to such-and-so directory paths. Would be nice if Drill could provide that feature as well, given the proper file metadata: in this case, the mapping of column names to path directories and file names. Does DFDL provide only schema information? Does it support versioning so that we know that "old.csv" lacks the "version" column, while "new.csv" includes that column? Does it also include the kinds of file metadata mentioned above? Or, perhaps DFDL is used in a different context in which the files have a fixed schema and are small in number? This would fit well the "desktop analytics" model that Charles and James suggested is where Drill is now most commonly used. The answers might suggest if DFDL can be the universal data description. or if DFDL applies just to individual file schemas, and Drill would still need a second system to track schema evolution and file metadata for large deployments. Further, if DFDL is kind of a stand-alone thing, with its own reader, then we end up with more complexity: the Drill JSON reader and the DFDL JSON reader. Same for CSV, etc. JSON is so complex that we'd find ourselves telling people that the quirks work one way with the native reader, another way with DFDL. Plus, the DFDL readers might not handle file splits the same way, or support the same set of formats that Drill's other readers support, and so on. It would be nice to separate the idea of schema description from reader implementation, so that DFDL can be used as a source of schema for any arbitrary reader: both at plan and scan times. If DFDL uses its own readers, then we'd need DFDL reader representations in Calcite, which would pick up DFDL schemas so that the schemas are reliably serialized out to each node as part of the physical plan. This is possible, but it does send us down the two-readers-for-every-format path. On the other hand, if DFDL mapped to Drill's existing schema description, then DFDL could be used with our existing readers and there would be just one schema description sent to readers: Drill's existing provided schema format that EVF can already consume. At present, just a few formats support provided schema in the Calcite layer: CSV for sure, maybe JSON? Any thoughts on
[jira] [Commented] (DRILL-8468) Drill doesn't perform drill.exec.storage.action_on_plugins_override_file action
[
https://issues.apache.org/jira/browse/DRILL-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17802009#comment-17802009
]
ASF GitHub Bot commented on DRILL-8468:
---
jnturton merged PR #2859:
URL: https://github.com/apache/drill/pull/2859
> Drill doesn't perform drill.exec.storage.action_on_plugins_override_file
> action
> ---
>
> Key: DRILL-8468
> URL: https://issues.apache.org/jira/browse/DRILL-8468
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Major
>
> After the Plugin Registry refactoring
> https://issues.apache.org/jira/browse/DRILL-7590, Drill stopped honoring
> {{drill.exec.storage.action_on_plugins_override_file}} property.
> After the refactoring, Drill performs
> {{drill.exec.storage.action_on_plugins_override_file}} action only once, on
> initial storage plugin registry initialization. After the initialization, on
> every next Drill restart, the action will not be performed. It means, that
> after {{storage-plugins-override.conf}} apply, this file is neither renamed
> nor removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-2835) IndexOutOfBoundsException in partition sender when doing streaming aggregate with LIMIT
[
https://issues.apache.org/jira/browse/DRILL-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17801835#comment-17801835
]
ASF GitHub Bot commented on DRILL-2835:
---
mbeckerle commented on code in PR #2836:
URL: https://github.com/apache/drill/pull/2836#discussion_r1439542636
##
contrib/format-daffodil/src/main/java/org/apache/drill/exec/store/daffodil/DaffodilBatchReader.java:
##
@@ -0,0 +1,184 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.drill.exec.store.daffodil;
+
+import java.io.InputStream;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.util.Objects;
+
+import org.apache.daffodil.japi.DataProcessor;
+import org.apache.drill.common.AutoCloseables;
+import org.apache.drill.common.exceptions.CustomErrorContext;
+import org.apache.drill.common.exceptions.UserException;
+import org.apache.drill.exec.physical.impl.scan.v3.ManagedReader;
+import org.apache.drill.exec.physical.impl.scan.v3.file.FileDescrip;
+import org.apache.drill.exec.physical.impl.scan.v3.file.FileSchemaNegotiator;
+import org.apache.drill.exec.physical.resultSet.RowSetLoader;
+import org.apache.drill.exec.record.metadata.TupleMetadata;
+import
org.apache.drill.exec.store.daffodil.schema.DaffodilDataProcessorFactory;
+import org.apache.drill.exec.store.dfs.DrillFileSystem;
+import org.apache.drill.exec.store.dfs.easy.EasySubScan;
+import org.apache.hadoop.fs.Path;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static
org.apache.drill.exec.store.daffodil.schema.DrillDaffodilSchemaUtils.daffodilDataProcessorToDrillSchema;
+
+
+public class DaffodilBatchReader implements ManagedReader {
+
+ private static final Logger logger =
LoggerFactory.getLogger(DaffodilBatchReader.class);
+ private final DaffodilFormatConfig dafConfig;
+ private final RowSetLoader rowSetLoader;
+ private final CustomErrorContext errorContext;
+ private final DaffodilMessageParser dafParser;
+ private final InputStream dataInputStream;
+
+ static class DaffodilReaderConfig {
+final DaffodilFormatPlugin plugin;
+DaffodilReaderConfig(DaffodilFormatPlugin plugin) {
+ this.plugin = plugin;
+}
+ }
+
+ public DaffodilBatchReader (DaffodilReaderConfig readerConfig, EasySubScan
scan, FileSchemaNegotiator negotiator) {
+
+errorContext = negotiator.parentErrorContext();
+this.dafConfig = readerConfig.plugin.getConfig();
+
+String schemaURIString = dafConfig.getSchemaURI(); //
"schema/complexArray1.dfdl.xsd";
+String rootName = dafConfig.getRootName();
+String rootNamespace = dafConfig.getRootNamespace();
+boolean validationMode = dafConfig.getValidationMode();
+
+URI dfdlSchemaURI;
+try {
+ dfdlSchemaURI = new URI(schemaURIString);
+} catch (URISyntaxException e) {
+ throw UserException.validationError(e)
+ .build(logger);
+}
+
+FileDescrip file = negotiator.file();
+DrillFileSystem fs = file.fileSystem();
+URI fsSchemaURI = fs.getUri().resolve(dfdlSchemaURI);
+
+
+DaffodilDataProcessorFactory dpf = new DaffodilDataProcessorFactory();
+DataProcessor dp;
+try {
+ dp = dpf.getDataProcessor(fsSchemaURI, validationMode, rootName,
rootNamespace);
+} catch (Exception e) {
+ throw UserException.dataReadError(e)
+ .message(String.format("Failed to get Daffodil DFDL processor for:
%s", fsSchemaURI))
+ .addContext(errorContext).addContext(e.getMessage()).build(logger);
+}
+// Create the corresponding Drill schema.
+// Note: this could be a very large schema. Think of a large complex RDBMS
schema,
+// all of it, hundreds of tables, but all part of the same metadata tree.
+TupleMetadata drillSchema = daffodilDataProcessorToDrillSchema(dp);
+// Inform Drill about the schema
+negotiator.tableSchema(drillSchema, true);
+
+//
+// DATA TIME: Next we construct the runtime objects, and open files.
+//
+// We get the DaffodilMessageParser, which is a stateful driver for
daffodil that
+// actually does the parsing.
+rowSetLoader = negotiator.build().writer();
+
+// We construct
[jira] [Commented] (DRILL-2835) IndexOutOfBoundsException in partition sender when doing streaming aggregate with LIMIT
[
https://issues.apache.org/jira/browse/DRILL-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17801834#comment-17801834
]
ASF GitHub Bot commented on DRILL-2835:
---
mbeckerle commented on PR #2836:
URL: https://github.com/apache/drill/pull/2836#issuecomment-1874213780
@cgivre yes, the next architectural-level issue is how to get a compiled
DFDL schema out to everyplace Drill will run a Daffodil parse. Every one of
those JVMs needs to reload it.
I'll do the various cleanups and such. The one issue I don't know how to fix
is the "typed setter" vs. (set-object) issue, so if you could steer me in the
right direction on that it would help.
> IndexOutOfBoundsException in partition sender when doing streaming aggregate
> with LIMIT
>
>
> Key: DRILL-2835
> URL: https://issues.apache.org/jira/browse/DRILL-2835
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - RPC
>Affects Versions: 0.8.0
>Reporter: Aman Sinha
>Assignee: Venki Korukanti
>Priority: Major
> Fix For: 0.9.0
>
> Attachments: DRILL-2835-1.patch, DRILL-2835-2.patch
>
>
> Following CTAS run on a TPC-DS 100GB scale factor on a 10-node cluster:
> {code}
> alter session set `planner.enable_hashagg` = false;
> alter session set `planner.enable_multiphase_agg` = true;
> create table dfs.tmp.stream9 as
> select cr_call_center_sk , cr_catalog_page_sk , cr_item_sk , cr_reason_sk ,
> cr_refunded_addr_sk , count(*) from catalog_returns_dri100
> group by cr_call_center_sk , cr_catalog_page_sk , cr_item_sk , cr_reason_sk
> , cr_refunded_addr_sk
> limit 100
> ;
> {code}
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: index: 1023, length: 1
> (expected: range(0, 0))
> at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:200)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
> at io.netty.buffer.DrillBuf.chk(DrillBuf.java:222)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
> at io.netty.buffer.DrillBuf.setByte(DrillBuf.java:621)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
> at
> org.apache.drill.exec.vector.UInt1Vector$Mutator.set(UInt1Vector.java:342)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.vector.NullableBigIntVector$Mutator.set(NullableBigIntVector.java:372)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.vector.NullableBigIntVector.copyFrom(NullableBigIntVector.java:284)
> ~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
> at
> org.apache.drill.exec.test.generated.PartitionerGen4$OutgoingRecordBatch.doEval(PartitionerTemplate.java:370)
> ~[na:na]
> at
> org.apache.drill.exec.test.generated.PartitionerGen4$OutgoingRecordBatch.copy(PartitionerTemplate.java:249)
> ~[na:na]
> at
> org.apache.drill.exec.test.generated.PartitionerGen4.doCopy(PartitionerTemplate.java:208)
> ~[na:na]
> at
> org.apache.drill.exec.test.generated.PartitionerGen4.partitionBatch(PartitionerTemplate.java:176)
> ~[na:na]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8467) Update netty to 4.1.101
[
https://issues.apache.org/jira/browse/DRILL-8467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17801802#comment-17801802
]
ASF GitHub Bot commented on DRILL-8467:
---
jnturton merged PR #2857:
URL: https://github.com/apache/drill/pull/2857
> Update netty to 4.1.101
> ---
>
> Key: DRILL-8467
> URL: https://issues.apache.org/jira/browse/DRILL-8467
> Project: Apache Drill
> Issue Type: Task
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Minor
> Fix For: 1.22.0
>
>
> Update Netty to the last available at this moment stable Netty build –
> {{{}4.1.101{}}}.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (DRILL-8443) upgrade netty to 4.1.94 due to CVE
[ https://issues.apache.org/jira/browse/DRILL-8443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton closed DRILL-8443. --- Resolution: Duplicate > upgrade netty to 4.1.94 due to CVE > -- > > Key: DRILL-8443 > URL: https://issues.apache.org/jira/browse/DRILL-8443 > Project: Apache Drill > Issue Type: Task > Components: Server >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > > https://github.com/apache/drill/security/dependabot/45 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (DRILL-8443) upgrade netty to 4.1.94 due to CVE
[ https://issues.apache.org/jira/browse/DRILL-8443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Turton reopened DRILL-8443: - Assignee: PJ Fanning > upgrade netty to 4.1.94 due to CVE > -- > > Key: DRILL-8443 > URL: https://issues.apache.org/jira/browse/DRILL-8443 > Project: Apache Drill > Issue Type: Task > Components: Server >Reporter: PJ Fanning >Assignee: PJ Fanning >Priority: Major > > https://github.com/apache/drill/security/dependabot/45 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (DRILL-8468) Drill doesn't perform drill.exec.storage.action_on_plugins_override_file action
[
https://issues.apache.org/jira/browse/DRILL-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17801782#comment-17801782
]
ASF GitHub Bot commented on DRILL-8468:
---
jnturton commented on PR #2859:
URL: https://github.com/apache/drill/pull/2859#issuecomment-1874010034
@rymarm okay let's rebase, get clean runs and merge!
> Drill doesn't perform drill.exec.storage.action_on_plugins_override_file
> action
> ---
>
> Key: DRILL-8468
> URL: https://issues.apache.org/jira/browse/DRILL-8468
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Major
>
> After the Plugin Registry refactoring
> https://issues.apache.org/jira/browse/DRILL-7590, Drill stopped honoring
> {{drill.exec.storage.action_on_plugins_override_file}} property.
> After the refactoring, Drill performs
> {{drill.exec.storage.action_on_plugins_override_file}} action only once, on
> initial storage plugin registry initialization. After the initialization, on
> every next Drill restart, the action will not be performed. It means, that
> after {{storage-plugins-override.conf}} apply, this file is neither renamed
> nor removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (DRILL-8468) Drill doesn't perform drill.exec.storage.action_on_plugins_override_file action
[
https://issues.apache.org/jira/browse/DRILL-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17801757#comment-17801757
]
ASF GitHub Bot commented on DRILL-8468:
---
rymarm commented on PR #2859:
URL: https://github.com/apache/drill/pull/2859#issuecomment-1873939057
> Are the two removed null checks not needed?
@jnturton Yes, they are redundant.
``` java
if (locatorPlugins != null) {
bootstrapPlugins.putAll(locatorPlugins);
}
```
It is redundant, because `StoragePlugins#putAll` himself checks the
argument nullability:
https://github.com/apache/drill/blob/15c446d7e8ebe6d97fc879103e1710d733318c6d/exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/StoragePlugins.java#L114-L117
```java
if (upgraded != null) {
upgraded.putAll(locatorPlugins);
}
```
It is redundant, because `upgraded` is just initialized with a constructor
just a few lines above and not overridden.
> Drill doesn't perform drill.exec.storage.action_on_plugins_override_file
> action
> ---
>
> Key: DRILL-8468
> URL: https://issues.apache.org/jira/browse/DRILL-8468
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.18.0
>Reporter: Maksym Rymar
>Assignee: Maksym Rymar
>Priority: Major
>
> After the Plugin Registry refactoring
> https://issues.apache.org/jira/browse/DRILL-7590, Drill stopped honoring
> {{drill.exec.storage.action_on_plugins_override_file}} property.
> After the refactoring, Drill performs
> {{drill.exec.storage.action_on_plugins_override_file}} action only once, on
> initial storage plugin registry initialization. After the initialization, on
> every next Drill restart, the action will not be performed. It means, that
> after {{storage-plugins-override.conf}} apply, this file is neither renamed
> nor removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
