[jira] [Commented] (CALCITE-6590) Run tests with java.security.manager=allow on JDK23+ in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17885231#comment-17885231
]
Istvan Toth commented on CALCITE-6590:
--
I merged the ByteBuddy bump, and re-based the reflection patch on top of it.
> Run tests with java.security.manager=allow on JDK23+ in Avatica
> ---
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-6601) Upgrade ByteBuddy version from 1.14.10 to 1.15.1 in Avatica
[ https://issues.apache.org/jira/browse/CALCITE-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved CALCITE-6601. -- Fix Version/s: avatica-1.26.0 Resolution: Fixed > Upgrade ByteBuddy version from 1.14.10 to 1.15.1 in Avatica > --- > > Key: CALCITE-6601 > URL: https://issues.apache.org/jira/browse/CALCITE-6601 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: avatica-1.26.0 > > > The old ByteBuddy version does not support Java 23 bytecode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6601) Upgrade ByteBuddy version from 1.14.10 to 1.15.1 in Avatica
Istvan Toth created CALCITE-6601: Summary: Upgrade ByteBuddy version from 1.14.10 to 1.15.1 in Avatica Key: CALCITE-6601 URL: https://issues.apache.org/jira/browse/CALCITE-6601 Project: Calcite Issue Type: Bug Components: avatica Reporter: Istvan Toth Assignee: Istvan Toth -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6590) Run tests with java.security.manager=allow on JDK23+ in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17885128#comment-17885128
]
Istvan Toth commented on CALCITE-6590:
--
Thank you.
I haven't noticed Compatible yet.
Should I add the SecurityManager related methods there instead of adding
SecurityUtil ?
I have the alternative fix using reflections up at
https://github.com/apache/calcite-avatica/pull/251
I will separate out the ByteBuddy upgrade, and obviously the JIRA description
and commit message will need to be updated to reflect the final solution chosen.
> Run tests with java.security.manager=allow on JDK23+ in Avatica
> ---
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6590) Run tests with java.security.manager=allow on JDK23+ in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17885134#comment-17885134
]
Istvan Toth commented on CALCITE-6590:
--
I have opened a separate JIRA and PR for the ByteBuddy version bump.
> Run tests with java.security.manager=allow on JDK23+ in Avatica
> ---
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-6601) Upgrade ByteBuddy version from 1.14.10 to 1.15.1 in Avatica
[ https://issues.apache.org/jira/browse/CALCITE-6601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated CALCITE-6601: - Description: The old ByteBuddy version does not support Java 23 bytecode. > Upgrade ByteBuddy version from 1.14.10 to 1.15.1 in Avatica > --- > > Key: CALCITE-6601 > URL: https://issues.apache.org/jira/browse/CALCITE-6601 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > The old ByteBuddy version does not support Java 23 bytecode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6590) Run tests with java.security.manager=allow on JDK23+ in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884865#comment-17884865
]
Istvan Toth commented on CALCITE-6590:
--
Kind of.
The code will not directly reference the methods, but they will be used
referenced and called indirectly via reflection if the new API is not available
(i.e. on older JDKs).
The compiler will not detect the old methods, will not generate warnings at
least.
> Run tests with java.security.manager=allow on JDK23+ in Avatica
> ---
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6590) Run tests with java.security.manager=allow on JDK23+ in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884616#comment-17884616
]
Istvan Toth commented on CALCITE-6590:
--
I have changed my mind.
I think that using a reflection based solution is better.
I originally thought (and said) that we only use the deprecated methods in the
Hadoop specific code, but I was wrong, we run the whole HTTP server in a
doAS(){} block.
We can lift it directly from Jetty 12:
https://github.com/jetty/jetty.project/blob/jetty-12.1.x/jetty-core/jetty-util/src/main/java/org/eclipse/jetty/util/security/SecurityUtils.java
> Run tests with java.security.manager=allow on JDK23+ in Avatica
> ---
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-6590) Run tests with java.security.manager=allow on JDK23+ in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884578#comment-17884578
]
Istvan Toth edited comment on CALCITE-6590 at 9/25/24 1:26 PM:
---
bq. Looking at the code, getSubject() only seems to be called when Kerberos
authentication is explicitly configured,
bq. which typically means Hadoop-related environments, where this option will
be needed anyway by the other libraries.
This is not true. We call Subject.doAs every time we start the Avatica HTTP
server.
was (Author: stoty):
bq. Looking at the code, getSubject() only seems to be called when Kerberos
authentication is explicitly configured,
bq. which typically means Hadoop-related environments, where this option will
be needed anyway by the other libraries.
This is not true. We call Subject.doAs every time we start the Avatica HTPP
server.
> Run tests with java.security.manager=allow on JDK23+ in Avatica
> ---
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6590) Run tests with java.security.manager=allow on JDK23+ in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884578#comment-17884578
]
Istvan Toth commented on CALCITE-6590:
--
bq. Looking at the code, getSubject() only seems to be called when Kerberos
authentication is explicitly configured,
bq. which typically means Hadoop-related environments, where this option will
be needed anyway by the other libraries.
This is not true. We call Subject.doAs every time we start the Avatica HTPP
server.
> Run tests with java.security.manager=allow on JDK23+ in Avatica
> ---
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-6590) Run tests with java.security.manager=allow on JDK23+ in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated CALCITE-6590:
-
Summary: Run tests with java.security.manager=allow on JDK23+ in Avatica
(was: Remove use of Java SecurityManager in Avatica)
> Run tests with java.security.manager=allow on JDK23+ in Avatica
> ---
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-6590) Remove use of Java SecurityManager in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884529#comment-17884529
]
Istvan Toth edited comment on CALCITE-6590 at 9/25/24 8:11 AM:
---
bq. The bytebuddy upgrade should be a separate commit (we should upgrade other
dependencies too).
OK, I will leave that out.
The tests won't run cleanly on JKD23 until that is done, though.
bq. “See CALCITE-” isn’t a very useful comment. I think the explanation
should be in the code.
OK
bq. The changes you make to gradle test launchers could solve this problem in
Calcite without requiring an Avatica upgrade.
I agree. Would you like me to open a PR for Calcite as well ?
was (Author: stoty):
bq. The bytebuddy upgrade should be a separate commit (we should upgrade other
dependencies too).
OK, I will leave that out.
The tests won't run cleanly on JKD23 until that is done, though.
bq. “See CALCITE-” isn’t a very useful comment. I think the explanation
should be in the code.
OK
bq. The changes you make to gradle test launchers could solve this problem in
Calcite without requiring an Avatica upgrade.
I agree. Would you like me to ope a PR for Calcite as well ?
> Remove use of Java SecurityManager in Avatica
> -
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-6590) Remove use of Java SecurityManager in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884529#comment-17884529
]
Istvan Toth edited comment on CALCITE-6590 at 9/25/24 8:10 AM:
---
bq. The bytebuddy upgrade should be a separate commit (we should upgrade other
dependencies too).
OK, I will leave that out.
The tests won't run cleanly on JKD23 until that is done, though.
bq. “See CALCITE-” isn’t a very useful comment. I think the explanation
should be in the code.
OK
bq. The changes you make to gradle test launchers could solve this problem in
Calcite without requiring an Avatica upgrade.
I agree. Would you like me to ope a PR for Calcite as well ?
was (Author: stoty):
bq. The bytebuddy upgrade should be a separate commit (we should upgrade other
dependencies too).
bq. “See CALCITE-” isn’t a very useful comment. I think the explanation
should be in the code.
bq. The changes you make to gradle test launchers could solve this problem in
Calcite without requiring an Avatica upgrade.
> Remove use of Java SecurityManager in Avatica
> -
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6590) Remove use of Java SecurityManager in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884530#comment-17884530
]
Istvan Toth commented on CALCITE-6590:
--
bq. The bytebuddy upgrade should be a separate commit (we should upgrade other
dependencies too).
bq. “See CALCITE-” isn’t a very useful comment. I think the explanation
should be in the code.
bq. The changes you make to gradle test launchers could solve this problem in
Calcite without requiring an Avatica upgrade.
> Remove use of Java SecurityManager in Avatica
> -
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] (CALCITE-6590) Remove use of Java SecurityManager in Avatica
[ https://issues.apache.org/jira/browse/CALCITE-6590 ]
Istvan Toth deleted comment on CALCITE-6590:
--
was (Author: stoty):
bq. The bytebuddy upgrade should be a separate commit (we should upgrade other
dependencies too).
bq. “See CALCITE-” isn’t a very useful comment. I think the explanation
should be in the code.
bq. The changes you make to gradle test launchers could solve this problem in
Calcite without requiring an Avatica upgrade.
> Remove use of Java SecurityManager in Avatica
> -
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6590) Remove use of Java SecurityManager in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884529#comment-17884529
]
Istvan Toth commented on CALCITE-6590:
--
bq. The bytebuddy upgrade should be a separate commit (we should upgrade other
dependencies too).
bq. “See CALCITE-” isn’t a very useful comment. I think the explanation
should be in the code.
bq. The changes you make to gradle test launchers could solve this problem in
Calcite without requiring an Avatica upgrade.
> Remove use of Java SecurityManager in Avatica
> -
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6590) Remove use of Java SecurityManager in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17884117#comment-17884117
]
Istvan Toth commented on CALCITE-6590:
--
IMO enabling the security manager for Java 23+ is the best option.
Looking at the code, getSubject() only seems to be called when Kerberos
authentication is explicitly configured,
which typically means Hadoop-related environments, where this option will be
needed anyway by the other libraries.
> Remove use of Java SecurityManager in Avatica
> -
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (CALCITE-6590) Remove use of Java SecurityManager in Avatica
[
https://issues.apache.org/jira/browse/CALCITE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth reassigned CALCITE-6590:
Assignee: Istvan Toth
> Remove use of Java SecurityManager in Avatica
> -
>
> Key: CALCITE-6590
> URL: https://issues.apache.org/jira/browse/CALCITE-6590
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Istvan Toth
>Priority: Major
> Fix For: 1.26.0
>
>
> Remove use of Java SecurityManager in Avatica.
> Running Avatica on JDK 23 (or JRE 23) we get the following runtime errors:
> {noformat}
> Caused by: java.lang.UnsupportedOperationException: getSubject is supported
> only if a security manager is allowed at
> java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at
> org.apache.calcite.avatica.server.SubjectPreservingPrivilegedThreadFactory.newThread(SubjectPreservingPrivilegedThreadFactory.java:43)
> {noformat}
> We were warned - the {{getSubject}} method has been deprecated since JDK 18.
> The deprecation did not cause a build failure, due to CALCITE-5136.
> HADOOP-19212 is an issue with the same underlying cause.
> The message "getSubject is supported only if a security manager is allowed"
> implies that another solution would be to enable a security manager
> (including during tests, and when Avatica is used in Calcite's tests). Should
> we consider that? If so, please change the case summary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-6529) Use persistent sessionContext in AvaticaCommonsHttpClientImpl
[ https://issues.apache.org/jira/browse/CALCITE-6529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved CALCITE-6529. -- Fix Version/s: avatica-1.26.0 Resolution: Fixed Committed. Thanks for the patch [~szucsvillo]. > Use persistent sessionContext in AvaticaCommonsHttpClientImpl > - > > Key: CALCITE-6529 > URL: https://issues.apache.org/jira/browse/CALCITE-6529 > Project: Calcite > Issue Type: Improvement >Reporter: Istvan Toth >Priority: Minor > Labels: pull-request-available > Fix For: avatica-1.26.0 > > > We are re-creating and setting up a SessionContext object for each HTTP call. > We could use a sessionContext object field for the same purpose, or even > better, just set those properties directly on the HttpClient object. > Every member of HttpSession is duplicated in HttpClient, HttpSession is only > used as an override. Since we have a 1:1 relationship between HttpClient and > HttpSession objects, HttpSession is redundant. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-6529) Use persistent sessionContext in AvaticaCommonsHttpClientImpl
[ https://issues.apache.org/jira/browse/CALCITE-6529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated CALCITE-6529: - Description: We are re-creating and setting up a SessionContext object for each HTTP call. We could use a sessionContext object field for the same purpose, or even better, just set those properties directly on the HttpClient object. Every member of HttpSession is duplicated in HttpClient, HttpSession is only used as an override. Since we have a 1:1 relationship between HttpClient and HttpSession objects, HttpSession is redundant. was: We are re-creating and setting up a SessionContext object for each HTTP call. We could use a sessionContext method for the same purpose, or even better, just set those properties directly on the HttpClient object. Every member of HttpSession is duplicated in HttpClient, HttpSession is only used as an override. Since we have a 1:1 relationship between HttpClient and HttpSession objects, HttpSession is redundant. > Use persistent sessionContext in AvaticaCommonsHttpClientImpl > - > > Key: CALCITE-6529 > URL: https://issues.apache.org/jira/browse/CALCITE-6529 > Project: Calcite > Issue Type: Improvement >Reporter: Istvan Toth >Priority: Minor > Labels: pull-request-available > > We are re-creating and setting up a SessionContext object for each HTTP call. > We could use a sessionContext object field for the same purpose, or even > better, just set those properties directly on the HttpClient object. > Every member of HttpSession is duplicated in HttpClient, HttpSession is only > used as an override. Since we have a 1:1 relationship between HttpClient and > HttpSession objects, HttpSession is redundant. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (CALCITE-6364) HttpClient SPENGO support is deprecated
[ https://issues.apache.org/jira/browse/CALCITE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth reassigned CALCITE-6364: Assignee: Istvan Toth > HttpClient SPENGO support is deprecated > --- > > Key: CALCITE-6364 > URL: https://issues.apache.org/jira/browse/CALCITE-6364 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Critical > > The Avatica Java client depends on Apache HttpClient's Kerberos/SPNEGO > implementation. > According to HTTPCLIENT-1625 that implementation is not secure, and is > deprecated in newer versions. > Unfortunately, HTTPCLIENT-1625 is very scant on details, and since the reason > given for deprecation is the lack of time to fix it, it is likely not a > trivial fix. > Unfortunately, Avatica depends heavily on httpclient, and replacing it would > it would be a big job. > While Avatica in theory has a configurable Http Client implementation, the > only non-httpclient implementation is more of a POC, and does not support ANY > authentication methods. > I can see these options: > 1. Find an another http client library, and use it in Avatica > 2. Copy the SPENGO auth code from httpclient, and fix it in Avatica > 3. Fix the SPENGO auth code in httpclient. > 4. Re-Implement SPENGO auth in Avatica (Hadoop does something like that, > though I'm, not sure how good that is) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (CALCITE-6530) HTTP Sessions are never expired in Avatica server
[ https://issues.apache.org/jira/browse/CALCITE-6530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved CALCITE-6530. -- Fix Version/s: avatica-1.26.0 Resolution: Fixed > HTTP Sessions are never expired in Avatica server > - > > Key: CALCITE-6530 > URL: https://issues.apache.org/jira/browse/CALCITE-6530 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: avatica-1.26.0 > > > Http Sessions created by the Avatica server are never expired. > This eventually causes the Avatica server to OOM. > Well designed clients using long-lived sessions may never generate enough > sessions between server restarts for this to become apparent, but ill-behaved > clients creating a lot of sessions can fill the heap in a short time. > As the Jetty default (when not using the web.xml descriptor) is infinite > lifetime, we need to set a reasonable default idle timeout when setting up > Jetty Session handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6530) HTTP Sessions are never expired in Avatica server
[ https://issues.apache.org/jira/browse/CALCITE-6530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17873659#comment-17873659 ] Istvan Toth commented on CALCITE-6530: -- The session timeout is set on the Jetty SessionHandler object via the setMaxInactiveInterval() method. This patch sets it to a fixed 1 hour. The session is only used for bypassing the potentially costly full authentication process. Before CALCITE-4152 Avatica had to authenticate every single request, so an extremely low timeout value would cause Avatica to revert to the pre CALCITE-4152 behaviour (and performance). We need to set the timeout to a value that is low enough so that we do not run into an OOM situation even when a lot of sessions are started, but high enough so that the majority of requests will be able to use the session cookie instead of having to re-authenticate. The main advantage of using a session is in machine-to-machine use cases like streaming or bulk loading data, or serving a web frontend, where a very high volume of requests need to be served, as that's the where the authentication costs can bottleneck performance. If clients only make requests rarely enough to hit the session timeout (like an interactive sql session), then the total volume of requests is expected to be low enough that the added cost re-authentication after timing out a session is unlikely to be a problem. So while there is nothing stopping us from making the value configurable, I cannot really see a use case where being able to set an exact timeout value matters a great deal. We could consider using 30 minutes instead of 60, so that we align with the default value when web.xml is used for configuring Jetty. > HTTP Sessions are never expired in Avatica server > - > > Key: CALCITE-6530 > URL: https://issues.apache.org/jira/browse/CALCITE-6530 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > > Http Sessions created by the Avatica server are never expired. > This eventually causes the Avatica server to OOM. > Well designed clients using long-lived sessions may never generate enough > sessions between server restarts for this to become apparent, but ill-behaved > clients creating a lot of sessions can fill the heap in a short time. > As the Jetty default (when not using the web.xml descriptor) is infinite > lifetime, we need to set a reasonable default idle timeout when setting up > Jetty Session handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6530) HTTP Sessions are never expired in Avatica server
[ https://issues.apache.org/jira/browse/CALCITE-6530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17873622#comment-17873622 ] Istvan Toth commented on CALCITE-6530: -- Yes, that's what the patch does. Added this to the description [~julianhyde]. > HTTP Sessions are never expired in Avatica server > - > > Key: CALCITE-6530 > URL: https://issues.apache.org/jira/browse/CALCITE-6530 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > > Http Sessions created by the Avatica server are never expired. > This eventually causes the Avatica server to OOM. > Well designed clients using long-lived sessions may never generate enough > sessions between server restarts for this to become apparent, but ill-behaved > clients creating a lot of sessions can fill the heap in a short time. > As the Jetty default (when not using the web.xml descriptor) is infinite > lifetime, we need to set a reasonable default idle timeout when setting up > Jetty Session handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-6530) HTTP Sessions are never expired in Avatica server
[ https://issues.apache.org/jira/browse/CALCITE-6530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated CALCITE-6530: - Description: Http Sessions created by the Avatica server are never expired. This eventually causes the Avatica server to OOM. Well designed clients using long-lived sessions may never generate enough sessions between server restarts for this to become apparent, but ill-behaved clients creating a lot of sessions can fill the heap in a short time. As the Jetty default (when not using the web.xml descriptor) is infinite lifetime, we need to set a reasonable default idle timeout when setting up Jetty Session handling. was: Http Sessions created by the Avatica server are never expired. This eventually causes the Avatica server to OOM. Well designed clients using long-lived sessions may never generate enough sessions between server restarts for this to become apparent, but ill-behaved clients creating a lot of sessions can fill the heap in a short time. > HTTP Sessions are never expired in Avatica server > - > > Key: CALCITE-6530 > URL: https://issues.apache.org/jira/browse/CALCITE-6530 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > > Http Sessions created by the Avatica server are never expired. > This eventually causes the Avatica server to OOM. > Well designed clients using long-lived sessions may never generate enough > sessions between server restarts for this to become apparent, but ill-behaved > clients creating a lot of sessions can fill the heap in a short time. > As the Jetty default (when not using the web.xml descriptor) is infinite > lifetime, we need to set a reasonable default idle timeout when setting up > Jetty Session handling. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-6530) HTTP Sessions are never expired in Avatica server
[ https://issues.apache.org/jira/browse/CALCITE-6530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated CALCITE-6530: - Summary: HTTP Sessions are never expired in Avatica server (was: HTTP Sessions are never exipred in Avatica server) > HTTP Sessions are never expired in Avatica server > - > > Key: CALCITE-6530 > URL: https://issues.apache.org/jira/browse/CALCITE-6530 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > > Http Sessions created by the Avatica server are never expired. > This eventually causes the Avatica server to OOM. > Well designed clients using long-lived sessions may never generate enough > sessions between server restarts for this to become apparent, but ill-behaved > clients creating a lot of sessions can fill the heap in a short time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6530) HTTP Sessions are never exipred in Avatica server
Istvan Toth created CALCITE-6530: Summary: HTTP Sessions are never exipred in Avatica server Key: CALCITE-6530 URL: https://issues.apache.org/jira/browse/CALCITE-6530 Project: Calcite Issue Type: Bug Components: avatica Reporter: Istvan Toth Assignee: Istvan Toth Http Sessions created by the Avatica server are never expired. This eventually causes the Avatica server to OOM. Well designed clients using long-lived sessions may never generate enough sessions between server restarts for this to become apparent, but ill-behaved clients creating a lot of sessions can fill the heap in a short time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6529) Use persistent sessionContext in AvaticaCommonsHttpClientImpl
Istvan Toth created CALCITE-6529: Summary: Use persistent sessionContext in AvaticaCommonsHttpClientImpl Key: CALCITE-6529 URL: https://issues.apache.org/jira/browse/CALCITE-6529 Project: Calcite Issue Type: Improvement Reporter: Istvan Toth We are re-creating and setting up a SessionContext object for each HTTP call. We could use a sessionContext method for the same purpose, or even better, just set those properties directly on the HttpClient object. Every member of HttpSession is duplicated in HttpClient, HttpSession is only used as an override. Since we have a 1:1 relationship between HttpClient and HttpSession objects, HttpSession is redundant. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-6364) HttpClient SPENGO support is deprecated
[ https://issues.apache.org/jira/browse/CALCITE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated CALCITE-6364: - Description: The Avatica Java client depends on Apache HttpClient's Kerberos/SPNEGO implementation. According to HTTPCLIENT-1625 that implementation is not secure, and is deprecated in newer versions. Unfortunately, HTTPCLIENT-1625 is very scant on details, and since the reason given for deprecation is the lack of time to fix it, it is likely not a trivial fix. Unfortunately, Avatica depends heavily on httpclient, and replacing it would it would be a big job. While Avatica in theory has a configurable Http Client implementation, the only non-httpclient implementation is more of a POC, and does not support ANY authentication methods. I can see these options: 1. Find an another http client library, and use it in Avatica 2. Copy the SPENGO auth code from httpclient, and fix it in Avatica 3. Fix the SPENGO auth code in httpclient. 4. Re-Implement SPENGO auth in Avatica (Hadoop does something like that, though I'm, not sure how good that is) was: The Avatica Java client depends on Apache HttpClient's Kerberos/SPNEGO implementation. According to HTTPCLIENT-1625 that implementation is not secure, and is deprecated in newer versions. Unfortunately, HTTPCLIENT-1625 is very scant on details, and since the reason given for deprecation is the lack of time to fix it, it is likely not a trivial fix. Unfortunately, Avatica depends heavily on httpclient, and replacing it would it would be a big job. While Avatica in theory has a configurable Http Client implementation, the only non-httpclient implementation is more of a POC, and does not support ANY authentication methods. I can see these options: 1. Find an another http client library, and use it in Avatica 2. Copy the SPENGO auth code from httpclient, and fix it in Avatica 3. Fix the SPENGO auth code in httpclient. > HttpClient SPENGO support is deprecated > --- > > Key: CALCITE-6364 > URL: https://issues.apache.org/jira/browse/CALCITE-6364 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Priority: Critical > > The Avatica Java client depends on Apache HttpClient's Kerberos/SPNEGO > implementation. > According to HTTPCLIENT-1625 that implementation is not secure, and is > deprecated in newer versions. > Unfortunately, HTTPCLIENT-1625 is very scant on details, and since the reason > given for deprecation is the lack of time to fix it, it is likely not a > trivial fix. > Unfortunately, Avatica depends heavily on httpclient, and replacing it would > it would be a big job. > While Avatica in theory has a configurable Http Client implementation, the > only non-httpclient implementation is more of a POC, and does not support ANY > authentication methods. > I can see these options: > 1. Find an another http client library, and use it in Avatica > 2. Copy the SPENGO auth code from httpclient, and fix it in Avatica > 3. Fix the SPENGO auth code in httpclient. > 4. Re-Implement SPENGO auth in Avatica (Hadoop does something like that, > though I'm, not sure how good that is) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6364) HttpClient SPENGO support is deprecated
[ https://issues.apache.org/jira/browse/CALCITE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838470#comment-17838470 ] Istvan Toth commented on CALCITE-6364: -- 5.3 even removes SPNEGO from DefaultAuthenticationStrategy, so that a custom AuthenticationStrategy is required to use it. > HttpClient SPENGO support is deprecated > --- > > Key: CALCITE-6364 > URL: https://issues.apache.org/jira/browse/CALCITE-6364 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Priority: Critical > > The Avatica Java client depends on Apache HttpClient's Kerberos/SPNEGO > implementation. > According to HTTPCLIENT-1625 that implementation is not secure, and is > deprecated in newer versions. > Unfortunately, HTTPCLIENT-1625 is very scant on details, and since the reason > given for deprecation is the lack of time to fix it, it is likely not a > trivial fix. > Unfortunately, Avatica depends heavily on httpclient, and replacing it would > it would be a big job. > While Avatica in theory has a configurable Http Client implementation, the > only non-httpclient implementation is more of a POC, and does not support ANY > authentication methods. > I can see these options: > 1. Find an another http client library, and use it in Avatica > 2. Copy the SPENGO auth code from httpclient, and fix it in Avatica > 3. Fix the SPENGO auth code in httpclient. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6364) HttpClient SPENGO support is deprecated
[ https://issues.apache.org/jira/browse/CALCITE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837098#comment-17837098 ] Istvan Toth commented on CALCITE-6364: -- Found this link in the HttpClient ticket > HttpClient SPENGO support is deprecated > --- > > Key: CALCITE-6364 > URL: https://issues.apache.org/jira/browse/CALCITE-6364 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Priority: Critical > > The Avatica Java client depends on Apache HttpClient's Kerberos/SPNEGO > implementation. > According to HTTPCLIENT-1625 that implementation is not secure, and is > deprecated in newer versions. > Unfortunately, HTTPCLIENT-1625 is very scant on details, and since the reason > given for deprecation is the lack of time to fix it, it is likely not a > trivial fix. > Unfortunately, Avatica depends heavily on httpclient, and replacing it would > it would be a big job. > While Avatica in theory has a configurable Http Client implementation, the > only non-httpclient implementation is more of a POC, and does not support ANY > authentication methods. > I can see these options: > 1. Find an another http client library, and use it in Avatica > 2. Copy the SPENGO auth code from httpclient, and fix it in Avatica > 3. Fix the SPENGO auth code in httpclient. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6364) HttpClient SPENGO support is deprecated
Istvan Toth created CALCITE-6364: Summary: HttpClient SPENGO support is deprecated Key: CALCITE-6364 URL: https://issues.apache.org/jira/browse/CALCITE-6364 Project: Calcite Issue Type: Bug Components: avatica Reporter: Istvan Toth The Avatica Java client depends on Apache HttpClient's Kerberos/SPNEGO implementation. According to HTTPCLIENT-1625 that implementation is not secure, and is deprecated in newer versions. Unfortunately, HTTPCLIENT-1625 is very scant on details, and since the reason given for deprecation is the lack of time to fix it, it is likely not a trivial fix. Unfortunately, Avatica depends heavily on httpclient, and replacing it would it would be a big job. While Avatica in theory has a configurable Http Client implementation, the only non-httpclient implementation is more of a POC, and does not support ANY authentication methods. I can see these options: 1. Find an another http client library, and use it in Avatica 2. Copy the SPENGO auth code from httpclient, and fix it in Avatica 3. Fix the SPENGO auth code in httpclient. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6209) Long queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825775#comment-17825775
]
Istvan Toth commented on CALCITE-6209:
--
Resolved in
https://github.com/apache/calcite-avatica/commit/4b9c8231b923fa7d7ea57131f0c5cb1dd2099613
.
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> ---
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Assignee: Istvan Toth
>Priority: Critical
> Labels: pull-request-available
> Fix For: avatica-1.25.0
>
> Attachments: screenshot-1.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> It's expected that they take more than 3 minutes and we need some way to
> increase timeout.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1611)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1586)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterable.iterator(MetaImpl.java:1581)
> at org.a
[jira] [Resolved] (CALCITE-6209) Long queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth resolved CALCITE-6209.
--
Fix Version/s: avatica-1.25.0
Resolution: Fixed
Committed to main.
Thanks for the review [~mbudiu].
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> ---
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Assignee: Istvan Toth
>Priority: Critical
> Labels: pull-request-available
> Fix For: avatica-1.25.0
>
> Attachments: screenshot-1.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> It's expected that they take more than 3 minutes and we need some way to
> increase timeout.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1611)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1586)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterable.iterator(MetaImpl.java:1581)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaIm
[jira] [Assigned] (CALCITE-6209) Long queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth reassigned CALCITE-6209:
Assignee: Istvan Toth
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> ---
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Assignee: Istvan Toth
>Priority: Critical
> Labels: pull-request-available
> Attachments: screenshot-1.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> It's expected that they take more than 3 minutes and we need some way to
> increase timeout.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1611)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1586)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterable.iterator(MetaImpl.java:1581)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:98)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:186)
> at
> or
[jira] [Resolved] (CALCITE-6280) Jetty version number leaked by Avatica http server
[
https://issues.apache.org/jira/browse/CALCITE-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth resolved CALCITE-6280.
--
Fix Version/s: 1.25.0
Resolution: Fixed
Committed to main.
Thank you, [~vjoshi].
> Jetty version number leaked by Avatica http server
> --
>
> Key: CALCITE-6280
> URL: https://issues.apache.org/jira/browse/CALCITE-6280
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: 1.24.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.25.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Unauthorised access to HTTP server using curl returns the Jerry server
> version. See sample response below
> {code:java}
>
>
>
> Error 401 Unauthorized
>
> HTTP ERROR 401 Unauthorized
>
> URI:/
> STATUS:401
> MESSAGE:Unauthorized
> SERVLET:-
>
> https://eclipse.org/jetty";>Powered by Jetty://
> 9.4.44.v20210927
>
> {code}
>
> For security reason, it's not advisable to return server version in the
> response.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-6280) Jetty version number leaked by Avatica http server
[
https://issues.apache.org/jira/browse/CALCITE-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated CALCITE-6280:
-
Summary: Jetty version number leaked by Avatica http server (was: The
Jetty's version number leak occurred while using the avatica http server)
> Jetty version number leaked by Avatica http server
> --
>
> Key: CALCITE-6280
> URL: https://issues.apache.org/jira/browse/CALCITE-6280
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: 1.24.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Unauthorised access to HTTP server using curl returns the Jerry server
> version. See sample response below
> {code:java}
>
>
>
> Error 401 Unauthorized
>
> HTTP ERROR 401 Unauthorized
>
> URI:/
> STATUS:401
> MESSAGE:Unauthorized
> SERVLET:-
>
> https://eclipse.org/jetty";>Powered by Jetty://
> 9.4.44.v20210927
>
> {code}
>
> For security reason, it's not advisable to return server version in the
> response.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-6280) The Jetty's version number leak occurred while using the avatica http server
[
https://issues.apache.org/jira/browse/CALCITE-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated CALCITE-6280:
-
Affects Version/s: 1.24.0
> The Jetty's version number leak occurred while using the avatica http server
>
>
> Key: CALCITE-6280
> URL: https://issues.apache.org/jira/browse/CALCITE-6280
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: 1.24.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Unauthorised access to HTTP server using curl returns the Jerry server
> version. See sample response below
> {code:java}
>
>
>
> Error 401 Unauthorized
>
> HTTP ERROR 401 Unauthorized
>
> URI:/
> STATUS:401
> MESSAGE:Unauthorized
> SERVLET:-
>
> https://eclipse.org/jetty";>Powered by Jetty://
> 9.4.44.v20210927
>
> {code}
>
> For security reason, it's not advisable to return server version in the
> response.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-6209) Long queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17821112#comment-17821112
]
Istvan Toth commented on CALCITE-6209:
--
I think that this is a very likely scenario, when using Avatica for data
analysis, and the easiest fix is making setResponseTimeout configurable.
I have looked a bit into this, and the only alternative I could find is adding
a keep-alive to the protocol, and sending them periodically.
Implementing that is a big change, and probably would not even work without
having support on both the client and server side.
While I agree that we should limit the number of exposed parameters, I cannot
see a viable alternative here.
Do you have some alternative suggestions, [~julianhyde] ?
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> ---
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Assignee: Istvan Toth
>Priority: Critical
> Labels: pull-request-available
> Attachments: screenshot-1.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> It's expected that they take more than 3 minutes and we need some way to
> increase timeout.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.
[jira] [Commented] (CALCITE-6209) Long queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17809469#comment-17809469
]
Istvan Toth commented on CALCITE-6209:
--
Also confirming that the problem is not related to CALCITE-5581, I can repro
the issue on 1.23.
(This is in line with [~konstantinx64]'s anaysis.)
httpclient has the following timeout parameters:
[setConnectTimeout|https://hc.apache.org/httpcomponents-client-5.2.x/current/httpclient5/apidocs/org/apache/hc/client5/http/config/RequestConfig.Builder.html#setConnectTimeout-long-java.util.concurrent.TimeUnit-]
This is connection timeout, as discussed above, and is not relevant here.
[setResponseTimeout|https://hc.apache.org/httpcomponents-client-5.2.x/current/httpclient5/apidocs/org/apache/hc/client5/http/config/RequestConfig.Builder.html#setResponseTimeout-long-java.util.concurrent.TimeUnit-]
This is at play here.
The java.net.SocketTimeoutException is thrown after this time has elapsed.
[setConnectionRequestTimeout|https://hc.apache.org/httpcomponents-client-5.2.x/current/httpclient5/apidocs/org/apache/hc/client5/http/config/RequestConfig.Builder.html#setConnectionRequestTimeout-long-java.util.concurrent.TimeUnit-]
I'm not sure what it does, it possibly comes into play when the httpclient
connection pool is full.
This does not have an effect on the java.net.SocketTimeoutException.
The simplest way to fix this is to expose the responseTimeout prameter in the
JDBC URL.
We could generate periodic null reponse segments on the server, or send kind of
ping packets from the client, but
that would be a much bigger task (and I am not familar enough with the HTTP
internals to tell if that would even generate traffic on the same TCP socket )
The JDBC Statement.setQueryTimeout() is somewhat related, but it's unlimited by
default, and I wouldn't tie that to the tcp timeout.
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> ---
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Assignee: Istvan Toth
>Priority: Critical
> Attachments: screenshot-1.png
>
>
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> It's expected that they take more than 3 minutes and we need some way to
> increase timeout.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> a
[jira] [Commented] (CALCITE-6209) Long queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17809423#comment-17809423
]
Istvan Toth commented on CALCITE-6209:
--
I am able to repro it now.
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> ---
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Assignee: Istvan Toth
>Priority: Critical
> Attachments: screenshot-1.png
>
>
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> It's expected that they take more than 3 minutes and we need some way to
> increase timeout.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1611)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1586)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterable.iterator(MetaImpl.java:1581)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:98)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:186)
> at
> org.apache.calcite.avatica.AvaticaConnection$1.ex
[jira] [Updated] (CALCITE-6209) Long queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated CALCITE-6209:
-
Attachment: (was: CALCITE-6209-test.patch)
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> ---
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Assignee: Istvan Toth
>Priority: Critical
> Attachments: screenshot-1.png
>
>
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> It's expected that they take more than 3 minutes and we need some way to
> increase timeout.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1611)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1586)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterable.iterator(MetaImpl.java:1581)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:98)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:186)
> at
> org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:669)
[jira] [Comment Edited] (CALCITE-6209) Long queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17809336#comment-17809336
]
Istvan Toth edited comment on CALCITE-6209 at 1/22/24 10:23 AM:
-I was not able to repro this.-
-I have created a tests which adds a 200 second delay before responding, and it
completes fine.-
-(The patch is attached to this ticket.)-
That mocked test doesn't actually go through the network.
was (Author: stoty):
I was not able to repro this.
I have created a tests which adds a 200 second delay before responding, and it
completes fine.
(The patch is attached to this ticket.)
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> ---
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Assignee: Istvan Toth
>Priority: Critical
> Attachments: CALCITE-6209-test.patch, screenshot-1.png
>
>
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> It's expected that they take more than 3 minutes and we need some way to
> increase timeout.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
> at
> org.apache.
[jira] [Commented] (CALCITE-6209) Long queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17809336#comment-17809336
]
Istvan Toth commented on CALCITE-6209:
--
I was not able to repro this.
I have created a tests which adds a 200 second delay before responding, and it
completes fine.
(The patch is attached to this ticket.)
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> ---
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Assignee: Istvan Toth
>Priority: Critical
> Attachments: CALCITE-6209-test.patch, screenshot-1.png
>
>
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> It's expected that they take more than 3 minutes and we need some way to
> increase timeout.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1611)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1586)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterable.iterator(MetaImpl.java:1581)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl
[jira] [Updated] (CALCITE-6209) Long queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated CALCITE-6209:
-
Attachment: CALCITE-6209-test.patch
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> ---
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Assignee: Istvan Toth
>Priority: Critical
> Attachments: CALCITE-6209-test.patch, screenshot-1.png
>
>
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> It's expected that they take more than 3 minutes and we need some way to
> increase timeout.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1611)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1586)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterable.iterator(MetaImpl.java:1581)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:98)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:186)
> at
> org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnect
[jira] [Assigned] (CALCITE-6209) Long queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth reassigned CALCITE-6209:
Assignee: Istvan Toth
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> ---
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Assignee: Istvan Toth
>Priority: Critical
> Attachments: screenshot-1.png
>
>
> Long queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> It's expected that they take more than 3 minutes and we need some way to
> increase timeout.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1611)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1586)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterable.iterator(MetaImpl.java:1581)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:98)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:186)
> at
> org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:669)
> at
> org.
[jira] [Commented] (CALCITE-6209) All queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17808556#comment-17808556
]
Istvan Toth commented on CALCITE-6209:
--
I find it strange that this hasn't come up yet, but it's feasible that we
should expose more timeout parameters (or perhaps enable http keepalive).
> All queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> --
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Priority: Critical
> Attachments: screenshot-1.png
>
>
> All queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1611)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1586)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterable.iterator(MetaImpl.java:1581)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:98)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:186)
> at
> org.apache.calcite.avatica.AvaticaConnection$1.execute(Avati
[jira] [Commented] (CALCITE-6209) All queries are failing with "java.net.SocketTimeoutException: Read timed out" after 3 minutes
[
https://issues.apache.org/jira/browse/CALCITE-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17808555#comment-17808555
]
Istvan Toth commented on CALCITE-6209:
--
Based on the above, please change the ticket description.
Something like "Long running queries..." would work.
"All queries are failing" implies that even fast queries are running for three
minutes the fail, which would definitely imply an user error.
> All queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes
> --
>
> Key: CALCITE-6209
> URL: https://issues.apache.org/jira/browse/CALCITE-6209
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.24.0
>Reporter: Konstantin
>Priority: Critical
> Attachments: screenshot-1.png
>
>
> All queries are failing with "java.net.SocketTimeoutException: Read timed
> out" after 3 minutes.
> I tried to change it via +http_connection_timeout+ property but it didn't
> help.
> Looks like that property is related to connect timeout only, but not to
> request timeout.
> I checked source code but didn't found any ways to change it there.
> Looks like it's always using default timeout=3minutes.
> !screenshot-1.png!
> Stack-trace:
> {code:java}
> org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error: Error while executing
> SQL "SELECT count(1) FROM trace": java.net.SocketTimeoutException: Read timed
> out
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:600)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:503)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:510)
> at
> org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:962)
> at
> org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4130)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
> at
> org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
> at
> org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:5148)
> at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
> at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
> Caused by: java.sql.SQLException: Error while executing SQL "SELECT count(1)
> FROM trace": java.net.SocketTimeoutException: Read timed out
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
> at
> org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:218)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330)
> at
> org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
> ... 12 more
> Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: Read
> timed out
> at
> org.apache.calcite.avatica.remote.AvaticaCommonsHttpClientImpl.send(AvaticaCommonsHttpClientImpl.java:155)
> at
> org.apache.calcite.avatica.remote.RemoteService.apply(RemoteService.java:36)
> at
> org.apache.calcite.avatica.remote.JsonService.apply(JsonService.java:140)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:310)
> at
> org.apache.calcite.avatica.remote.RemoteMeta$14.call(RemoteMeta.java:307)
> at
> org.apache.calcite.avatica.AvaticaConnection.invokeWithRetries(AvaticaConnection.java:795)
> at
> org.apache.calcite.avatica.remote.RemoteMeta.fetch(RemoteMeta.java:306)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.moveNext(MetaImpl.java:1643)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1611)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterator.(MetaImpl.java:1586)
> at
> org.apache.calcite.avatica.MetaImpl$FetchIterable.iterator(MetaImpl.java:1581)
> at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:98)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execut
[jira] [Commented] (CALCITE-6135) BEARER authentication support
[ https://issues.apache.org/jira/browse/CALCITE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17790140#comment-17790140 ] Istvan Toth commented on CALCITE-6135: -- IMO Avatica should have an option to specify the token directly in the connection URL as a string. This wouldn't work with short lifetime tokens, but would be ok for long lived ones. It may also be worth adding an option to specify the class name of a token provide implmenetation, and perhaps parameters for it, so that Avatica could be used to connect to OAUTH or similar. (when the additional classes are provided on the classpath) > BEARER authentication support > - > > Key: CALCITE-6135 > URL: https://issues.apache.org/jira/browse/CALCITE-6135 > Project: Calcite > Issue Type: New Feature > Components: avatica >Reporter: Aron Attila Meszaros >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Add support for Bearer authentication scheme (RFC 6750). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6125) Automate generation of contributor names in release notes
[
https://issues.apache.org/jira/browse/CALCITE-6125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17787845#comment-17787845
]
Istvan Toth commented on CALCITE-6125:
--
How is the RM supposed to figure out the real name ?
Not every contributor name may be available, and tasking the RM with
investigating this seems to be too much.
Is this a requirement by ASF policies ?
> Automate generation of contributor names in release notes
> -
>
> Key: CALCITE-6125
> URL: https://issues.apache.org/jira/browse/CALCITE-6125
> Project: Calcite
> Issue Type: Improvement
>Reporter: Julian Hyde
>Priority: Major
> Labels: pull-request-available
>
> Automate generation of contributor names in release notes. Currently the
> process requires manual effort because people do not use their real name in
> the author field of their git commit.
> For example, consider generating the list for release 1.32.0. Here is a rough
> process:
> {noformat}
> $ git log calcite-1.31.0..calcite-1.32.0 | egrep 'Author:|Co-authored-by:' |
> sort -u
> Author: Alessandro Solimando
> Author: almansour
> Author: Andrei Sereda <25229979+asereda...@users.noreply.github.com>
> Author: Benchao Li
> Author: Bertil Chapuis
> Author: chunwei.lcw
> Author: dssysolyatin
> Author: Julian Hyde
> Author: lincoln.lil
> Author: rubenada
> Author: strongduanmu
> Author: TJ Banghart
> Author: wumou.wm
> Author: xiejiajun
> Co-authored-by: David Handermann
> {noformat}
> If we add a [git mailmap file|https://git-scm.com/docs/gitmailmap],
> {{.mailmap}}, we can translate some of the names 'lincolon.lil' to 'Lincoln
> Lee' and 'wumou.wm' to 'Mou Wu':
> {noformat}
> $ git log calcite-1.31.0..calcite-1.32.0 |egrep 'Author:|Co-authored-by:'
> |sort -u
> Author: Alessandro Solimando
> Author: Ali Mansour
> Author: Andrei Sereda <25229979+asereda...@users.noreply.github.com>
> Author: Benchao Li
> Author: Bertil Chapuis
> Author: chunwei.lcw
> Author: dssysolyatin
> Author: Jiajun Xie
> Author: Julian Hyde
> Author: Lincoln Lee
> Author: Mou Wu
> Author: Ruben Quesada Lopez
> Author: TJ Banghart
> Author: Zhengqiang Duan
> Co-authored-by: David Handermann
> {noformat}
> I propose to add a mailmap file, and change the release process so that the
> release manager must modify the mailmap to achieve the right names in the
> release notes (not edit the names manually).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-6113) Update HttpComponents Core to 5.2.3 and HttpComponents Client to 5.2.1 in Avatica
[ https://issues.apache.org/jira/browse/CALCITE-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved CALCITE-6113. -- Fix Version/s: avatica-1.24.0 Resolution: Fixed Committed to main. Thanks for the review [~francischuang] . > Update HttpComponents Core to 5.2.3 and HttpComponents Client to 5.2.1 in > Avatica > -- > > Key: CALCITE-6113 > URL: https://issues.apache.org/jira/browse/CALCITE-6113 > Project: Calcite > Issue Type: Task > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: avatica-1.24.0 > > Time Spent: 20m > Remaining Estimate: 0h > > We are currently using 5.1.x for both, which is quite old. > Upgrade to the latest GA versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-6113) Update HttpComponents Core to 5.2.3 and HttpComponents Client to 5.2.1 in Avatica
[ https://issues.apache.org/jira/browse/CALCITE-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated CALCITE-6113: - Issue Type: Task (was: Bug) > Update HttpComponents Core to 5.2.3 and HttpComponents Client to 5.2.1 in > Avatica > -- > > Key: CALCITE-6113 > URL: https://issues.apache.org/jira/browse/CALCITE-6113 > Project: Calcite > Issue Type: Task > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > We are currently using 5.1.x for both, which is quite old. > Upgrade to the latest GA versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-6113) Update HttpComponents Core to 5.2.3 and HttpComponents Client to 5.2.1 in Avatica
[ https://issues.apache.org/jira/browse/CALCITE-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated CALCITE-6113: - Description: We are currently using 5.1.x for both, which is quite old. Upgrade to the latest GA versions. was:We are currently using 5.1.x for both, which is quite old. > Update HttpComponents Core to 5.2.3 and HttpComponents Client to 5.2.1 in > Avatica > -- > > Key: CALCITE-6113 > URL: https://issues.apache.org/jira/browse/CALCITE-6113 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > We are currently using 5.1.x for both, which is quite old. > Upgrade to the latest GA versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-6113) Update HttpComponents Core to 5.2.3 and HttpComponents Client to 5.2.1 in Avatica
[ https://issues.apache.org/jira/browse/CALCITE-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated CALCITE-6113: - Description: We are currently using 5.1.x for both, which is quite old. > Update HttpComponents Core to 5.2.3 and HttpComponents Client to 5.2.1 in > Avatica > -- > > Key: CALCITE-6113 > URL: https://issues.apache.org/jira/browse/CALCITE-6113 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > We are currently using 5.1.x for both, which is quite old. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6113) Update HttpComponents Core to 5.2.3 and HttpComponents Client to 5.2.1 in Avatica
Istvan Toth created CALCITE-6113: Summary: Update HttpComponents Core to 5.2.3 and HttpComponents Client to 5.2.1 in Avatica Key: CALCITE-6113 URL: https://issues.apache.org/jira/browse/CALCITE-6113 Project: Calcite Issue Type: Bug Components: avatica Reporter: Istvan Toth Assignee: Istvan Toth -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (CALCITE-5536) Clean up some of the magic numbers in AvaticaResultSetConversionsTest
[ https://issues.apache.org/jira/browse/CALCITE-5536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved CALCITE-5536. -- Fix Version/s: avatica-1.24.0 Resolution: Fixed Committed. Thank you [~wnoble] for the patch. > Clean up some of the magic numbers in AvaticaResultSetConversionsTest > - > > Key: CALCITE-5536 > URL: https://issues.apache.org/jira/browse/CALCITE-5536 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Will Noble >Assignee: Will Noble >Priority: Trivial > Labels: pull-request-available > Fix For: avatica-1.24.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Retroactively filing this bug for some [cleanup I > did|https://github.com/apache/calcite-avatica/pull/208] in preparation for > the fix to CALCITE-5446. This cleanup makes the fix and accompanied test > adjustments easier to review. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (CALCITE-5536) Clean up some of the magic numbers in AvaticaResultSetConversionsTest
[ https://issues.apache.org/jira/browse/CALCITE-5536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth reassigned CALCITE-5536: Assignee: Will Noble > Clean up some of the magic numbers in AvaticaResultSetConversionsTest > - > > Key: CALCITE-5536 > URL: https://issues.apache.org/jira/browse/CALCITE-5536 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Will Noble >Assignee: Will Noble >Priority: Trivial > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Retroactively filing this bug for some [cleanup I > did|https://github.com/apache/calcite-avatica/pull/208] in preparation for > the fix to CALCITE-5446. This cleanup makes the fix and accompanied test > adjustments easier to review. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (CALCITE-6072) OpenTelemetry Trace Context Propagation for Avatica
[ https://issues.apache.org/jira/browse/CALCITE-6072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth reassigned CALCITE-6072: Assignee: Istvan Toth > OpenTelemetry Trace Context Propagation for Avatica > --- > > Key: CALCITE-6072 > URL: https://issues.apache.org/jira/browse/CALCITE-6072 > Project: Calcite > Issue Type: New Feature > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > It would be super useful to be able to propagate the trace context via > Avatica. > Investigate whether the existing Opentelemetry agent can provide this > functionality transparently, and if not, what do we need to add to Avatica to > enable this. > Can/does opentelemetry-agent add instrumentation to add the context as HTTP > headers in httpclient ? > Can/does opentelemetry-agent add instrumentation to retrieve the context in > Jetty ? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-6072) OpenTelemetry Trace Context Propagation for Avatica
[ https://issues.apache.org/jira/browse/CALCITE-6072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17779759#comment-17779759 ] Istvan Toth commented on CALCITE-6072: -- Opentelemetry seems to have agent-based instrumentation for HttpClient 5: [https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/apache-httpclient/apache-httpclient-5.0/javaagent/src/main/java/io/opentelemetry/javaagent/instrumentation/apachehttpclient/v5_0] It doesn't seem to have instrumentation for Jetty server, but context propagation could possibly be added to Jetty when setting up the Avatica server object. Keeping this open until I have a working POC at least. > OpenTelemetry Trace Context Propagation for Avatica > --- > > Key: CALCITE-6072 > URL: https://issues.apache.org/jira/browse/CALCITE-6072 > Project: Calcite > Issue Type: New Feature > Components: avatica >Reporter: Istvan Toth >Priority: Major > > It would be super useful to be able to propagate the trace context via > Avatica. > Investigate whether the existing Opentelemetry agent can provide this > functionality transparently, and if not, what do we need to add to Avatica to > enable this. > Can/does opentelemetry-agent add instrumentation to add the context as HTTP > headers in httpclient ? > Can/does opentelemetry-agent add instrumentation to retrieve the context in > Jetty ? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-6072) OpenTelemetry Trace Context Propagation for Avatica
[ https://issues.apache.org/jira/browse/CALCITE-6072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated CALCITE-6072: - Summary: OpenTelemetry Trace Context Propagation for Avatica (was: OpenTelemetry trace context propagation for Avatica) > OpenTelemetry Trace Context Propagation for Avatica > --- > > Key: CALCITE-6072 > URL: https://issues.apache.org/jira/browse/CALCITE-6072 > Project: Calcite > Issue Type: New Feature > Components: avatica >Reporter: Istvan Toth >Priority: Major > > It would be super useful to be able to delegate the trace context via Avatica. > Investigate whether the existing Opentelemetry agent can provide this > functionality transparently, and if not, what do we need to add to Avatica to > enable this. > Can/does opentelemtry-agent add instrumentation to add the context as HTTP > headers in httpclient ? > Can/does opentelemetry-agent add instrumentation to retrieve the context in > Jetty ? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-6072) OpenTelemetry Trace Context Propagation for Avatica
[ https://issues.apache.org/jira/browse/CALCITE-6072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated CALCITE-6072: - Description: It would be super useful to be able to propagate the trace context via Avatica. Investigate whether the existing Opentelemetry agent can provide this functionality transparently, and if not, what do we need to add to Avatica to enable this. Can/does opentelemetry-agent add instrumentation to add the context as HTTP headers in httpclient ? Can/does opentelemetry-agent add instrumentation to retrieve the context in Jetty ? was: It would be super useful to be able to delegate the trace context via Avatica. Investigate whether the existing Opentelemetry agent can provide this functionality transparently, and if not, what do we need to add to Avatica to enable this. Can/does opentelemtry-agent add instrumentation to add the context as HTTP headers in httpclient ? Can/does opentelemetry-agent add instrumentation to retrieve the context in Jetty ? > OpenTelemetry Trace Context Propagation for Avatica > --- > > Key: CALCITE-6072 > URL: https://issues.apache.org/jira/browse/CALCITE-6072 > Project: Calcite > Issue Type: New Feature > Components: avatica >Reporter: Istvan Toth >Priority: Major > > It would be super useful to be able to propagate the trace context via > Avatica. > Investigate whether the existing Opentelemetry agent can provide this > functionality transparently, and if not, what do we need to add to Avatica to > enable this. > Can/does opentelemetry-agent add instrumentation to add the context as HTTP > headers in httpclient ? > Can/does opentelemetry-agent add instrumentation to retrieve the context in > Jetty ? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6072) OpenTelemetry trace context propagation for Avatica
Istvan Toth created CALCITE-6072: Summary: OpenTelemetry trace context propagation for Avatica Key: CALCITE-6072 URL: https://issues.apache.org/jira/browse/CALCITE-6072 Project: Calcite Issue Type: New Feature Components: avatica Reporter: Istvan Toth It would be super useful to be able to delegate the trace context via Avatica. Investigate whether the existing Opentelemetry agent can provide this functionality transparently, and if not, what do we need to add to Avatica to enable this. Can/does opentelemtry-agent add instrumentation to add the context as HTTP headers in httpclient ? Can/does opentelemetry-agent add instrumentation to retrieve the context in Jetty ? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5947) Use existing ticket cache for kerberos login
Istvan Toth created CALCITE-5947: Summary: Use existing ticket cache for kerberos login Key: CALCITE-5947 URL: https://issues.apache.org/jira/browse/CALCITE-5947 Project: Calcite Issue Type: New Feature Components: avatica Affects Versions: avatica-1.23.0 Reporter: Istvan Toth Avatica currently requires that a kerberos principal and keytab is supplied in the JDBC URL for for connecting to a kerberized PQS. This is often sub-optimal solution. It would be much more user-friendly, if Avatica could simply use an existing ticket from the ticket cache. The algorithm could be something like this: * if principal and keytab is supplied in the URL: current behaviour * If only principal is supplied: try to load the ticket for the prinicpal from cache * If neither is supplied: Use first/default principal in the cache Most of the logic could be directly lifted from Phoenix Query Server: https://github.com/apache/phoenix-queryserver/blob/master/phoenix-queryserver-client/src/main/java/org/apache/phoenix/queryserver/client/KerberosLoginFromTicketCache.java -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-5896) Document TLS client certificate JDBC URL options
[
https://issues.apache.org/jira/browse/CALCITE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated CALCITE-5896:
-
Summary: Document TLS client certificate JDBC URL options (was: Document
client certificate JDBC URL options)
> Document TLS client certificate JDBC URL options
>
>
> Key: CALCITE-5896
> URL: https://issues.apache.org/jira/browse/CALCITE-5896
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Reporter: Istvan Toth
>Priority: Minor
> Labels: doc
>
> The client reference does not include the {_}keystore{_}, _keystore_password_
> and _key_password_ options.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-5890) Handle non-JKS truststores in Avatica client
[ https://issues.apache.org/jira/browse/CALCITE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved CALCITE-5890. -- Fix Version/s: avatica-1.24.0 Resolution: Fixed Pushed to main. Thanks for the review [~snuyanzin] . > Handle non-JKS truststores in Avatica client > > > Key: CALCITE-5890 > URL: https://issues.apache.org/jira/browse/CALCITE-5890 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: avatica-1.24.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Avatica can handle non-JKS truststores on the server side. > However, the client fails if we try to use a non-JKS keystore. > Either add a connection property to specify the keystore format, or enable > autodetection (if it is possible) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-5896) Document client certificate JDBC URL options
[
https://issues.apache.org/jira/browse/CALCITE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated CALCITE-5896:
-
Labels: doc (was: )
> Document client certificate JDBC URL options
>
>
> Key: CALCITE-5896
> URL: https://issues.apache.org/jira/browse/CALCITE-5896
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Reporter: Istvan Toth
>Priority: Minor
> Labels: doc
>
> The client reference does not include the {_}keystore{_}, _keystore_password_
> and _key_password_ options.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (CALCITE-5896) Document client certificate JDBC URL options
Istvan Toth created CALCITE-5896:
Summary: Document client certificate JDBC URL options
Key: CALCITE-5896
URL: https://issues.apache.org/jira/browse/CALCITE-5896
Project: Calcite
Issue Type: Bug
Components: avatica
Reporter: Istvan Toth
The client reference does not include the {_}keystore{_}, _keystore_password_
and _key_password_ options.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5890) Handle non-JKS truststores in Avatica client
[ https://issues.apache.org/jira/browse/CALCITE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated CALCITE-5890: - Description: Avatica can handle non-JKS truststores on the server side. However, the client fails if we try to use a non-JKS keystore. Either add a connection property to specify the keystore format, or enable autodetection (if it is possible) was: Avatica can handle non-JKS truststores on the server side. However, the client fails if we try to use non-JKS keystore. Either add a connection property to specify the keystore format, or enable autodetection (if it is possible) > Handle non-JKS truststores in Avatica client > > > Key: CALCITE-5890 > URL: https://issues.apache.org/jira/browse/CALCITE-5890 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Avatica can handle non-JKS truststores on the server side. > However, the client fails if we try to use a non-JKS keystore. > Either add a connection property to specify the keystore format, or enable > autodetection (if it is possible) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5890) Handle non-JKS truststores in Avatica client
[
https://issues.apache.org/jira/browse/CALCITE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750785#comment-17750785
]
Istvan Toth edited comment on CALCITE-5890 at 8/3/23 4:03 PM:
--
Testing on a system with a JVM configured for FIPS with bouncycastle:
Trying to load a bcfks truststore without the patch:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK
[main] ERROR org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
HTTPS registry configuration failed
java.lang.RuntimeException: java.io.IOException: Invalid keystore format
{noformat}
Trying to load the same with the patch and the new property:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK;keystore_type=bcfks
[main] INFO org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
Trustore loaded from:
/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks{noformat}
The trustsore file is bcfks, despite the file name extension.
was (Author: stoty):
Testing on a system with a JVM configure for FIPS with bouncycastle:
Trying to load a bcfks truststore without the patch:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK
[main] ERROR org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
HTTPS registry configuration failed
java.lang.RuntimeException: java.io.IOException: Invalid keystore format
{noformat}
Trying to load the same with the patch and the new property:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK;keystore_type=bcfks
[main] INFO org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
Trustore loaded from:
/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks{noformat}
The trustsore file is bcfks, despite the file name extension.
> Handle non-JKS truststores in Avatica client
>
>
> Key: CALCITE-5890
> URL: https://issues.apache.org/jira/browse/CALCITE-5890
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Avatica can handle non-JKS truststores on the server side.
> However, the client fails if we try to use non-JKS keystore.
> Either add a connection property to specify the keystore format, or enable
> autodetection (if it is possible)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5890) Handle non-JKS truststores in Avatica client
[
https://issues.apache.org/jira/browse/CALCITE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750785#comment-17750785
]
Istvan Toth edited comment on CALCITE-5890 at 8/3/23 4:03 PM:
--
Testing on a system with a JVM configure for FIPS with bouncycastle:
Trying to load a bcfks truststore without the patch:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK
[main] ERROR org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
HTTPS registry configuration failed
java.lang.RuntimeException: java.io.IOException: Invalid keystore format
{noformat}
Trying to load the same with the patch and the new property:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK;keystore_type=bcfks
[main] INFO org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
Trustore loaded from:
/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks{noformat}
The trustsore file is bcfks, despite the file name extension.
was (Author: stoty):
Testing ona system with a JVM configure for FIPS with bouncycastle:
Trying to load a bcfks truststore without the patch:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK
[main] ERROR org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
HTTPS registry configuration failed
java.lang.RuntimeException: java.io.IOException: Invalid keystore format
{noformat}
Trying to load the same with the patch and the new property:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK;keystore_type=bcfks
[main] INFO org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
Trustore loaded from:
/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks{noformat}
The trustsore file is bcfks, despite the file name extension.
> Handle non-JKS truststores in Avatica client
>
>
> Key: CALCITE-5890
> URL: https://issues.apache.org/jira/browse/CALCITE-5890
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Avatica can handle non-JKS truststores on the server side.
> However, the client fails if we try to use non-JKS keystore.
> Either add a connection property to specify the keystore format, or enable
> autodetection (if it is possible)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5890) Handle non-JKS truststores in Avatica client
[
https://issues.apache.org/jira/browse/CALCITE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750785#comment-17750785
]
Istvan Toth edited comment on CALCITE-5890 at 8/3/23 3:02 PM:
--
Testing ona system with a JVM configure for FIPS with bouncycastle:
Trying to load a bcfks truststore without the patch:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK
[main] ERROR org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
HTTPS registry configuration failed
java.lang.RuntimeException: java.io.IOException: Invalid keystore format
{noformat}
Trying to load the same with the patch and the new property:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK;keystore_type=bcfks
[main] INFO org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
Trustore loaded from:
/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks{noformat}
The trustsore file is bcfks, despite the file name extension.
was (Author: stoty):
Testing ona system with a JVM configure for FIPS with bouncycastle:
Trying to load a bcfks truststore without the patch:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK
[main] ERROR org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
HTTPS registry configuration failed
java.lang.RuntimeException: java.io.IOException: Invalid keystore format
{noformat}
Trying to load the same with the patch and the new property:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK;keystore_type=bcfks
[main] INFO org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
Trustore loaded from:
/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks{noformat}
> Handle non-JKS truststores in Avatica client
>
>
> Key: CALCITE-5890
> URL: https://issues.apache.org/jira/browse/CALCITE-5890
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Avatica can handle non-JKS truststores on the server side.
> However, the client fails if we try to use non-JKS keystore.
> Either add a connection property to specify the keystore format, or enable
> autodetection (if it is possible)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5890) Handle non-JKS truststores in Avatica client
[
https://issues.apache.org/jira/browse/CALCITE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750785#comment-17750785
]
Istvan Toth commented on CALCITE-5890:
--
Testing ona system with a JVM configure for FIPS with bouncycastle:
Trying to load a bcfks truststore without the patch:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK
[main] ERROR org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
HTTPS registry configuration failed
java.lang.RuntimeException: java.io.IOException: Invalid keystore format
{noformat}
Trying to load the same with the patch and the new property:
{noformat}
Connecting to
jdbc:phoenix:thin:url=https://quasar-uablrr-1.vpc.cloudera.com:8765;serialization=PROTOBUF;authentication=SPNEGO;truststore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks;truststore_password=86cjugByTIj4IUGj4CD9SWwWeYXVwnx9PMNWVwyGECK;keystore_type=bcfks
[main] INFO org.apache.calcite.avatica.remote.CommonsHttpClientPoolCache -
Trustore loaded from:
/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks{noformat}
> Handle non-JKS truststores in Avatica client
>
>
> Key: CALCITE-5890
> URL: https://issues.apache.org/jira/browse/CALCITE-5890
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Avatica can handle non-JKS truststores on the server side.
> However, the client fails if we try to use non-JKS keystore.
> Either add a connection property to specify the keystore format, or enable
> autodetection (if it is possible)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5890) Handle non-JKS truststores in Avatica client
[ https://issues.apache.org/jira/browse/CALCITE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750762#comment-17750762 ] Istvan Toth commented on CALCITE-5890: -- The API doesn't seem to support autodetection of keystore types. We could probably do some kind of enumeration based on the installed providers, and try each one in turn, but that sounds slow and hacky. On the other hand, adding a new JDBC URL option to set the keystore type seems straightforward. > Handle non-JKS truststores in Avatica client > > > Key: CALCITE-5890 > URL: https://issues.apache.org/jira/browse/CALCITE-5890 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > Avatica can handle non-JKS truststores on the server side. > However, the client fails if we try to use non-JKS keystore. > Either add a connection property to specify the keystore format, or enable > autodetection (if it is possible) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-5890) Handle non-JKS truststores in Avatica client
[ https://issues.apache.org/jira/browse/CALCITE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated CALCITE-5890: - Summary: Handle non-JKS truststores in Avatica client (was: Handle non-JKS keystores in client) > Handle non-JKS truststores in Avatica client > > > Key: CALCITE-5890 > URL: https://issues.apache.org/jira/browse/CALCITE-5890 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > Avatica can handle non-JKS truststores on the server side. > However, the client fails if we try to use non-JKS keystore. > Either add a connection property to specify the keystore format, or enable > autodetection (if it is possible) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-5890) Handle non-JKS keystores in client
[ https://issues.apache.org/jira/browse/CALCITE-5890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17750293#comment-17750293 ] Istvan Toth commented on CALCITE-5890: -- FYI [~richardantal] > Handle non-JKS keystores in client > -- > > Key: CALCITE-5890 > URL: https://issues.apache.org/jira/browse/CALCITE-5890 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > Avatica can handle non-JKS truststores on the server side. > However, the client fails if we try to use non-JKS keystore. > Either add a connection property to specify the keystore format, or enable > autodetection (if it is possible) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5890) Handle non-JKS keystores in client
Istvan Toth created CALCITE-5890: Summary: Handle non-JKS keystores in client Key: CALCITE-5890 URL: https://issues.apache.org/jira/browse/CALCITE-5890 Project: Calcite Issue Type: Bug Components: avatica Reporter: Istvan Toth Assignee: Istvan Toth Avatica can handle non-JKS truststores on the server side. However, the client fails if we try to use non-JKS keystore. Either add a connection property to specify the keystore format, or enable autodetection (if it is possible) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CALCITE-5581) Implement Basic client side load balancing in Avatica Driver
[
https://issues.apache.org/jira/browse/CALCITE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated CALCITE-5581:
-
Affects Version/s: avatica-1.23.0
(was: 1.33.0)
> Implement Basic client side load balancing in Avatica Driver
>
>
> Key: CALCITE-5581
> URL: https://issues.apache.org/jira/browse/CALCITE-5581
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
>Affects Versions: avatica-1.23.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.24.0
>
> Attachments: Avatica Basic client side load balancing.docx
>
> Time Spent: 12h 20m
> Remaining Estimate: 0h
>
> Implement mechanism to support Client side load balancing by configuring
> connection properties for load balancing
> {code:java}
> jdbc:phoenix:thin:
> url=https://:;
> serialization=PROTOBUF;
> authentication=BASIC;
> avatica_user=;
> avatica_password=
> useClientSideLB: [true/false]
> lbStrategy: [RandomDistribution/Sequential/RoundRobin]
> lbURLs: {code}
> Client should connect to any of the URLs specified in lbURLs ( depending on
> LB strategy selected). Please refer to [^Avatica Basic client side load
> balancing.docx]for more details
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (CALCITE-5581) Implement Basic client side load balancing in Avatica Driver
[
https://issues.apache.org/jira/browse/CALCITE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated CALCITE-5581:
-
Fix Version/s: avatica-1.24.0
(was: 1.24.0)
> Implement Basic client side load balancing in Avatica Driver
>
>
> Key: CALCITE-5581
> URL: https://issues.apache.org/jira/browse/CALCITE-5581
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
>Affects Versions: avatica-1.23.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Major
> Labels: pull-request-available
> Fix For: avatica-1.24.0
>
> Attachments: Avatica Basic client side load balancing.docx
>
> Time Spent: 12h 20m
> Remaining Estimate: 0h
>
> Implement mechanism to support Client side load balancing by configuring
> connection properties for load balancing
> {code:java}
> jdbc:phoenix:thin:
> url=https://:;
> serialization=PROTOBUF;
> authentication=BASIC;
> avatica_user=;
> avatica_password=
> useClientSideLB: [true/false]
> lbStrategy: [RandomDistribution/Sequential/RoundRobin]
> lbURLs: {code}
> Client should connect to any of the URLs specified in lbURLs ( depending on
> LB strategy selected). Please refer to [^Avatica Basic client side load
> balancing.docx]for more details
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-5581) Implement Basic client side load balancing in Avatica Driver
[
https://issues.apache.org/jira/browse/CALCITE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth resolved CALCITE-5581.
--
Fix Version/s: 1.24.0
Resolution: Fixed
Committed.
Thank you [~vjoshi] .
> Implement Basic client side load balancing in Avatica Driver
>
>
> Key: CALCITE-5581
> URL: https://issues.apache.org/jira/browse/CALCITE-5581
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
>Affects Versions: 1.33.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.24.0
>
> Attachments: Avatica Basic client side load balancing.docx
>
> Time Spent: 12h 20m
> Remaining Estimate: 0h
>
> Implement mechanism to support Client side load balancing by configuring
> connection properties for load balancing
> {code:java}
> jdbc:phoenix:thin:
> url=https://:;
> serialization=PROTOBUF;
> authentication=BASIC;
> avatica_user=;
> avatica_password=
> useClientSideLB: [true/false]
> lbStrategy: [RandomDistribution/Sequential/RoundRobin]
> lbURLs: {code}
> Client should connect to any of the URLs specified in lbURLs ( depending on
> LB strategy selected). Please refer to [^Avatica Basic client side load
> balancing.docx]for more details
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-5609) Document Avatica Client Side Load balancing parameters ( CALCITE-5581 )
[ https://issues.apache.org/jira/browse/CALCITE-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved CALCITE-5609. -- Resolution: Duplicate The main patch already includes the docs. > Document Avatica Client Side Load balancing parameters ( CALCITE-5581 ) > --- > > Key: CALCITE-5609 > URL: https://issues.apache.org/jira/browse/CALCITE-5609 > Project: Calcite > Issue Type: New Feature > Components: avatica >Reporter: Vaibhav Joshi >Assignee: Vaibhav Joshi >Priority: Major > > We need to update [site/_docs/client_reference.md > |https://github.com/apache/calcite-avatica/pull/199/files#diff-141a82cee99f7d96b403700fcfc1b03d0bd5d40ea29f925a5b4ed3544a6d7164]once > we resolve the Client Side load balancing issue. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-5581) Implement Basic client side load balancing in Avatica Driver
[
https://issues.apache.org/jira/browse/CALCITE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706377#comment-17706377
]
Istvan Toth commented on CALCITE-5581:
--
Sounds fine.
Maybe call the second one LB_CONNECTION_FAILOVER_RETRIES , to indicate that it
only applies to the initial connection.
> Implement Basic client side load balancing in Avatica Driver
>
>
> Key: CALCITE-5581
> URL: https://issues.apache.org/jira/browse/CALCITE-5581
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
>Affects Versions: 1.33.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Major
> Labels: pull-request-available
> Attachments: Avatica Basic client side load balancing.docx
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Implement mechanism to support Client side load balancing by configuring
> connection properties for load balancing
> {code:java}
> jdbc:phoenix:thin:
> url=https://:;
> serialization=PROTOBUF;
> authentication=BASIC;
> avatica_user=;
> avatica_password=
> useClientSideLB: [true/false]
> lbStrategy: [RandomDistribution/Sequential/RoundRobin]
> lbURLs: {code}
> Client should connect to any of the URLs specified in lbURLs ( depending on
> LB strategy selected). Please refer to [^Avatica Basic client side load
> balancing.docx]for more details
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5581) Implement Basic client side load balancing in Avatica Driver
[
https://issues.apache.org/jira/browse/CALCITE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706245#comment-17706245
]
Istvan Toth edited comment on CALCITE-5581 at 3/29/23 7:08 AM:
---
No, I wouldn't use system properties.
Use the existing JDBC connection property mechanism for the timeout setting.
I'd also suggest including "http" and "connection" in the timeout property
name, there may be other timeouts we need to make configurable later.
was (Author: stoty):
No, I wouldn't use system properties.
Use the existing JDBC connection property mechanism for the timeout setting.
I'd also suggest including "http" in the timeout property name, there may be
other timeouts we need to make configurable later.
> Implement Basic client side load balancing in Avatica Driver
>
>
> Key: CALCITE-5581
> URL: https://issues.apache.org/jira/browse/CALCITE-5581
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
>Affects Versions: 1.33.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Major
> Labels: pull-request-available
> Attachments: Avatica Basic client side load balancing.docx
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Implement mechanism to support Client side load balancing by configuring
> connection properties for load balancing
> {code:java}
> jdbc:phoenix:thin:
> url=https://:;
> serialization=PROTOBUF;
> authentication=BASIC;
> avatica_user=;
> avatica_password=
> useClientSideLB: [true/false]
> lbStrategy: [RandomDistribution/Sequential/RoundRobin]
> lbURLs: {code}
> Client should connect to any of the URLs specified in lbURLs ( depending on
> LB strategy selected). Please refer to [^Avatica Basic client side load
> balancing.docx]for more details
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5581) Implement Basic client side load balancing in Avatica Driver
[
https://issues.apache.org/jira/browse/CALCITE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706245#comment-17706245
]
Istvan Toth commented on CALCITE-5581:
--
No, I wouldn't use system properties.
Use the existing JDBC connection property mechanism for the timeout setting.
I'd also suggest including "http" in the timeout property name, there may be
other timeouts we need to make configurable later.
> Implement Basic client side load balancing in Avatica Driver
>
>
> Key: CALCITE-5581
> URL: https://issues.apache.org/jira/browse/CALCITE-5581
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
>Affects Versions: 1.33.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Major
> Labels: pull-request-available
> Attachments: Avatica Basic client side load balancing.docx
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Implement mechanism to support Client side load balancing by configuring
> connection properties for load balancing
> {code:java}
> jdbc:phoenix:thin:
> url=https://:;
> serialization=PROTOBUF;
> authentication=BASIC;
> avatica_user=;
> avatica_password=
> useClientSideLB: [true/false]
> lbStrategy: [RandomDistribution/Sequential/RoundRobin]
> lbURLs: {code}
> Client should connect to any of the URLs specified in lbURLs ( depending on
> LB strategy selected). Please refer to [^Avatica Basic client side load
> balancing.docx]for more details
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-5581) Implement Basic client side load balancing in Avatica Driver
[
https://issues.apache.org/jira/browse/CALCITE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705184#comment-17705184
]
Istvan Toth edited comment on CALCITE-5581 at 3/27/23 5:28 AM:
---
Sounds good.
Note that the timeout handling is not trivial.
Essentialy we have two timeouts: One is the http(s) session establishmet, but
potentially we can also timeout after the https session is established, while
the Connection object is being built on the server side (for example Phoenix
initializing metadata on the very first connect)
was (Author: stoty):
Sounds good.
Note that the timeout handling is not trivial.
Essentialy we have two timeouts: One is the http(s) session establishmet, but
potentially we can also timeout after the https session is established, while
the Connection object is built on the server side (for example Phoenix
initializing metadata on the very first connect)
> Implement Basic client side load balancing in Avatica Driver
>
>
> Key: CALCITE-5581
> URL: https://issues.apache.org/jira/browse/CALCITE-5581
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
>Affects Versions: 1.33.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Major
> Labels: pull-request-available
> Attachments: Avatica Basic client side load balancing.docx
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Implement mechanism to support Client side load balancing by configuring
> connection properties for load balancing
> {code:java}
> jdbc:phoenix:thin:
> url=https://:;
> serialization=PROTOBUF;
> authentication=BASIC;
> avatica_user=;
> avatica_password=
> useClientSideLB: [true/false]
> lbStrategy: [RandomDistribution/Sequential/RoundRobin]
> lbURLs: {code}
> Client should connect to any of the URLs specified in lbURLs ( depending on
> LB strategy selected). Please refer to [^Avatica Basic client side load
> balancing.docx]for more details
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5581) Implement Basic client side load balancing in Avatica Driver
[
https://issues.apache.org/jira/browse/CALCITE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705185#comment-17705185
]
Istvan Toth commented on CALCITE-5581:
--
Also, while avatica is already not consistent, most parameters use undescrores
to separate words, not CamelCase,we should follow the more used convention.
> Implement Basic client side load balancing in Avatica Driver
>
>
> Key: CALCITE-5581
> URL: https://issues.apache.org/jira/browse/CALCITE-5581
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
>Affects Versions: 1.33.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Major
> Labels: pull-request-available
> Attachments: Avatica Basic client side load balancing.docx
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Implement mechanism to support Client side load balancing by configuring
> connection properties for load balancing
> {code:java}
> jdbc:phoenix:thin:
> url=https://:;
> serialization=PROTOBUF;
> authentication=BASIC;
> avatica_user=;
> avatica_password=
> useClientSideLB: [true/false]
> lbStrategy: [RandomDistribution/Sequential/RoundRobin]
> lbURLs: {code}
> Client should connect to any of the URLs specified in lbURLs ( depending on
> LB strategy selected). Please refer to [^Avatica Basic client side load
> balancing.docx]for more details
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5581) Implement Basic client side load balancing in Avatica Driver
[
https://issues.apache.org/jira/browse/CALCITE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705184#comment-17705184
]
Istvan Toth commented on CALCITE-5581:
--
Sounds good.
Note that the timeout handling is not trivial.
Essentialy we have two timeouts: One is the http(s) session establishmet, but
potentially we can also timeout after the https session is established, while
the Connection object is built on the server side (for example Phoenix
initializing metadata on the very first connect)
> Implement Basic client side load balancing in Avatica Driver
>
>
> Key: CALCITE-5581
> URL: https://issues.apache.org/jira/browse/CALCITE-5581
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
>Affects Versions: 1.33.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Major
> Labels: pull-request-available
> Attachments: Avatica Basic client side load balancing.docx
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Implement mechanism to support Client side load balancing by configuring
> connection properties for load balancing
> {code:java}
> jdbc:phoenix:thin:
> url=https://:;
> serialization=PROTOBUF;
> authentication=BASIC;
> avatica_user=;
> avatica_password=
> useClientSideLB: [true/false]
> lbStrategy: [RandomDistribution/Sequential/RoundRobin]
> lbURLs: {code}
> Client should connect to any of the URLs specified in lbURLs ( depending on
> LB strategy selected). Please refer to [^Avatica Basic client side load
> balancing.docx]for more details
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5581) Implement Basic client side load balancing in Avatica Driver
[
https://issues.apache.org/jira/browse/CALCITE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17704166#comment-17704166
]
Istvan Toth commented on CALCITE-5581:
--
I agree that the current state of the patch is already useful, but I think that
we improve on it
I would like to achieve feature parity with the (already pretty bare-bones)
Knox implementation at least. (where it makes sense)
*Identifiying failed hosts*
We can sidestep most of the complexity if we don't try to process the reasons
for failure.
We can ignore when an alredy active connection fails, and only mark a host
failed if the original connection attempt is unsuccessful. We know that the
other end is either an avatica server, or some kind of proxy, and the only
processing we need to do ignoring 401/403 errors.
*Handling failed hosts*
One option is to move the failed host to the end of the list, like Knox does.
This of course is not feasible for random. One option is to simply not maintain
a failed list for the random strategy.
*Timeouts*
If we want to handle failover during connection establishment, then we must
make sure to use a relatively short timeout. I.e. if the normal connect tiemout
is 30 seconds, then we do not want to wait a total of 150 seconds if we try 5
hosts.
> Implement Basic client side load balancing in Avatica Driver
>
>
> Key: CALCITE-5581
> URL: https://issues.apache.org/jira/browse/CALCITE-5581
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
>Affects Versions: 1.33.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Major
> Labels: pull-request-available
> Attachments: Avatica Basic client side load balancing.docx
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Implement mechanism to support Client side load balancing by configuring
> connection properties for load balancing
> {code:java}
> jdbc:phoenix:thin:
> url=https://:;
> serialization=PROTOBUF;
> authentication=BASIC;
> avatica_user=;
> avatica_password=
> useClientSideLB: [true/false]
> lbStrategy: [RandomDistribution/Sequential/RoundRobin]
> lbURLs: {code}
> Client should connect to any of the URLs specified in lbURLs ( depending on
> LB strategy selected). Please refer to [^Avatica Basic client side load
> balancing.docx]for more details
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5581) Implement Basic client side load balancing in Avatica Driver
[
https://issues.apache.org/jira/browse/CALCITE-5581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703554#comment-17703554
]
Istvan Toth commented on CALCITE-5581:
--
Thank you [~vjoshi] .
I have left some comments on the PR.
However, there are some broader issues that may be better discussed directly on
the ticket.
Your current PR is very minimal, I would consider adding two more features,
that would add useful functionality without bloating the client, or adding too
much complexity:
1.
Maintaining a list of servers with recently failed connections.
This does not need a backround liveness check, we can use two sources:
- Note whenever a new connection attempt fails, which is simple
- Note when an existing connection fails, evaluate the reason, and check if it
was caused by a server failure. This one less straightforward, and probably not
worth it for a first cut.
We can
* simply move failing servers to the end of the list. (Only works for round
robin)
* or maintain a failed server list for a period of time. This one is more
complex, we have to handle each server getting on the failed list, and removing
servers from there.
2.
Automatically re-trying the connection on failure.
The current implementation leaves the reconnection attempt to the application,
which it should ideally handle anyway.
However, we could also handle the failing connections within Avatica.
This could work by defining a timeout for connecting, and automatically failing
over to another server if the connection returns an error, or if it is not
connecting successfully within the timout.
We need to decide or mak configurable things like what the connection timeout
is, how long do we keep trying before reporting failure to the client, etc.
> Implement Basic client side load balancing in Avatica Driver
>
>
> Key: CALCITE-5581
> URL: https://issues.apache.org/jira/browse/CALCITE-5581
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
>Affects Versions: 1.33.0
>Reporter: Vaibhav Joshi
>Assignee: Vaibhav Joshi
>Priority: Major
> Labels: pull-request-available
> Attachments: Avatica Basic client side load balancing.docx
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Implement mechanism to support Client side load balancing by configuring
> connection properties for load balancing
> {code:java}
> jdbc:phoenix:thin:
> url=https://:;
> serialization=PROTOBUF;
> authentication=BASIC;
> avatica_user=;
> avatica_password=
> useClientSideLB: [true/false]
> lbStrategy: [RandomDistribution/Sequential/RoundRobin]
> lbURLs: {code}
> Client should connect to any of the URLs specified in lbURLs ( depending on
> LB strategy selected). Please refer to [^Avatica Basic client side load
> balancing.docx]for more details
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5534) Add `userAgent` JDBC param to Avatica
[
https://issues.apache.org/jira/browse/CALCITE-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17690925#comment-17690925
]
Istvan Toth commented on CALCITE-5534:
--
If having the parameter exposed as an HTTP header is important (i.e. for a load
balancer or similar proxy), then we could consider allowing adding free-form
HTTP header lines.
My colleague is working on implementing bearer HTTP auth for Avatica, and being
able to specify arbitrary full HTTP header(s) would have been the easiest route.
i.e something like
_jdbc:avatica:remote:url=remoteserver;extra_http_header=User-Agent: agent-info_
(with http encoding or some other escaping for the colon)
> Add `userAgent` JDBC param to Avatica
> -
>
> Key: CALCITE-5534
> URL: https://issues.apache.org/jira/browse/CALCITE-5534
> Project: Calcite
> Issue Type: Improvement
> Components: avatica
>Reporter: TJ Banghart
>Assignee: TJ Banghart
>Priority: Minor
>
> It could be beneficial to add a {{userAgent}} parameter to the JDBC URL
> recognized options. This {{userAgent}} value would be passed from the client
> to server as a {{User-Agent}} HTTP header overriding Avatica's current
> default of {{Apache-HttpClient/ ()}}. This would allow
> Avatica servers to more easily identify requesting applications.
> Here's [Denodo's {{userAgent}}
> |https://community.denodo.com/docs/html/browse/6.0/vdp/administration/monitoring_the_virtual_dataport_server/monitoring_with_a_java_management_extensions_jmx_agent/setting_the_user_agent_of_an_application]
> as an example.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (CALCITE-5534) Add `userAgent` JDBC param to Avatica
[
https://issues.apache.org/jira/browse/CALCITE-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17690858#comment-17690858
]
Istvan Toth commented on CALCITE-5534:
--
Instead of adding new properties, IMO wo should transparently relay all
non-Avatica properties to the server, and let the proxied driver handle them.
That would solve both this problem, and add the ability to set any driver
specific properties from the Avatica client.
(We could also introduce a special property where the relayed properties could
be set instead)
i.e
_jdbc:avatica:remote:url=remoteserver;driverspecific1=a;driverspecifc2=b_
or
_jdbc:avatica:remote:url=remoteserver;relayoption=driverspecifc1ESCEQaESCCOMMAdrivespecific2ESCEQb_
and these would be set for the proxied connection as
as jdbc:proxied?driverspecific1=a&driverspecific2=b
See https://issues.apache.org/jira/browse/CALCITE-4121
> Add `userAgent` JDBC param to Avatica
> -
>
> Key: CALCITE-5534
> URL: https://issues.apache.org/jira/browse/CALCITE-5534
> Project: Calcite
> Issue Type: Improvement
> Components: avatica
>Reporter: TJ Banghart
>Assignee: TJ Banghart
>Priority: Minor
>
> It could be beneficial to add a {{userAgent}} parameter to the JDBC URL
> recognized options. This {{userAgent}} value would be passed from the client
> to server as a {{User-Agent}} HTTP header overriding Avatica's current
> default of {{Apache-HttpClient/ ()}}. This would allow
> Avatica servers to more easily identify requesting applications.
> Here's [Denodo's {{userAgent}}
> |https://community.denodo.com/docs/html/browse/6.0/vdp/administration/monitoring_the_virtual_dataport_server/monitoring_with_a_java_management_extensions_jmx_agent/setting_the_user_agent_of_an_application]
> as an example.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (CALCITE-5438) Document transparent_reconnection property
[ https://issues.apache.org/jira/browse/CALCITE-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved CALCITE-5438. -- Fix Version/s: 1.23.0 Resolution: Fixed > Document transparent_reconnection property > -- > > Key: CALCITE-5438 > URL: https://issues.apache.org/jira/browse/CALCITE-5438 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: 1.23.0 > > Time Spent: 40m > Remaining Estimate: 0h > > Add documentation for the transparent_reconnection property added in > CALCITE-5009 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (CALCITE-5065) Update phoenixdb information on website
[ https://issues.apache.org/jira/browse/CALCITE-5065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved CALCITE-5065. -- Fix Version/s: 1.23.0 Resolution: Fixed > Update phoenixdb information on website > --- > > Key: CALCITE-5065 > URL: https://issues.apache.org/jira/browse/CALCITE-5065 > Project: Calcite > Issue Type: Task >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Minor > Fix For: 1.23.0 > > Time Spent: 40m > Remaining Estimate: 0h > > The information at > [https://calcite.apache.org/avatica/docs/#apache-phoenix-database-adapter-for-python] > is out of date, update it to reflect the current status. > (new homepage, maintained by the Phoenix project, supports up to avatica 1.19) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (CALCITE-5065) Update phoenixdb information on website
[ https://issues.apache.org/jira/browse/CALCITE-5065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth reassigned CALCITE-5065: Assignee: Istvan Toth > Update phoenixdb information on website > --- > > Key: CALCITE-5065 > URL: https://issues.apache.org/jira/browse/CALCITE-5065 > Project: Calcite > Issue Type: Task >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > The information at > [https://calcite.apache.org/avatica/docs/#apache-phoenix-database-adapter-for-python] > is out of date, update it to reflect the current status. > (new homepage, maintained by the Phoenix project, supports up to avatica 1.19) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-5438) Document transparent_reconnection property
Istvan Toth created CALCITE-5438: Summary: Document transparent_reconnection property Key: CALCITE-5438 URL: https://issues.apache.org/jira/browse/CALCITE-5438 Project: Calcite Issue Type: Bug Components: avatica Reporter: Istvan Toth Assignee: Istvan Toth Add documentation for the transparent_reconnection property added in CALCITE-5009 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (CALCITE-4651) Try to get the SPENGO specific Jetty fixes/overrides in Avatica into Jetty
[ https://issues.apache.org/jira/browse/CALCITE-4651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630219#comment-17630219 ] Istvan Toth edited comment on CALCITE-4651 at 11/8/22 6:41 AM: --- Since CALCITE-4152 we are directly using a newer style SPENGO support provided by Jetty, and workarounds are no longer needed or used. was (Author: stoty): Since CALCITE-4152 we are directly using a newer style SPENGO support provided for Jetty, and workarounds are no longer needed or used. > Try to get the SPENGO specific Jetty fixes/overrides in Avatica into Jetty > -- > > Key: CALCITE-4651 > URL: https://issues.apache.org/jira/browse/CALCITE-4651 > Project: Calcite > Issue Type: Task > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > Analyze which of the the SPENGO specific fixes in the overriden Jetty classes > in Avatica are fit for inclusion in Jetty, and try to get them fixed in Jetty. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (CALCITE-4651) Try to get the SPENGO specific Jetty fixes/overrides in Avatica into Jetty
[ https://issues.apache.org/jira/browse/CALCITE-4651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth reassigned CALCITE-4651: Assignee: Istvan Toth > Try to get the SPENGO specific Jetty fixes/overrides in Avatica into Jetty > -- > > Key: CALCITE-4651 > URL: https://issues.apache.org/jira/browse/CALCITE-4651 > Project: Calcite > Issue Type: Task > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > Analyze which of the the SPENGO specific fixes in the overriden Jetty classes > in Avatica are fit for inclusion in Jetty, and try to get them fixed in Jetty. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (CALCITE-4651) Try to get the SPENGO specific Jetty fixes/overrides in Avatica into Jetty
[ https://issues.apache.org/jira/browse/CALCITE-4651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved CALCITE-4651. -- Resolution: Won't Do Since CALCITE-4152 we are directly using a newer style SPENGO support provided for Jetty, and workarounds are no longer needed or used. > Try to get the SPENGO specific Jetty fixes/overrides in Avatica into Jetty > -- > > Key: CALCITE-4651 > URL: https://issues.apache.org/jira/browse/CALCITE-4651 > Project: Calcite > Issue Type: Task > Components: avatica >Reporter: Istvan Toth >Priority: Major > > Analyze which of the the SPENGO specific fixes in the overriden Jetty classes > in Avatica are fit for inclusion in Jetty, and try to get them fixed in Jetty. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (CALCITE-4971) update httpclient and httpcore to latest 5.1 release
[ https://issues.apache.org/jira/browse/CALCITE-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517572#comment-17517572 ] Istvan Toth commented on CALCITE-4971: -- The patch is ready for review. > update httpclient and httpcore to latest 5.1 release > > > Key: CALCITE-4971 > URL: https://issues.apache.org/jira/browse/CALCITE-4971 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Apache commons httpclient/httpcomponent 4.5 depend on commons-logging and not > slf4j. This means that phoenix-thin requires explicit log4j configuration to > work. > We want all logging to go through SLF4j, and to be able to use any supported > backend. > Based on an an offline conversation with [~elserj] > As these are new major version, it's probably going to involve more than a > version bump. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (CALCITE-4971) update httpclient and httpcore to latest 5.1 release
[ https://issues.apache.org/jira/browse/CALCITE-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth reassigned CALCITE-4971: Assignee: Istvan Toth > update httpclient and httpcore to latest 5.1 release > > > Key: CALCITE-4971 > URL: https://issues.apache.org/jira/browse/CALCITE-4971 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > Apache commons httpclient/httpcomponent 4.5 depend on commons-logging and not > slf4j. This means that phoenix-thin requires explicit log4j configuration to > work. > We want all logging to go through SLF4j, and to be able to use any supported > backend. > Based on an an offline conversation with [~elserj] > As these are new major version, it's probably going to involve more than a > version bump. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (CALCITE-5009) Transparent JDBC connection re-creation may lead to data loss
[
https://issues.apache.org/jira/browse/CALCITE-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated CALCITE-5009:
-
Docs Text:
Avatica will no longer tranparently create a new connection if the proxied
connection object expires from the server side cache.
The transparent reconnection feature can be enabled setting the
"transparent_reconnection" JDBC property to true, but this is only recommended
for connections where losing the connection state, including transactional
state, is not a problem.
However, it is recommended to tune the connection and server side cache
parameters to avoid the cache expiry alltogether.
was:
Avatica will no longer tranparently create a new connection if the proxied
connection object times out from the server side cache.
The transparent reconnection feature can be enabled setting the
"transparent_reconnection" JDBC property to true, but this is only recommended
for connections where losing the connection state, including transactional
state, is not a problem.
However, it is recommended to tune the connection and server side cache
parameters to avoid the cache expiry alltogether.
> Transparent JDBC connection re-creation may lead to data loss
> -
>
> Key: CALCITE-5009
> URL: https://issues.apache.org/jira/browse/CALCITE-5009
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently, if the server-side JDBC connection goes away because it is expored
> from the server-side connection cache we attempt to transparently create a
> new "real" JDBC connection, and continue using that instead of the original
> connection
> [https://github.com/apache/calcite-avatica/blob/fbdcc62745a0e8920db759fb6bdce564d854e407/core/src/main/java/org/apache/calcite/avatica/AvaticaConnection.java#L796]
> This is fine for most read-only connections, but it can break transaction
> semantics, which is captured in the "real" connection object.
> {noformat}
> conn.setAutocommit(false)
> stmt = conn.createStatement()
> execute(insert A)
> //Connection lost and object recreated which now proxies a new "real"
> connection
> execute(insert B)
> conn.commit()
> //We have lost "insert A"{noformat}
> I'm not sure if we synchronize autocommit state of the new connection to the
> lost one or not, but it's bad either way.
>
> We should either completely drop this feature, add some logic that avoids it
> if there is an open transaction and/or only allow it for connections that
> have the readOnly flag set.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
