[jira] [Comment Edited] (CALCITE-3621) JDBC adapter can't push down sort to DB
[
https://issues.apache.org/jira/browse/CALCITE-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002088#comment-17002088
]
Lei Jiang edited comment on CALCITE-3621 at 12/23/19 3:59 AM:
--
Thank you very much Jin Xing, I updated some comment in [Pull
Request|https://github.com/apache/calcite/pull/1680#discussion_r360756860]
was (Author: lei jiang):
Thank you very much Jin Xing, I updated some comment in [Pull
Request|https://github.com/apache/calcite/pull/1680#discussion_r360756860]
[http|http://dict.youdao.com/search?q=http=chrome.extension] [ˌeɪtʃ
tiː tiː ˈpiː]
[详细|http://dict.youdao.com/search?q=http=chrome.extension]X
基本翻译
abbr. 超文本传输协议 (hypertext transport protocol)
网络释义
[HTTP:|http://dict.youdao.com/search?q=HTTP=chrome.extension=eng]
超文本传输协议(Hyper Text Transfer Protocol)
[HTTP
Pipelining:|http://dict.youdao.com/search?q=HTTP%20Pipelining=chrome.extension=eng]
管线化
[HTTP头:|http://dict.youdao.com/search?q=HTTP%E5%A4%B4=chrome.extension=eng]
HTTP header
> JDBC adapter can't push down sort to DB
> ---
>
> Key: CALCITE-3621
> URL: https://issues.apache.org/jira/browse/CALCITE-3621
> Project: Calcite
> Issue Type: Bug
> Components: jdbc-adapter
>Affects Versions: 1.21.0
>Reporter: Lei Jiang
>Assignee: Lei Jiang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> JDBC adapter can't push down sort to DB
> {code:java}
> select ename from scott.emp order by empno
> {code}
> {code:java}
> PLAN=EnumerableSort(sort0=[$1], dir0=[ASC])
> JdbcToEnumerableConverter
> JdbcProject(ENAME=[$1], EMPNO=[$0])
> JdbcTableScan(table=[[SCOTT, EMP]])
> {code}
> It should be:
> {code:java}
> PLAN=JdbcToEnumerableConverter
> JdbcSort(sort0=[$1], dir0=[ASC])
> JdbcProject(ENAME=[$1], EMPNO=[$0])
> JdbcTableScan(table=[[SCOTT, EMP]])
> {code}
> I think the root cause is JdbcSortRule, it convert input's trait to "JDBC,
> {color:#ff}[1]{color}". that is, input's relset will add a "JDBC, [1]"
> subset. But there is nothing rule can convert that input to a rel with "JDBC,
> {color:#ff}[1]{color}", so EnumerableSort win.
> {code:java}
> public RelNode convert(Sort sort, boolean convertInputTraits) {
> final RelTraitSet traitSet = sort.getTraitSet().replace(out);
> final RelNode input;
> if (convertInputTraits) {
> input = convert(sort.getInput(), traitSet);
> } else {
> input = sort.getInput();
> }
> return new JdbcSort(sort.getCluster(), traitSet,
> input, sort.getCollation(), sort.offset, sort.fetch);
> }
> {code}
> This is my a part of change: convert input's trait to "JDBC, []"
> {code:java}
> public RelNode convert(Sort sort, boolean convertInputTraits) {
> final RelTraitSet traitSet = sort.getTraitSet().replace(out);
> //update again
> final RelTraitSet inputTraitSet =
> sort.getInput().getTraitSet().replace(out);
> final RelNode input;
> if (convertInputTraits) {
> //update
> input = convert(sort.getInput(), inputTraitSet);
> } else {
> input = sort.getInput();
> }
> return new JdbcSort(sort.getCluster(), traitSet,
> input, sort.getCollation(), sort.offset, sort.fetch);
> }
> {code}
>
> When JdbcSort is used, my solution will trigger a regression issue
> JdbcTest#testSelfJoinDifferentColumns(): Project->Sort->Join(left.A,
> right.A), Join.left and Join.right have same field A. My solution: select
> list of Sort contains explicit column name instead of *
>
> {code:java}
> RelToSqlConverter#visit(Sort e)
> ...
> Result x = visitChild(0, e.getInput());
> Builder builder = x.builder(e, Clause.ORDER_BY);
> //my update
> if (builder.select.getSelectList() == null) {
> final List selectList = Expressions.list();
> for (RelDataTypeField field : e.getRowType().getFieldList()) {
> addSelect(selectList, builder.context.field(field.getIndex()),
> e.getRowType());
> }
> builder.select.setSelectList(new SqlNodeList(selectList, POS));
> }
> //end
> ...{code}
> REL:
> JdbcToEnumerableConverter
> --JdbcProject(full_name=[$1], last_name=[$3])
> JdbcSort(sort0=[$3], dir0=[ASC], fetch=[3])
> --JdbcJoin(condition=[=($2, $0)], joinType=[inner])
> JdbcProject(last_name=[$3])
> --JdbcTableScan(table=[[foodmart, employee]])
> JdbcProject(full_name=[$1], first_name=[$2], last_name=[$3])
> --JdbcTableScan(table=[[foodmart, employee]])
> Before Change: field last_name{color:#de350b}0{color} doesn't exist in DB
> SELECT "t2"."full_name", "t2"."last_name{color:#ff}0{color}" AS
> "last_name"
> FROM (SELECT {color:#ff}*{color}
> FROM (SELECT "last_name"
> FROM "foodmart"."employee") AS "t"
> INNER JOIN (SELECT "full_name", "first_name", "last_name"
[jira] [Commented] (CALCITE-3621) JDBC adapter can't push down sort to DB
[
https://issues.apache.org/jira/browse/CALCITE-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17002088#comment-17002088
]
Lei Jiang commented on CALCITE-3621:
Thank you very much Jin Xing, I updated some comment in [Pull
Request|https://github.com/apache/calcite/pull/1680#discussion_r360756860]
[http|http://dict.youdao.com/search?q=http=chrome.extension] [ˌeɪtʃ
tiː tiː ˈpiː]
[详细|http://dict.youdao.com/search?q=http=chrome.extension]X
基本翻译
abbr. 超文本传输协议 (hypertext transport protocol)
网络释义
[HTTP:|http://dict.youdao.com/search?q=HTTP=chrome.extension=eng]
超文本传输协议(Hyper Text Transfer Protocol)
[HTTP
Pipelining:|http://dict.youdao.com/search?q=HTTP%20Pipelining=chrome.extension=eng]
管线化
[HTTP头:|http://dict.youdao.com/search?q=HTTP%E5%A4%B4=chrome.extension=eng]
HTTP header
> JDBC adapter can't push down sort to DB
> ---
>
> Key: CALCITE-3621
> URL: https://issues.apache.org/jira/browse/CALCITE-3621
> Project: Calcite
> Issue Type: Bug
> Components: jdbc-adapter
>Affects Versions: 1.21.0
>Reporter: Lei Jiang
>Assignee: Lei Jiang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> JDBC adapter can't push down sort to DB
> {code:java}
> select ename from scott.emp order by empno
> {code}
> {code:java}
> PLAN=EnumerableSort(sort0=[$1], dir0=[ASC])
> JdbcToEnumerableConverter
> JdbcProject(ENAME=[$1], EMPNO=[$0])
> JdbcTableScan(table=[[SCOTT, EMP]])
> {code}
> It should be:
> {code:java}
> PLAN=JdbcToEnumerableConverter
> JdbcSort(sort0=[$1], dir0=[ASC])
> JdbcProject(ENAME=[$1], EMPNO=[$0])
> JdbcTableScan(table=[[SCOTT, EMP]])
> {code}
> I think the root cause is JdbcSortRule, it convert input's trait to "JDBC,
> {color:#ff}[1]{color}". that is, input's relset will add a "JDBC, [1]"
> subset. But there is nothing rule can convert that input to a rel with "JDBC,
> {color:#ff}[1]{color}", so EnumerableSort win.
> {code:java}
> public RelNode convert(Sort sort, boolean convertInputTraits) {
> final RelTraitSet traitSet = sort.getTraitSet().replace(out);
> final RelNode input;
> if (convertInputTraits) {
> input = convert(sort.getInput(), traitSet);
> } else {
> input = sort.getInput();
> }
> return new JdbcSort(sort.getCluster(), traitSet,
> input, sort.getCollation(), sort.offset, sort.fetch);
> }
> {code}
> This is my a part of change: convert input's trait to "JDBC, []"
> {code:java}
> public RelNode convert(Sort sort, boolean convertInputTraits) {
> final RelTraitSet traitSet = sort.getTraitSet().replace(out);
> //update again
> final RelTraitSet inputTraitSet =
> sort.getInput().getTraitSet().replace(out);
> final RelNode input;
> if (convertInputTraits) {
> //update
> input = convert(sort.getInput(), inputTraitSet);
> } else {
> input = sort.getInput();
> }
> return new JdbcSort(sort.getCluster(), traitSet,
> input, sort.getCollation(), sort.offset, sort.fetch);
> }
> {code}
>
> When JdbcSort is used, my solution will trigger a regression issue
> JdbcTest#testSelfJoinDifferentColumns(): Project->Sort->Join(left.A,
> right.A), Join.left and Join.right have same field A. My solution: select
> list of Sort contains explicit column name instead of *
>
> {code:java}
> RelToSqlConverter#visit(Sort e)
> ...
> Result x = visitChild(0, e.getInput());
> Builder builder = x.builder(e, Clause.ORDER_BY);
> //my update
> if (builder.select.getSelectList() == null) {
> final List selectList = Expressions.list();
> for (RelDataTypeField field : e.getRowType().getFieldList()) {
> addSelect(selectList, builder.context.field(field.getIndex()),
> e.getRowType());
> }
> builder.select.setSelectList(new SqlNodeList(selectList, POS));
> }
> //end
> ...{code}
> REL:
> JdbcToEnumerableConverter
> --JdbcProject(full_name=[$1], last_name=[$3])
> JdbcSort(sort0=[$3], dir0=[ASC], fetch=[3])
> --JdbcJoin(condition=[=($2, $0)], joinType=[inner])
> JdbcProject(last_name=[$3])
> --JdbcTableScan(table=[[foodmart, employee]])
> JdbcProject(full_name=[$1], first_name=[$2], last_name=[$3])
> --JdbcTableScan(table=[[foodmart, employee]])
> Before Change: field last_name{color:#de350b}0{color} doesn't exist in DB
> SELECT "t2"."full_name", "t2"."last_name{color:#ff}0{color}" AS
> "last_name"
> FROM (SELECT {color:#ff}*{color}
> FROM (SELECT "last_name"
> FROM "foodmart"."employee") AS "t"
> INNER JOIN (SELECT "full_name", "first_name", "last_name"
> FROM "foodmart"."employee") AS "t0" ON "t"."last_name" = "t0"."first_name"
> ORDER BY "t0"."last_name" NULLS LAST
> After Change:
> SELECT "t2"."full_name", "t2"."last_name0" AS "last_name"
> FROM (SELECT
[jira] [Updated] (CALCITE-3621) JDBC adapter can't push down sort to DB
[
https://issues.apache.org/jira/browse/CALCITE-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei Jiang updated CALCITE-3621:
---
Description:
JDBC adapter can't push down sort to DB
{code:java}
select ename from scott.emp order by empno
{code}
{code:java}
PLAN=EnumerableSort(sort0=[$1], dir0=[ASC])
JdbcToEnumerableConverter
JdbcProject(ENAME=[$1], EMPNO=[$0])
JdbcTableScan(table=[[SCOTT, EMP]])
{code}
It should be:
{code:java}
PLAN=JdbcToEnumerableConverter
JdbcSort(sort0=[$1], dir0=[ASC])
JdbcProject(ENAME=[$1], EMPNO=[$0])
JdbcTableScan(table=[[SCOTT, EMP]])
{code}
I think the root cause is JdbcSortRule, it convert input's trait to "JDBC,
{color:#ff}[1]{color}". that is, input's relset will add a "JDBC, [1]"
subset. But there is nothing rule can convert that input to a rel with "JDBC,
{color:#ff}[1]{color}", so EnumerableSort win.
{code:java}
public RelNode convert(Sort sort, boolean convertInputTraits) {
final RelTraitSet traitSet = sort.getTraitSet().replace(out);
final RelNode input;
if (convertInputTraits) {
input = convert(sort.getInput(), traitSet);
} else {
input = sort.getInput();
}
return new JdbcSort(sort.getCluster(), traitSet,
input, sort.getCollation(), sort.offset, sort.fetch);
}
{code}
This is my a part of change: convert input's trait to "JDBC, []"
{code:java}
public RelNode convert(Sort sort, boolean convertInputTraits) {
final RelTraitSet traitSet = sort.getTraitSet().replace(out);
//update again
final RelTraitSet inputTraitSet = sort.getInput().getTraitSet().replace(out);
final RelNode input;
if (convertInputTraits) {
//update
input = convert(sort.getInput(), inputTraitSet);
} else {
input = sort.getInput();
}
return new JdbcSort(sort.getCluster(), traitSet,
input, sort.getCollation(), sort.offset, sort.fetch);
}
{code}
When JdbcSort is used, my solution will trigger a regression issue
JdbcTest#testSelfJoinDifferentColumns(): Project->Sort->Join(left.A, right.A),
Join.left and Join.right have same field A. My solution: select list of Sort
contains explicit column name instead of *
{code:java}
RelToSqlConverter#visit(Sort e)
...
Result x = visitChild(0, e.getInput());
Builder builder = x.builder(e, Clause.ORDER_BY);
//my update
if (builder.select.getSelectList() == null) {
final List selectList = Expressions.list();
for (RelDataTypeField field : e.getRowType().getFieldList()) {
addSelect(selectList, builder.context.field(field.getIndex()),
e.getRowType());
}
builder.select.setSelectList(new SqlNodeList(selectList, POS));
}
//end
...{code}
REL:
JdbcToEnumerableConverter
--JdbcProject(full_name=[$1], last_name=[$3])
JdbcSort(sort0=[$3], dir0=[ASC], fetch=[3])
--JdbcJoin(condition=[=($2, $0)], joinType=[inner])
JdbcProject(last_name=[$3])
--JdbcTableScan(table=[[foodmart, employee]])
JdbcProject(full_name=[$1], first_name=[$2], last_name=[$3])
--JdbcTableScan(table=[[foodmart, employee]])
Before Change: field last_name{color:#de350b}0{color} doesn't exist in DB
SELECT "t2"."full_name", "t2"."last_name{color:#ff}0{color}" AS "last_name"
FROM (SELECT {color:#ff}*{color}
FROM (SELECT "last_name"
FROM "foodmart"."employee") AS "t"
INNER JOIN (SELECT "full_name", "first_name", "last_name"
FROM "foodmart"."employee") AS "t0" ON "t"."last_name" = "t0"."first_name"
ORDER BY "t0"."last_name" NULLS LAST
After Change:
SELECT "t2"."full_name", "t2"."last_name0" AS "last_name"
FROM (SELECT {color:#ff}"t"."last_name", "t0"."full_name",
"t0"."first_name", "t0"."last_name" AS "last_name0"{color}
FROM (SELECT "last_name"
FROM "foodmart"."employee") AS "t"
INNER JOIN (SELECT "full_name", "first_name", "last_name"
FROM "foodmart"."employee") AS "t0" ON "t"."last_name" = "t0"."first_name"
ORDER BY "t0"."last_name" NULLS LAST
LIMIT 3) AS "t2"
was:
JDBC adapter can't push down sort to DB
{code:java}
select ename from scott.emp order by empno
{code}
{code:java}
PLAN=EnumerableSort(sort0=[$1], dir0=[ASC])
JdbcToEnumerableConverter
JdbcProject(ENAME=[$1], EMPNO=[$0])
JdbcTableScan(table=[[SCOTT, EMP]])
{code}
It should be:
{code:java}
PLAN=JdbcToEnumerableConverter
JdbcSort(sort0=[$1], dir0=[ASC])
JdbcProject(ENAME=[$1], EMPNO=[$0])
JdbcTableScan(table=[[SCOTT, EMP]])
{code}
I think the root cause is JdbcSortRule, it convert input's trait to "JDBC,
{color:#ff}[1]{color}". that is, input's relset will add a "JDBC, [1]"
subset. But there is nothing rule can convert that input to a rel with "JDBC,
{color:#ff}[1]{color}", so EnumerableSort win.
{code:java}
public RelNode convert(Sort sort, boolean convertInputTraits) {
final RelTraitSet traitSet = sort.getTraitSet().replace(out);
final RelNode input;
if (convertInputTraits) {
input = convert(sort.getInput(), traitSet);
} else {
input =
[jira] [Resolved] (CALCITE-3387) Query with GROUP BY and JOIN ... USING wrongly fails with "Column 'DEPTNO' is ambiguous" error
[
https://issues.apache.org/jira/browse/CALCITE-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chunwei Lei resolved CALCITE-3387.
--
Fix Version/s: 1.22.0
Resolution: Fixed
Fixed in
[https://github.com/apache/calcite/commit/c4d18de778713850f7d00f2ed418dca6922a0bff].
Thanks for your review, [~jinxing6...@126.com], [~julianhyde], [~hyuan].
> Query with GROUP BY and JOIN ... USING wrongly fails with "Column 'DEPTNO' is
> ambiguous" error
> --
>
> Key: CALCITE-3387
> URL: https://issues.apache.org/jira/browse/CALCITE-3387
> Project: Calcite
> Issue Type: Bug
>Reporter: Julian Hyde
>Assignee: Chunwei Lei
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Query with GROUP BY and JOIN ... USING wrongly fails with "Column 'DEPTNO' is
> ambiguous" error. Here is the query:
> {code}
> select deptno, count(*)
> from emp
> join dept using (deptno)
> group by deptno;
> {code}
> Because of USING, the two deptno fields should be merged into one, and
> therefore {{deptno}} is not ambiguous. That query works in Oracle. Also note
> that
> {code}select deptno
> from emp
> join dept using (deptno)
> {code}
> works correctly in Calcite, and has since CALCITE-2227 was fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3623) Replace Spotless with Autostyle
[ https://issues.apache.org/jira/browse/CALCITE-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3623: Labels: pull-request-available (was: ) > Replace Spotless with Autostyle > --- > > Key: CALCITE-3623 > URL: https://issues.apache.org/jira/browse/CALCITE-3623 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.21.0 >Reporter: Vladimir Sitnikov >Priority: Major > Labels: pull-request-available > > Spotless has certain drawbacks: > 1) It is not able to verify license headers for non-Java files. For instance, > it skips `package-info.java`, it skips `*.kts` and so on :( > 2) Its error messages are too verbose. Sometimes it prints the full > stacktrace when just one line was enough: "file X line Y column Z has error: > ..." > 3) It uses unsafe Gradle APIs, so it will be incompatible with Gradle 7.0 > I suggest to replace it with https://github.com/autostyle/autostyle > Note: I tried to contact Spotless authors, and their way to manage code makes > it very hard to co-operate :(( -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3623) Replace Spotless with Autostyle
Vladimir Sitnikov created CALCITE-3623: -- Summary: Replace Spotless with Autostyle Key: CALCITE-3623 URL: https://issues.apache.org/jira/browse/CALCITE-3623 Project: Calcite Issue Type: Improvement Components: core Affects Versions: 1.21.0 Reporter: Vladimir Sitnikov Spotless has certain drawbacks: 1) It is not able to verify license headers for non-Java files. For instance, it skips `package-info.java`, it skips `*.kts` and so on :( 2) Its error messages are too verbose. Sometimes it prints the full stacktrace when just one line was enough: "file X line Y column Z has error: ..." 3) It uses unsafe Gradle APIs, so it will be incompatible with Gradle 7.0 I suggest to replace it with https://github.com/autostyle/autostyle Note: I tried to contact Spotless authors, and their way to manage code makes it very hard to co-operate :(( -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-1385) Support password-based convenience logins for Kerberos-authenticated clients
[ https://issues.apache.org/jira/browse/CALCITE-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1385: Fix Version/s: (was: avatica-1.16.0) avatica-1.17.0 > Support password-based convenience logins for Kerberos-authenticated clients > > > Key: CALCITE-1385 > URL: https://issues.apache.org/jira/browse/CALCITE-1385 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.17.0 > > > Had a request for the automatic Kerberos login process to also support > password-based authentication (instead of just keytab). > This would require some extra logic in KerberosConnection to do the Jaas > Configuration with the provided password, but not attempt to prompt the user > for it (as it might be a headless client, like some GUI application). > One problem is that GUI apps will likely try to use pass this value in via > the "password" key in some properties (in addition to the "user" field). Will > actually have to try to test this out to make sure it works as intended. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-1341) Improve mechanism for client/server to unwrap protobuf RPC message
[
https://issues.apache.org/jira/browse/CALCITE-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-1341:
Fix Version/s: (was: avatica-1.16.0)
avatica-1.17.0
> Improve mechanism for client/server to unwrap protobuf RPC message
> --
>
> Key: CALCITE-1341
> URL: https://issues.apache.org/jira/browse/CALCITE-1341
> Project: Calcite
> Issue Type: Improvement
> Components: avatica
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Critical
> Fix For: avatica-1.17.0
>
>
> We just ran into a funny bug in PHOENIX-3136 as a fallout of some changes
> which relocated Avatica classes.
> Because the Avatica RPC classes were also relocated (from
> org.apache.calcite.avatica to org.apache.phoenix.org.apache.calcite.avatica),
> clients of an older version could no longer communicate with a server of the
> newer version (and vice versa). This stems from a decision made where the
> wire API included the class name of the Request and Response Java protobuf
> class in the message. The server would send back the Response class name with
> the relocated name, but the client would not know what that class is (because
> it doesn't have the same relocation). In other words, the current protobuf
> serialization approach requires that Avatica classes are not shaded and
> relocated.
> The JSON serialization doesn't have this problem because it uses the
> {{JsonSubType}} Jackson annotation to map the Request/Response class to a
> logical name (e.g. OpenConnectionResponse to openConnection).
> We could do a similar approach for protobuf that the JSON serialization does
> (maintain this mapping and guarantee that it is not changed incompatibly).
> Long-term, I believe I'd like to see specific Requests dispatched to certain
> URLs (instead of all HTTP requests send to {{/}}) and do away with this
> additional logic in the serialization layer, but I'm not sure if we have to
> re-architect the URL scheme just to fix this issue in the near-term.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-1286) Create self-contained test-harness for TCK
[ https://issues.apache.org/jira/browse/CALCITE-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1286: Fix Version/s: (was: avatica-1.16.0) avatica-1.17.0 > Create self-contained test-harness for TCK > -- > > Key: CALCITE-1286 > URL: https://issues.apache.org/jira/browse/CALCITE-1286 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.17.0 > > > Should make a Vagrant VM or a Docker image which is capable of automatically > running the Avatica TCK. > Ideally, running the TCK should be as simple as a single command. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-2983) Add Avatica compatibility page for TLS and IBM Java
[ https://issues.apache.org/jira/browse/CALCITE-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-2983: Fix Version/s: (was: avatica-1.16.0) avatica-1.17.0 > Add Avatica compatibility page for TLS and IBM Java > --- > > Key: CALCITE-2983 > URL: https://issues.apache.org/jira/browse/CALCITE-2983 > Project: Calcite > Issue Type: Task > Components: avatica, site >Reporter: Kevin Risden >Priority: Major > Fix For: avatica-1.17.0 > > > With the Jetty upgrade in CALCITE-2972, there are some compatibility issues > between TLS support and IBM Java. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-1350) Avoid closeConnection when openConnection doesn't finish/succeed
[ https://issues.apache.org/jira/browse/CALCITE-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1350: Fix Version/s: (was: avatica-1.16.0) avatica-1.17.0 > Avoid closeConnection when openConnection doesn't finish/succeed > > > Key: CALCITE-1350 > URL: https://issues.apache.org/jira/browse/CALCITE-1350 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.17.0 > > > I've noticed during testing of Avatica, often times when SPNEGO > authentication is misconfigured, the client will get stuck in > openConnection(). > If we consider sqlline and the user control-C's it, sqlline will try to close > the driver as well which would do a closeConnection() (which would also > obviously fail). > I believe we should be able to short-circuit the the closeConnection() when > we know that the openConnection() didn't succeed properly. > Another scenario is when the Avatica server is down. openConnection will > fail, but we'll still attempt the closeConnection on exit. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-1164) Use setObject(int, Object, int) when binding parameters
[
https://issues.apache.org/jira/browse/CALCITE-1164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-1164:
Fix Version/s: (was: avatica-1.16.0)
avatica-1.17.0
> Use setObject(int, Object, int) when binding parameters
> ---
>
> Key: CALCITE-1164
> URL: https://issues.apache.org/jira/browse/CALCITE-1164
> Project: Calcite
> Issue Type: Improvement
> Components: avatica
>Reporter: Josh Elser
>Priority: Minor
> Fix For: avatica-1.17.0
>
>
> Trying to capture some discussion from a recent pull request:
> https://github.com/apache/calcite/pull/209#issuecomment-195025402
> In a few places (such as
> https://github.com/apache/calcite/blob/master/avatica/server/src/main/java/org/apache/calcite/avatica/jdbc/JdbcMeta.java#L795-L800),
> we perform:
> {code}
> TypedValue o = parameterValues.get(i);
> preparedStatement.setObject(i + 1, o.toJdbc(calendar));
> {code}
> Vladimir stated that this is ambiguous (stored procedures differing by
> argument list and differentiating between the actual type when the value is
> null) and would be remedied by passing along the desired type when setting
> the object.
> We may also have to invoke setNull explicitly? This is unclear to me.
> h5. Reasons why "explicit sql type" is important
> h6. Calling the proper function
> Consider database has two functions that differ in type of argument only.
> For instance {{compute(varchar)}}, {{compute(numeric)}}, and
> {{compute(user_defined_struct)}}
> Which one should be executed if calling with just
> {{preparedStatement.setObject(i, null)}}?
> There is not enough information for the database to choose between varchar
> and numeric function.
> h6. Performance
> Execution plan depends on the types of bind parameters. For instance, in
> PostgreSQL, you must tell all the datatypes of the bind variables right in
> {{PREPARE}} message.
> That basically means, if you flip between datatypes, you have to use
> different prepared statement IDs.
> If just {{String val = ...; ps.setObject(1, val)}} is used, then for non-null
> it can result in {{String}} execution plan, while for null it can flip to
> unknown.
> Same for batched statement execution. PostgreSQL just cannot handle datatype
> flips right in the middle of the batch. It is handled in the pgjdbc driver,
> so it cuts batch in several sub batches, so it becomes less efficient.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3412) FLOOR(timestamp TO WEEK) gives wrong result
[
https://issues.apache.org/jira/browse/CALCITE-3412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-3412:
Fix Version/s: (was: avatica-1.16.0)
avatica-1.17.0
> FLOOR(timestamp TO WEEK) gives wrong result
> ---
>
> Key: CALCITE-3412
> URL: https://issues.apache.org/jira/browse/CALCITE-3412
> Project: Calcite
> Issue Type: Bug
> Components: avatica, core
>Reporter: huaicui
>Assignee: Julian Hyde
>Priority: Major
> Fix For: 1.22.0, avatica-1.17.0
>
> Attachments: image-2019-10-15-13-33-34-896.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Floor timestamp to week exist error:
> The start day of the week is not Sunday or Monday, calcite use Thursday to
> split a week.
> Example:
> sql: select FLOOR(CAST('2017-01-28' AS TIMESTAMP) TO WEEK);
> Response:
> 2017-01-26 00:00:00.0
>
> 2017-01-26 is Thursday that is not our expect result. As result, 2017-01-22
> is the first day that we expect.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3162) Reading Arrays from Calcite through JdbcMeta generates AvaticaSqlException
[
https://issues.apache.org/jira/browse/CALCITE-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-3162:
Fix Version/s: (was: avatica-1.16.0)
avatica-1.17.0
> Reading Arrays from Calcite through JdbcMeta generates AvaticaSqlException
> --
>
> Key: CALCITE-3162
> URL: https://issues.apache.org/jira/browse/CALCITE-3162
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.15.0
> Environment: Tested on OS X 10.14.5, OpenJDK Runtime Environment
> Zulu11.29+3-CA (build 11.0.2+7-LTS)
> Issue occurs with both Apache Calcite 1.19.0 & 1.20.0.
>Reporter: Ralph Gasser
>Priority: Major
> Labels: pull-request-available
> Fix For: avatica-1.17.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I'm trying to use _Apache Calcite_ as SQL Parser and Query Planner for a
> custom data store and in addition, I'm using _Apache Avatica_ to expose the
> entire functionality via JDBC for an arbitrary, potential remote client to
> use.
> We're working a lot with Array types, since we're using our backend to store
> high-dimensional vectors. However, it seems that currently, Apache Avatica
> has troubles handling such arrays.
> Take the following test-case that reproduces the problem pretty well.
> {code:java}
> @Test
> public void test() throws Exception {
> HttpServer server = null;
> try {
> Class.forName("org.apache.calcite.jdbc.Driver");
> server = new HttpServer.Builder<>()
> .withHandler(new LocalService(new JdbcMeta("jdbc:calcite:",
> newProperties())),Driver.Serialization.PROTOBUF)
> .withPort(1234)
> .build();
> server.start();
> final Connection connection =
> DriverManager.getConnection("jdbc:avatica:remote:serialization=protobuf;url=http://127.0.0.1:1234;);
> final Statement stmt = connection.createStatement();
> final ResultSet resultSet = stmt.executeQuery("select ARRAY[1.0, 1.0,
> 3.0, 2.0]");
> resultSet.next();
> resultSet.getArray(1);
> } catch (Exception e) {
> System.out.println(e.getMessage());
> } finally {
> if (server != null) {
>server.stop();
> }
> }
> }
> {code}
> Executing the statement will throw an AvaticaSqlException:
> {code:java}
> org.apache.calcite.avatica.AvaticaSqlException: Error -1 (0) : Error
> while executing SQL "select ARRAY[1.0, 1.0, 3.0, 2.0]": Remote driver error:
> RuntimeException: java.sql.SQLException: invalid column ordinal: 2 ->
> SQLException: invalid column ordinal: 2
> {code}
> The culprit seems to be the _org.apache.calcite.avatica.jdbc.JdbcResultSet_
> class. More specifically, the _JdbcResultSet#extractUsingResultSet()_ method.
> I am actually testing a potential fix but first I wanted to make sure, that
> there is nothing wrong with the way I'm using the two components.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-1318) Build/Document Avatica Kerberos/SPNEGO authentication behind load balancer.
[ https://issues.apache.org/jira/browse/CALCITE-1318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1318: Fix Version/s: (was: avatica-1.16.0) avatica-1.17.0 > Build/Document Avatica Kerberos/SPNEGO authentication behind load balancer. > --- > > Key: CALCITE-1318 > URL: https://issues.apache.org/jira/browse/CALCITE-1318 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.17.0 > > > I was talking with [~_alexm] this weekend about some work that he had > recently done in getting Apache Impala set up behind a load balancer. When > Kerberos is in the picture, he told me that the way that works is that the > impalad daemons actually have two Kerberos identities: one for the hostname > that the impalad service is actually running on and another for the load > balancer host. The load-balancer continues to just do a simple pass-through. > Right now, the Avatica server can only accept a single Kerberos > principal+keytab. This means that we can't use the Kerberos authentication > when the client can access the server via multiple hostnames -- invalidating > the use of 'dumb' load balancers (hypothetically, a smart loadbalancer could > make it work). We could configure the Avatica server to use a principal with > the load-balancer's hostname, but then users would be unable to connect > directly to the server. > I know that Impala uses (or at least exposes) Thrift which has its own SASL > implementation; maybe they do something tricky there? Maybe we can glean > something from their implementation (even though it's not HTTP based). I > don't think JAAS lets us have multiple active logins, so I'm not even sure > where to begin. > Ideally, this is something that would be great to understand and provide some > deployment guidance for users to have identical deployment scenario for > "secure" and "unsecure" scenarios. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-1244) `offset` is ignored in FetchRequest
[
https://issues.apache.org/jira/browse/CALCITE-1244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-1244:
Fix Version/s: (was: avatica-1.16.0)
avatica-1.17.0
> `offset` is ignored in FetchRequest
> ---
>
> Key: CALCITE-1244
> URL: https://issues.apache.org/jira/browse/CALCITE-1244
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Reporter: Josh Elser
>Priority: Major
> Fix For: avatica-1.17.0
>
>
> In looking into some issues reported by [~onpduo], we both noticed that
> {{FetchRequest.offset}} is never actually used by the server.
> We should make sure the semantics on this value are clear (is it an offset
> from the beginning of the ResultSet or the current position of the
> ResultSet?) and make sure it is used.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-1928) Calling previous() on a ResultSet that came from an Array gives NullPointerException
[ https://issues.apache.org/jira/browse/CALCITE-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1928: Fix Version/s: (was: avatica-1.16.0) avatica-1.17.0 > Calling previous() on a ResultSet that came from an Array gives > NullPointerException > > > Key: CALCITE-1928 > URL: https://issues.apache.org/jira/browse/CALCITE-1928 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Julian Hyde >Priority: Major > Fix For: avatica-1.17.0 > > > Calling previous() on a ResultSet that came from an Array gives > NullPointerException. I broke this when I committed CALCITE-1902 and changed > 'Helper.INSTANCE' to 'statement.connection.helper', forgetting that > 'statement' can be null. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-1352) Clarify documentation for avatica's max_rows_total
[
https://issues.apache.org/jira/browse/CALCITE-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-1352:
Fix Version/s: (was: avatica-1.16.0)
avatica-1.17.0
> Clarify documentation for avatica's max_rows_total
> --
>
> Key: CALCITE-1352
> URL: https://issues.apache.org/jira/browse/CALCITE-1352
> Project: Calcite
> Issue Type: Improvement
> Components: avatica
>Affects Versions: avatica-1.8.0
>Reporter: Francis Chuang
>Priority: Minor
> Fix For: avatica-1.17.0
>
>
> The {{max_rows_total}} parameter in the protocol buffer definitions should be
> updated to include more information on values that result in unlimited rows
> being returned.
> When testing against calcite 1.8.0, I observed the following:
> {code}
> -1: Unlimited number of rows
> 0: 0 rows
> 1: 1 row
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3163) Mapping of Types in AbstractCursor#convertValue() does not adhere to JDBC specifications
[ https://issues.apache.org/jira/browse/CALCITE-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-3163: Fix Version/s: (was: avatica-1.16.0) avatica-1.17.0 > Mapping of Types in AbstractCursor#convertValue() does not adhere to JDBC > specifications > > > Key: CALCITE-3163 > URL: https://issues.apache.org/jira/browse/CALCITE-3163 > Project: Calcite > Issue Type: Bug > Components: avatica >Affects Versions: avatica-1.15.0 >Reporter: Ralph Gasser >Priority: Major > Labels: pull-request-available > Fix For: avatica-1.17.0 > > Attachments: patch0001.diff > > Time Spent: 10m > Remaining Estimate: 0h > > This is an issue I found while working on CALCITE-3162: According to my > understanding, the method _AbstractCursor#convertValue()_, used for > converting the component types of ARRAYs, does not adhere to what is > specified in the [JDBC specification (Appendix > B)|https://download.oracle.com/otn-pub/jcp/jdbc-4_1-mrel-spec/jdbc4.1-fr-spec.pdf?AuthParam=1561978725_eefb6fb1703d96f182674508c44126ea] > regarding mapping of types: > * The JDBC types FLOAT (6) and DOUBLE (8) should actually both map to Java > _double_ or _Double_ respectively. Currently, FLOAT (6) is converted to a > _float_. > * The implementation is currently missing support for JDBC type REAL (7), > which should convert to _float_ or _Float_ respectively > This issue will cause an IllegalStateException to be thrown in clients using > the Avatica Driver if they encounter REAL datatype in arrays. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-1308) Implement remaining DatabaseMetaData operations in RemoteMeta
[ https://issues.apache.org/jira/browse/CALCITE-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1308: Fix Version/s: (was: avatica-1.16.0) avatica-1.17.0 > Implement remaining DatabaseMetaData operations in RemoteMeta > - > > Key: CALCITE-1308 > URL: https://issues.apache.org/jira/browse/CALCITE-1308 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Critical > Fix For: avatica-1.17.0 > > > Noticed in CALCITE-1291: sqlline normally highlights the column(s) which are > (part of the) primary key. Running a debugger over it quickly, showed that no > keys were returned over the DatabaseMetaData.getPrimaryKeys call. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3324) Infinite loop when calling DDL from remote Avatica driver to LocalService wrapping CalciteMetaImpl
[
https://issues.apache.org/jira/browse/CALCITE-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-3324:
Fix Version/s: (was: avatica-1.16.0)
avatica-1.17.0
> Infinite loop when calling DDL from remote Avatica driver to LocalService
> wrapping CalciteMetaImpl
> --
>
> Key: CALCITE-3324
> URL: https://issues.apache.org/jira/browse/CALCITE-3324
> Project: Calcite
> Issue Type: Bug
> Components: avatica, core
>Affects Versions: 1.20.0
>Reporter: Robert Yokota
>Priority: Major
> Labels: pull-request-available
> Fix For: avatica-1.17.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> I encountered an infinite loop when submitting DDL from a remote Avatica
> driver (using sqlline) against a {{LocalService}} wrapping
> {{CalciteMetaImpl}}.
> I was able to debug the issue and verified a fix.
> The problem was that in {{CalciteMetaImpl.prepareAndExecute}}, the
> updateCount is not being passed to the {{MetaResultSet}} constructor.
> Instead, the updateCount is always set to -1. This incorrectly instructs the
> remote Avatica driver to fetch results when there aren't any, which causes it
> to loop.
> By fixing the {{CalciteMetaImpl.prepareAndExecute}} method to set the
> updateCount in {{MetaResultSet}}, I was able to fix the infinite loop.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-1242) Allow configuration of maximum allowed value for maxRowCount
[ https://issues.apache.org/jira/browse/CALCITE-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1242: Fix Version/s: (was: avatica-1.16.0) avatica-1.17.0 > Allow configuration of maximum allowed value for maxRowCount > > > Key: CALCITE-1242 > URL: https://issues.apache.org/jira/browse/CALCITE-1242 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Duo Xu >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.17.0 > > > There are several APIs having the maxRowCount parameter. Currently this > setting is hard coded in Avatica as 100. So if the user set maxRowCount > 100 > in PrepareAndExecuteRequest, for example, the server will still only return > 100 results. So the APIs are currently broken. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-1811) Work around "latest" tag for Docker images
[
https://issues.apache.org/jira/browse/CALCITE-1811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-1811:
Fix Version/s: (was: avatica-1.16.0)
avatica-1.17.0
> Work around "latest" tag for Docker images
> --
>
> Key: CALCITE-1811
> URL: https://issues.apache.org/jira/browse/CALCITE-1811
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.10.0
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Minor
> Fix For: avatica-1.17.0
>
>
> One wrinkle with the dockerhub integration with the ASF is that we only get a
> tag for the explicit version we released (e.g. 1.10.0).
> However, in the other Dockerfiles we have in the 1.10.0 release, I had
> (incorrectly) relied on that. Need to come up with a plan for how to better
> manage this in the future. I think that we need to just have {{FROM
> apache/calcite-avatica:$tag}} everywhere, relying on the parent eventually
> being published to dockerhub by the ASF infra.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-839) Remove Jackson annotations from POJO classes in Meta
[ https://issues.apache.org/jira/browse/CALCITE-839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-839: --- Fix Version/s: (was: avatica-1.16.0) avatica-1.17.0 > Remove Jackson annotations from POJO classes in Meta > > > Key: CALCITE-839 > URL: https://issues.apache.org/jira/browse/CALCITE-839 > Project: Calcite > Issue Type: Bug > Components: avatica >Reporter: Julian Hyde >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.17.0 > > > The Meta interface contains several POJO classes that represent RPC requests > or responses. Currently a few of those classes have Jackson annotations such > as @JsonCreator, @JsonProperty to help Jackson serialize the POJO to JSON and > de-serialize from JSON to the object. > As [~ndimiduk] pointed out in > http://mail-archives.apache.org/mod_mbox/calcite-dev/201503.mbox/%3CCANZa=gvkgd+bkj4+ejmuo6ivhs+okgskg1vwdazcy-zijyy...@mail.gmail.com%3E > these annotations are a "code smell" and should be removed. It makes it look > as if Jackson is the only possible transport, which is not the case. We can > continue to use Jackson as a transport, just specify the mappings elsewhere, > not as annotations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-2704) Multilingual decoded problem
[
https://issues.apache.org/jira/browse/CALCITE-2704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Chuang updated CALCITE-2704:
Fix Version/s: (was: avatica-1.16.0)
avatica-1.17.0
> Multilingual decoded problem
>
>
> Key: CALCITE-2704
> URL: https://issues.apache.org/jira/browse/CALCITE-2704
> Project: Calcite
> Issue Type: Bug
> Components: avatica
>Affects Versions: avatica-1.11.0, avatica-1.12.0, avatica-1.13.0
>Reporter: pufan
>Priority: Blocker
> Labels: pull-request-available
> Fix For: avatica-1.17.0
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> When we execute SQL with Chinese characters, we find that the server parses
> it in gibberish.
> After checking the code, we found that:
> org.apache.calcite.avatica.server.AvaticaJsonHandler.java line 109：
> {code:java}
> //The readFully method USES utf-8 encoding internally.
> rawRequest = AvaticaUtils.readFully(inputStream, buffer);
> {code}
> But in the org.apache.calcite.avatica.server.AvaticaJsonHandler.java line 116：
> {code:java}
> // Decoded using iso-8859-1 Cause Chinese garble.
> final String jsonRequest = new String(rawRequest.getBytes("ISO-8859-1"),
> "UTF-8");
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-1316) Better control over retried operations in Avatica client
[ https://issues.apache.org/jira/browse/CALCITE-1316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-1316: Fix Version/s: (was: avatica-1.16.0) avatica-1.17.0 > Better control over retried operations in Avatica client > > > Key: CALCITE-1316 > URL: https://issues.apache.org/jira/browse/CALCITE-1316 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: avatica-1.17.0 > > > We have at least two places in the Avatica client now where we will try to > re-issue "RPCs" in the attempt to work seamlessly with load-balanced servers. > Between these two places, we have finite retries, infinite retries and no > standardized back-off strategies. We should try to centralize this into one > place and make sure that users can override the logic, heaven forbid they > come up with some situation where it's necessary. > Need to investigate the retry-loops we have in the Avatica client now, > categorize the loops and come up with a minimal set of knobs to configure the > retries, expose those knobs in the JDBC URL string options, and update the > documentation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3610) Fix avatica dockerfiles
[ https://issues.apache.org/jira/browse/CALCITE-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang updated CALCITE-3610: Fix Version/s: avatica-1.17.0 > Fix avatica dockerfiles > --- > > Key: CALCITE-3610 > URL: https://issues.apache.org/jira/browse/CALCITE-3610 > Project: Calcite > Issue Type: Task > Components: avatica >Reporter: Francis Chuang >Assignee: Francis Chuang >Priority: Major > Fix For: avatica-1.17.0 > > > The dockerfiles current reference vatica-standalone-server-1.16.0-shaded.jar. > These jars no longer exist and are now avatica-standalone-server-1.16.0.jar. > The dockerifles should be updated to point to these new filenames. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (CALCITE-3610) Fix avatica dockerfiles
[ https://issues.apache.org/jira/browse/CALCITE-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Chuang resolved CALCITE-3610. - Resolution: Fixed > Fix avatica dockerfiles > --- > > Key: CALCITE-3610 > URL: https://issues.apache.org/jira/browse/CALCITE-3610 > Project: Calcite > Issue Type: Task > Components: avatica >Reporter: Francis Chuang >Assignee: Francis Chuang >Priority: Major > Fix For: avatica-1.17.0 > > > The dockerfiles current reference vatica-standalone-server-1.16.0-shaded.jar. > These jars no longer exist and are now avatica-standalone-server-1.16.0.jar. > The dockerifles should be updated to point to these new filenames. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3621) JDBC adapter can't push down sort to DB
[
https://issues.apache.org/jira/browse/CALCITE-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001957#comment-17001957
]
Jin Xing commented on CALCITE-3621:
---
# Why the change in
[RelToSqlConverter.java|https://github.com/apache/calcite/pull/1680/files/3c0dd3f7cd750cd9af2035726a131464bbac229f#diff-79ccdc597ccf355912fea1a60683eea2]
is necessary ?
# How about update the change in JdbcSortRule as
{code:java}
final RelTraitSet inputTraitSet = sort.getInput().getTraitSet().replace(out);
{code}
> JDBC adapter can't push down sort to DB
> ---
>
> Key: CALCITE-3621
> URL: https://issues.apache.org/jira/browse/CALCITE-3621
> Project: Calcite
> Issue Type: Bug
> Components: jdbc-adapter
>Affects Versions: 1.21.0
>Reporter: Lei Jiang
>Assignee: Lei Jiang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> JDBC adapter can't push down sort to DB
> {code:java}
> select ename from scott.emp order by empno
> {code}
> {code:java}
> PLAN=EnumerableSort(sort0=[$1], dir0=[ASC])
> JdbcToEnumerableConverter
> JdbcProject(ENAME=[$1], EMPNO=[$0])
> JdbcTableScan(table=[[SCOTT, EMP]])
> {code}
> It should be:
> {code:java}
> PLAN=JdbcToEnumerableConverter
> JdbcSort(sort0=[$1], dir0=[ASC])
> JdbcProject(ENAME=[$1], EMPNO=[$0])
> JdbcTableScan(table=[[SCOTT, EMP]])
> {code}
> I think the root cause is JdbcSortRule, it convert input's trait to "JDBC,
> {color:#ff}[1]{color}". that is, input's relset will add a "JDBC, [1]"
> subset. But there is nothing rule can convert that input to a rel with "JDBC,
> {color:#ff}[1]{color}", so EnumerableSort win.
> {code:java}
> public RelNode convert(Sort sort, boolean convertInputTraits) {
> final RelTraitSet traitSet = sort.getTraitSet().replace(out);
> final RelNode input;
> if (convertInputTraits) {
> input = convert(sort.getInput(), traitSet);
> } else {
> input = sort.getInput();
> }
> return new JdbcSort(sort.getCluster(), traitSet,
> input, sort.getCollation(), sort.offset, sort.fetch);
> }
> {code}
> This is my a part of change: convert input's trait to "JDBC, []"
> {code:java}
> public RelNode convert(Sort sort, boolean convertInputTraits) {
> final RelTraitSet traitSet = sort.getTraitSet().replace(out);
> //update
> RelTraitSet inputTraitSet = traitSet.replace(RelCollations.EMPTY);
> final RelNode input;
> if (convertInputTraits) {
> //update
> input = convert(sort.getInput(), inputTraitSet);
> } else {
> input = sort.getInput();
> }
> return new JdbcSort(sort.getCluster(), traitSet,
> input, sort.getCollation(), sort.offset, sort.fetch);
> }
> {code}
> I have updated some classes to reslove this issue
> When JdbcSort is used, my solution will trigger a regression issue
> JdbcTest#testSelfJoinDifferentColumns(): Project->Sort->Join(left.A,
> right.A), Join.left and Join.right have same field A. My solution: select
> list of Sort contains explicit column name instead of *
>
> {code:java}
> RelToSqlConverter#visit(Sort e)
> ...
> Result x = visitChild(0, e.getInput());
> Builder builder = x.builder(e, Clause.ORDER_BY);
> //my update
> if (builder.select.getSelectList() == null) {
> final List selectList = Expressions.list();
> for (RelDataTypeField field : e.getRowType().getFieldList()) {
> addSelect(selectList, builder.context.field(field.getIndex()),
> e.getRowType());
> }
> builder.select.setSelectList(new SqlNodeList(selectList, POS));
> }
> //end
> ...{code}
> Before:
> SELECT "t2"."full_name", "t2"."last_name{color:#ff}0{color}" AS
> "last_name"
> FROM (SELECT {color:#ff}*{color}
> FROM (SELECT "last_name"
> FROM "foodmart"."employee") AS "t"
> INNER JOIN (SELECT "full_name", "first_name", "last_name"
> FROM "foodmart"."employee") AS "t0" ON "t"."last_name" = "t0"."first_name"
> ORDER BY "t0"."last_name" NULLS LAST
> After:
> SELECT "t2"."full_name", "t2"."last_name0" AS "last_name"
> FROM (SELECT {color:#ff}"t"."last_name", "t0"."full_name",
> "t0"."first_name", "t0"."last_name" AS "last_name0"{color}
> FROM (SELECT "last_name"
> FROM "foodmart"."employee") AS "t"
> INNER JOIN (SELECT "full_name", "first_name", "last_name"
> FROM "foodmart"."employee") AS "t0" ON "t"."last_name" = "t0"."first_name"
> ORDER BY "t0"."last_name" NULLS LAST
> LIMIT 3) AS "t2"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (CALCITE-3622) Update geode tests upgrade from junit4 to junit5
[ https://issues.apache.org/jira/browse/CALCITE-3622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu reassigned CALCITE-3622: --- Assignee: Forward Xu > Update geode tests upgrade from junit4 to junit5 > > > Key: CALCITE-3622 > URL: https://issues.apache.org/jira/browse/CALCITE-3622 > Project: Calcite > Issue Type: Improvement >Reporter: Forward Xu >Assignee: Forward Xu >Priority: Major > > Update `geode` tests upgrade from junit4 to junit5. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (CALCITE-3622) Update geode tests upgrade from junit4 to junit5
Forward Xu created CALCITE-3622: --- Summary: Update geode tests upgrade from junit4 to junit5 Key: CALCITE-3622 URL: https://issues.apache.org/jira/browse/CALCITE-3622 Project: Calcite Issue Type: Improvement Reporter: Forward Xu Update `geode` tests upgrade from junit4 to junit5. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CALCITE-3609) Wrong query results in Hive due to wrong struct nullability
[
https://issues.apache.org/jira/browse/CALCITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001878#comment-17001878
]
hezhang commented on CALCITE-3609:
--
[~zabetak] thanks (y)
> Wrong query results in Hive due to wrong struct nullability
>
>
> Key: CALCITE-3609
> URL: https://issues.apache.org/jira/browse/CALCITE-3609
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.10.0, 1.16.0
>Reporter: hezhang
>Priority: Blocker
> Attachments: calcite-3608.patch
>
>
> student table：
> {code:java}
> CREATE TABLE `student`(
> `id` int,
> `info` struct)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
> WITH SERDEPROPERTIES (
> 'colelction.delim'=':',
> 'field.delim'=',',
> 'serialization.format'=',',
> 'serialization.null.format'='NULL')
> {code}
> the sql:
>
> {code:java}
> select * from student where info is not null;
> {code}
> result:
>
> {code:java}
> 1 {"name":"zhou","age":30}
> 2 {"name":"yan","age":30}
> 3 {"name":"chen","age":20}
> 4 {"name":"li","age":80}
> NULLNULL
> NULL{"name":null,"age":null}
> {code}
> cause:calcite Ineffective optimization
> {code:java}
> HiveProject(id=[$0], info=[$1])
> HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
>
> after fix RelRecordType isNullable,the plan is
> {code:java}
> HiveProject(id=[$0], info=[$1])
> HiveFilter(condition=[IS NOT NULL($1)])
> HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
> then the result is:
> {code:java}
> 1 {"name":"zhou","age":30}
> 2 {"name":"yan","age":30}
> 3 {"name":"chen","age":20}
> 4 {"name":"li","age":80}
> NULL{"name":null,"age":null}
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CALCITE-3609) Wrong query results in Hive due to wrong struct nullability
[
https://issues.apache.org/jira/browse/CALCITE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001877#comment-17001877
]
hezhang commented on CALCITE-3609:
--
[~amaliujia] thanks for your advice :D
> Wrong query results in Hive due to wrong struct nullability
>
>
> Key: CALCITE-3609
> URL: https://issues.apache.org/jira/browse/CALCITE-3609
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.10.0, 1.16.0
>Reporter: hezhang
>Priority: Blocker
> Attachments: calcite-3608.patch
>
>
> student table：
> {code:java}
> CREATE TABLE `student`(
> `id` int,
> `info` struct)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
> WITH SERDEPROPERTIES (
> 'colelction.delim'=':',
> 'field.delim'=',',
> 'serialization.format'=',',
> 'serialization.null.format'='NULL')
> {code}
> the sql:
>
> {code:java}
> select * from student where info is not null;
> {code}
> result:
>
> {code:java}
> 1 {"name":"zhou","age":30}
> 2 {"name":"yan","age":30}
> 3 {"name":"chen","age":20}
> 4 {"name":"li","age":80}
> NULLNULL
> NULL{"name":null,"age":null}
> {code}
> cause:calcite Ineffective optimization
> {code:java}
> HiveProject(id=[$0], info=[$1])
> HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
>
> after fix RelRecordType isNullable,the plan is
> {code:java}
> HiveProject(id=[$0], info=[$1])
> HiveFilter(condition=[IS NOT NULL($1)])
> HiveTableScan(table=[[default.student]], table:alias=[student])
> {code}
> then the result is:
> {code:java}
> 1 {"name":"zhou","age":30}
> 2 {"name":"yan","age":30}
> 3 {"name":"chen","age":20}
> 4 {"name":"li","age":80}
> NULL{"name":null,"age":null}
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (CALCITE-3601) Update elasticsearch tests upgrade from junit4 to junit5
[ https://issues.apache.org/jira/browse/CALCITE-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CALCITE-3601: Labels: pull-request-available (was: ) > Update elasticsearch tests upgrade from junit4 to junit5 > > > Key: CALCITE-3601 > URL: https://issues.apache.org/jira/browse/CALCITE-3601 > Project: Calcite > Issue Type: Test >Reporter: Forward Xu >Assignee: Forward Xu >Priority: Major > Labels: pull-request-available > > Update `elasticsearch` tests upgrade from junit4 to junit5. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3601) Update elasticsearch tests upgrade from junit4 to junit5
[ https://issues.apache.org/jira/browse/CALCITE-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu updated CALCITE-3601: Summary: Update elasticsearch tests upgrade from junit4 to junit5 (was: Update elasticsearch test upgrade from junit4 to junit5) > Update elasticsearch tests upgrade from junit4 to junit5 > > > Key: CALCITE-3601 > URL: https://issues.apache.org/jira/browse/CALCITE-3601 > Project: Calcite > Issue Type: Test >Reporter: Forward Xu >Assignee: Forward Xu >Priority: Major > > Update `elasticsearch` tests upgrade from junit4 to junit5. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3601) Update elasticsearch test upgrade from junit4 to junit5
[ https://issues.apache.org/jira/browse/CALCITE-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu updated CALCITE-3601: Description: Update `elasticsearch` tests upgrade from junit4 to junit5. (was: Update `cassandra, elasticsearch, geode, linq4j, and mongodb` tests upgrade from junit4 to junit5.) > Update elasticsearch test upgrade from junit4 to junit5 > --- > > Key: CALCITE-3601 > URL: https://issues.apache.org/jira/browse/CALCITE-3601 > Project: Calcite > Issue Type: Test >Reporter: Forward Xu >Assignee: Forward Xu >Priority: Major > > Update `elasticsearch` tests upgrade from junit4 to junit5. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CALCITE-3601) Update elasticsearch test upgrade from junit4 to junit5
[ https://issues.apache.org/jira/browse/CALCITE-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Forward Xu updated CALCITE-3601: Summary: Update elasticsearch test upgrade from junit4 to junit5 (was: Update mongodb test upgrade from junit4 to junit5) > Update elasticsearch test upgrade from junit4 to junit5 > --- > > Key: CALCITE-3601 > URL: https://issues.apache.org/jira/browse/CALCITE-3601 > Project: Calcite > Issue Type: Test >Reporter: Forward Xu >Assignee: Forward Xu >Priority: Major > > Update `cassandra, elasticsearch, geode, linq4j, and mongodb` tests upgrade > from junit4 to junit5. -- This message was sent by Atlassian Jira (v8.3.4#803005)
