[
https://issues.apache.org/jira/browse/CALCITE-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Lee updated CALCITE-5964:
--------------------------------
Description:
The goal is for Avatica to be able to handle additional columns in the standard
JDBC getTables() and getColumns() responses.
After testing, it appears that the Avatica client in RemoteMeta is able to
handle Signatures of different shapes that don't necessarily conform to the
JDBC standard
[getTables.|https://docs.oracle.com/javase/8/docs/api/java/sql/ResultSet.html#getTables-java.lang.String-java.lang.String-java.lang.String-java.lang.String:A-)]
I have opened a PR that adds a test to RemoteDriverMockTest that verifies that
if a Signature comes in with additional columns, that Avatica is able to handle
it fluently.
This ticket is related to CALCITE-5982 , and includes some additions to
connection properties to allow Calcite to create and send Signatures of
different shapes.
-The goal is to add to Avatica a mechanism such that additional metadata fields
pertaining to tables and columns can be transmitted alongside the standard JDBC
[getTables|https://docs.oracle.com/javase/8/docs/api/java/sql/ResultSet.html#getTables-java.lang.String-java.lang.String-java.lang.String-java.lang.String:A-)]
and {{getColumns}} calls.-
-The Avatica client needs the response to be extensible such that revisions to
metadata fields send and future additions does not require a new JAR file.-
-Requirements:-
# -Avatica user does not need to download new jar files if the server decides
to send over new metadata data in the future-
# -If the client makes modifications to support additional columns, they
should always be present in the call and appear with null values, as opposed to
complete omission (Number of columns in response stays the same)-
# -Can handle attributes of varying types i.e. {{numberOne: int}} and
{{booleanOne: boolean}}-
# -Allows value retrieval from the {{ResultSet}} through calling
{{resultSet.getInt(“booleanOne”)}} or {{resultSet.getBoolean(“booleanOne”)}}-
-Current proposal is to modify the {{MetaTable}} and {{MetaColumn}} classes to
include a map.-
-{{{}HashMap<String, Object>{}}}, such that when instantiating the
{{CalciteMetaTable}} in the {{{}ResultSet{}}}, new entries could be added in
the future without changes to Avatica.-
-One we have a list of additional metadata fields to be emitted in
{{{}CalciteMetaImpl{}}}, the {{ResultSet}} would be created with the
appropriate values.-
--
-There are still some challenges identified below and I would love some input:-
-Challenges:-
* -Currently the {{MetaTable}} class that is instantiated is a
{{{}CalciteMetaImpl{}}}. For the {{getTables()}} call, the response will be a
list composed of schema tables of class {{CalciteMetaTable}} and database
tables which can potentially be overloaded into 1 or more different subclasses.
From this one heterogeneous list, we must determine the full list of columns to
be included in the additional metadata hash. My initial plan was to provide a
function in Calcite’s {{Table}} class such as {{getAdditionalColumns}} and
allow it to be overloaded, but then I discovered the heterogeneity of the list.-
* -Modifying the MetaTable class to include the hashmap of values could be
easily done, but the challenge lies at {{{}RemoteMeta{}}}, to be able to
serialize this cleanly so that requirement (4) is met and users can retrieve
the values nicely. {{RemoteMeta}} currently serializes the response using
reflection by looking at MetaTable.class and its attributes. The addition of
one map is not immediately compatible with iterating over the keys of the map
and turning each of those into fields. I’m looking into the idea of processing
the enumerable in {{CalciteMetaImpl}} before the Frame gets created-
was:
The goal is for Avatica to be able to handle additional columns in the standard
JDBC getTables() and getColumns() responses.
After testing, it appears that the Avatica client in RemoteMeta is able to
handle Signatures of different shapes that don't necessarily conform to the
JDBC standard
[getTables.|https://docs.oracle.com/javase/8/docs/api/java/sql/ResultSet.html#getTables-java.lang.String-java.lang.String-java.lang.String-java.lang.String:A-)]
I have opened a PR that adds a test to RemoteDriverMockTest that verifies that
if a Signature comes in with additional columns, that Avatica is able to handle
it fluently.
-The goal is to add to Avatica a mechanism such that additional metadata fields
pertaining to tables and columns can be transmitted alongside the standard JDBC
[getTables|https://docs.oracle.com/javase/8/docs/api/java/sql/ResultSet.html#getTables-java.lang.String-java.lang.String-java.lang.String-java.lang.String:A-)]
and {{getColumns}} calls.-
-The Avatica client needs the response to be extensible such that revisions to
metadata fields send and future additions does not require a new JAR file.-
-Requirements:-
# -Avatica user does not need to download new jar files if the server decides
to send over new metadata data in the future-
# -If the client makes modifications to support additional columns, they
should always be present in the call and appear with null values, as opposed to
complete omission (Number of columns in response stays the same)-
# -Can handle attributes of varying types i.e. {{numberOne: int}} and
{{booleanOne: boolean}}-
# -Allows value retrieval from the {{ResultSet}} through calling
{{resultSet.getInt(“booleanOne”)}} or {{resultSet.getBoolean(“booleanOne”)}}-
-Current proposal is to modify the {{MetaTable}} and {{MetaColumn}} classes to
include a map.-
-{{{}HashMap<String, Object>{}}}, such that when instantiating the
{{CalciteMetaTable}} in the {{{}ResultSet{}}}, new entries could be added in
the future without changes to Avatica.-
-One we have a list of additional metadata fields to be emitted in
{{{}CalciteMetaImpl{}}}, the {{ResultSet}} would be created with the
appropriate values.-
--
-There are still some challenges identified below and I would love some input:-
-Challenges:-
* -Currently the {{MetaTable}} class that is instantiated is a
{{{}CalciteMetaImpl{}}}. For the {{getTables()}} call, the response will be a
list composed of schema tables of class {{CalciteMetaTable}} and database
tables which can potentially be overloaded into 1 or more different subclasses.
From this one heterogeneous list, we must determine the full list of columns to
be included in the additional metadata hash. My initial plan was to provide a
function in Calcite’s {{Table}} class such as {{getAdditionalColumns}} and
allow it to be overloaded, but then I discovered the heterogeneity of the list.-
* -Modifying the MetaTable class to include the hashmap of values could be
easily done, but the challenge lies at {{{}RemoteMeta{}}}, to be able to
serialize this cleanly so that requirement (4) is met and users can retrieve
the values nicely. {{RemoteMeta}} currently serializes the response using
reflection by looking at MetaTable.class and its attributes. The addition of
one map is not immediately compatible with iterating over the keys of the map
and turning each of those into fields. I’m looking into the idea of processing
the enumerable in {{CalciteMetaImpl}} before the Frame gets created-
> Support additional metadata attributes in GET_TABLES and GET_COLUMNS
> --------------------------------------------------------------------
>
> Key: CALCITE-5964
> URL: https://issues.apache.org/jira/browse/CALCITE-5964
> Project: Calcite
> Issue Type: New Feature
> Components: avatica
> Reporter: Oliver Lee
> Assignee: Oliver Lee
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The goal is for Avatica to be able to handle additional columns in the
> standard JDBC getTables() and getColumns() responses.
> After testing, it appears that the Avatica client in RemoteMeta is able to
> handle Signatures of different shapes that don't necessarily conform to the
> JDBC standard
> [getTables.|https://docs.oracle.com/javase/8/docs/api/java/sql/ResultSet.html#getTables-java.lang.String-java.lang.String-java.lang.String-java.lang.String:A-)]
> I have opened a PR that adds a test to RemoteDriverMockTest that verifies
> that if a Signature comes in with additional columns, that Avatica is able to
> handle it fluently.
>
> This ticket is related to CALCITE-5982 , and includes some additions to
> connection properties to allow Calcite to create and send Signatures of
> different shapes.
>
>
>
>
> -The goal is to add to Avatica a mechanism such that additional metadata
> fields pertaining to tables and columns can be transmitted alongside the
> standard JDBC
> [getTables|https://docs.oracle.com/javase/8/docs/api/java/sql/ResultSet.html#getTables-java.lang.String-java.lang.String-java.lang.String-java.lang.String:A-)]
> and {{getColumns}} calls.-
> -The Avatica client needs the response to be extensible such that revisions
> to metadata fields send and future additions does not require a new JAR
> file.-
> -Requirements:-
> # -Avatica user does not need to download new jar files if the server
> decides to send over new metadata data in the future-
> # -If the client makes modifications to support additional columns, they
> should always be present in the call and appear with null values, as opposed
> to complete omission (Number of columns in response stays the same)-
> # -Can handle attributes of varying types i.e. {{numberOne: int}} and
> {{booleanOne: boolean}}-
> # -Allows value retrieval from the {{ResultSet}} through calling
> {{resultSet.getInt(“booleanOne”)}} or {{resultSet.getBoolean(“booleanOne”)}}-
> -Current proposal is to modify the {{MetaTable}} and {{MetaColumn}} classes
> to include a map.-
> -{{{}HashMap<String, Object>{}}}, such that when instantiating the
> {{CalciteMetaTable}} in the {{{}ResultSet{}}}, new entries could be added in
> the future without changes to Avatica.-
> -One we have a list of additional metadata fields to be emitted in
> {{{}CalciteMetaImpl{}}}, the {{ResultSet}} would be created with the
> appropriate values.-
> --
> -There are still some challenges identified below and I would love some
> input:-
> -Challenges:-
> * -Currently the {{MetaTable}} class that is instantiated is a
> {{{}CalciteMetaImpl{}}}. For the {{getTables()}} call, the response will be a
> list composed of schema tables of class {{CalciteMetaTable}} and database
> tables which can potentially be overloaded into 1 or more different
> subclasses. From this one heterogeneous list, we must determine the full list
> of columns to be included in the additional metadata hash. My initial plan
> was to provide a function in Calcite’s {{Table}} class such as
> {{getAdditionalColumns}} and allow it to be overloaded, but then I discovered
> the heterogeneity of the list.-
> * -Modifying the MetaTable class to include the hashmap of values could be
> easily done, but the challenge lies at {{{}RemoteMeta{}}}, to be able to
> serialize this cleanly so that requirement (4) is met and users can retrieve
> the values nicely. {{RemoteMeta}} currently serializes the response using
> reflection by looking at MetaTable.class and its attributes. The addition of
> one map is not immediately compatible with iterating over the keys of the map
> and turning each of those into fields. I’m looking into the idea of
> processing the enumerable in {{CalciteMetaImpl}} before the Frame gets
> created-
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
