RE: Re: Ability to access Calcite JDBC remotely from another JDBC client such as DBeaver/DataGrip
Thank you, TJ and Gavin.
Appreciate you taking the time to provide such helpful suggestions.
I will try these out and revert, in case I face any more issues.
Best Regards,
Raja Senapati
On 2023/11/08 18:53:54 TJ Banghart wrote:
> Hello Raja and welcome to Calcite!
>
> I have found it helpful to think of an Avatica server in terms of its main
> interfaces Service
> <
https://github.com/apache/calcite-avatica/blob/519d1ceeb04cd99530bceb60c1a8e0966c413541/core/src/main/java/org/apache/calcite/avatica/remote/Service.java#L59C7-L59C7
>
> and Meta
> <
https://github.com/apache/calcite-avatica/blob/519d1ceeb04cd99530bceb60c1a8e0966c413541/core/src/main/java/org/apache/calcite/avatica/Meta.java#L53
>.
> The service implementation determines what RPC message format to use as
> well as what RPCs are available. The meta implementation determines how
the
> server retrieves metadata, executes queries, etc.
> When starting the server we can supply both the service and meta
> implementations we want to use. In this case we want the server to access
> Calcite locally while using the generic Avatica driver from a JDBC client
> like DBeaver. For example (in Kotlin):
>
> class MyJdbcServer(port: Int) {
> val meta: Meta = MyCalciteMetaFactory().create()
> val service: Service = LocalService(meta)
> val server: HttpServer = HttpServer(port, AvaticaJsonHandler(service))
>
> ... methods to start/stop server etc...
> }
>
> class MyCalciteMetaFactory() : Meta.Factory {
> override fun create(args: List?): Meta {
> val info: Properties = ...some method to generate Calcite
> connection props...
> val driver: Driver = Driver()
> val connection: AvaticaConnection =
driver.connect("jdbc:calcite:",
> info) as AvaticaConnection?
>
> return driver.createMeta(connection)
> }
> }
>
>
> All RPCs from the JDBC client using the generic Avatica driver should be
> handled by the local CalciteMetaImpl
> <
https://github.com/apache/calcite/blob/main/core/src/main/java/org/apache/calcite/jdbc/CalciteMetaImpl.java
>
> instance
> associated with the server. This is the same as if you had a local Calcite
> connection. Note that this will not work behind a load-balancer as local
> Calcite connections are "stateful" (see CALCITE-668
> ).
> I'm sure others might have a better way but this has worked for us so far!
>
> On Wed, Nov 8, 2023 at 9:00 AM Raja Ranjan Senapati
> wrote:
>
> > Team,
> > I am a newbie to Calcite and am excited about its potential. I have a
> > question. According to the status section on the docs page
> >
Re: JOIN and "Type mismatch: rowtype of new rel: RecordType ... NOT NULL rowtype of set"
If the issue can be reproduced using a SQL query on a standard data set, the best test case is often a Quidem fragment. A recent commit that did that: https://github.com/apache/calcite/commit/1245db6201. They also added a test to the applicable unit test. I confess it’s not very easy to run Quidem tests from the command line. Maybe someone can fix that. Julian > On Nov 9, 2023, at 6:30 AM, Egor Ryashin wrote: > > I wonder which *Test.java should I try to use as a template? I need smth that > parses the SQL and uses Volcano optimizer as the stacktrace shows it fails > during optimization. > > BTW, it happened during Apache Druid query execution and I saw the Druid > custom join rule was engaged so I thought initially it was on the Druid side. > But Apache Druid fixed that with Apache Calcite update to 1.35, so now I’m > not sure was it Calcite or Druid or both. > >> On 9 Nov 2023, at 14:52, Ran Tao wrote: >> >> Hi, Egor, I agree with Julian's opinion, maybe you can use calcite's test >> set for this case, such as emp, dept, hr and other schema/tables. >> then developers will easily locate the problem. >> >> Some bugs may have persisted in earlier versions to this day, some were >> later fixed, and some were covered up. >> But as long as a specific standard case/sql and a certain calcite version >> are given, >> we can easily draw from the commit/code history which commit, which version >> introduced the problem or it has never worked correctly. >> >> Best Regards, >> Ran Tao >> >> >> Egor Ryashin 于2023年11月9日周四 18:10写道： >> >>> FYI, it’s not reproduced with Calcite 1.35 >>> On 7 Nov 2023, at 03:10, Julian Hyde wrote: Did you find an existing Jira case? If not, could you log one? You >>> should create a minimal test case and figure out what are the essential >>> features that are necessary to reproduce this problem. Ideally convert it >>> to run on a built-in schema such as Scott (emp and dept tables). This will >>> help someone fix the case, help someone suggest a workaround, and help >>> others identify that they are hitting the same issue. > On Nov 6, 2023, at 3:51 AM, Egor Ryashin >>> wrote: > > Hi all, > > I’ve just run into this exception: > Type mismatch: rowtype of new rel: RecordType(VARCHAR pub, VARCHAR >>> pub0, DOUBLE NOT NULL $f2) NOT NULL rowtype of set: RecordType(VARCHAR pub, >>> VARCHAR pub0, DOUBLE rate) NOT NULL > > > executing this SQL: > SELECT base."pub”, >Any_value(base.”rate”), > Any_value(comparison."rate") > FROM( > SELECT "pub", > Min(total)/Max(price) AS "rate" > FROM "test_data" > WHERE"__time" >= timestamp '2022-03-01 >>> 00:00:00' > GROUP BY "pub" > ORDER BY TRUE, > 2 limit 500 ) base > left outer join > ( > SELECT "pub", > min(total)/max(price) AS "rate" > FROM "test_data" > WHERE"__time" < timestamp '2022-03-01 >>> 00:00:00' > GROUP BY "pub" ) comparison > ON base."pub" = comparison."pub" > GROUP BY1 > > looking through JIRA I concluded that bug is still an issue, I wonder >>> if there’s a known workaround for that? > > > >>> >>> >
[jira] [Created] (CALCITE-6101) SqlCollectionTypeNameSpec should preserve the nullability property of the element type
Dmitry Sysolyatin created CALCITE-6101: -- Summary: SqlCollectionTypeNameSpec should preserve the nullability property of the element type Key: CALCITE-6101 URL: https://issues.apache.org/jira/browse/CALCITE-6101 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.35.0, 1.36.0 Reporter: Dmitry Sysolyatin SqlCollectionTypeNameSpec uses SqlTypeNameSpec for the elementType property, but SqlTypeNameSpec does not provide information about nullability. Therefore, SqlDataTypeSpec should be used for elementType instead of SqlTypeNameSpec, as was done for SqlMapTypeNameSpec and SqlRowTypeNameSpec. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (CALCITE-6100) The equalsDeep of SqlRowTypeNameSpec returns wrong result
Dmitry Sysolyatin created CALCITE-6100:
--
Summary: The equalsDeep of SqlRowTypeNameSpec returns wrong result
Key: CALCITE-6100
URL: https://issues.apache.org/jira/browse/CALCITE-6100
Project: Calcite
Issue Type: Bug
Components: core
Affects Versions: 1.35.0, 1.36.0
Reporter: Dmitry Sysolyatin
SqlRowTypeNameSpec#equalsDeep method uses reference comparison for objects
inside fieldTypes list instead of using SqlDataTypeSpec#equalsDeep.
Testcase: Add to `SqlEqualsDeepTest` the following lines of code:
{code:java}
@Test
void testRowEqualsDeep() throws SqlParseException {
assertEqualsDeep("CAST(a AS ROW(field INTEGER))",
"CAST(a AS ROW(field INTEGER))", true);
}
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: JOIN and "Type mismatch: rowtype of new rel: RecordType ... NOT NULL rowtype of set"
I wonder which *Test.java should I try to use as a template? I need smth that parses the SQL and uses Volcano optimizer as the stacktrace shows it fails during optimization. BTW, it happened during Apache Druid query execution and I saw the Druid custom join rule was engaged so I thought initially it was on the Druid side. But Apache Druid fixed that with Apache Calcite update to 1.35, so now I’m not sure was it Calcite or Druid or both. > On 9 Nov 2023, at 14:52, Ran Tao wrote: > > Hi, Egor, I agree with Julian's opinion, maybe you can use calcite's test > set for this case, such as emp, dept, hr and other schema/tables. > then developers will easily locate the problem. > > Some bugs may have persisted in earlier versions to this day, some were > later fixed, and some were covered up. > But as long as a specific standard case/sql and a certain calcite version > are given, > we can easily draw from the commit/code history which commit, which version > introduced the problem or it has never worked correctly. > > Best Regards, > Ran Tao > > > Egor Ryashin 于2023年11月9日周四 18:10写道： > >> FYI, it’s not reproduced with Calcite 1.35 >> >>> On 7 Nov 2023, at 03:10, Julian Hyde wrote: >>> >>> Did you find an existing Jira case? If not, could you log one? You >> should create a minimal test case and figure out what are the essential >> features that are necessary to reproduce this problem. Ideally convert it >> to run on a built-in schema such as Scott (emp and dept tables). This will >> help someone fix the case, help someone suggest a workaround, and help >> others identify that they are hitting the same issue. >>> On Nov 6, 2023, at 3:51 AM, Egor Ryashin >> wrote: Hi all, I’ve just run into this exception: Type mismatch: rowtype of new rel: RecordType(VARCHAR pub, VARCHAR >> pub0, DOUBLE NOT NULL $f2) NOT NULL rowtype of set: RecordType(VARCHAR pub, >> VARCHAR pub0, DOUBLE rate) NOT NULL executing this SQL: SELECT base."pub”, Any_value(base.”rate”), Any_value(comparison."rate") FROM( SELECT "pub", Min(total)/Max(price) AS "rate" FROM "test_data" WHERE"__time" >= timestamp '2022-03-01 >> 00:00:00' GROUP BY "pub" ORDER BY TRUE, 2 limit 500 ) base left outer join ( SELECT "pub", min(total)/max(price) AS "rate" FROM "test_data" WHERE"__time" < timestamp '2022-03-01 >> 00:00:00' GROUP BY "pub" ) comparison ON base."pub" = comparison."pub" GROUP BY1 looking through JIRA I concluded that bug is still an issue, I wonder >> if there’s a known workaround for that? >>> >> >>
Re: JOIN and "Type mismatch: rowtype of new rel: RecordType ... NOT NULL rowtype of set"
Hi, Egor, I agree with Julian's opinion, maybe you can use calcite's test set for this case, such as emp, dept, hr and other schema/tables. then developers will easily locate the problem. Some bugs may have persisted in earlier versions to this day, some were later fixed, and some were covered up. But as long as a specific standard case/sql and a certain calcite version are given, we can easily draw from the commit/code history which commit, which version introduced the problem or it has never worked correctly. Best Regards, Ran Tao Egor Ryashin 于2023年11月9日周四 18:10写道： > FYI, it’s not reproduced with Calcite 1.35 > > > On 7 Nov 2023, at 03:10, Julian Hyde wrote: > > > > Did you find an existing Jira case? If not, could you log one? You > should create a minimal test case and figure out what are the essential > features that are necessary to reproduce this problem. Ideally convert it > to run on a built-in schema such as Scott (emp and dept tables). This will > help someone fix the case, help someone suggest a workaround, and help > others identify that they are hitting the same issue. > > > >> On Nov 6, 2023, at 3:51 AM, Egor Ryashin > wrote: > >> > >> Hi all, > >> > >> I’ve just run into this exception: > >> Type mismatch: rowtype of new rel: RecordType(VARCHAR pub, VARCHAR > pub0, DOUBLE NOT NULL $f2) NOT NULL rowtype of set: RecordType(VARCHAR pub, > VARCHAR pub0, DOUBLE rate) NOT NULL > >> > >> > >> executing this SQL: > >> SELECT base."pub”, > >> Any_value(base.”rate”), > >> Any_value(comparison."rate") > >> FROM( > >>SELECT "pub", > >> Min(total)/Max(price) AS "rate" > >>FROM "test_data" > >>WHERE"__time" >= timestamp '2022-03-01 > 00:00:00' > >>GROUP BY "pub" > >>ORDER BY TRUE, > >> 2 limit 500 ) base > >> left outer join > >> ( > >>SELECT "pub", > >> min(total)/max(price) AS "rate" > >>FROM "test_data" > >>WHERE"__time" < timestamp '2022-03-01 > 00:00:00' > >>GROUP BY "pub" ) comparison > >> ON base."pub" = comparison."pub" > >> GROUP BY1 > >> > >> looking through JIRA I concluded that bug is still an issue, I wonder > if there’s a known workaround for that? > >> > >> > >> > > > >
Re: JOIN and "Type mismatch: rowtype of new rel: RecordType ... NOT NULL rowtype of set"
FYI, it’s not reproduced with Calcite 1.35 > On 7 Nov 2023, at 03:10, Julian Hyde wrote: > > Did you find an existing Jira case? If not, could you log one? You should > create a minimal test case and figure out what are the essential features > that are necessary to reproduce this problem. Ideally convert it to run on a > built-in schema such as Scott (emp and dept tables). This will help someone > fix the case, help someone suggest a workaround, and help others identify > that they are hitting the same issue. > >> On Nov 6, 2023, at 3:51 AM, Egor Ryashin wrote: >> >> Hi all, >> >> I’ve just run into this exception: >> Type mismatch: rowtype of new rel: RecordType(VARCHAR pub, VARCHAR pub0, >> DOUBLE NOT NULL $f2) NOT NULL rowtype of set: RecordType(VARCHAR pub, >> VARCHAR pub0, DOUBLE rate) NOT NULL >> >> >> executing this SQL: >> SELECT base."pub”, >> Any_value(base.”rate”), >> Any_value(comparison."rate") >> FROM( >>SELECT "pub", >> Min(total)/Max(price) AS "rate" >>FROM "test_data" >>WHERE"__time" >= timestamp '2022-03-01 00:00:00' >>GROUP BY "pub" >>ORDER BY TRUE, >> 2 limit 500 ) base >> left outer join >> ( >>SELECT "pub", >> min(total)/max(price) AS "rate" >>FROM "test_data" >>WHERE"__time" < timestamp '2022-03-01 00:00:00' >>GROUP BY "pub" ) comparison >> ON base."pub" = comparison."pub" >> GROUP BY1 >> >> looking through JIRA I concluded that bug is still an issue, I wonder if >> there’s a known workaround for that? >> >> >> >
