Re: [DISCUSS] Upgrading Elasticsearch in Calcite from 7.0.1 to 7.15
Hi, Julian, Andrei. I’ve upgraded the embedded Elasticsearch version to 7.15.2, so does the RestClient, and all the existing tests could run through. But when I try new ES features in unit test(such as multi_terms、top_metrics), results show that they are not available(cannot be parsed). I’ve also installed ES 7.15.2 locally, and I can get correct results with the above new features by running client codes, which are the same with the codes in calcite. So I’m wandering if I missed something necessary. It will be very helpful if you guys have some suggestions or directions. Best, Zhe Hu On 11/23/2021 09:08，Zhe Hu wrote： That’s very helpful, thanks for your reminder. I will look into whether ES rest client is backward compatible. On 11/23/2021 07:34，Andrei Sereda wrote： +1 Also take a look at vendor's official support schedule: https://www.elastic.co/support/eol I think it is safe to upgrade ES rest client to currently supported version. On Mon, Nov 22, 2021, 19:00 Julian Hyde wrote: +1 Sounds like a good idea. On Nov 22, 2021, at 6:40 AM, Zhe Hu wrote: Hi, community. Recently, when I tried to fix bugs in Elasticsearch Adapter, I’ve found that some new features in released Elasticsearch cannot be applied in Calcite, since the Elasticsearch version in Calcite is 7.0.1. For instance, as CALCITE-4868 mentioned, I want to sort aggregation results by multi_terms(supported after ES 7.1.11) as it’s more suitable under such circumstances than terms aggregation, however the DSL is not supported in Calcite currently. Moreover, I've browsed Elasticsearch's release notes roughly, and there are many new SQL features were added from 7.0.1 to 7.15.2, and lots of bugs were fixed meanwhile(if needed, I will spend some time to collect such info that benefits Calcite users). Since we use RestClient in Elasticsearch Adapter, there are still some cons about upgrading ES version, like new DSL couldn’t be used in old version Elasticsearch cluster, current use case might need to be adjusted. Nevertheless, I’m still wondering if we can upgrade the Elasticsearch version to 7.15, which could enrich ES Adapter's ablility as much as possible. Best regards, Zhe Hu
Re: [DISCUSS] Upgrading Elasticsearch in Calcite from 7.0.1 to 7.15
That’s very helpful, thanks for your reminder. I will look into whether ES rest client is backward compatible. On 11/23/2021 07:34，Andrei Sereda wrote： +1 Also take a look at vendor's official support schedule: https://www.elastic.co/support/eol I think it is safe to upgrade ES rest client to currently supported version. On Mon, Nov 22, 2021, 19:00 Julian Hyde wrote: +1 Sounds like a good idea. On Nov 22, 2021, at 6:40 AM, Zhe Hu wrote: Hi, community. Recently, when I tried to fix bugs in Elasticsearch Adapter, I’ve found that some new features in released Elasticsearch cannot be applied in Calcite, since the Elasticsearch version in Calcite is 7.0.1. For instance, as CALCITE-4868 mentioned, I want to sort aggregation results by multi_terms(supported after ES 7.1.11) as it’s more suitable under such circumstances than terms aggregation, however the DSL is not supported in Calcite currently. Moreover, I've browsed Elasticsearch's release notes roughly, and there are many new SQL features were added from 7.0.1 to 7.15.2, and lots of bugs were fixed meanwhile(if needed, I will spend some time to collect such info that benefits Calcite users). Since we use RestClient in Elasticsearch Adapter, there are still some cons about upgrading ES version, like new DSL couldn’t be used in old version Elasticsearch cluster, current use case might need to be adjusted. Nevertheless, I’m still wondering if we can upgrade the Elasticsearch version to 7.15, which could enrich ES Adapter's ablility as much as possible. Best regards, Zhe Hu
Re: [DISCUSS] Upgrading Elasticsearch in Calcite from 7.0.1 to 7.15
+1 Also take a look at vendor's official support schedule: https://www.elastic.co/support/eol I think it is safe to upgrade ES rest client to currently supported version. On Mon, Nov 22, 2021, 19:00 Julian Hyde wrote: > +1 > > Sounds like a good idea. > > > On Nov 22, 2021, at 6:40 AM, Zhe Hu wrote: > > > > Hi, community. > > > > > > Recently, when I tried to fix bugs in Elasticsearch Adapter, I’ve found > that some new features in released Elasticsearch cannot be applied in > Calcite, since the Elasticsearch version in Calcite is 7.0.1. > > > > > > For instance, as CALCITE-4868 mentioned, I want to sort aggregation > results by multi_terms(supported after ES 7.1.11) as it’s more suitable > under such circumstances than terms aggregation, however the DSL is not > supported in Calcite currently. > > > > > > Moreover, I've browsed Elasticsearch's release notes roughly, and there > are many new SQL features were added from 7.0.1 to 7.15.2, and lots of bugs > were fixed meanwhile(if needed, I will spend some time to collect such info > that benefits Calcite users). > > > > > > Since we use RestClient in Elasticsearch Adapter, there are still some > cons about upgrading ES version, like new DSL couldn’t be used in old > version Elasticsearch cluster, current use case might need to be adjusted. > > > > > > Nevertheless, I’m still wondering if we can upgrade the Elasticsearch > version to 7.15, which could enrich ES Adapter's ablility as much as > possible. > > > > > > Best regards, > > Zhe Hu > > > > > > >
Re: [DISCUSS] Upgrading Elasticsearch in Calcite from 7.0.1 to 7.15
+1 Sounds like a good idea. > On Nov 22, 2021, at 6:40 AM, Zhe Hu wrote: > > Hi, community. > > > Recently, when I tried to fix bugs in Elasticsearch Adapter, I’ve found that > some new features in released Elasticsearch cannot be applied in Calcite, > since the Elasticsearch version in Calcite is 7.0.1. > > > For instance, as CALCITE-4868 mentioned, I want to sort aggregation results > by multi_terms(supported after ES 7.1.11) as it’s more suitable under such > circumstances than terms aggregation, however the DSL is not supported in > Calcite currently. > > > Moreover, I've browsed Elasticsearch's release notes roughly, and there are > many new SQL features were added from 7.0.1 to 7.15.2, and lots of bugs were > fixed meanwhile(if needed, I will spend some time to collect such info that > benefits Calcite users). > > > Since we use RestClient in Elasticsearch Adapter, there are still some cons > about upgrading ES version, like new DSL couldn’t be used in old version > Elasticsearch cluster, current use case might need to be adjusted. > > > Nevertheless, I’m still wondering if we can upgrade the Elasticsearch version > to 7.15, which could enrich ES Adapter's ablility as much as possible. > > > Best regards, > Zhe Hu > > >
[DISCUSS] Upgrading Elasticsearch in Calcite from 7.0.1 to 7.15
Hi, community. Recently, when I tried to fix bugs in Elasticsearch Adapter, I’ve found that some new features in released Elasticsearch cannot be applied in Calcite, since the Elasticsearch version in Calcite is 7.0.1. For instance, as CALCITE-4868 mentioned, I want to sort aggregation results by multi_terms(supported after ES 7.1.11) as it’s more suitable under such circumstances than terms aggregation, however the DSL is not supported in Calcite currently. Moreover, I've browsed Elasticsearch's release notes roughly, and there are many new SQL features were added from 7.0.1 to 7.15.2, and lots of bugs were fixed meanwhile(if needed, I will spend some time to collect such info that benefits Calcite users). Since we use RestClient in Elasticsearch Adapter, there are still some cons about upgrading ES version, like new DSL couldn’t be used in old version Elasticsearch cluster, current use case might need to be adjusted. Nevertheless, I’m still wondering if we can upgrade the Elasticsearch version to 7.15, which could enrich ES Adapter's ablility as much as possible. Best regards, Zhe Hu
