Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table

2018-06-29 Thread Andrei Sereda
That's a reasonable alternative. On Fri, Jun 29, 2018 at 7:57 PM Julian Hyde wrote: > Maybe there could be a separator char as one of the adapter’s parameters. > People should choose a value, say ‘$’ or ‘#’, that is legal in an unquoted > SQL identifier but does not occur in any of their index o

Re: [DISCUSS] Towards Calcite 1.17.0

2018-06-29 Thread Francis Chuang
Thanks Julian + Sergey! Can someone please confirm that the following PRs can be closed? https://github.com/apache/calcite/pull/180 https://github.com/apache/calcite/pull/17 https://github.com/apache/calcite/pull/422 If so, I'll open a case with INFRA to close them. Francis On 30/06/2018 7:14

Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table

2018-06-29 Thread Julian Hyde
Maybe there could be a separator char as one of the adapter’s parameters. People should choose a value, say ‘$’ or ‘#’, that is legal in an unquoted SQL identifier but does not occur in any of their index or type names. If not specified, the adapter would end up in a simple mode, say looking for

Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table

2018-06-29 Thread Andrei Sereda
That's a valid point. Then user would define a different pattern like "i$index_t$type" for his cluster. I think we should first answer wherever such scenarios should be supported by calcite (given that they're already deprecated by the vendor). If yes, what should be collision strategy ? User def

Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table

2018-06-29 Thread Julian Hyde
> In elastic (index/type) pair is guaranteed to be unique therefore > "${index}_${type}" will be also unique (as string). This is only necessary > when we have several types per index. Valid question is wherever user > should be allowed such flexibility. Uniqueness is not my concern. Suppose ther

Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table

2018-06-29 Thread Andrei Sereda
> Can you show how those examples affect SQL against the ES adapter and/or how they affect JSON models? The discussion is how to properly bridge (index/type) concept from ES into relational world. Proposal to use placeholders ($index / $type) affects only how table is named in calcite. They're not

Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table

2018-06-29 Thread Andrei Sereda
Let's assume we have the following indexes / types in ES (same cluster) (Index with two types. Legacy format. Not supported in ES8+) - index1: t1 + t2 (Index with single type. Default and most used format) - index2: t1 (Index with two types having same name as index1) - index3: t1 + t2 Technic

Re: [DISCUSS] Towards Calcite 1.17.0

2018-06-29 Thread Julian Hyde
Thanks for bringing this up, Francis. Note that sometimes there are comments on the JIRA case (and so sometimes the ball might be in the contributor's court). But yes, let's either review all of these or close them as stale. As it happens, I picked up a half-finished patch https://issues.apache.or

Calcite-Master - Build # 317 - Still Failing

2018-06-29 Thread Apache Jenkins Server
The Apache Jenkins build system has built Calcite-Master (build #317) Status: Still Failing Check console output at https://builds.apache.org/job/Calcite-Master/317/ to view the results.

Re: Gandiva

2018-06-29 Thread Julian Hyde
We already have two JIRA cases for Arrow integration: https://issues.apache.org/jira/browse/CALCITE-2040 and https://issues.apache.org/jira/browse/CALCITE-2173. I think this is an extremely important area of work for the Calcite project, because it helps us realize the vision of a deconstructed da

Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table

2018-06-29 Thread Christian Beikov
I'm not sure what the benefit of allowing users to specify this scheme would be. We'd have to parse it, interpret it, make sure the expressions don't result conflicting names etc. IMO a simple mode configuration would be way easier to implement and probably cover 99% of the use cases. Mit f

Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table

2018-06-29 Thread Julian Hyde
Andrei, I'm not an ES user so I don't fully understand this issue, but my two cents anyway... Can you show how those examples affect SQL against the ES adapter and/or how they affect JSON models? You seem to be using '_' as a separator character. Are we sure that people will never use it in inde

Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table

2018-06-29 Thread Andrei Sereda
Plus allow to filter indexes using regexp ? On Fri, Jun 29, 2018 at 1:58 PM Andrei Sereda wrote: > I agree there should be a configuration option. How about the following > approach. > > Expose both variables ${index} and ${type} in configuration (JSON) and > user will use them to generate table

Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table

2018-06-29 Thread Andrei Sereda
I agree there should be a configuration option. How about the following approach. Expose both variables ${index} and ${type} in configuration (JSON) and user will use them to generate table name in calcite schema. Example "table_name": "${type}" // current "table_name": "${index}" // new (default

Re: [jira] [Created] (CALCITE-2389) Drill query fails when file name contains semicolon

2018-06-29 Thread Vova Vysotskyi
Sorry, accidentally chose the wrong project. Kind regards, Volodymyr Vysotskyi пт, 29 черв. 2018 о 19:09 Volodymyr Vysotskyi (JIRA) пише: > Volodymyr Vysotskyi created CALCITE-2389: > > > Summary: Drill query fails when file name contai

[jira] [Created] (CALCITE-2389) Drill query fails when file name contains semicolon

2018-06-29 Thread Volodymyr Vysotskyi (JIRA)
Volodymyr Vysotskyi created CALCITE-2389: Summary: Drill query fails when file name contains semicolon Key: CALCITE-2389 URL: https://issues.apache.org/jira/browse/CALCITE-2389 Project: Calcite

Re: [DISCUSS] Towards Calcite 1.17.0

2018-06-29 Thread Sergey Nuyanzin
>>[CALCITE-1866] adapted existing changes to current master + added tests On Fri, Jun 29, 2018 at 2:14 AM, Francis Chuang wrote: > I would love to see if we can close out/merge some of the really stale PRs > on Github. > > Here are a few that I think we should resolve before the release: > > [CA

Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table

2018-06-29 Thread Michael Mior
I think it sounds like you and Andrei are in a good position to tackle this one so I'm happy to have you both work on whatever solution you think is best. -- Michael Mior mm...@apache.org Le ven. 29 juin 2018 à 04:19, Christian Beikov a écrit : > IMO the best solution would be to make it conf

Re: Elasticsearch Adapter. Removal of Mapping Types (by vendor). Index == Table

2018-06-29 Thread Christian Beikov
IMO the best solution would be to make it configurable by introducing a "table_mapping" config with values * type - every type in the known indices is mapped as table * index - every known index is mapped as table We'd probably also need a "type_field" configuration for defining which field