[jira] [Commented] (FLINK-17859) [flink-s3-fs-presto] Upgrade the dependent presto-hive jar to latest version
[ https://issues.apache.org/jira/browse/FLINK-17859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358954#comment-17358954 ] sam lin commented on FLINK-17859: - Cool, thanks! Closing this issue. > [flink-s3-fs-presto] Upgrade the dependent presto-hive jar to latest version > > > Key: FLINK-17859 > URL: https://issues.apache.org/jira/browse/FLINK-17859 > Project: Flink > Issue Type: New Feature > Components: Connectors / FileSystem >Affects Versions: 1.7.2 >Reporter: sam lin >Priority: Major > Fix For: 1.8.0 > > > The current version of presto-hive is 0.187.[ > |https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] > > [https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] > [ > |https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] > The latest version is 0.234. [https://github.com/prestodb/presto/releases > ] > There are some nice features we want to use after 0.187. One of them is the > CredentialProviderChain support when using AWS S3 client added in this > [pr]([https://github.com/prestodb/presto/pull/13858]) > Do you have any concerns to upgrade the `presto-hive` to the latest version? > Could you please upgrade it in the latest release? Thanks. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (FLINK-17859) [flink-s3-fs-presto] Upgrade the dependent presto-hive jar to latest version
[ https://issues.apache.org/jira/browse/FLINK-17859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam lin resolved FLINK-17859. - Resolution: Fixed > [flink-s3-fs-presto] Upgrade the dependent presto-hive jar to latest version > > > Key: FLINK-17859 > URL: https://issues.apache.org/jira/browse/FLINK-17859 > Project: Flink > Issue Type: New Feature > Components: Connectors / FileSystem >Affects Versions: 1.7.2 >Reporter: sam lin >Priority: Major > Fix For: 1.8.0 > > > The current version of presto-hive is 0.187.[ > |https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] > > [https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] > [ > |https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] > The latest version is 0.234. [https://github.com/prestodb/presto/releases > ] > There are some nice features we want to use after 0.187. One of them is the > CredentialProviderChain support when using AWS S3 client added in this > [pr]([https://github.com/prestodb/presto/pull/13858]) > Do you have any concerns to upgrade the `presto-hive` to the latest version? > Could you please upgrade it in the latest release? Thanks. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (FLINK-19196) FlinkSQL aggregation function to aggregate array typed column to Map/Multiset
[
https://issues.apache.org/jira/browse/FLINK-19196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sam lin closed FLINK-19196.
---
Resolution: Fixed
> FlinkSQL aggregation function to aggregate array typed column to Map/Multiset
> -
>
> Key: FLINK-19196
> URL: https://issues.apache.org/jira/browse/FLINK-19196
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API
>Affects Versions: 1.11.1
>Reporter: sam lin
>Priority: Major
>
> Hi,
>
> For FlinkSQL, how do I aggregate an array typed column to a Map/Multiset
> type? e.g. Given the source events:
> userID, arr_types
> 1, [a, b]
> 2, [a]
> 1, [b, c]
>
> ```
> SELECT userID, collect_arr(arr_types) FROM src_table GROUP BY userID
> ```
> Using the above SQL could produce results like:
> 1, \{a: 1, b: 2, c:1}
> 2, \{a: 1}
> My question is do we have existing UDF to do that? If not, what is the
> recommended way to achieve this? I'm happy to write a UDF if you could
> provide some code pointers. Thanks in advance!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-19196) FlinkSQL aggregation function to aggregate array typed column to Map/Multiset
[
https://issues.apache.org/jira/browse/FLINK-19196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17198591#comment-17198591
]
sam lin commented on FLINK-19196:
-
Thanks a lot, [~jark]! I will implement a UDF for this case then since my
actual SQL will be more complicated than what described above. Nested subquery
may have perf downgrade.
> FlinkSQL aggregation function to aggregate array typed column to Map/Multiset
> -
>
> Key: FLINK-19196
> URL: https://issues.apache.org/jira/browse/FLINK-19196
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API
>Affects Versions: 1.11.1
>Reporter: sam lin
>Priority: Major
>
> Hi,
>
> For FlinkSQL, how do I aggregate an array typed column to a Map/Multiset
> type? e.g. Given the source events:
> userID, arr_types
> 1, [a, b]
> 2, [a]
> 1, [b, c]
>
> ```
> SELECT userID, collect_arr(arr_types) FROM src_table GROUP BY userID
> ```
> Using the above SQL could produce results like:
> 1, \{a: 1, b: 2, c:1}
> 2, \{a: 1}
> My question is do we have existing UDF to do that? If not, what is the
> recommended way to achieve this? I'm happy to write a UDF if you could
> provide some code pointers. Thanks in advance!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-19196) FlinkSQL aggregation function to aggregate array typed column to Map/Multiset
sam lin created FLINK-19196:
---
Summary: FlinkSQL aggregation function to aggregate array typed
column to Map/Multiset
Key: FLINK-19196
URL: https://issues.apache.org/jira/browse/FLINK-19196
Project: Flink
Issue Type: New Feature
Components: Table SQL / API
Affects Versions: 1.11.1
Reporter: sam lin
Hi,
For FlinkSQL, how do I aggregate an array typed column to a Map/Multiset type?
e.g. Given the source events:
userID, arr_types
1, [a, b]
2, [a]
1, [b, c]
```
SELECT userID, collect_arr(arr_types) FROM src_table GROUP BY userID
```
Using the above SQL could produce results like:
1, \{a: 1, b: 2, c:1}
2, \{a: 1}
My question is do we have existing UDF to do that? If not, what is the
recommended way to achieve this? I'm happy to write a UDF if you could provide
some code pointers. Thanks in advance!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (FLINK-17852) FlinkSQL support WITH clause in query statement
[ https://issues.apache.org/jira/browse/FLINK-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam lin closed FLINK-17852. --- Resolution: Fixed > FlinkSQL support WITH clause in query statement > --- > > Key: FLINK-17852 > URL: https://issues.apache.org/jira/browse/FLINK-17852 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API >Reporter: sam lin >Priority: Major > > Many modern SQL language support WITH clause in query statement. e.g. > ``` > WITH myName AS ( > select * from table where ... > ) > select column1 from myName > ``` > e.g. In BeamSQL supports this: > [https://beam.apache.org/documentation/dsls/sql/calcite/query-syntax/] > query_statement: [ WITH with_query_name AS ( query_expr ) [, ...] ] > query_expr. > In presto, supports this as well: > [https://prestodb.io/docs/current/sql/select.html#with-clause] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17859) [flink-s3-fs-presto] Upgrade the dependent presto-hive jar to latest version
[ https://issues.apache.org/jira/browse/FLINK-17859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam lin updated FLINK-17859: Component/s: Connectors / Hive > [flink-s3-fs-presto] Upgrade the dependent presto-hive jar to latest version > > > Key: FLINK-17859 > URL: https://issues.apache.org/jira/browse/FLINK-17859 > Project: Flink > Issue Type: New Feature > Components: Connectors / Hive >Reporter: sam lin >Priority: Major > > The current version of presto-hive is 0.187.[ > |https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] > > [https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] > [ > |https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] > The latest version is 0.234. [https://github.com/prestodb/presto/releases > ] > There are some nice features we want to use after 0.187. One of them is the > CredentialProviderChain support when using AWS S3 client added in this > [pr]([https://github.com/prestodb/presto/pull/13858]) > Do you have any concerns to upgrade the `presto-hive` to the latest version? > Could you please upgrade it in the latest release? Thanks. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17859) [flink-s3-fs-presto] Upgrade the dependent presto-hive jar to latest version
[ https://issues.apache.org/jira/browse/FLINK-17859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam lin updated FLINK-17859: Component/s: Connectors / FileSystem > [flink-s3-fs-presto] Upgrade the dependent presto-hive jar to latest version > > > Key: FLINK-17859 > URL: https://issues.apache.org/jira/browse/FLINK-17859 > Project: Flink > Issue Type: New Feature > Components: Connectors / FileSystem, Connectors / Hive >Reporter: sam lin >Priority: Major > > The current version of presto-hive is 0.187.[ > |https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] > > [https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] > [ > |https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] > The latest version is 0.234. [https://github.com/prestodb/presto/releases > ] > There are some nice features we want to use after 0.187. One of them is the > CredentialProviderChain support when using AWS S3 client added in this > [pr]([https://github.com/prestodb/presto/pull/13858]) > Do you have any concerns to upgrade the `presto-hive` to the latest version? > Could you please upgrade it in the latest release? Thanks. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (FLINK-17859) [flink-s3-fs-presto] Upgrade the dependent presto-hive jar to latest version
sam lin created FLINK-17859: --- Summary: [flink-s3-fs-presto] Upgrade the dependent presto-hive jar to latest version Key: FLINK-17859 URL: https://issues.apache.org/jira/browse/FLINK-17859 Project: Flink Issue Type: New Feature Reporter: sam lin The current version of presto-hive is 0.187.[ |https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] [https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] [ |https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-presto/pom.xml#L36] The latest version is 0.234. [https://github.com/prestodb/presto/releases ] There are some nice features we want to use after 0.187. One of them is the CredentialProviderChain support when using AWS S3 client added in this [pr]([https://github.com/prestodb/presto/pull/13858]) Do you have any concerns to upgrade the `presto-hive` to the latest version? Could you please upgrade it in the latest release? Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-17852) FlinkSQL support WITH clause in query statement
[ https://issues.apache.org/jira/browse/FLINK-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112836#comment-17112836 ] sam lin commented on FLINK-17852: - [~jark]: thanks for the code pointer. Yeah, it will be great if we could document it here: [https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html#supported-syntax]. Maybe this issue can change to document this handy feature. :) > FlinkSQL support WITH clause in query statement > --- > > Key: FLINK-17852 > URL: https://issues.apache.org/jira/browse/FLINK-17852 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API >Reporter: sam lin >Priority: Major > > Many modern SQL language support WITH clause in query statement. e.g. > ``` > WITH myName AS ( > select * from table where ... > ) > select column1 from myName > ``` > e.g. In BeamSQL supports this: > [https://beam.apache.org/documentation/dsls/sql/calcite/query-syntax/] > query_statement: [ WITH with_query_name AS ( query_expr ) [, ...] ] > query_expr. > In presto, supports this as well: > [https://prestodb.io/docs/current/sql/select.html#with-clause] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-17852) FlinkSQL support WITH clause in query statement
[ https://issues.apache.org/jira/browse/FLINK-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17112588#comment-17112588 ] sam lin commented on FLINK-17852: - [~f.pompermaier]: Nope, this is for SELECT statement, the issue you pointed is CREATE statement. The name after WITH acts like a variable stores the resulting values of sub-sql. > FlinkSQL support WITH clause in query statement > --- > > Key: FLINK-17852 > URL: https://issues.apache.org/jira/browse/FLINK-17852 > Project: Flink > Issue Type: New Feature > Components: Table SQL / API >Reporter: sam lin >Priority: Major > > Many modern SQL language support WITH clause in query statement. e.g. > ``` > WITH myName AS ( > select * from table where ... > ) > select column1 from myName > ``` > e.g. In BeamSQL supports this: > [https://beam.apache.org/documentation/dsls/sql/calcite/query-syntax/] > query_statement: [ WITH with_query_name AS ( query_expr ) [, ...] ] > query_expr. > In presto, supports this as well: > [https://prestodb.io/docs/current/sql/select.html#with-clause] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-17852) FlinkSQL support WITH clause in query statement
[
https://issues.apache.org/jira/browse/FLINK-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sam lin updated FLINK-17852:
Description:
Many modern SQL language support WITH clause in query statement. e.g.
```
WITH myName AS (
select * from table where ...
)
select column1 from myName
```
e.g. In BeamSQL supports this:
[https://beam.apache.org/documentation/dsls/sql/calcite/query-syntax/]
query_statement: [ WITH with_query_name AS ( query_expr ) [, ...] ] query_expr.
In presto, supports this as well:
[https://prestodb.io/docs/current/sql/select.html#with-clause]
was:
Many modern SQL language support WITH clause in query statement. e.g.
```
WITH myName AS (
select * from table where ...
)
select column1 from myName
```
e.g. In BeamSQL supports
`{{query_statement:
[ WITH with_query_name AS ( query_expr ) [, ...] ]
query_expr`.
}}[https://beam.apache.org/documentation/dsls/sql/calcite/query-syntax/]
> FlinkSQL support WITH clause in query statement
> ---
>
> Key: FLINK-17852
> URL: https://issues.apache.org/jira/browse/FLINK-17852
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API
>Reporter: sam lin
>Priority: Major
>
> Many modern SQL language support WITH clause in query statement. e.g.
> ```
> WITH myName AS (
> select * from table where ...
> )
> select column1 from myName
> ```
> e.g. In BeamSQL supports this:
> [https://beam.apache.org/documentation/dsls/sql/calcite/query-syntax/]
> query_statement: [ WITH with_query_name AS ( query_expr ) [, ...] ]
> query_expr.
> In presto, supports this as well:
> [https://prestodb.io/docs/current/sql/select.html#with-clause]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (FLINK-17852) FlinkSQL support WITH clause in query statement
sam lin created FLINK-17852:
---
Summary: FlinkSQL support WITH clause in query statement
Key: FLINK-17852
URL: https://issues.apache.org/jira/browse/FLINK-17852
Project: Flink
Issue Type: New Feature
Components: Table SQL / API
Reporter: sam lin
Many modern SQL language support WITH clause in query statement. e.g.
```
WITH myName AS (
select * from table where ...
)
select column1 from myName
```
e.g. In BeamSQL supports
`{{query_statement:
[ WITH with_query_name AS ( query_expr ) [, ...] ]
query_expr`.
}}[https://beam.apache.org/documentation/dsls/sql/calcite/query-syntax/]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
