[jira] [Commented] (FLINK-947) Add support for Named Datasets
[
https://issues.apache.org/jira/browse/FLINK-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325686#comment-14325686
]
ASF GitHub Bot commented on FLINK-947:
--
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/405#issuecomment-74842606
Yeah, I'm not sure about linq as well. I like the name but realise that it
might be problematic. What do the others think. I could call it
flink-expressions.
I will add documentation about which types are supported and a good error
message for unsupported types as @rmetzger mentioned.
Add support for Named Datasets
Key: FLINK-947
URL: https://issues.apache.org/jira/browse/FLINK-947
Project: Flink
Issue Type: New Feature
Components: Java API
Reporter: Aljoscha Krettek
Assignee: Aljoscha Krettek
Priority: Minor
This would create an API that is a mix between SQL like declarativity and the
power of user defined functions. Example user code could look like this:
{code:Java}
NamedDataSet one = ...
NamedDataSet two = ...
NamedDataSet result = one.join(two).where(key).equalTo(otherKey)
.project(a, b, c)
.map( (UserTypeIn in) - return new UserTypeOut(...) )
.print();
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-947) Add support for Named Datasets
[
https://issues.apache.org/jira/browse/FLINK-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325705#comment-14325705
]
ASF GitHub Bot commented on FLINK-947:
--
Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/405#issuecomment-74845397
To me, `flink-expressions` sounds much better than `linq` and it mitigates
the risk of law suites :)
Add support for Named Datasets
Key: FLINK-947
URL: https://issues.apache.org/jira/browse/FLINK-947
Project: Flink
Issue Type: New Feature
Components: Java API
Reporter: Aljoscha Krettek
Assignee: Aljoscha Krettek
Priority: Minor
This would create an API that is a mix between SQL like declarativity and the
power of user defined functions. Example user code could look like this:
{code:Java}
NamedDataSet one = ...
NamedDataSet two = ...
NamedDataSet result = one.join(two).where(key).equalTo(otherKey)
.project(a, b, c)
.map( (UserTypeIn in) - return new UserTypeOut(...) )
.print();
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-947) Add support for Named Datasets
[
https://issues.apache.org/jira/browse/FLINK-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325060#comment-14325060
]
ASF GitHub Bot commented on FLINK-947:
--
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/405#issuecomment-74773124
Very very nice work! I've played around a bit with it and the first
impression is very good.
+1 to merge the pull request. The change is very big, but stable enough to
be merged to master. Not merging it soon would probably cause a lot of work on
@aljoscha side.
Add support for Named Datasets
Key: FLINK-947
URL: https://issues.apache.org/jira/browse/FLINK-947
Project: Flink
Issue Type: New Feature
Components: Java API
Reporter: Aljoscha Krettek
Assignee: Aljoscha Krettek
Priority: Minor
This would create an API that is a mix between SQL like declarativity and the
power of user defined functions. Example user code could look like this:
{code:Java}
NamedDataSet one = ...
NamedDataSet two = ...
NamedDataSet result = one.join(two).where(key).equalTo(otherKey)
.project(a, b, c)
.map( (UserTypeIn in) - return new UserTypeOut(...) )
.print();
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-947) Add support for Named Datasets
[
https://issues.apache.org/jira/browse/FLINK-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324948#comment-14324948
]
ASF GitHub Bot commented on FLINK-947:
--
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/405#discussion_r24857326
--- Diff: flink-staging/flink-linq/pom.xml ---
@@ -0,0 +1,227 @@
+?xml version=1.0 encoding=UTF-8?
+!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+License); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+--
+project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;
+
+ modelVersion4.0.0/modelVersion
+
+ parent
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-staging/artifactId
+ version0.9-SNAPSHOT/version
+ relativePath../relativePath
+ /parent
+
+ artifactIdflink-linq/artifactId
+ nameflink-linq/name
+
+ packagingjar/packaging
+
+ dependencies
+
+ dependency
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-scala/artifactId
+ version${project.version}/version
+ /dependency
+
+ dependency
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-streaming-scala/artifactId
+ version${project.version}/version
+ /dependency
+
+ dependency
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-scala-examples/artifactId
+ version${project.version}/version
+ /dependency
+
+ dependency
+ groupIdorg.scala-lang/groupId
+ artifactIdscala-reflect/artifactId
+ /dependency
+
+ dependency
+ groupIdorg.scala-lang/groupId
+ artifactIdscala-library/artifactId
+ /dependency
+
+ dependency
+ groupIdorg.scala-lang/groupId
+ artifactIdscala-compiler/artifactId
+ /dependency
+
--- End diff --
I think its really not an issue to directly add your dependencies to the
pom.
Imagine we change something in the `flink-scala` module.
I was actually thinking about adding a check to maven that every dependency
has to be added directly. I'm pretty sure there are cases in the project where
we use stuff Apache Commons libraries which come from external dependencies.
Add support for Named Datasets
Key: FLINK-947
URL: https://issues.apache.org/jira/browse/FLINK-947
Project: Flink
Issue Type: New Feature
Components: Java API
Reporter: Aljoscha Krettek
Assignee: Aljoscha Krettek
Priority: Minor
This would create an API that is a mix between SQL like declarativity and the
power of user defined functions. Example user code could look like this:
{code:Java}
NamedDataSet one = ...
NamedDataSet two = ...
NamedDataSet result = one.join(two).where(key).equalTo(otherKey)
.project(a, b, c)
.map( (UserTypeIn in) - return new UserTypeOut(...) )
.print();
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-947) Add support for Named Datasets
[
https://issues.apache.org/jira/browse/FLINK-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323372#comment-14323372
]
ASF GitHub Bot commented on FLINK-947:
--
Github user mbalassi commented on the pull request:
https://github.com/apache/flink/pull/405#issuecomment-74579541
Great additions! Really looking to trying it out. :)
Add support for Named Datasets
Key: FLINK-947
URL: https://issues.apache.org/jira/browse/FLINK-947
Project: Flink
Issue Type: New Feature
Components: Java API
Reporter: Aljoscha Krettek
Assignee: Aljoscha Krettek
Priority: Minor
This would create an API that is a mix between SQL like declarativity and the
power of user defined functions. Example user code could look like this:
{code:Java}
NamedDataSet one = ...
NamedDataSet two = ...
NamedDataSet result = one.join(two).where(key).equalTo(otherKey)
.project(a, b, c)
.map( (UserTypeIn in) - return new UserTypeOut(...) )
.print();
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-947) Add support for Named Datasets
[
https://issues.apache.org/jira/browse/FLINK-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323370#comment-14323370
]
ASF GitHub Bot commented on FLINK-947:
--
Github user mbalassi commented on a diff in the pull request:
https://github.com/apache/flink/pull/405#discussion_r24778932
--- Diff: flink-staging/flink-linq/pom.xml ---
@@ -0,0 +1,227 @@
+?xml version=1.0 encoding=UTF-8?
+!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+License); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+--
+project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;
+
+ modelVersion4.0.0/modelVersion
+
+ parent
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-staging/artifactId
+ version0.9-SNAPSHOT/version
+ relativePath../relativePath
+ /parent
+
+ artifactIdflink-linq/artifactId
+ nameflink-linq/name
+
+ packagingjar/packaging
+
+ dependencies
+
+ dependency
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-scala/artifactId
+ version${project.version}/version
+ /dependency
+
+ dependency
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-streaming-scala/artifactId
+ version${project.version}/version
+ /dependency
+
+ dependency
+ groupIdorg.apache.flink/groupId
+ artifactIdflink-scala-examples/artifactId
+ version${project.version}/version
+ /dependency
+
+ dependency
+ groupIdorg.scala-lang/groupId
+ artifactIdscala-reflect/artifactId
+ /dependency
+
+ dependency
+ groupIdorg.scala-lang/groupId
+ artifactIdscala-library/artifactId
+ /dependency
+
+ dependency
+ groupIdorg.scala-lang/groupId
+ artifactIdscala-compiler/artifactId
+ /dependency
+
--- End diff --
You transitively depend on the scala stuff through flink-scala, so you
could omit these.
Add support for Named Datasets
Key: FLINK-947
URL: https://issues.apache.org/jira/browse/FLINK-947
Project: Flink
Issue Type: New Feature
Components: Java API
Reporter: Aljoscha Krettek
Assignee: Aljoscha Krettek
Priority: Minor
This would create an API that is a mix between SQL like declarativity and the
power of user defined functions. Example user code could look like this:
{code:Java}
NamedDataSet one = ...
NamedDataSet two = ...
NamedDataSet result = one.join(two).where(key).equalTo(otherKey)
.project(a, b, c)
.map( (UserTypeIn in) - return new UserTypeOut(...) )
.print();
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
