[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/885 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-142527730 Thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-142344379 The version of `maven-shade-plugin` is reverted. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-142230271 Thanks a lot for working on this @chiwanpark. I'll check out your pull request and try to integrate it with our nightly deployment to the snapshot repository. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/885#discussion_r40068488 --- Diff: flink-dist/pom.xml --- @@ -305,6 +305,7 @@ under the License. org.apache.maven.plugins maven-shade-plugin + 2.4.1 --- End diff -- Are you sure the wrong version was used here? I intentionally removed the version tag of the `maven-shade-plugin` from the sub-modules and set the version centrally in the `dependencyManagement` section of the main pom. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on a diff in the pull request: https://github.com/apache/flink/pull/885#discussion_r40068542 --- Diff: flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml --- @@ -96,7 +96,7 @@ under the License. org.apache.maven.plugins maven-shade-plugin - 2.3 + 2.4.1 --- End diff -- updating the maven-shade-version in the quickstarts is a good idea. They are not inheriting the version from the flink-parent pom. so the change is good ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on a diff in the pull request: https://github.com/apache/flink/pull/885#discussion_r40078159 --- Diff: flink-dist/pom.xml --- @@ -305,6 +305,7 @@ under the License. org.apache.maven.plugins maven-shade-plugin + 2.4.1 --- End diff -- Oh I didn't catch the version tag is intentionally removed. I'll update this except quickstart pom. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-141810617 I updated this PR. The changes are following: * Add a bash script named `change-scala-version.sh` to change artifact id * Add a missing flag `-Dscala-2.11` to description of building with custom scala version * Add some documentation * Update maven-shade-plugin version to 2.4.1 I have tested this changes with `mvn deploy -DskipTests -Dscala-2.11` command. In snapshots repository, there are uploaded libraries with `_2.11` suffix. I have also tested `_2.11` libraries using `flink-quickstart-scala_2.11` archetype. Simple WordCount job works. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-141404870 I would suggest to change the scope of this pull request to enable deploying a `_2.11` artifact for each module with the nightly snapshot builds. You can test the deployment to the maven snapshot repository also locally (as a Flink commiter, you should be able to do a mvn deploy call). If you want, I can also take over the issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-141425832 Sure, I don't have time over the weekend to work on this anyways ;) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-141416793 I hope to create a shell script and make setting for deploying `_2.11_ artifact. Could you wait a while to update this? Maybe I can update this PR until this weekend. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-140731019 I agree. We need to use the script. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-138894613 I think that using property in artifact id causes many problems. Maybe Spark faces the same problems, and uses shell script. How about using the shell script for Flink? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-134624663 Cool, that was quick ;) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/885#issuecomment-134658771
Is the Maven shade plugin bug the reason why this fails:
```
ERROR] Failed to execute goal
org.apache.maven.plugins:maven-shade-plugin:2.4.1:shade (shade-hadoop) on
project flink-yarn-tests_2.11: Error creating shaded jar: 3 problems were
encountered while building the effective model for
org.apache.flink:flink-yarn-tests_2.11:[unknown-version]
[ERROR] [WARNING] 'artifactId' contains an expression but should be a
constant. @ org.apache.flink:flink-yarn-tests${scala.suffix}:[unknown-version],
/home/robert/incubator-flink/flink-yarn-tests/pom.xml, line 36, column 14
[ERROR] [WARNING] 'parent.relativePath' of POM
org.apache.flink:flink-yarn-tests_2.11:[unknown-version]
(/home/robert/incubator-flink/flink-yarn-tests/target/dependency-reduced-pom.xml)
points at org.apache.flink:flink-yarn-tests${scala.suffix} instead of
org.apache.flink:flink-parent${scala.suffix}, please verify your project
structure @ line 3, column 11
[ERROR] [FATAL] Non-resolvable parent POM for
org.apache.flink:flink-yarn-tests_2.11:[unknown-version]: Could not find
artifact org.apache.flink:flink-parent${scala.suffix}:pom:0.10-SNAPSHOT in
apache.snapshots (http://repository.apache.org/snapshots) and
'parent.relativePath' points at wrong local POM @ line 3, column 11
[ERROR] for project
org.apache.flink:flink-yarn-tests_2.11:[unknown-version] at
/home/robert/incubator-flink/flink-yarn-tests/target/dependency-reduced-pom.xml
```
?
About the shading artifacts, your guess is right. Because Hadoop packages
don't need Scala dependencies, I didn't add suffix to them. But if we need the
suffix for them to maintain uniformity, we can add the suffix. How do you think?
I think its fine to leave them as they are.
As you see, there are property expressions (${scala.suffix}) in
artifactId. I think that it can be a problem. How can I solve this?
Yes, that is certainly a problem.
Also, the artifact for flink-parent is not created properly in my local
maven repository. Its name is now `flink-parent${scala.suffix}/`.
Maybe we have to look at other projects which are doing the same... if
there are any projects ;)
Kafka for example is offering builds for different scala versions. Sadly,
they are using sbt for building their project.
Spark doesn't deploy its _2.11 artifacts to maven central.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/885#discussion_r37874193
--- Diff: flink-staging/flink-gelly/pom.xml ---
@@ -37,17 +37,17 @@ under the License.
dependencies
dependency
groupIdorg.apache.flink/groupId
- artifactIdflink-java/artifactId
+ artifactIdflink-java${scala.suffix}/artifactId
version${project.version}/version
/dependency
dependency
groupIdorg.apache.flink/groupId
- artifactIdflink-clients/artifactId
+ artifactIdflink-clients${scala.suffix}/artifactId
version${project.version}/version
/dependency
dependency
groupIdorg.apache.flink/groupId
- artifactIdflink-test-utils/artifactId
+ artifactIdflink-test-utils${scala.suffix}/artifactId
version${project.version}/version
scopetest/scope
/dependency
--- End diff --
I can not comment below this line, but you forgot the `${scala.suffix}` for
the `flink-optimizer`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-134623006 @rmetzger Thanks! I addressed your comment and rebased on master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-134667459 There is a Spark 2.11 artifact in mvn central. I think they are doing a similar thing as we are already doing with the hadoop1/hadoop2 versions: They generate specific pom files when deploying Spark to maven central: https://github.com/apache/spark/blob/master/dev/change-scala-version.sh --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-132934697 Thank you for opening an issue for the shade plugin. I didn't forget this issue here. I'll look into it once I have a bit more time. Sorry for the delay. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-125187262 Okay, I tried it again and you are right. It seems to create the artifacts correctly. I was confused because a) the reactor summary didn't contain the variable: ``` [INFO] Reactor Summary: [INFO] [INFO] flink .. SUCCESS [ 5.004 s] [INFO] flink-shaded-hadoop SUCCESS [ 0.927 s] [INFO] flink-shaded-include-yarn .. SUCCESS [ 10.527 s] [INFO] flink-shaded-include-yarn-tests SUCCESS [ 11.211 s] [INFO] flink-core . SUCCESS [ 7.121 s] [INFO] flink-java . SUCCESS [ 6.419 s] [INFO] flink-runtime .. SUCCESS [ 40.310 s] [INFO] flink-optimizer SUCCESS [ 1.987 s] [INFO] flink-clients .. SUCCESS [ 1.735 s] [INFO] flink-test-utils ... SUCCESS [ 5.362 s] ``` but that's because you are not setting the `name` tag. (That is fine, no need to change it) b) the shading artifacts (such as `flink-shaded-include-yarn`) are not producing any `_2.11` suffixed artifacts. (But I guess that's also fine because Hadoop doesn't have any scala dependencies. This is the command I was using ``` git checkout -b flink2200-chiwan chiwan/FLINK-2200 mvn clean install -DskipTests -Pscala-2.11 ``` I got the following error while running it ``` [ERROR] Failed to execute goal on project flink-scala_2.11: Could not resolve dependencies for project org.apache.flink:flink-scala_2.11:jar:0.10-SNAPSHOT: Could not find artifact org.scalamacros:quasiquotes_2.11:jar:2.0.1 in central (https://repo.maven.apache.org/maven2) - [Help 1] ``` Are you getting this error as well? It seems that maven is not activating the right profile in `flink-scala`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on the pull request:
https://github.com/apache/flink/pull/885#issuecomment-125271970
There is wrong description in building documentation. Because Scala 2.11
profile activation is determined by property scala-2.11, Scala 2.10 profile
is activated when we execute `mvn -Pscala-2.11`. Following commands is right:
```
mvn clean install -DskipTests -Dscala-2.11
```
About the shading artifacts, your guess is right. Because Hadoop packages
don't need Scala dependencies, I didn't add suffix to them. But if we need the
suffix for them to maintain uniformity, we can add the suffix. How do you think?
I just found another problem. I opened the dependency reduced pom in my
maven repository, there are some property expressions in artifact id. For
example, the following is head of flink-runtime module's pom.xml:
```
project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;
parent
artifactIdflink-parent${scala.suffix}/artifactId
groupIdorg.apache.flink/groupId
version0.10-SNAPSHOT/version
relativePath../../pom.xml/relativePath
/parent
modelVersion4.0.0/modelVersion
artifactIdflink-runtime${scala.suffix}/artifactId
nameflink-runtime/name
build
plugins
plugin
```
As you see, there are property expressions (${scala.suffix}) in artifactId.
I think that it can be a problem. How can I solve this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-125319617 When we upgrade version of maven-shade-plugin to 2.4.1, the property in current project artifactId is interpolated properly. But the property in parent artifactId is not interpolated. I think that it is a bug of maven-shade-plugin. I posted an [issue](https://issues.apache.org/jira/browse/MSHADE-200) to maven-shade-plugin JIRA. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-124521433 Could you post your command to compile Flink with Scala 2.11? The current setting works well in my environment. Maven module definitions is not artifact id but directory name. So we should keep current setting. I'm adding the suffix into the pom except quickstart. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/885#discussion_r35194868
--- Diff:
flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/src/assembly/word-count-storm.xml
---
@@ -36,7 +36,7 @@ under the License.
includes
!-- need to be added explicitly to get 'defaults.yaml' --
includeorg.apache.storm:storm-core:jar/include
-
includeorg.apache.flink:flink-storm-examples:jar/include
+
includeorg.apache.flink:flink-storm-compatibility-examples${scala.suffix}:jar/include
--- End diff --
good finding ;)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-123643760 No need to hurry .. I needed 10 days to look at it .. so a few hours don't matter ;) I tried to install the artifacts to my local repository, but the _2.11 suffix is not added to the artifacts. There are a few points still missing: the parent pom probably also needs the suffix, + all the parent definitions in the modules and the module.../module definitions in the pom projects. In this commit you can see what I mean: https://github.com/rmetzger/flink/commit/77622269a50babfdc85f46a4235bcdf093cb8e50 Please let me know if you have further questions. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-123640287 Sorry for the delay. I will deploy the artifacts from this branch to the maven snapshot repository to see if everything works as expected. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-123642852 Hi, currently this PR is not ready to merge, because this PR doesn't contain changes for #677. I'll update soon. Unfortunately I'm outside now. Maybe I can update this PR in 4-5 hours. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-120720565 Let's wait for @rmetzger to give a +1 to this. He has done most of the POM magic and can probably give a good review... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-120554092 If I get 1 more LGTM, I'll merge this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-120411775 Hi, I updated PR :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user aalexandrov commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-120423639 Looks very neat! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/885#issuecomment-119578556
I agree with @aalexandrov to do the name switch using
`artifactIdflink-clients${scala.suffix}/artifactId`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user aalexandrov commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-119622710 @rmetzger if you can hack your way with regex's in IntelliJ / Eclipse most of the work should be handled by the IDE. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on a diff in the pull request: https://github.com/apache/flink/pull/885#discussion_r34067611 --- Diff: docs/apis/programming_guide.md --- @@ -187,7 +187,17 @@ that creates the type information for Flink operations. /div /div + Scala Dependency Versions +Because Scala 2.10 binary is not compatible with Scala 2.11 binary, we provide multiple artifacts +to support both Scala versions. If you want to run your program on Flink with Scala 2.11, you need +to add a suffix `_2.11` to all Flink artifact ids in your dependencies. You should be careful with +this difference of artifact id. All modules with Scala 2.11 have a suffix `_2.11` in artifact id. +For example, `flink-java` should be changed to `flink-java_2.11` and `flink-clients` should be +changed to `flink-clients_2.11`. --- End diff -- @aalexandrov I'm not sure it is good to use the keyword Scala-dependent. We cross-build all modules including java-based but linked with Scala modules. I think that omitting Scala-dependent is much better. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user chiwanpark commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-118611879 @aalexandrov Thank you for review. :) I will applying your suggestion and update this PR. But in this changes, all modules require the suffix if the module is linked with Scala 2.11. So we don't need list of modules. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/885#issuecomment-118635826 Very good suggestions, @aalexandrov --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
GitHub user chiwanpark opened a pull request: https://github.com/apache/flink/pull/885 [FLINK-2200] Add Flink with Scala 2.11 in Maven Repository Hi, This PR contains following changes: * Add a suffix `_2.11` to all maven modules by profile setting. (`-Pscala-2.11`) * Add documentation about using Flink with Scala 2.11. I defined rules to add the suffix `_2.11` to maven modules. (From suggestion and discussion in mailing list, thanks @rmetzger) 1. All modules with Scala 2.10 don't have any suffix. (`flink-java`, `flink-core`, `flink-scala`, ..., etc.) 2. All modules except packaging as pom with Scala 2.11 have the suffix `_2.11`. (`flink-java_2.11`, `flink-core_2.11`, `flink-scala_2.11`, ..., etc.) * `flink-parent`, `flink-quickstart`, `flink-examples`, ..., and etc. don't have any suffix although they are built with Scala 2.11 because this modules are for only structuring project. 3. Currently, `flink-scala-shell` don't have any suffix. But in later, we have to add suffix to `flink-scala-shell`. I think that providing quickstart with Scala version variation such as `flink-quickstart-java`, `flink-quickstart-java_2.11` is good. But I cannot find the method to change architype result file (`pom.xml` in `src/main/resources/archetype-resources`) by maven profile. So I add explanation of default scala version in quickstart to documentation. You can merge this pull request into a Git repository by running: $ git pull https://github.com/chiwanpark/flink FLINK-2200 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/885.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #885 commit e15ad6749c24a7bc6e2d37370c926981066a52e1 Author: Chiwan Park chiwanp...@apache.org Date: 2015-07-02T17:28:51Z [FLINK-2200] [build system] Add scala version suffix to artifact id for Scala 2.11 commit 3bfa184af168d538cbafb527b6238ff945997d95 Author: Chiwan Park chiwanp...@apache.org Date: 2015-07-03T09:16:16Z [FLINK-2200] [docs] Add documentation for Flink with Scala 2.11 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user aalexandrov commented on a diff in the pull request: https://github.com/apache/flink/pull/885#discussion_r33857654 --- Diff: docs/apis/programming_guide.md --- @@ -136,7 +136,7 @@ mvn archetype:generate / /div /div -The archetypes are working for stable releases and preview versions (`-SNAPSHOT`) +The archetypes are working for stable releases and preview versions. (`-SNAPSHOT`) --- End diff -- The dot should be after (`-SNAPSHOT`). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user aalexandrov commented on the pull request:
https://github.com/apache/flink/pull/885#issuecomment-118326627
I think that a cleaner solution would be to
1. Add a `scala.suffix` to the Scala profiles. This should be set as the
empty string for 2.10 and as `_${scala.binary.version}` for 2.11.
2. Append the suffix to the required `artifactId` elements. For example,
instead of
```
artifactId${flink.clients.artifactId}/artifactId
```
you would write
```
artifactIdflink-clients${scala.suffix}/artifactId
```
IMHO this makes everything a bit easier to read and then a straight-forward
to migrate to the best practice way of having a suffix for all profiles in
the future.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user aalexandrov commented on a diff in the pull request: https://github.com/apache/flink/pull/885#discussion_r33858383 --- Diff: docs/apis/programming_guide.md --- @@ -187,7 +187,17 @@ that creates the type information for Flink operations. /div /div + Scala Dependency Versions +Because Scala 2.10 binary is not compatible with Scala 2.11 binary, we provide multiple artifacts +to support both Scala versions. If you want to run your program on Flink with Scala 2.11, you need +to add a suffix `_2.11` to all Flink artifact ids in your dependencies. You should be careful with +this difference of artifact id. All modules with Scala 2.11 have a suffix `_2.11` in artifact id. +For example, `flink-java` should be changed to `flink-java_2.11` and `flink-clients` should be +changed to `flink-clients_2.11`. --- End diff -- Because Scala 2.10 binary is not compatible with Scala 2.11 binary, we provide multiple artifacts to support both Scala versions. Starting from the 0.10 line, we cross-build all Scala-dependent Flink modules for both 2.10 and 2.11. If you want to run your program on Flink with Scala 2.11, you need to add a `_2.11` suffix to the `artifactId` values of the Scala-dependent Flink modules in your `dependencies` section. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink pull request: [FLINK-2200] Add Flink with Scala 2.11 in Mave...
Github user aalexandrov commented on a diff in the pull request: https://github.com/apache/flink/pull/885#discussion_r33858475 --- Diff: docs/apis/programming_guide.md --- @@ -187,7 +187,17 @@ that creates the type information for Flink operations. /div /div + Scala Dependency Versions +Because Scala 2.10 binary is not compatible with Scala 2.11 binary, we provide multiple artifacts +to support both Scala versions. If you want to run your program on Flink with Scala 2.11, you need +to add a suffix `_2.11` to all Flink artifact ids in your dependencies. You should be careful with +this difference of artifact id. All modules with Scala 2.11 have a suffix `_2.11` in artifact id. +For example, `flink-java` should be changed to `flink-java_2.11` and `flink-clients` should be +changed to `flink-clients_2.11`. --- End diff -- Maybe a list of all modules indicating which requires a suffix and which not will be helpful (or you can just link to Maven central). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
