Re: Package multiple jobs in a single jar

2015-05-19 Thread Flavio Pompermaier
Nice feature Matthias! My suggestion is to create a specific Flink interface to get also description of a job and standardize parameter passing. Then, somewhere (e.g. Manifest) you could specify the list of packages (or also directly the classes) to inspect with reflection to extract the list of

Re: Package multiple jobs in a single jar

2015-05-19 Thread Stephan Ewen
We actually has an interface like that before (Program). It is still supported, but in all new programs we simply use the Java main method. The advantage is that most IDEs can create executable JARs automatically, setting the JAR manifest attributes, etc. The Program interface still works,

Re: Package multiple jobs in a single jar

2015-05-19 Thread Matthias J. Sax
Supporting an interface like this seems to be a nice idea. Any other opinions on it? It seems to be some more work to get it done right. I don't want to start working on it, before it's clear that it has a chance to be included in Flink. @Flavio: I moved the discussion to dev mailing list (user

[jira] [Created] (FLINK-2042) Increase font size for stack figure on website home

2015-05-19 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-2042: Summary: Increase font size for stack figure on website home Key: FLINK-2042 URL: https://issues.apache.org/jira/browse/FLINK-2042 Project: Flink Issue

Re: Gelly Roadmap

2015-05-19 Thread Andra Lungu
Apache TinkerPop made it to the wish-list :) Thanks, Fabian! On Tue, May 19, 2015 at 9:42 AM, Flavio Pompermaier pomperma...@okkam.it wrote: You're definitely right Fabian +1 On Mon, May 18, 2015 at 2:03 PM, Fabian Hueske fhue...@gmail.com wrote: Integration with Apache TinkerPop3 could

[jira] [Created] (FLINK-2044) Implementation of Gelly Algorithm HITS

2015-05-19 Thread Ahamd Javid (JIRA)
Ahamd Javid created FLINK-2044: -- Summary: Implementation of Gelly Algorithm HITS Key: FLINK-2044 URL: https://issues.apache.org/jira/browse/FLINK-2044 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-2046) Quickstarts are missing on the new Website

2015-05-19 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-2046: --- Summary: Quickstarts are missing on the new Website Key: FLINK-2046 URL: https://issues.apache.org/jira/browse/FLINK-2046 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-2047) Rename CoCoA to SVM

2015-05-19 Thread Theodore Vasiloudis (JIRA)
Theodore Vasiloudis created FLINK-2047: -- Summary: Rename CoCoA to SVM Key: FLINK-2047 URL: https://issues.apache.org/jira/browse/FLINK-2047 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-2048) Enhance Twitter Stream support

2015-05-19 Thread Hilmi Yildirim (JIRA)
Hilmi Yildirim created FLINK-2048: - Summary: Enhance Twitter Stream support Key: FLINK-2048 URL: https://issues.apache.org/jira/browse/FLINK-2048 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-2049) KafkaSink sporadically fails to send message

2015-05-19 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-2049: - Summary: KafkaSink sporadically fails to send message Key: FLINK-2049 URL: https://issues.apache.org/jira/browse/FLINK-2049 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-2050) Add pipelining mechanism for chainable transformers and estimators

2015-05-19 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-2050: Summary: Add pipelining mechanism for chainable transformers and estimators Key: FLINK-2050 URL: https://issues.apache.org/jira/browse/FLINK-2050 Project: Flink

Re: Problems building the current master

2015-05-19 Thread Chiwan Park
Hi. I think that you are building in encrypted file system such as ecryptfs. Some encrypted file systems do not support long file name, but scala classes have long file name frequently. You can choose two options to solve this problem. 1. Build in non-encrypted file system. 2. Add

[jira] [Created] (FLINK-2051) Fix disabled test in ComplexIntegrationTest

2015-05-19 Thread JIRA
Márton Balassi created FLINK-2051: - Summary: Fix disabled test in ComplexIntegrationTest Key: FLINK-2051 URL: https://issues.apache.org/jira/browse/FLINK-2051 Project: Flink Issue Type: Bug

Re: Problems building the current master

2015-05-19 Thread Robert Metzger
We could actually add a check into the tools/qa-check.sh script. That way we'll see at least a warning in pull requests introducing too long lines. On Tue, May 19, 2015 at 6:03 PM, Stephan Ewen se...@apache.org wrote: Chiwan proposed a good workaround. I think that, as a long term fix, we

[jira] [Created] (FLINK-2052) Clear up Serializable warnings in streaming operators

2015-05-19 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-2052: - Summary: Clear up Serializable warnings in streaming operators Key: FLINK-2052 URL: https://issues.apache.org/jira/browse/FLINK-2052 Project: Flink Issue Type:

Re: Problems building the current master

2015-05-19 Thread Robert Metzger
Okay .. it seems that maven is downloading the flink-compiler artifacts from a snapshots repository: [INFO] [INFO] Building flink-language-binding-generic 0.9-SNAPSHOT [INFO]

Re: SparseVector.fromCOO keeps zero entries

2015-05-19 Thread Till Rohrmann
Hi Christoph, the thing with the current implementation of the SparseVector is that you can only modify entries which are “non-zero”. All other entries are not represented in the underlying data structures. This means that you have to create a new SparseVector if you want to set a zero entry to

Re: Problems building the current master

2015-05-19 Thread Stephan Ewen
Chiwan proposed a good workaround. I think that, as a long term fix, we should rewrite the code a bit such that it does not generate such long names. On Tue, May 19, 2015 at 5:37 PM, Chiwan Park chiwanp...@icloud.com wrote: Hi. I think that you are building in encrypted file system such as

Re: Problems building the current master

2015-05-19 Thread Alexander Alexandrov
I think that these two should be renamed to flink-optimizer, no? ./flink-staging/flink-language-binding/flink-python/pom.xml: artifactIdflink-compiler/artifactId ./flink-staging/flink-language-binding/flink-language-binding-generic/pom.xml: artifactIdflink-compiler/artifactId 2015-05-19 21:07

Re: Problems building the current master

2015-05-19 Thread Alexander Alexandrov
I had a different issue related to the fact that flink-language-binding-generic was not able to find (a potentially outdated) flink-compiler dependency. I had to wipe out the local flink artifacts from my .m2/repository to make this work. 2015-05-19 18:06 GMT+02:00 Robert Metzger

Re: Problems building the current master

2015-05-19 Thread Stephan Ewen
Yes, those should definitely be renamed. It is amazing that maven actually compiles this. I am a bit puzzled there... On Tue, May 19, 2015 at 9:09 PM, Alexander Alexandrov alexander.s.alexand...@gmail.com wrote: I think that these two should be renamed to flink-optimizer, no?

Re: Problems building the current master

2015-05-19 Thread Robert Metzger
Probably all Flink dev's have the flink-compiler jar in their local .m2 directory. Also on travis, we are caching the .m2 directory to improve build times. Maybe it would make sense to disable that on travis to detect those issues in the future. I'll try it out to see how it affects the build