Re: Extending and improving our "How to contribute" page

2016-02-19 Thread Martin Liesenberg
Cool, if no one objects, I'll create a JIRA ticket and open a corresponding PR during the weekend. Best regards Martin On Thu, 18 Feb 2016, 17:36 Maximilian Michels wrote: > Hi Martin, > > Sounds like a good idea to me to create a checklist like this. It > would be a nice reminder for people wh

Re: Dense matricies in FlinkML

2016-02-19 Thread Theodore Vasiloudis
Just to note: This should be a separate PR if you plan on contributing this. On Thu, Feb 18, 2016 at 7:54 PM, Márton Balassi wrote: > Hi guys, > > They are at least already registered for serialization [1], so there should > be no intentional conflict as Theo has suggested. > > [1] > > https://g

Re: Extending and improving our "How to contribute" page

2016-02-19 Thread Fabian Hueske
Thanks Martin! can you add two more fields? - Builds locally (mvn clean verify) - Documentation updated or not updates necessary Best, Fabian 2016-02-19 9:35 GMT+01:00 Martin Liesenberg : > Cool, if no one objects, I'll create a JIRA ticket and open a corresponding > PR during the weekend. > >

[jira] [Created] (FLINK-3448) Move initialization of handlers to start() method

2016-02-19 Thread Maximilian Michels (JIRA)
Maximilian Michels created FLINK-3448: - Summary: Move initialization of handlers to start() method Key: FLINK-3448 URL: https://issues.apache.org/jira/browse/FLINK-3448 Project: Flink Iss

cancel running stream job

2016-02-19 Thread Nikolaas s
Hi guys, I'm trying to connect flink streaming and zeppelin, however if I try to resubmit a streaming program from zeppelin I receive a "org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException". Is there a way to cancel a running Streaming Job ? And would cancelling the Streami

Re: cancel running stream job

2016-02-19 Thread Ufuk Celebi
Yes, you can cancel it via the web frontend or the CLI interface [1]. If you can send messages to the JobManager, you can also send a CancelJob message yourself. [1] https://ci.apache.org/projects/flink/flink-docs-master/apis/cli.html On Fri, Feb 19, 2016 at 12:08 PM, Nikolaas s wrote: > Hi guy

Re: cancel running stream job

2016-02-19 Thread Márton Balassi
Adding to Ufuk's answer: yes, cancelling the job frees up the resources. :) Best: Marton On Fri, Feb 19, 2016 at 12:10 PM, Ufuk Celebi wrote: > Yes, you can cancel it via the web frontend or the CLI interface [1]. > > If you can send messages to the JobManager, you can also send a > CancelJob

[jira] [Created] (FLINK-3449) createInput swallows exception if TypeExtractor fails

2016-02-19 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-3449: --- Summary: createInput swallows exception if TypeExtractor fails Key: FLINK-3449 URL: https://issues.apache.org/jira/browse/FLINK-3449 Project: Flink Iss

Re: Dense matricies in FlinkML

2016-02-19 Thread Till Rohrmann
The idea was actually to leverage existing linear algebra libraries such as breeze instead of building another blas implementation which will never be as good as the ones out there. Cheers, Till On Fri, Feb 19, 2016 at 9:48 AM, Theodore Vasiloudis < theodoros.vasilou...@gmail.com> wrote: > Just

Re: cancel running stream job

2016-02-19 Thread Till Rohrmann
Furthermore, if all of your sources are stoppable, then you can even stop the streaming job gracefully now either using the CLI or the StopJob message. Cheers, Till ​ On Fri, Feb 19, 2016 at 12:16 PM, Márton Balassi wrote: > Adding to Ufuk's answer: yes, cancelling the job frees up the resource

Re: Extending and improving our "How to contribute" page

2016-02-19 Thread Maximilian Michels
+1 for the documentation check box. Are we requiring local builds? Travis builds are fine, right? So what about "Builds locally or on Travis"? Could we add more subpoints from the How to Contribute guide? [X] General - JIRA issue associated - Single PR per change - Meaningful commit messag

Re: Extending and improving our "How to contribute" page

2016-02-19 Thread Fabian Hueske
Max, you're right. Not necessarily a local build, but a least "some" build to verify that the code compiles and most tests pass. I said local builds because this is the easiest way to check for somebody not familiar with our setup. I think it is a good idea to explicitly state the command to run:

[jira] [Created] (FLINK-3450) RocksDB Backed Window Fails with KryoSerializer

2016-02-19 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-3450: --- Summary: RocksDB Backed Window Fails with KryoSerializer Key: FLINK-3450 URL: https://issues.apache.org/jira/browse/FLINK-3450 Project: Flink Issue Typ

Re: Dense matricies in FlinkML

2016-02-19 Thread Trevor Grant
That makes sense, a more accurate question would be, why does Vector.scala provide BLAS methods (functions, routines, whatever you call them), and Matrix doesn't? I assume they are there for speed (?) so does Matrix need them? Trevor Grant Data Scientist https://github.com/rawkintrevo http://st

Re: Dense matricies in FlinkML

2016-02-19 Thread Trevor Grant
Agree. Since the current matrices will support what I'm trying to do, I'm going to drive on with the Neural nets. I think the BLAS should speed up matrix computations ( someone more familiar with Java math things can correct me) so I'll make that its own issue. Trevor Grant Data Scientist https://

Re: Dense matricies in FlinkML

2016-02-19 Thread Theodore Vasiloudis
Yes, what I meant by BLAS calls is to have bindings to the relevant Breeze or netlib-java implementation . -- Sent from a mobile device. May contain autocorrect errors. On Feb 19, 2016 3:06 PM, "Trevor Grant" wrote: > That makes sense, a more accurate question would be, why does Vector.scala >

[jira] [Created] (FLINK-3451) NettyServer port is determined via NetUtils.getAvailablePort()

2016-02-19 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-3451: -- Summary: NettyServer port is determined via NetUtils.getAvailablePort() Key: FLINK-3451 URL: https://issues.apache.org/jira/browse/FLINK-3451 Project: Flink Iss

[jira] [Created] (FLINK-3452) Procuring more Travis CI build capacity for Flink

2016-02-19 Thread Wes McKinney (JIRA)
Wes McKinney created FLINK-3452: --- Summary: Procuring more Travis CI build capacity for Flink Key: FLINK-3452 URL: https://issues.apache.org/jira/browse/FLINK-3452 Project: Flink Issue Type: Wis

[jira] [Created] (FLINK-3453) Fix TaskManager logs exception when sampling backpressure while task completes

2016-02-19 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-3453: - Summary: Fix TaskManager logs exception when sampling backpressure while task completes Key: FLINK-3453 URL: https://issues.apache.org/jira/browse/FLINK-3453 Project: Flink

Introducing a PR template

2016-02-19 Thread Martin Liesenberg
Based on the recent discussion in the email thread 'Extending and improving our "How to contribute" page', I propose to introduce the following template for PRs Thanks for contributing to Apache Flink, before you open your PR please kindly take into consideration the following check list. Onc

Re: Introducing a PR template

2016-02-19 Thread Fabian Hueske
Hi Martin, "mvn install" does include the goals "test" and "verify". In fact, "verify" is enough, because "install" does only copy the results into the local Maven repository (~/.m2/repository). So I think - [ ] Tests pass (`mvn test`) - [ ] Build passes (`mvn install`) - [ ] Check style passe

[jira] [Created] (FLINK-3454) Add test dependencies on packaged jars

2016-02-19 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-3454: - Summary: Add test dependencies on packaged jars Key: FLINK-3454 URL: https://issues.apache.org/jira/browse/FLINK-3454 Project: Flink Issue Type: Bug Comp

[jira] [Created] (FLINK-3455) Bump Kafka 0.9 connector dependency to Kafka 0.9.0.1

2016-02-19 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-3455: - Summary: Bump Kafka 0.9 connector dependency to Kafka 0.9.0.1 Key: FLINK-3455 URL: https://issues.apache.org/jira/browse/FLINK-3455 Project: Flink Issue Ty

Re: Adding TaskManager's

2016-02-19 Thread Deepak Jha
Hi Ufuk, I'm planning to build Flink HA cluster and I may need to autoscale taskmanager based on the the requirement. I feel it makes more sense for me to start each taskmanager & jobmanager individually using taskmanager.sh and jobmanager.sh and let these taskmanager's discover jobmanager's using

Re: Adding TaskManager's

2016-02-19 Thread Ufuk Celebi
OK, nice! :-) Then you can just skip the "slaves" file and directly work with the scripts. I'm curious to know if everything works as expected. If you encounter something that seems wrong, let us know. – Ufuk On Fri, Feb 19, 2016 at 9:02 PM, Deepak Jha wrote: > Hi Ufuk, > I'm planning to build

[jira] [Created] (FLINK-3456) Backpressure Monitoring does not work for Kafka Sources

2016-02-19 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-3456: --- Summary: Backpressure Monitoring does not work for Kafka Sources Key: FLINK-3456 URL: https://issues.apache.org/jira/browse/FLINK-3456 Project: Flink Issue Typ

Re: Adding TaskManager's

2016-02-19 Thread Deepak Jha
Hi Ufuk, Sure... I will let you know. I'm planning to use centralized zookeeper. That way Flink and ZK will have the separation. On Fri, Feb 19, 2016 at 12:06 PM, Ufuk Celebi wrote: > OK, nice! :-) Then you can just skip the "slaves" file and directly > work with the scripts. > > I'm curious to

RE: Guarantees for object reuse modes and documentation

2016-02-19 Thread Ken Krugler
Not sure how useful this is, but we'd run into similar issues with Cascading over the years. This wasn't an issue for input data, as Cascading "locks" the Tuple such that attempts to modify it will fail. And in general Hadoop always re-uses the data container being passed to operations, so you

Re: Introducing a PR template

2016-02-19 Thread Jamie Grier
+1 On Fri, Feb 19, 2016 at 9:30 AM, Fabian Hueske wrote: > Hi Martin, > > "mvn install" does include the goals "test" and "verify". > In fact, "verify" is enough, because "install" does only copy the results > into the local Maven repository (~/.m2/repository). > > So I think > - [ ] Tests pass