Re: Moving from single-node, Maven examples to cluster execution

2016-06-16 Thread Prez Cannady
All right, I figured I’d have to do shading, but hadn’t gotten around to experimenting. I’ll try it out. Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH: https://github.com/opencorrelate <https://github.com/opencorrelate>

Moving from single-node, Maven examples to cluster execution

2016-06-16 Thread Prez Cannady
<https://ci.apache.org/projects/flink/flink-docs-release-1.0/quickstart/run_example_quickstart.html> Any help in getting me on the right, preferably best practices path would be appreciated. Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH:

Thanks everyone

2016-04-22 Thread Prez Cannady
We’ve completed our first full sweep on a five node Flink cluster and it went beautifully. On behalf of my team, thought I’d say thanks for all the support. Lots more learning and work to do, so we look forward to working with you all. Prez Cannady p: 617 500 3378 e: revp

Re: Configuring task slots and parallelism for single node Maven executed

2016-04-18 Thread Prez Cannady
Thank you both. Will let you guys know how it works out. Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH: https://github.com/opencorrelate <https://github.com/opencorrelate> LI: https://www.linkedin.com/in/revprez <https://w

Configuring task slots and parallelism for single node Maven executed

2016-04-17 Thread Prez Cannady
a rabbit hole. I figure task slots is limited to the number of processors/cores/whatever available (and the JVM can get at). Is this accurate? Any feedback would be appreciated. Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH:

Re: DataSet -> DataStream

2016-03-11 Thread Prez Cannady
This is roughly the solution I have now. On the other hand, I was hoping for a solution that doesn’t involve checking whether a file has updated. Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH: https://github.com/opencorrelate

DataSet -> DataStream

2016-03-10 Thread Prez Cannady
I’d like to pour some data I’ve collected into a DataSet via JDBC into a Kafka topic, but I think I need to transform my DataSet into a DataStream first. If anyone has a clue how to proceed, I’d appreciate it; or let me know if I’m completely off track. Prez Cannady p: 617 500 3378 e

Re: JDBCInputFormat preparation with Flink 1.1-SNAPSHOT and Scala 2.11

2016-03-09 Thread Prez Cannady
you’ll be operating on whole result sets. 2. Kafka is useful for mating batch processes (like slurping a database) with stream ones (reading out the results of a database query then distributed to various processing nodes). Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:r

Re: JDBCInputFormat preparation with Flink 1.1-SNAPSHOT and Scala 2.11

2016-03-09 Thread Prez Cannady
available (even if it is Java). I might glean a hint from it. Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH: https://github.com/opencorrelate <https://github.com/opencorrelate> LI: https://www.linkedin.com/in/re

Re: Attempting to refactor Kafka Example using Flink 0.10.2 with Scala 2.11

2016-03-09 Thread Prez Cannady
issue seems to have been attempting to publish and subscribe to an older (~0.8.x.x) Kafka instance. Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH: https://github.com/opencorrelate <https://github.com/opencorrelate> LI: https:

JDBCInputFormat preparation with Flink 1.1-SNAPSHOT and Scala 2.11

2016-03-08 Thread Prez Cannady
rking, what should I supply to `createInput`? Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH: https://github.com/opencorrelate <https://github.com/opencorrelate> LI: https://www.linkedin.com/in/revprez <https://www.linkedin.com/in/revprez>

Attempting to refactor Kafka Example using Flink 0.10.2 with Scala 2.11

2016-03-03 Thread Prez Cannady
topics, KeyedDeserializationSchema deserializer, Properties props)`, and is probably due to me missing something in the classpath. Any hints on how to proceed would be appreciated. Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH: https://github.com/o

Re: Results of testing Flink quickstart against 0.10-SNAPSHOT and 1.0-SNAPSHOT (re. Dependency on non-existent org.scalamacros:quasiquotes_2.11:)

2016-01-21 Thread Prez Cannady
s://github.com/apache/flink/commit/cfcb5d7ba5c22295c0ae628f8d9a2447e2286985>). Perhaps there’s a fresher repo I should be using? Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH: https://github.com/opencorrelate <https://gith

Re: Results of testing Flink quickstart against 0.10-SNAPSHOT and 1.0-SNAPSHOT (re. Dependency on non-existent org.scalamacros:quasiquotes_2.11:)

2016-01-20 Thread Prez Cannady
eFlinkAccumulators() throws IOException, ClassNotFoundException { return flinkAccumulators.deserializeValue(ClassLoader.getSystemClassLoader()); } ClassLoader.getSystemClassLoader() should be getClass().getClassLoader(). Not sure why it’s not taking in the build, but there’s the problem. Pre

Re: Compile fails with scala 2.11.4

2016-01-18 Thread Prez Cannady
these conditions resolved my issue which emitted the same error. Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH: https://github.com/opencorrelate <https://github.com/opencorrelate> LI: https://www.linkedin.com/in/re

Re: Results of testing Flink quickstart against 0.10-SNAPSHOT and 1.0-SNAPSHOT (re. Dependency on non-existent org.scalamacros:quasiquotes_2.11:)

2016-01-18 Thread Prez Cannady
One correction, line 3 of 1.0-SHAPSHOT source build should read “checked out master branch (snapshot version 1.0-SNAPSHOT)." Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH: https://github.com/opencorrelate <https://github.com

Results of testing Flink quickstart against 0.10-SNAPSHOT and 1.0-SNAPSHOT (re. Dependency on non-existent org.scalamacros:quasiquotes_2.11:)

2016-01-18 Thread Prez Cannady
not sure as to why 1.0-SNAPSHOT is suffering from a macros incident and not 0.10-SNAPSHOT. Both are bringing in dependencies (quasiquotes, scala-library, scala-reflect, and a bunch of akka-* stuff) compiled for 2.10, but 0.10-SNAPSHOT does not invite the macro-related errors some of us h

Re: Flink QuickStart: On start up, running into ClassNotFoundException: org.apache.flink.streaming.runtime.tasks.OneInputStreamTask

2016-01-14 Thread Prez Cannady
Sure thing. Opened pull requests #1506 against master. Also submitted pull request #1507 against release-0.9.1.rc1 seeing as we’re still pinned to 0.9.1 for our project. Not sure if you guys are interested in hot fixes to previous releases, but there you have it. Prez Cannady p: 617 500

Re: Flink QuickStart: On start up, running into ClassNotFoundException: org.apache.flink.streaming.runtime.tasks.OneInputStreamTask

2016-01-13 Thread Prez Cannady
this actually an appropriate fix, or just a way to highlight a configuration problem? I assume that injecting a parent class loader when registering a task might break things, but I don’t know nearly enough about Flink and this code to say one way or another. Prez Cannady p: 617 500 3378 e

Flink QuickStart: On start up, running into ClassNotFoundException: org.apache.flink.streaming.runtime.tasks.OneInputStreamTask

2016-01-13 Thread Prez Cannady
ild.gradle>) 3. Gist containing the full exception (https://gist.github.com/revprez/2c1fb01c40e5d6790247 <https://gist.github.com/revprez/2c1fb01c40e5d6790247>) Prez Cannady p: 617 500 3378 e: revp...@opencorrelate.org <mailto:revp...@opencorrelate.org> GH: https://github.co

Re: What port should I choose to run the SocketTextStreamWordCount sample

2016-01-12 Thread Prez Cannady
Thanks. That summed it up nicely. I’m able to move forward now. Prez Cannady e: revp...@enerscore.com <mailto:revp...@enerscore.com> p: 617 500 3378 w: http://www.enerscore.com/ <http://www.enerscore.com/> d: http://demo2.enerscore.com/#/ <http://demo2.enerscore.com/#/>

What port should I choose to run the SocketTextStreamWordCount sample

2016-01-12 Thread Prez Cannady
[2] Flink Setup: Configuration (https://ci.apache.org/projects/flink/flink-docs-master/setup/config.html) Also asked at Stack Overflow. http://stackoverflow.com/questions/34753226/what-port-should-i-choose-to-run-the-sockettextstreamwordcount-sample Prez Cannady e: revp...@enerscore.com p: 617 50