Re: Windows, watermarks, and late data

2016-03-04 Thread Aljoscha Krettek
Hi Mike, RichWindowFunction not being supported for the Scala API is an oversight on our side. We’re working to fix it. For ReduceFunction and FoldFunction it’s a bit more tricky, since they are right now not proper operator functions but only used inside the internal window state to incrementa

Re: Windows, watermarks, and late data

2016-03-04 Thread Stephan Ewen
We have a fix open for support of the RichFunctions. Later today, the 1.0-SNAPSHOT and 1.1-SNAPSHOT will probably support that. Seems we need to get a 1.0.1 out very soon, because this is a quite important fix. On Fri, Mar 4, 2016 at 12:37 PM, Aljoscha Krettek wrote: > Hi Mike, > RichWindowFunc

InetSocketAddress is not serializable

2016-03-04 Thread HungChang
Hi, I'm building the connector for ElasticSearch2. One main issue for me now is that List transports = new ArrayList(); transports.add(new InetSocketTransportAddress(new InetSocketAddress(TransportAddress, 9300))); throws java.io.NotSerializableException: org.elasticsearch.common.transport.Inet

Re: InetSocketAddress is not serializable

2016-03-04 Thread Zach Cox
I ran into the same issue upgrading to Elasticsearch 2, here's how I solved it: https://gist.github.com/zcox/59e486be7aeeca381be0#file-elasticsearch2sink-java-L110 -Zach On Fri, Mar 4, 2016 at 7:30 AM HungChang wrote: > Hi, > > I'm building the connector for ElasticSearch2. One main issue for

Re: InetSocketAddress is not serializable

2016-03-04 Thread HungChang
Thank you. Very nice usage and It works! -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/InetSocketAddress-is-not-serializable-when-building-ElasticSearch2-connector-tp5296p5298.html Sent from the Apache Flink User Mailing List archive. maili

Re: InetSocketAddress is not serializable

2016-03-04 Thread HungChang
Can I ask why List can become serilizable? Best, Sendoh -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/InetSocketAddress-is-not-serializable-when-building-ElasticSearch2-connector-tp5296p5299.html Sent from the Apache Flink User Mailing L

Re: Mixing Batch & Streaming

2016-03-04 Thread sskhiri
Nick, Fabian, Is there any update on that point ? Is the pattern described by Nick is still the only way to enrich an event with information in a DB ? Is there any way to load a table through a table and to query it from the Stream ? Thanks, Sabri. -- View this message in context: http://apa

Re: InetSocketAddress is not serializable

2016-03-04 Thread Zach Cox
java.net.InetSocketAddress is Serializable [1] and implementations of java.util.List (e.g. java.util.ArrayList [2]) are also usually Serializable. Unfortunately, for some reason InetSocketTransportAddress [3] is no longer Serializable in Elasticsearch 2.x. :( -Zach [1] http://docs.oracle.com/jav

Re: Mixing Batch & Streaming

2016-03-04 Thread Jeyhun Karimov
Hi all, We are currently working on this issue to make efficient mixing between datastream window and dataset. However, the simplest solution would be, to output each window in a sequential file to HDFS and do computation on that datasource as dataset. On Fri, Mar 4, 2016 at 4:05 PM sskhiri w

Re: InetSocketAddress is not serializable

2016-03-04 Thread HungChang
Thank for your explanation. Yes the InetSocketAddress you used is imported from java.net instead of elaticsearh2. Very cool! -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/InetSocketAddress-is-not-serializable-when-building-ElasticSearch2-c

Re: Periodic actions

2016-03-04 Thread shikhar
Wow that's embarassing :D That was indeed the issue -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Periodic-actions-tp5290p5304.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: java.lang.NoClassDefFoundError for Keys Class

2016-03-04 Thread Madhire, Naveen
Thanks Stephan. I am using the new dependencies from the 1.0.0 release candidate. Now, my application is failing with the below error, java.lang.NoClassDefFoundError: scala/concurrent/ExecutionContext The error is here, https://github.com/apache/flink/blob/b54499b1cf6d14d01e448ea9ce8edabaaba83

Re: java.lang.NoClassDefFoundError for Keys Class

2016-03-04 Thread Stephan Ewen
Hi! You actually don't need an extra Akka or Scala dependency. My suspicion is that you have still some incorrect artifact in the dependencies. Good thing is that these problems disappear when you can reference a stable release. Is it possible for you to share the complete dependency section of

Re: java.lang.NoClassDefFoundError for Keys Class

2016-03-04 Thread Madhire, Naveen
Here it is, 1.0.0 apacherelease Apache release https://repository.apache.org/content/repositories/orgapacheflink-1069/ true false org.apache.kafka kafka_2.9.1 0.8.2.0 commons-configuration commons-con

Re: java.lang.NoClassDefFoundError for Keys Class

2016-03-04 Thread Robert Metzger
The problem is probably this dependency: org.apache.kafka kafka_2.9.1 0.8.2.0 Since it pulls Scala 2.9. You can probably remove this dependency because our kafka connector will also pull also that kafka dependency. On Fri, Mar 4, 2016 at 8:07 PM, Madhire, Naveen < naveen

Re: java.lang.NoClassDefFoundError for Keys Class

2016-03-04 Thread Madhire, Naveen
Its working now. Thanks Robert and Stephan. From: Robert Metzger mailto:rmetz...@apache.org>> Reply-To: "user@flink.apache.org" mailto:user@flink.apache.org>> Date: Friday, March 4, 2016 at 1:09 PM To: "user@flink.apache.org" mailto:use