Re: Broadcasted Variable not updated in a Iterative Data Set

Can you share the respective part of the code, then we can have a look? In general, a DataSet is re-broadcasted in each iteration, if it depends on the IterativeDataSet. You have to re-grab it from the RuntimeContext in each superstep. If you call getBroadcastSet() in the open() method, then it wi

Re: flink ml - k-means

Paul! Can you use the KMeans example? The code is for three-dimensional points, but you should be able to generalize it easily. That would be the fastest way to go. without waiting for any release dates... Stephan On Mon, May 11, 2015 at 2:46 PM, Pa Rö wrote: > hi, > > now i want implement km

Re: Memory exception

What you see is that the effect of extremely little memory arriving at the memory manager. Given that you are saying that restarting the machine helps (or using a downloaded version of Flink), the reason is probably that you do not configure the JVM memory (in which case Java grabs a bit of the sy

Re: EOF reading file from HDFS

Working around dependency mismatches is a bit tricky such large systems that have many (transitive) dependencies. You are probably colliding with the thrift version packaged by Hadoop (which in turn is packaged by Flink). You can fix that for now by compiling your own version of Flink, after setti

Broadcasted Variable not updated in a Iterative Data Set

Hello, I am writing a Flink application whose purpose is to train neural networks. I have implemented a version of SDG based on mini batches using the IterativeDataSet class. In a nutshell, my code groups the input features in a set called mini batch (i have as many mini batches as the parallelism

Re: How to make a generic key for groupBy

My pleasure, happy to hear that it solved your problem. I was wondering: Do more people have a similar structure in their types and utilities? If yes, it may make sense to add a generic version of these utilities to the Fink codebase. On Fri, May 1, 2015 at 11:51 AM, LINZ, Arnaud wrote: > Hi S

Re: flink ml k means relase

Hi, the community didn't decide on a plan for releasing Flink 0.9 yet. Here, you can track the progress for the Flink ML variant of KMeans: https://issues.apache.org/jira/browse/FLINK-1731 There is also a KMeans implementation in the examples of Flink. Maybe that is sufficient for now? --Robert

Re: flink ml - k-means

hi, now i want implement kmeans with flink, maybe you know a release date for flink ml kmeans? best regards paul 2015-04-27 9:36 GMT+02:00 Pa Rö : > Hi Alexander and Till, > > thanks for your informations, I look forward to the release. > I'm curious how well is flink ml against mahout und spar

flink ml k means relase

hi, now i want implement kmeans with flink, maybe you know a release date for flink ml kmeans? best regards paul

Re: Error when use tow datasink

This looks like a dependency issue of the asm dependency. Did you upgrade all flink dependencies to the 0.8.1 version? I think you also refresh / reimport the project in your IDE so that it can pick up the new dependencies. On Mon, May 11, 2015 at 10:21 AM, hagersaleh wrote: > when upgrade from

Re: Error when use tow datasink

when upgrade from flink 0.7.0 to 0.8.1 display error message I use netbeanse when replace lib flink 0.7.0 to to lib 0.8.1 Exception in thread "main" java.lang.NoClassDefFoundError: org/objectweb/asm/ClassVisitor at org.apache.flink.api.java.DataSet.clean(DataSet.java:133) at org.apache.fl