Re: FastR-Flink: a new open source Truffle project

2015-10-26 Thread Juan Fumero
previous operation (object) flink.sum(object) Sum previous > operation (object) flink.execute() It cosumes the operation pipeline > defined flink.collect(object) It consumes the pipeline and return the > R result flink.filter(object, func) Filter previous operation (only > map supported) acco

FastR-Flink: a new open source Truffle project

2015-10-26 Thread Juan Fumero
Hi everyone, we have just published a new open source Truffle project, FastR-Flink. It is available in https://bitbucket.org/allr/fastr-flink FastR is an implementation of the R language on top of Truffle and Graal [3] developed by Purdue University, Johannes Kepler University and Oracle Labs

Options for the JVM in the TaskManagers

2015-07-29 Thread Juan Fumero
Hi, is there any option where I can start the VM with preconfigured values? Something similar to env.java.home: "/path/to/jdk" But java.bootstrap: "-Xbootclasspath " and more options. Thanks Juan

Re: Cluster Execution - log

2015-07-23 Thread Juan Fumero
that you did a code update locally > (client), and forgot to update the cluster (or the other way around)? > > > Greetings, > Stephan > > > > On Thu, Jul 23, 2015 at 3:10 PM, Juan Fumero > wrote: > Hi, >When I execute from Java the app

Cluster Execution - log

2015-07-23 Thread Juan Fumero
Hi, When I execute from Java the app in a cluster, the user app is blocked in this point: log4j:WARN No appenders could be found for logger (org.apache.flink.api.java.ExecutionEnvironment). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.

Re: Cluster execution -jar files-

2015-07-16 Thread Juan Fumero
Missing reference: [1] https://ci.apache.org/projects/flink/flink-docs-release-0.9/apis/cluster_execution.html On Don, 2015-07-16 at 16:04 +0200, Juan Fumero wrote: > Hi, > I would like to use the createRemoteEnvironment to run the application > in a cluster and I have some

Cluster execution -jar files-

2015-07-16 Thread Juan Fumero
Hi, I would like to use the createRemoteEnvironment to run the application in a cluster and I have some questions. Following the documentation in [1] It is not clear to me how to use it. What should be the content of the jar file? All the external libraries that I use? or need to include the pr

Re: Deterministic map?

2015-07-14 Thread Juan Fumero
t; > [1] > https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/api/java/DataSet.html#partitionCustom(org.apache.flink.api.common.functions.Partitioner,%20int) > > > On Jul 15, 2015, at 1:10 AM, Juan Fumero > > wrote: > > > > Hi,

Deterministic map?

2015-07-14 Thread Juan Fumero
Hi, I am doing pure map computation with typical benchmarks like BlackScholes and NBody. I am using local configuration with multiple threads. It seems like, inside the chuck (total size / numThreads) the order is correct. But the ordering between chunks is not correct, giving an incorrect res

Re: Logs meaning states

2015-06-29 Thread Juan Fumero
urn result; Cheers Juan On Mon, 2015-06-29 at 16:04 +0200, Stephan Ewen wrote: > It is not quite easy to understand what you are trying to do. > > > > Can you post your program here? Then we can take a look and give you a > good answer... > > > On Mon, Jun 29

Re: Logs meaning states

2015-06-29 Thread Juan Fumero
el any more. > > > Try to do as much on the DataSet as possible. > > On Mon, Jun 29, 2015 at 2:58 PM, Juan Fumero > wrote: > Hi Stephan, > so should I use another method instead of collect? It seems > multithread is not working with this. >

Re: Logs meaning states

2015-06-29 Thread Juan Fumero
in the flatMap() and the sink is a dummy that executes > nothing. The flatMap writes the data to be collected to the > "accumulator" that delivers it back. > > > Greetings, > Stephan > > > > On Mon, Jun 29, 2015 at 2:30 PM, Juan Fumero > wrote: >

Logs meaning states

2015-06-29 Thread Juan Fumero
Hi, I am starting with Flink. I have tried to look for the documentation but I havent found it clear. I wonder the difference between these two states: FlatMap RUNNING vs DataSink RUNNIG. FlatMap is doing data any data transformation? Compilation? In which point is actually executing the f

Re: Log messages - redirect

2015-06-22 Thread Juan Fumero
> Regards, > Chiwan Park > > > > On Jun 19, 2015, at 8:05 PM, Juan Fumero > wrote: > > > > Hi, > > is there any option (from API level) to redirect the log > messages to a

Log messages - redirect

2015-06-19 Thread Juan Fumero
Hi, is there any option (from API level) to redirect the log messages to a file instead of stdout? Thanks log4j:WARN No appenders could be found for logger (org.apache.flink.api.java.ExecutionEnvironment). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.