Re: How to get latency info from benchmark

2016-09-03 Thread Eric Fukuda
I got the same error with this commit too. Weird :-( I will try picking the necessary classes. Thanks anyway. On Sat, Sep 3, 2016 at 7:41 AM, Robert Metzger wrote: > I also can't checkout the commit locally... which is weird, because GitHub > still seems to be able to somehow access it. > > Can

Re: How to get latency info from benchmark

2016-09-03 Thread Robert Metzger
I also can't checkout the commit locally... which is weird, because GitHub still seems to be able to somehow access it. Can you try this commit: df42160832ff65ae2a85b478d1dd0b398fa6ef3f ? I actually believe its probably easier to just pick the classes you need from the "benchmark" repository and

Re: How to get latency info from benchmark

2016-09-02 Thread Eric Fukuda
Thanks Robert, I tried to checkout the commit you mentioned, but git returns an error "fatal: reference if not a tree: 547e7490fb99562ca15a2127f0ce1e784db97f3e". I've searched for a solution but could not find any. Am I doing something wrong? - $ git clone https://github.com/rmetz

Re: How to get latency info from benchmark

2016-09-02 Thread Robert Metzger
Hi Eric, I'm sorry that you are running into these issues. I think the version is 0.10-SNAPSHOT, and I think I've used this commit: https://github.com/rmetzger/flink/commit/547e749 for some of the runs (of the throughput / latency tests, not for the yahoo benchmark). The commit should at least poi

Re: How to get latency info from benchmark

2016-09-02 Thread Eric Fukuda
Hi Robert, I've been trying to build the "performance" project using various versions of Flink, but failing. It seems that I need both KafkaZKStringSerializer class and FlinkKafkaConsumer082 class to build the project, but none of the branches has both of them. KafkaZKStringSerializer existed in 0

Re: How to get latency info from benchmark

2016-08-24 Thread Robert Metzger
Hi, Version 0.10-SNAPSHOT is pretty old. The snapshot repository of Apache probably doesn't keep old artifacts around forever. Maybe you can migrate the tests to Flink 0.10.0, or maybe even to a higher version. Regards, Robert On Wed, Aug 24, 2016 at 10:32 PM, Eric Fukuda wrote: > Hi Max, Robe

Re: How to get latency info from benchmark

2016-08-24 Thread Eric Fukuda
Hi Max, Robert, Thanks for the advice. I'm trying to build the "performance" project, but failing with the following error. Is there a solution for this? [ERROR] Failed to execute goal on project streaming-state-demo: Could not resolve dependencies for project com.dataartisans.flink: streaming-st

Re: How to get latency info from benchmark

2016-08-24 Thread Robert Metzger
Hi Eric, Max is right, the tool has been used for a different benchmark [1]. The throughput logger that should produce the right output is this one [2]. Very recently, I've opened a pull request for adding metric-measuring support into the engine [3]. Maybe that's helpful for your experiments. [

Re: How to get latency info from benchmark

2016-08-24 Thread Maximilian Michels
I believe the AnaylzeTool is for processing logs of a different benchmark. CC Jamie and Robert who worked on the benchmark. On Wed, Aug 24, 2016 at 3:25 AM, Eric Fukuda wrote: > Hi, > > I'm trying to benchmark Flink without Kafka as mentioned in this post > (http://data-artisans.com/extending-th

How to get latency info from benchmark

2016-08-23 Thread Eric Fukuda
Hi, I'm trying to benchmark Flink without Kafka as mentioned in this post ( http://data-artisans.com/extending-the-yahoo-streaming-benchmark/). After running flink.benchmark.state.AdvertisingTopologyFlinkState with user.local.event.generator in localConf.yaml set to 1, I ran flink.benchmark.utils.