re: some questions about submit flink job on flink-yarn

2017-01-15 Thread huangwei (G)
is always overwritten and set to 0 which means random port selection when starting a yarn session. I hope this clarifies things a little bit. Cheers, Till On Sat, Jan 14, 2017 at 2:52 AM, huangwei (G) <huangwei...@huawei.com> wrote: > Hi Till, > > The "9-96-101-177"

re: some questions about submit flink job on flink-yarn

2017-01-13 Thread huangwei (G)
replaced by -. Could you maybe tell me which version of Flink you’re running and also share the complete JobManager log with us? I tested it with the latest 1.2 SNAPSHOT version and there it seemed to work. Cheers, Till ​ On Fri, Jan 13, 2017 at 9:02 AM, huangwei (G) <huangwei...@huawei.com&

some questions about submit flink job on flink-yarn

2017-01-13 Thread huangwei (G)
Dear All, I get an error in jobmanage.log following when I submit a flink job (batch/WordCount.jar) by using command : "./bin/flink run -m 9.96.101.177:39180 ./examples/batch/WordCount.jar". And the flink is on yarn cluster. Error in jobmanage.log : 2017-01-13 15:28:27,402 ERROR

Re: Some questions about storm metrics

2015-09-21 Thread huangwei (G)
extend the constructor of FlinkTopologyContext by a RuntimeContext parameter an add a member to FlinkTopologyContext that hold a refrence to the RuntimeContext. -Matthias On 09/19/2015 10:28 AM, huangwei (G) wrote: > Hi Matthias, > > Very sorry for the email in weekend. > I try to a

Re: Some questions about storm metrics

2015-09-19 Thread huangwei (G)
topologies to be collected and displayed in Flink WebUI. -Matthias On 09/07/2015 02:51 PM, huangwei (G) wrote: > Hi Kostas, > > Thanks for your support! > I created the JIRA and pushed a PR for it. > > @Matthias > Could you give me a storm example in using the cust metrics?

答复: Some questions about storm metrics

2015-09-08 Thread huangwei (G)
, huangwei (G) wrote: > Hi Kostas, > > Thanks for your support! > I created the JIRA and pushed a PR for it. > > @Matthias > Could you give me a storm example in using the cust metrics? > Since it is needed for follow-up development. > Many thanks!

Re: Some questions about storm metrics

2015-09-01 Thread huangwei (G)
thias > > On 08/31/2015 01:01 PM, huangwei (G) wrote: > > Hi, > > I think I can take a simple design for it. > > Do you have any idea? > > > > > > > > Maybe. I did not put any thoughts into this... > >

答复: Some questions about storm metrics

2015-08-31 Thread huangwei (G)
ility layer. >> See README here: >> >> https://github.com/apache/flink/tree/master/flink-contrib/flink-storm >> -compatibility/flink-storm-compatibility-core >> >> It is an open question, if/how Flink can support this feature. >> >> -Matthias >> &g

Some questions about storm metrics

2015-08-31 Thread huangwei (G)
Hi Matthias and all, I`d like to do the custom metrics in flink-storm-compatibility. And there are some questions following: Is there anything like storm metrics in Flink? If not, does Flink need to have metrics mechanism in internal first? Or just let the metrics use in

Re: a problem about flink cluster

2015-08-24 Thread huangwei (G)
Hi Stephan, Specifying taskmanager.hostname works. Thank you! Hi! Usually, TaskManagers should automatically choose a network interface from which they can communicate to the JobManager. They also use that network interface to talk to each other, then. If I understand

Some new problems about flink-storm-compatibility

2015-08-20 Thread huangwei (G)
Hi, I got some new problems about the storm compatibility currently. These occurred when I ran the “storm-wordcount” in the storm compatibility on a flink-0.10SNAPSHOT which I built it over a latest flink project. First, I start a local flink: $ cd bin $ ./start-local.sh Then I ran the

Re: Some new problems about flink-storm-compatibility

2015-08-20 Thread huangwei (G)
/2015 01:19 PM, huangwei (G) wrote: Hi, I got some new problems about the storm compatibility currently. These occurred when I ran the “storm-wordcount” in the storm compatibility on a flink-0.10SNAPSHOT which I built it over a latest flink project. First, I start a local flink

re: re: re: Some problems about Flink applications

2015-08-18 Thread huangwei (G)
Exception { if (value.f0.f1 + value.f1.f1 10){ c.collect(new Tuple2String, Integer(value.f0.f0, value.f0.f1 + value.f1.f1)); } } }); On 13.08.2015 15:24, huangwei (G) wrote: Hi Stephan

Some thoughts about query max min block count in CompactingHashTable

2015-08-18 Thread huangwei (G)
Hi, I found some like the code following may affect performance( Time complexity is O(n) ): private int getMaxPartition() { int maxPartition = 0; for(InMemoryPartitionT p1 : this.partitions) { if(p1.getBlockCount() maxPartition) { maxPartition = p1.getBlockCount(); }

Re: Some thoughts about query max min block count in CompactingHashTable

2015-08-18 Thread huangwei (G)
traversal checks fewer elements, but has more memory jumps (cache misses) on the way. Greetings, Stephan On Tue, Aug 18, 2015 at 11:33 AM, huangwei (G) huangwei...@huawei.com wrote: Hi, I found some like the code following may affect performance( Time complexity is O(n) ): private int

答复: 答复: Some problems about Flink applications

2015-08-13 Thread huangwei (G)
Spout/Bolts with Flink streaming program. -- Maybe we should add some comments in the pom file...) 3) How do you build? The whole source code? Or only parts of it? Did you run install? -Matthias On 08/11/2015 02:47 PM, huangwei (G) wrote: Hi Stephan and Matthias, Sorry for replying

答复: Some problems about Flink applications

2015-08-11 Thread huangwei (G)
to the latest master) You can also have a look here how to package jars correctly (even if this example is about Flink ML): https://stackoverflow.com/questions/31661900/maven-build-has-missing-p ackage-linkages/31662642#31662642 -Matthias On 08/08/2015 11:15 AM, huangwei (G) wrote: Hi

Some problems about Flink applications

2015-08-08 Thread huangwei (G)
Hi, I get some trouble in developing Flink applications. 1. I want to test the performance between Storm and flink-storm-compatibility using the test program: https://github.com/project-flink/flink-perf/blob/master/storm-jobs/src/jvm/experiments/Throughput.java. And there is a bit of my changes

答复: Need help with CI problem

2015-08-05 Thread huangwei (G)
, 2015 at 8:36 AM, huangwei (G) huangwei...@huawei.com wrote: Hi, I cannot see the CI detail as I click the link. It was like: [image: cid:image001.png@01D0CF8B.60774F60] Does anyone help me?