Odd error

Getting this: DataStream stream =                env.addSource(new FlinkKafkaConsumer08<>("raw", schema, properties)                ).setParallelism(30).flatMap(new RecordSplit()).setParallelism(30).                        name("Raw splitter").keyBy("id","keyByHelper","schema"); Field expressio

RE: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

That worked.. Thanks Chesnay, [Verizon] Kanagaraj Vengidasamy RTCI 7701 E Telecom PKWY Temple Terrace, FL 33637 O 813.978.4372 | M 813.455.9757 [Twitter] [LinkedIn]

Re: Kinesis connector SHARD_GETRECORDS_MAX default value

Hi Steffan, I have to admit that I didn’t put too much thoughts in the default values for the Kinesis consumer. I’d say it would be reasonable to change the default values to follow KCL’s settings. Could you file a JIRA for this? In general, we might want to reconsider all the default values f

Kinesis connector SHARD_GETRECORDS_MAX default value

Hi there, I recently ran into problems with a Flink job running on an EMR cluster consuming events from a Kinesis stream receiving roughly 15k event/second. Although the EMR cluster was substantially scaled and CPU utilization and system load were well below any alarming threshold, the proces

Re: Cassandra Sink version

Hi Nancy, For both Flink 1.2 and Flink 1.3, our tests are written against Cassandra 2.2.5. We use the version 3.0 of this https://github.com/datastax/java-driver/tree/3.0.x driver. So please check there to see which Cassandra versions they sup

Re: RocksDB segfaults

Hi, for the first checkpoint, from the stacktrace I assume that the backend is not accessed as part of processing an element, but by another thread. Is that correct? RocksDB requires accessing threads to hold the task’s checkpointing lock, otherwise they might call methods on an instance that i

Cassandra Sink version

Hi all, Which Cassandra version is currently supported by the Flink 1.2 connector? and I am wondering which version Flink 1.3 will be supporting? Thank you! -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Cassandra-Sink-version-tp12329.html

Re: RocksDB segfaults

Hi Stephen, you are right, the second stack trace is indeed from a run of Flink 1.1.4. Sorry, my bad. That leaves us with the first trace of a segfault for which I can guarantee that it brought down a 1.2.0 instance. Unfortunately I cannot reproduce the problem. It has happened twice so far, b

Re: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

I guess that's because the grouping is wrong. ^server\.*([0-9])+*[[:space:]]*\=([^: \#]+) should probably be ^server\.*([0-9]+)*[[:space:]]*\=([^: \#]+) Could you modify the .sh script as such and try again? Regards, Chesnay On 22.03.2017 16:10, kanagaraj.vengidas...@verizon.com wrote: Gre

RE: [E] Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Greg, Sorry about that.. when I copy the config and replaced the real server names . I don't have space in my configurations. The issue is when I have server.10=, server.11= And tried to start the zookeeper, Myid replaced with 0 and 1 for these servers which is conflicting with server.0 and s

Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Side note: It's quite uncommon to have more than 9 ZK nodes (at least if the ZK is dedicated to Flink alone). The ZooKeeper docs suggest to add "observers" to scale out ZooKeeper further, not quorum peers, because many quorum peers make ZooKeeper transactions slower (because the quorum that has to

Re: RocksDB segfaults

Hi! It looks like you are running the RocksDB state backend 1.1 (is still an old version packaged into your JAR file?) This line indicates that: org.apache.flink.contrib.streaming.state. RocksDBStateBackend.performSemiAsyncSnapshot (the method does not exist in 1.2 any more) Can you try and run

RocksDB segfaults

Hi, I have experienced two crashes of Flink 1.2 by SIGSEGV, caused by something in RocksDB. What is the preferred way to report them? All I got at the moment are two hs_err_pid12345.log files. They are over 4000 lines long each. Is there anything significant that I should extract to help you gu

Re: unable to add more servers in zookeeper quorum peers in flink 1.2

Kanagaraj, None of the server lines are matching since the regex in start-zookeeper-quorum.sh does not allow for spaces after the equals sign. ^server\.([0-9])+[[:space:]]*\=([^: \#]+) Greg > On Mar 22, 2017, at 8:49 AM, kanagaraj.vengidas...@verizon.com wrote: > > Hi All, > > We are using

unable to add more servers in zookeeper quorum peers in flink 1.2

Hi All, We are using flink 1.2 . Unable to add more than one digit in server.x. when I have more than one digit it is not allowing to start the zookeeper. What I need to do if I want to keep more servers? # ZooKeeper quorum peers server.0= server1:2888:3888 server.1= server2:2888:3888