Re: Flink 1.5, failed to instantiate S3 FS

Thanks Amit for checking. I do not use hadoop, but I am using Flink with bundled HDP 2.8 binary. I think this article is right, I mixed 2.7 lib and 2.8 binary somehow. On Sat, Jun 2, 2018 at 1:05 AM Amit Jain wrote: > Hi Hao, > > Have look over > https://issues.apache.org/jira/browse/HADOOP-1381

Using Flink RocksDBStateBackend with EFS as an NFS mount for large image data

Hi, We have big image data(about 20 MB each) coming in at high frequency/volume from a video stream from many cameras. The current design thought is to store this data in the 1st step of the Flink Dataflow in EFS(NAS) and access the EFS data from the 3rd step in the dataflow(may be in a totally d

Java Code for Kafka Flink SQL

Hi, Could any one help me by providing some sample java code which Flink subscribes data data from kafka and then doing SQL queries using SQL APIs. Also, what are the compatible versions for java/kafka/flink. Since, I am beginner and there are many exceptions in my code public class FlinkKaf

Re: Ask for SQL using kafka in Flink

Thanks Rong, I used Flink 1.3.0 in case of using Flink 1.5 how can I define jsonschema? Yes, there are two names but now I put one name only and I want to define jsonschema. Rong Rong wrote > Hi Radhya, > > Can you provide which Flink version you are using? Based on the latest > FLINK 1.5 r

Re: Ask for SQL using kafka in Flink

Thanks Rong, I used Flink 1.3.0 in case of using Flink 1.5 how can I define jsonschema?Yes, there are two names but now I put one name only and I want to define jsonschema. Rong Rong wrote > Hi Radhya,Can you provide which Flink version you are using? Based on the > latestFLINK 1.5 release, Kafka0

Re: Ask for SQL using kafka in Flink

Thanks Rong, I used Flink 1.3.0 in case of using Flink 1.5 how can I define jsonschema? Yes, there are two names but now I put one name only and I want to define jsonschema. Rong Rong wrote > Hi Radhya, > > Can you provide which Flink version you are using? Based on the latest > FLINK 1.5 rel

Re: ConcurrentModificationException while accessing managed keyed state

Hi Garvit, thanks for you feedback, I see you are using the 1.4.1 with Heap state backend, and there are actually two bugs in 1.4.1 related to the kryo serializer and DefaultOperateStateBackend which may cause the ConcurrentModificationException(when checkpointing), they both have been fixed

Re: ConcurrentModificationException while accessing managed keyed state

Hi ,Garvit Sharma Just read your code snippet, I think it look ok. May need your reproduce senior and exception stackstrace then. Best wishes Aitozi Garvit Sharma wrote > Hi, > > Sorry guys for the delay. I was trying to reproduce the complete error on > my local machine but could not get it

Re: ConcurrentModificationException while accessing managed keyed state

Hi, Sorry guys for the delay. I was trying to reproduce the complete error on my local machine but could not get it though. I will try again with actual traffic and let you the exception Stacktrace. For now, I have the following details available to me. Flink version: 1.4.1 State backend: Heap

Re: ConcurrentModificationException while accessing managed keyed state

Hi Garvit Sharma, Flink run with per parallel with a single thread. Can you show a little code about how you use the keyed state in processFunction Garvit Sharma wrote > Hi, > > I have a use case where I am keeping the keyed state in ProcessFunction. > > Key: Integer personId; > > /** > * The

Re: Flink 1.5, failed to instantiate S3 FS

Hi Hao, Have look over https://issues.apache.org/jira/browse/HADOOP-13811?focusedCommentId=15703276&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15703276 What version of Hadoop are you using? Could you provide classpath used by Flink Job Manager, it is present in