Re: Queryable state feature in latest version!!

2023-11-06 Thread Hangxiang Yu
Hi, Puneet. Queryable State has been deprecated in the latest version which will be removed in Flink 2.0. The Interface/Usage is freezed in the 1.x, so you still could reference the documents of previous versions to use it. BTW, Could you also share something about your scenarios using

Re: Queryable state feature in latest version!!

2023-11-06 Thread Junrui Lee
Hi, Puneet Thank you for reaching out. In the latest release of Flink (version 1.18), we have marked Queryable State as @Deprecated and removed the related content from the stable documentation. This means that Queryable State is no longer actively supported or recommended for use. More details

Queryable state feature in latest version!!

2023-11-06 Thread Puneet Kinra
Hi All We are using flink 1.10 version which were having Queryable state for querying the in-memory state. we are planning to migrate our old applications to newer version of the flink ,In latest version documents I can't find any reference to it. can anyone highlight what was approach to query

Re: Queryable State Deprecation

2022-03-11 Thread Ron Crocker
tters who could spend > time on getting the Queryable state to a production ready state. I might be > speaking for myself here, but I think the main use case for the queryable > state is to have an insight into the current state of the application for > debugging purposes. If it is use

Re: Queryable State Deprecation

2022-02-12 Thread Frank Dekervel
r use case into looking at queryable state is that we have many readers and a very few writers (readers to writers ratio in the 1000s). Each consuming application (reader) needs a live view of a subset of the state and these applications come online and go offline many times a day. What would be a

Re: Queryable State Deprecation

2022-02-12 Thread Jatti, Karthik
Hi Frank, What sink did you end up choosing for materializing the state ? Our use case into looking at queryable state is that we have many readers and a very few writers (readers to writers ratio in the 1000s). Each consuming application (reader) needs a live view of a subset of the state

Re: Queryable State Deprecation

2022-02-11 Thread Jatti, Karthik
Thank you Frank and Dawid for providing the context here. From: Frank Dekervel Date: Friday, February 4, 2022 at 9:56 AM To: user@flink.apache.org Subject: Re: Queryable State Deprecation EXTERNAL SENDER Hello, To give an extra datapoint: after a not so

Re: Queryable State Deprecation

2022-02-04 Thread Frank Dekervel
Hello, To give an extra datapoint: after a not so successful experiment with faust-streaming we moved our application to flink. Since flinks queryable state was apparently  stagnant, we implemented what was needed to sink the state to an external data store for querying. However

Re: Queryable State Deprecation

2022-02-04 Thread Dawid Wysakowicz
Hi Karthik, The reason we deprecated it is because we lacked committers who could spend time on getting the Queryable state to a production ready state. I might be speaking for myself here, but I think the main use case for the queryable state is to have an insight into the current state

Queryable State Deprecation

2022-02-01 Thread Jatti, Karthik
Hi, I see on the Flink Roadmap that Queryable state API is scheduled to be deprecated but I couldn’t find much information on confluence or this mailing group’s archives to understand the background as to why it’s being deprecated and what would be a an alternative. Any pointers to help me

Re: Regarding Queryable state in Flink

2022-01-25 Thread Martijn Visser
Hi Jessy, Queryable State is considered approaching end of life [1] per the Flink Roadmap. There are currently no development activities planned for it. Best regards, Martijn [1] https://flink.apache.org/roadmap.html Op di 25 jan. 2022 om 18:00 schreef Jessy Ping > Hi Matthias, > &g

Re: Regarding Queryable state in Flink

2022-01-25 Thread Jessy Ping
Hi Matthias, I want to query the current state of the application at real-time. Hence, state processor API won't fit here. I have the following questions, * Is the queryable state stable enough to use in production systems ?. Are there any improvements or development activities planned or going

RE: Regarding Queryable state in Flink

2022-01-25 Thread Schwalbe Matthias
To: user Subject: Regarding Queryable state in Flink Hi Team, We are currently running our streaming application based Flink(Datastream API ) on a non-prod cluster.And planning to move it to production cluster soon.. We are keeping cerating keyed state backed by rocksdb in the flink application

Regarding Queryable state in Flink

2022-01-24 Thread Jessy Ping
for debugging and troubleshooting. Is it a good idea to use Queryable state for a single link-job running in application-mode on kubernetes for an average load of 10k events/second ?. Or is it a better idea to keep these state values in an external k,v store ?. So in short --> Is the querya

Re: Queryable State Lookup Failure

2021-07-26 Thread Roman Khachatryan
ze: 512m > > > > But now I have the below issue at exactly or approximately at the same time > i.e. about after 5000 records. It doesn’t matter whether I send them in a > burst or stagger them, strangely after that limit, it always blows up i.e. > approx near to 4.5

Queryable State Lookup Failure

2021-07-23 Thread Sandeep khanzode
i.e. approx near to 4.5 to 5.5 records. Now I am doing multiple state lookups for the Queryable State. Previously I used to do about 50% compared to what I did not and I could ingest millions of records. But simply doubling the number of lookups has caused the Queryable State to fail. What

Re: Queryable State unavailable after Kubernetes HA State cleanup

2021-04-29 Thread Till Rohrmann
Hi Sandeep, I don't fully understand the problematic scenario yet. What exactly is the HA state maintained by Kubernetes in S3? Queryable state works by asking for the current state of an operator. If you use asQueryableState, then you create a reducing state which appends all stream elements

Queryable State unavailable after Kubernetes HA State cleanup

2021-04-28 Thread Sandeep khanzode
Hello, Stuck at this time. Any help will be appreciated. I am able to create a queryable state and also query the state. Everything works correctly. KeyedStream, Key> stream = sourceStream.keyBy(t2 -> t2.f0); stream.asQueryableState("queryableVO"); I deploy this on a Ku

Re: Flink Native Kubernetes 部署模式下,如何开启Queryable State?

2021-03-25 Thread Yang Wang
Native Kubernets部署模式下,如何开启Queryable > State呢?官网提供了Standaleon K8S下开启的说明( > https://ci.apache.org/projects/flink/flink-docs-stable/deployment/resource-providers/standalone/kubernetes.html#enabling-queryable-state),但Native > K8S部署模式下,无论是Session还是Application 模式,Flink相关k8s > spec基本都是由flink代码生成,尤其

Flink Native Kubernetes 部署模式下,如何开启Queryable State?

2021-03-25 Thread tian lin
各位好:  请教Flink 1.12.1 在Flink Native Kubernets部署模式下,如何开启Queryable State呢?官网提供了Standaleon K8S下开启的说明(https://ci.apache.org/projects/flink/flink-docs-stable/deployment/resource-providers/standalone/kubernetes.html#enabling-queryable-state),但Native K8S部署模式下,无论是Session还是Application 模式,Flink相关k8s

Re: timeWindow()s and queryable state

2021-03-04 Thread Dawid Wysakowicz
Hey Ron, I am pretty sure the queryable state will not do any pruning. It will keep the state for all windows seen so far. The allowedLateness applies to the window computation not the queryable state part. The `asQueryableState` will create a downstream operator that will keep updating a state

timeWindow()s and queryable state

2021-03-01 Thread Ron Crocker
Hi all - I’m trying to keep some state around for a little while after a window fires to use as queryable state. I am intending on using something like: .keyBy() .timeWindow(Time.minutes(1)).allowedLateness(Time.minutes(90)) .aggregate(…) .keyBy() .asQueryableState(...) My intent is to keep

Re: Cannot connect to queryable state proxy

2021-02-08 Thread Khachatryan Roman
Hi ChangZhuo, Queryable state is exposed on the same address as the TM RPC. You can change this address by modifying taskmanager.host [1]. However, I'm not sure if setting it to 127.0.0.1 or localhost will not break connectivity with the other components. [1] https://ci.apache.org/projects/flink

Re: Cannot connect to queryable state proxy

2021-02-07 Thread 陳昌倬
On Thu, Feb 04, 2021 at 04:26:42PM +0800, ChangZhuo Chen (陳昌倬) wrote: > Hi, > > We have problem connecting to queryable state client proxy as described > in [0]. Any help is appreciated. > > * The port 6125 is opened in taskmanager pod. > > ``` > root@-654

Cannot connect to queryable state proxy

2021-02-04 Thread 陳昌倬
Hi, We have problem connecting to queryable state client proxy as described in [0]. Any help is appreciated. The following is our setup: * Flink 1.12.1 * Standalone Kubernetes * Related config in flink-conf.yaml ``` queryable-state.enable: true queryable-state.proxy.ports: 6125

Re: Queryable state on task managers that are not running the job

2020-12-23 Thread Yun Tang
Hi Martin, What kind of deploy mode you choose? If you use per-job mode [1] to launch jobs, there might exist only idle slots instead of idle taskmanagers. Currently, queryable state is bounded to specific job and if the idle taskmanager is not registered in the target's resource manager

Queryable state on task managers that are not running the job

2020-12-21 Thread Martin Boyanov
Hi, I'm running a long-running flink job in cluster mode and I'm interested in using the queryable state functionality. I have the following problem: when I query the flink task managers (i.e. the queryable state proxy), it is possible to hit a task manager which doesn't have the requested state

Re: Using Queryable State within 1 job + docs suggestion

2020-05-26 Thread Annemarie Burger
, the Queryable state CompletableFuture objects always return empty. Below is the relevant code. Any idea what I'm doing wrong? Any help much appreciated. The state is a MapState>. This represents a edge list of a graph, sorted by source vertex id, and then by target vertex id. // The method c

Re: Using Queryable State within 1 job + docs suggestion

2020-05-22 Thread m@xi
Hi Gordon, Yes we are well aware of the inconsistencies that can (and will) emerge while using queryable state like this. However, we will treat them manually for ensuring the correctness of our targeting applications. Therefore, could you help with Annemarie's question or are you aware

Re: Using Queryable State within 1 job + docs suggestion

2020-05-21 Thread Tzu-Li (Gordon) Tai
This in general is not a good idea, as the state you query using queryable state within a job does not provide any consistency guarantees at all. Would it be possible to have some trigger that emits state of the windows, and join the states downstream? In general, that is a better approach

Re: Using Queryable State within 1 job + docs suggestion

2020-05-21 Thread Tzu-Li (Gordon) Tai
Hi, That in general is not a good idea, with the problem you mentioned as well as the fact that the state you query within the same job using queryable state does not provide any means of consistency guarantee. When it comes to "querying state from another operator", it is a hint tha

Re: Using Queryable State within 1 job + docs suggestion

2020-05-21 Thread Annemarie Burger
Hi, So what I meant was that I have a keyed stream, and from each thread/keygroup/PU I want to query the state of the other threads/keygroups/PUs. Does anybody have any experience with this? I'm currently working on it, and the main problem seems to be that the Queryable State Client requires

Re: Using Queryable State within 1 job + docs suggestion

2020-05-21 Thread Yun Tang
that they could query each other, which provide better performance. Best Yun Tang From: Annemarie Burger Sent: Thursday, May 21, 2020 19:45 To: user@flink.apache.org Subject: Re: Using Queryable State within 1 job + docs suggestion Hi, Thanks for your response! What if I'm

Re: Using Queryable State within 1 job + docs suggestion

2020-05-21 Thread Annemarie Burger
Hi, Thanks for your response! What if I'm using regular state instead of windowState, is there any way to use query this state of a PU from another PU in the same Flink job? Best, Annemarie -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Using Queryable State within 1 job + docs suggestion

2020-05-18 Thread Yun Tang
this feature on server side as Flink job needs to access the queryable-state classes. If you're just running your Flink job locally, add dependency could let your local job access the queryable-state classes which is actually the doc wanted to tell users. Best Yun Tang

Using Queryable State within 1 job + docs suggestion

2020-05-18 Thread Annemarie Burger
Hi, I want to use Queryable State to communicate between PU's in the same Flink job. I'm aware this is not the intended use of Queryable State, but I was wondering if and how it could be done. More specifically, I want to query the (event-time) window state of one PU, from another PU, while both

Re: Windowed Stream Queryable State Support

2020-05-18 Thread Annemarie Burger
Hi, I was wondering that since it is possible to "query the state of an in-flight window", if it is also possible to make sure we query *every* window at the proper time. So how to access in flight window state of a window of a PU from another PU with Queryable State. I want to query

????????Queryable State ????????????????

2019-11-14 Thread claylin

Re: Queryable State 查询反序列化问题

2019-11-12 Thread Congxian Qiu
Hi 从错误栈来看,应该是 serializer 不一致导致的,可以再检查下相应的 key/namespace serialzier Best, Congxian chengwenfeng 于2019年11月12日周二 下午2:47写道: > 大家好: > 我在测试Querable State功能的时候,发现 > 语法 > dataStream.keyby(key).process(); 这种语法下,简单的状态和复杂的POJO都可以查询 > 但在 >

Queryable State 查询反序列化问题

2019-11-11 Thread chengwenfeng
大家好: 我在测试Querable State功能的时候,发现 语法 dataStream.keyby(key).process(); 这种语法下,简单的状态和复杂的POJO都可以查询 但在 studentAnswerDataStream.connect(learningStrategyDataStream) .keyBy(val->val.getCourseId()+"_"+val.getTaskId() ,

Re: Is Queryable State not working in standalone cluster with 1 task manager?

2019-07-30 Thread Oytun Tez
Fabian Hueske wrote: > >> Hi Oytun, >> >> Is QS enabled in your Docker image or did you enable QS by copying/moving >> flink-queryable-state-runtime_2.11-1.8.0.jar from ./opt to ./lib [1]? >> >> Best, Fabian >> >> [1] >> https://ci.apa

Re: Is Queryable State not working in standalone cluster with 1 task manager?

2019-07-30 Thread Oytun Tez
On Tue, Jul 30, 2019 at 3:42 AM Fabian Hueske wrote: > Hi Oytun, > > Is QS enabled in your Docker image or did you enable QS by copying/moving > flink-queryable-state-runtime_2.11-1.8.0.jar from ./opt to ./lib [1]? > > Best, Fabian > > [1] > https://ci.apache.or

Re: Is Queryable State not working in standalone cluster with 1 task manager?

2019-07-30 Thread Fabian Hueske
Hi Oytun, Is QS enabled in your Docker image or did you enable QS by copying/moving flink-queryable-state-runtime_2.11-1.8.0.jar from ./opt to ./lib [1]? Best, Fabian [1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#activating-queryable-state Am

Re: Is Queryable State not working in standalone cluster with 1 task manager?

2019-07-29 Thread Oytun Tez
> queryable-state.enable: > true config directly via Configuration, then I can at least see that the > queryable state server is spinning up. > > Any pointers? With standalone-job.sh, it seems that it doesn't care about > queryable-state.enable config inside flink-conf.yaml

Is Queryable State not working in standalone cluster with 1 task manager?

2019-07-29 Thread Oytun Tez
and provide queryable-state.enable: true config directly via Configuration, then I can at least see that the queryable state server is spinning up. Any pointers? With standalone-job.sh, it seems that it doesn't care about queryable-state.enable config inside flink-conf.yaml. --- Oytun Tez *M O T A W

Re: Flink queryable state - io.netty4 errors

2019-07-07 Thread Yun Tang
@flink.apache.org Subject: Flink queryable state - io.netty4 errors I am using Flink queryable state client to read a dummy valuestate I created. The code is quite simple, just created a stream out of a kafka topic, keys it as inputStream.keyBy(value -> 0L).map(new mapFucnt

Flink queryable state - io.netty4 errors

2019-07-07 Thread Shivam Dubey
I am using Flink queryable state client to read a dummy valuestate I created. The code is quite simple, just created a stream out of a kafka topic, keys it as     inputStream.keyBy(value -> 0L).map(new mapFucntion()).print() Withing the mapFunction I create a value state, and decl

Flink queryable state - io.netty4 errors

2019-07-07 Thread Shivam Dubey
I am using Flink queryable state client to read a dummy valuestate I created. The code is quite simple, just created a stream out of a kafka topic, keys it as inputStream.keyBy(value -> 0L).map(new mapFucntion()).print() Withing the mapFunction I create a value st

Re: Queryable state and TTL

2019-07-06 Thread Avi Levi
Thanks, I'll check it out. On Sun, Jul 7, 2019 at 5:40 AM Eron Wright wrote: > *This Message originated outside your organization.* > -- > Here's a PR for queryable state TLS that I closed because I didn't have > time, and because I get th

Re: Queryable state and TTL

2019-07-06 Thread Eron Wright
Here's a PR for queryable state TLS that I closed because I didn't have time, and because I get the impression that the queryable state feature is used very often.Feel free to take it up, if you like. https://github.com/apache/flink/pull/6626 -Eron On Wed, Jul 3, 2019 at 11:21 PM Avi Levi

Re: Queryable state and TTL

2019-07-04 Thread Avi Levi
e road map but I am not aware about plans of any contributor >> to work on it for the next releases. >> I think the community will firstly work on the event time support for TTL. >> I will loop Yu in, maybe he has some plans to work on TTL for the >> queryable state. >> >

Re: Queryable state and TTL

2019-07-03 Thread Yu Li
are about plans of any contributor to > work on it for the next releases. > I think the community will firstly work on the event time support for TTL. > I will loop Yu in, maybe he has some plans to work on TTL for the > queryable state. > > Best, > Andrey > > On Wed, Ju

Re: Queryable state and TTL

2019-07-03 Thread Andrey Zagrebin
Hi Avi, It is on the road map but I am not aware about plans of any contributor to work on it for the next releases. I think the community will firstly work on the event time support for TTL. I will loop Yu in, maybe he has some plans to work on TTL for the queryable state. Best, Andrey On Wed

Queryable state and TTL

2019-07-03 Thread Avi Levi
Hi, Adding queryable state to state with ttl is not supported at 1.8.0 (throwing java.lang.IllegalArgumentException: Queryable state is currently not supported with TTL) I saw in previous mailing thread <http://mail-archives.apache.org/mod_mbox/flink-user/201808.mbox/%3c300f09f0-053e-43ba-a

Re: RE: [DISCUSS] Improve Queryable State and introduce a QueryServerProxy component

2019-07-02 Thread vino yang
af_uI/edit?usp=sharing yanghua1127 于2019年6月7日周五 下午11:32写道: > Hi Georgi, > > Thanks for your feedback. And glad to hear you are using queryable state. > > I agree that implementation of option 1 is easier than others. However, > when we design the new architecture we need t

Re: RE: [DISCUSS] Improve Queryable State and introduce a QueryServerProxy component

2019-06-07 Thread yanghua1127
Hi Georgi, Thanks for your feedback. And glad to hear you are using queryable state. I agree that implementation of option 1 is easier than others. However, when we design the new architecture we need to consider more aspects .e.g. scalability. So it seems option 3 is more suitable. Actually

RE: [DISCUSS] Improve Queryable State and introduce a QueryServerProxy component

2019-06-07 Thread Georgi Stoyanov
5:18 PM To: dev ; user Cc: Stefan Richter ; Aljoscha Krettek ; kklou...@gmail.com Subject: [DISCUSS] Improve Queryable State and introduce a QueryServerProxy component Hi all, I want to share my thought with you about improving the queryable state and introducing a QueryServerProxy component

Re: Queryable State race condition or serialization errors?

2019-05-27 Thread Tzu-Li (Gordon) Tai
Hi Burgess, Would you be able to provide a minimal project that can reproduce your error? That would help a lot with figuring out the issue. If you prefer to share that only privately, please feel free to send me a private email with the details. Another thing you can do is set logging level to

Re: Queryable State race condition or serialization errors?

2019-05-21 Thread burgesschen
Hi Gary. Thanks for the reply. I am using RocksDBStateBackend though. Best, Chen -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Queryable State race condition or serialization errors?

2019-05-21 Thread Gary Yao
/queryable_state.html On Tue, May 21, 2019 at 5:06 AM burgesschen wrote: > Hi Guys, > > I observed some strange behaviors while using Queryable state with Flink > 1.6.2. Here is the story: > > My state is of type MapState[String, Map[String, String]]. the inner map is > frequently u

Queryable State race condition or serialization errors?

2019-05-20 Thread burgesschen
Hi Guys, I observed some strange behaviors while using Queryable state with Flink 1.6.2. Here is the story: My state is of type MapState[String, Map[String, String]]. the inner map is frequently updated. Upon querying, sometimes the returned inner map can miss some fields. What's more, sometimes

Re: Queryable state formal release plan

2019-05-20 Thread Aljoscha Krettek
Hi, Currently no committers (or PMC members) are focusing on the queryable state feature. This will probably mean that not much is going to happen there in the near future. However, there is some discussion on the development by the larger community about QS: https://lists.apache.org

Queryable state formal release plan

2019-05-20 Thread Praveen Chandna
Hi As the Queryable state is in the Beta state, Can you Please confirm the plan for the formal release of the feature Queryable state. Is there any timeline by when this would be added to Flink. Thanks !!! /// Regards Praveen Chandna Product Owner, Mobile +91 9873597204 | ECN: 2864

Re: Queryable state support in Flink 1.9

2019-04-16 Thread Boris Lublinsky
Thanks thats it. Boris Lublinsky FDP Architect boris.lublin...@lightbend.com https://www.lightbend.com/ > On Apr 16, 2019, at 8:31 AM, Guowei Ma wrote: > > AbstractQueryableStateTestBase

Re: Queryable state support in Flink 1.9

2019-04-16 Thread Guowei Ma
Hi, 1. I think Mini cluster supports queryable state. 2. You could set queryable-state.enable to true and try again. You could check AbstractQueryableStateTestBase and there are some tests. :) Best, Guowei Boris Lublinsky 于2019年4月16日周二 下午9:09写道: > Thanks Guowei > The questions tha

Re: Queryable state support in Flink 1.9

2019-04-16 Thread Boris Lublinsky
Thanks Guowei The questions that I am asking is slightly different: 1. Does Mini cluster support queryable state? 2. If the answer is yes, how to set it up? Boris Lublinsky FDP Architect boris.lublin...@lightbend.com https://www.lightbend.com/ > On Apr 15, 2019, at 12:07 AM, Guowei Ma wr

Re: Queryable state support in Flink 1.9

2019-04-14 Thread Guowei Ma
Hi, I think you should check TM log first and check if there are some info like: 1430 [main] INFO org.apache.flink.queryablestate.server.KvStateServerImpl - Started Queryable State Server @ /127.0.0.1:9069. 1436 [main] INFO org.apache.flink.queryablestate.client.proxy.KvStateClientProxyImpl

Queryable state support in Flink 1.9

2019-04-14 Thread Boris Lublinsky
(JobManagerOptions.ADDRESS, "localhost"); config.setInteger(TaskManagerOptions.NUM_TASK_SLOTS, parallelism); // In a non MiniCluster setup queryable state is enabled by default. config.setString(QueryableStateOptions.PROXY_PORT_RAN

Re: Do we have an example of setting up Queryable state ( proxies, client etc ) on k8s ?

2019-04-01 Thread Vishal Santoshi
g towards Queryable State is the architectural > preference of Prometheus to scrape ( pull rather then push model ) and our > intent to expose aggregations. That said your idea makes sense. The worry I > had is the ip resolution of TMs that QueryableStateClient does and our > wanting to av

Re: Do we have an example of setting up Queryable state ( proxies, client etc ) on k8s ?

2019-03-29 Thread Vishal Santoshi
Thanks Konstantin, That makes sense. To give you some context, the reason we are gravitating towards Queryable State is the architectural preference of Prometheus to scrape ( pull rather then push model ) and our intent to expose aggregations. That said your idea makes

Re: Do we have an example of setting up Queryable state ( proxies, client etc ) on k8s ?

2019-03-29 Thread Konstantin Knauf
the > local TM query server. > > . > > . > > - name: queryable-state > > image: _IMAGE_ > > args: ["queryable-state"] > > env: > > - name: POD_IP > > valueFrom: > > fieldRef: >

Re: Do we have an example of setting up Queryable state ( proxies, client etc ) on k8s ?

2019-03-27 Thread Vishal Santoshi
the local TM query server. . . - name: queryable-state image: _IMAGE_ args: ["queryable-state"] env: - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP ports: - container

Do we have an example of setting up Queryable state ( proxies, client etc ) on k8s ?

2019-03-25 Thread Vishal Santoshi
I have 2 options 1. A Rest Based, in my case a Jetty/REST based QueryableStateClient in a side car container colocated on JM ( Though it could on all TMs but that looks to an overkill ) 2.A Rest Based, in my case a Jetty/REST based QueryableStateClient in a side car container colocated on

Re: Queryable state when key is UUID - getting Kyro Exception

2018-11-14 Thread Jayant Ameta
er, which is usually >>> used in POJO class, but from the error messages, I think you can add >>> flink-avro as a dependency and try again. >>> >>> >>> Best, >>> >>> Jiayi Liao >>> >>> Original Message >>> *Sender

Re: Queryable state when key is UUID - getting Kyro Exception

2018-11-13 Thread Till Rohrmann
add flink-avro >> as a dependency and try again. >> >> >> Best, >> >> Jiayi Liao >> >> Original Message >> *Sender:* Jayant Ameta >> *Recipient:* bupt_ljy >> *Cc:* trohrmann; Tzu-Li (Gordon) Tai< >> tzuli...@apache.org>; user &

Re: Queryable state when key is UUID - getting Kyro Exception

2018-11-13 Thread Jayant Ameta
hink you can add flink-avro > as a dependency and try again. > > > Best, > > Jiayi Liao > > Original Message > *Sender:* Jayant Ameta > *Recipient:* bupt_ljy > *Cc:* trohrmann; Tzu-Li (Gordon) Tai< > tzuli...@apache.org>; user > *Date:* Tuesday, Nov

Re: Queryable state when key is UUID - getting Kyro Exception

2018-11-13 Thread bupt_ljy
...@163.com Cc:trohrmanntrohrm...@apache.org; Tzu-Li (Gordon) taitzuli...@apache.org; useru...@flink.apache.org Date:Tuesday, Nov 13, 2018 16:15 Subject:Re: Queryable state when key is UUID - getting Kyro Exception Thanks Jiayi, I updated the client code to use keyed stream key. The key is a Tuple2UUID

Re: Queryable state when key is UUID - getting Kyro Exception

2018-11-13 Thread Jayant Ameta
aged keyed state. > >By the way, I think we should optimize the error messages with which > what Jayant met. > > Best, > Jiayi Liao > > Original Message > *Sender:* Jayant Ameta > *Recipient:* trohrmann > *Cc:* bupt_ljy; Tzu-Li (Gordon) Tai; > user > *Date:*

Re: Queryable state when key is UUID - getting Kyro Exception

2018-11-12 Thread bupt_ljy
Date:Tuesday, Nov 13, 2018 13:39 Subject:Re: Queryable state when key is UUID - getting Kyro Exception Hi Till, Here is the client snippet. Here Rule is a custom POJO that I use. public static void main(String[] args) throws IOException, InterruptedException, ExecutionException { UUID uuid

Re: Queryable state when key is UUID - getting Kyro Exception

2018-11-12 Thread Jayant Ameta
ll looks okay. >>>> >>>> I’m not very sure about this problem. Maybe you can write a very >>>> simple demo to try if it works. >>>> >>>> >>>> Jiayi Liao, Best >>>> >>>> Original Message >>>> *

Re: Queryable state when key is UUID - getting Kyro Exception

2018-11-09 Thread Till Rohrmann
rite a very >>> simple demo to try if it works. >>> >>> >>> Jiayi Liao, Best >>> >>> Original Message >>> *Sender:* Jayant Ameta >>> *Recipient:* bupt_ljy >>> *Cc:* Tzu-Li (Gordon) Tai; user< >>> user@flink.a

Re: Queryable state when key is UUID - getting Kyro Exception

2018-11-08 Thread Jayant Ameta
’m not very sure about this problem. Maybe you can write a very >> simple demo to try if it works. >> >> >> Jiayi Liao, Best >> >> Original Message >> *Sender:* Jayant Ameta >> *Recipient:* bupt_ljy >> *Cc:* Tzu-Li (Gordon) Tai; user< >>

Re: Queryable state when key is UUID - getting Kyro Exception

2018-11-07 Thread Till Rohrmann
:* Jayant Ameta > *Recipient:* bupt_ljy > *Cc:* Tzu-Li (Gordon) Tai; user > > *Date:* Monday, Oct 29, 2018 11:53 > *Subject:* Re: Queryable state when key is UUID - getting Kyro Exception > > Hi Jiayi, > Any further help on this? > > Jayant Ameta > > > On Fri, O

Re: Queryable state when key is UUID - getting Kyro Exception

2018-10-30 Thread bupt_ljy
...@gmail.com Recipient:bupt_ljybupt_...@163.com Cc:Tzu-Li (Gordon) taitzuli...@apache.org; useru...@flink.apache.org Date:Monday, Oct 29, 2018 11:53 Subject:Re: Queryable state when key is UUID - getting Kyro Exception Hi Jiayi, Any further help on this? Jayant Ameta On Fri, Oct 26, 2018 at 9:22 AM

Re: Queryable state when key is UUID - getting Kyro Exception

2018-10-28 Thread Jayant Ameta
omString("3b3f17a0-d81a-11e8-bb91-7fd1412de84d"), >>> TypeInformation.of(new TypeHint() {}), descriptor); >>> >>> >> Jiayi Liao, Best >> >> >> Original Message >> *Sender:* Jayant Ameta >> *Recipient:* bupt_ljy >> *Cc:

Re: Queryable state when key is UUID - getting Kyro Exception

2018-10-25 Thread Jayant Ameta
Original Message > *Sender:* Jayant Ameta > *Recipient:* bupt_ljy > *Cc:* Tzu-Li (Gordon) Tai; user > > *Date:* Friday, Oct 26, 2018 02:26 > *Subject:* Re: Queryable state when key is UUID - getting Kyro Exception > > Also, I haven't provided any custom serializer in my flink job

Re: Queryable state when key is UUID - getting Kyro Exception

2018-10-25 Thread bupt_ljy
iao, Best Original Message Sender:Jayant ametawittyam...@gmail.com Recipient:bupt_ljybupt_...@163.com Cc:Tzu-Li (Gordon) taitzuli...@apache.org; useru...@flink.apache.org Date:Friday, Oct 26, 2018 02:26 Subject:Re: Queryable state when key is UUID - getting Kyro Exception Also, I haven't provid

Re: Queryable state when key is UUID - getting Kyro Exception

2018-10-25 Thread Jayant Ameta
Also, I haven't provided any custom serializer in my flink job. Shouldn't the same configuration work for queryable state client? Jayant Ameta On Thu, Oct 25, 2018 at 4:15 PM Jayant Ameta wrote: > Hi Gordon, > Following is the stack trace that I'm getting: > > *Exception in

Re: Queryable state when key is UUID - getting Kyro Exception

2018-10-25 Thread Jayant Ameta
e > *Sender:* Tzu-Li (Gordon) Tai > *Recipient:* Jayant Ameta; bupt_ljy > > *Cc:* user > *Date:* Thursday, Oct 25, 2018 17:18 > *Subject:* Re: Queryable state when key is UUID - getting Kyro Exception > > Hi Jayant, > > What is the Kryo exception message that you are

Re: Queryable state when key is UUID - getting Kyro Exception

2018-10-25 Thread bupt_ljy
, Best Original Message Sender:Tzu-Li (Gordon) taitzuli...@apache.org Recipient:Jayant ametawittyam...@gmail.com; bupt_ljybupt_...@163.com Cc:useru...@flink.apache.org Date:Thursday, Oct 25, 2018 17:18 Subject:Re: Queryable state when key is UUID - getting Kyro Exception Hi Jayant, What

Re: Queryable state when key is UUID - getting Kyro Exception

2018-10-25 Thread Tzu-Li (Gordon) Tai
Recipient: user Date: Thursday, Oct 25, 2018 14:17 Subject: Queryable state when key is UUID - getting Kyro Exception I get Kyro exception when querying the state.   Key: UUID MapState Client code snippet: CompletableFuture> resultFuture = client.getKvState(JobID.fromHexStr

Re: Queryable state when key is UUID - getting Kyro Exception

2018-10-25 Thread Jayant Ameta
he > same Serializer as the Flink program do? Could you show the serializer in > descriptor? > > > > Jiayi Liao, Best > > Original Message > *Sender:* Jayant Ameta > *Recipient:* user > *Date:* Thursday, Oct 25, 2018 14:17 > *Subject:* Queryable state when key is

Re: Queryable state when key is UUID - getting Kyro Exception

2018-10-25 Thread bupt_ljy
Hi, It seems that your codes are right. Are you sure that you’re using the same Serializer as the Flink program do? Could you show the serializer in descriptor? Jiayi Liao, Best Original Message Sender:Jayant ametawittyam...@gmail.com Recipient:useru...@flink.apache.org Date:Thursday, Oct

Queryable state when key is UUID - getting Kyro Exception

2018-10-25 Thread Jayant Ameta
I get Kyro exception when querying the state. Key: UUID MapState Client code snippet: CompletableFuture> resultFuture = client.getKvState(JobID.fromHexString("c7b8af14b8afacf4fac16cdd0da7e997"), "rule", UUID.fromString("3b3f17a0-d81a-11e8-bb91-7fd1412de84d"),

Re: Flink 1.4: Queryable State Client

2018-10-15 Thread Kostas Kloudas
8 um 18:02 schrieb Seye Jin : >> >> I recently upgraded to flink 1.4 from 1.3 and leverage Queryable State >> client in my application. I have 1 jm and 5 tm all serviced behind >> kubernetes. A large state is built and distributed evenly across task >> mangers and

Re: Flink 1.4: Queryable State Client

2018-10-14 Thread Jörn Franke
; > I recently upgraded to flink 1.4 from 1.3 and leverage Queryable State client > in my application. I have 1 jm and 5 tm all serviced behind kubernetes. A > large state is built and distributed evenly across task mangers and the > client can query state for specified key >

Re: Flink 1.4: Queryable State Client

2018-10-13 Thread vino yang
age Queryable State > client in my application. I have 1 jm and 5 tm all serviced behind > kubernetes. A large state is built and distributed evenly across task > mangers and the client can query state for specified key > > Issue: if a task manager dies and a new one gets spu

Flink 1.4: Queryable State Client

2018-10-13 Thread Seye Jin
I recently upgraded to flink 1.4 from 1.3 and leverage Queryable State client in my application. I have 1 jm and 5 tm all serviced behind kubernetes. A large state is built and distributed evenly across task mangers and the client can query state for specified key Issue: if a task manager dies

Re: Should Queryable State Server be listening on 127.0.1.1?

2018-09-26 Thread Till Rohrmann
sue where I am starting a standalone flink cluster > in an lxc container. When my TaskManager starts up, the queryable state > proxy starts listening on 127.0.1.1:9069. Attempting to connect to that > port from outside the container fails. I'm totally willing to believe this > is

Should Queryable State Server be listening on 127.0.1.1?

2018-09-25 Thread Andrew Kowpak
I'm running into an issue where I am starting a standalone flink cluster in an lxc container. When my TaskManager starts up, the queryable state proxy starts listening on 127.0.1.1:9069. Attempting to connect to that port from outside the container fails. I'm totally willing to believe

  1   2   3   >