Re: QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

2019-04-29 Thread Ufuk Celebi 
Actually, I couldn't even find a mention of this flag in the docs here:
https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/stream/state/queryable_state.html

– Ufuk


On Mon, Apr 29, 2019 at 8:45 AM Ufuk Celebi  wrote:

> I didn't find this as part of the
> https://flink.apache.org/news/2019/04/09/release-1.8.0.html notes.
>
> I think an update to the Important Changes section would be valuable for
> users upgrading to 1.8 from earlier releases. Also, logging that the
> library is on the classpath but the feature flag is set to false would be a
> helpful.
>
> – Ufuk
>
>
> On Thu, Apr 25, 2019 at 4:13 PM Vishal Santoshi 
> wrote:
>
>> Ditto that, queryable-state.enable to true works.
>>
>> Thanks everyone.
>>
>> On Thu, Apr 25, 2019 at 6:28 AM Dawid Wysakowicz 
>> wrote:
>>
>>> Hi Vishal,
>>>
>>> As Guowei mentioned you have to enable the Queryable state. The default
>>> setting was changed in 1.8.0. There is an open JIRA[1] for changing the
>>> documentation accordingly.
>>>
>>> Best,
>>>
>>> Dawid
>>>
>>> [1] https://issues.apache.org/jira/browse/FLINK-12274
>>> On 25/04/2019 03:27, Guowei Ma wrote:
>>>
>>> You could try to set queryable-state.enable to true. And check again.
>>>
>>> Vishal Santoshi 于2019年4月25日 周四上午1:40写道：
>>>
 Any one ?

 On Wed, Apr 24, 2019 at 12:02 PM Vishal Santoshi <
 vishal.santo...@gmail.com> wrote:

> Hello folks,
>
>  Following
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state
>  .
> for setting up the Queryable Server and proxy, I have my classpath ( the
> lib directory ) that has the  required jar, But I do not see the mentioned
> log and of course am not able to set up the QS server/Proxy . This has
> worked on 1.7.2 and I think I have everything as advised, see the logs
> below. I do not  see this log  "Started the Queryable State Proxy
> Server @ ...".  Any one with this issue...
>
>
>
> 2019-04-24 15:54:26,296 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   - -Dtaskmanager.numberOfTaskSlots=1
>
> 2019-04-24 15:54:26,296 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   - --configDir
>
> 2019-04-24 15:54:26,296 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   - /usr/local/flink/conf
>
> 2019-04-24 15:54:26,296 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   -  Classpath:
> /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:
> */usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar*
> :/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::
>
> 2019-04-24 15:54:26,296 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   -
> 
>
> 2019-04-24 15:54:26,298 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   - Registered UNIX signal handlers for [TERM, HUP, INT]
>
> 2019-04-24 15:54:26,300 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   - Maximum number of open file descriptors is 65536.
>
> 2019-04-24 15:54:26,305 INFO
> org.apache.flink.configuration.GlobalConfiguration-
> Loading configuration property: state.backend.fs.checkpointdir,
> hdfs:///flink-checkpoints_k8s_test/prod
>
> 2
>
>
>
>
> --
>>> Best,
>>> Guowei
>>>
>>>

Re: QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

2019-04-28 Thread Ufuk Celebi 
I didn't find this as part of the
https://flink.apache.org/news/2019/04/09/release-1.8.0.html notes.

I think an update to the Important Changes section would be valuable for
users upgrading to 1.8 from earlier releases. Also, logging that the
library is on the classpath but the feature flag is set to false would be a
helpful.

– Ufuk


On Thu, Apr 25, 2019 at 4:13 PM Vishal Santoshi 
wrote:

> Ditto that, queryable-state.enable to true works.
>
> Thanks everyone.
>
> On Thu, Apr 25, 2019 at 6:28 AM Dawid Wysakowicz 
> wrote:
>
>> Hi Vishal,
>>
>> As Guowei mentioned you have to enable the Queryable state. The default
>> setting was changed in 1.8.0. There is an open JIRA[1] for changing the
>> documentation accordingly.
>>
>> Best,
>>
>> Dawid
>>
>> [1] https://issues.apache.org/jira/browse/FLINK-12274
>> On 25/04/2019 03:27, Guowei Ma wrote:
>>
>> You could try to set queryable-state.enable to true. And check again.
>>
>> Vishal Santoshi 于2019年4月25日 周四上午1:40写道：
>>
>>> Any one ?
>>>
>>> On Wed, Apr 24, 2019 at 12:02 PM Vishal Santoshi <
>>> vishal.santo...@gmail.com> wrote:
>>>
 Hello folks,

  Following
 https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state
  .
 for setting up the Queryable Server and proxy, I have my classpath ( the
 lib directory ) that has the  required jar, But I do not see the mentioned
 log and of course am not able to set up the QS server/Proxy . This has
 worked on 1.7.2 and I think I have everything as advised, see the logs
 below. I do not  see this log  "Started the Queryable State Proxy
 Server @ ...".  Any one with this issue...



 2019-04-24 15:54:26,296 INFO  
 org.apache.flink.runtime.taskexecutor.TaskManagerRunner
   - -Dtaskmanager.numberOfTaskSlots=1

 2019-04-24 15:54:26,296 INFO  
 org.apache.flink.runtime.taskexecutor.TaskManagerRunner
   - --configDir

 2019-04-24 15:54:26,296 INFO  
 org.apache.flink.runtime.taskexecutor.TaskManagerRunner
   - /usr/local/flink/conf

 2019-04-24 15:54:26,296 INFO  
 org.apache.flink.runtime.taskexecutor.TaskManagerRunner
   -  Classpath:
 /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:
 */usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar*
 :/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::

 2019-04-24 15:54:26,296 INFO  
 org.apache.flink.runtime.taskexecutor.TaskManagerRunner
   -
 

 2019-04-24 15:54:26,298 INFO  
 org.apache.flink.runtime.taskexecutor.TaskManagerRunner
   - Registered UNIX signal handlers for [TERM, HUP, INT]

 2019-04-24 15:54:26,300 INFO  
 org.apache.flink.runtime.taskexecutor.TaskManagerRunner
   - Maximum number of open file descriptors is 65536.

 2019-04-24 15:54:26,305 INFO
 org.apache.flink.configuration.GlobalConfiguration-
 Loading configuration property: state.backend.fs.checkpointdir,
 hdfs:///flink-checkpoints_k8s_test/prod

 2




 --
>> Best,
>> Guowei
>>
>>

Re: QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

2019-04-25 Thread Vishal Santoshi 
Ditto that, queryable-state.enable to true works.

Thanks everyone.

On Thu, Apr 25, 2019 at 6:28 AM Dawid Wysakowicz 
wrote:

> Hi Vishal,
>
> As Guowei mentioned you have to enable the Queryable state. The default
> setting was changed in 1.8.0. There is an open JIRA[1] for changing the
> documentation accordingly.
>
> Best,
>
> Dawid
>
> [1] https://issues.apache.org/jira/browse/FLINK-12274
> On 25/04/2019 03:27, Guowei Ma wrote:
>
> You could try to set queryable-state.enable to true. And check again.
>
> Vishal Santoshi 于2019年4月25日 周四上午1:40写道：
>
>> Any one ?
>>
>> On Wed, Apr 24, 2019 at 12:02 PM Vishal Santoshi <
>> vishal.santo...@gmail.com> wrote:
>>
>>> Hello folks,
>>>
>>>  Following
>>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state
>>>  .
>>> for setting up the Queryable Server and proxy, I have my classpath ( the
>>> lib directory ) that has the  required jar, But I do not see the mentioned
>>> log and of course am not able to set up the QS server/Proxy . This has
>>> worked on 1.7.2 and I think I have everything as advised, see the logs
>>> below. I do not  see this log  "Started the Queryable State Proxy
>>> Server @ ...".  Any one with this issue...
>>>
>>>
>>>
>>> 2019-04-24 15:54:26,296 INFO  
>>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>>   - -Dtaskmanager.numberOfTaskSlots=1
>>>
>>> 2019-04-24 15:54:26,296 INFO  
>>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>>   - --configDir
>>>
>>> 2019-04-24 15:54:26,296 INFO  
>>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>>   - /usr/local/flink/conf
>>>
>>> 2019-04-24 15:54:26,296 INFO  
>>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>>   -  Classpath:
>>> /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:
>>> */usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar*
>>> :/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::
>>>
>>> 2019-04-24 15:54:26,296 INFO  
>>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>>   -
>>> 
>>>
>>> 2019-04-24 15:54:26,298 INFO  
>>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>>   - Registered UNIX signal handlers for [TERM, HUP, INT]
>>>
>>> 2019-04-24 15:54:26,300 INFO  
>>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>>   - Maximum number of open file descriptors is 65536.
>>>
>>> 2019-04-24 15:54:26,305 INFO
>>> org.apache.flink.configuration.GlobalConfiguration- Loading
>>> configuration property: state.backend.fs.checkpointdir,
>>> hdfs:///flink-checkpoints_k8s_test/prod
>>>
>>> 2
>>>
>>>
>>>
>>>
>>> --
> Best,
> Guowei
>
>

Re: QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

2019-04-25 Thread Dawid Wysakowicz 
Hi Vishal,

As Guowei mentioned you have to enable the Queryable state. The default
setting was changed in 1.8.0. There is an open JIRA[1] for changing the
documentation accordingly.

Best,

Dawid

[1] https://issues.apache.org/jira/browse/FLINK-12274

On 25/04/2019 03:27, Guowei Ma wrote:
> You could try to set queryable-state.enable to true. And check again. 
>
> Vishal Santoshi  >于2019年4月25日 周四上午1:40写道：
>
> Any one ?
>
> On Wed, Apr 24, 2019 at 12:02 PM Vishal Santoshi
> mailto:vishal.santo...@gmail.com>> wrote:
>
> Hello folks, 
>
>        
>  Following 
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state
>  . 
> for setting up the Queryable Server and proxy, I have my
> classpath ( the lib directory ) that has the  required jar,
> But I do not see the mentioned log and of course am not able
> to set up the QS server/Proxy . This has worked on 1.7.2 and I
> think I have everything as advised, see the logs below. I do
> not  see this log  |"Started the Queryable State Proxy Server
> @ ..."|.  Any one with this issue...
>
>
>
> 2019-04-24 15:54:26,296 INFO 
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner      
> -     -Dtaskmanager.numberOfTaskSlots=1
>
> 2019-04-24 15:54:26,296 INFO 
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner      
> -     --configDir
>
> 2019-04-24 15:54:26,296 INFO 
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner      
> -     /usr/local/flink/conf
>
> 2019-04-24 15:54:26,296 INFO 
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner      
> -  Classpath:
> 
> /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:*/usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar*:/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::
>
> 2019-04-24 15:54:26,296 INFO 
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner      
> -
> 
> 
>
> 2019-04-24 15:54:26,298 INFO 
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner      
> - Registered UNIX signal handlers for [TERM, HUP, INT]
>
> 2019-04-24 15:54:26,300 INFO 
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner      
> - Maximum number of open file descriptors is 65536.
>
> 2019-04-24 15:54:26,305 INFO 
> org.apache.flink.configuration.GlobalConfiguration           
> - Loading configuration property:
> state.backend.fs.checkpointdir,
> hdfs:///flink-checkpoints_k8s_test/prod
>
> 2
>
>
>
>
>
> -- 
> Best,
> Guowei


signature.asc
Description: OpenPGP digital signature

Re: QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

2019-04-24 Thread Guowei Ma 
You could try to set queryable-state.enable to true. And check again.

Vishal Santoshi 于2019年4月25日 周四上午1:40写道：

> Any one ?
>
> On Wed, Apr 24, 2019 at 12:02 PM Vishal Santoshi <
> vishal.santo...@gmail.com> wrote:
>
>> Hello folks,
>>
>>  Following
>> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state
>>  .
>> for setting up the Queryable Server and proxy, I have my classpath ( the
>> lib directory ) that has the  required jar, But I do not see the mentioned
>> log and of course am not able to set up the QS server/Proxy . This has
>> worked on 1.7.2 and I think I have everything as advised, see the logs
>> below. I do not  see this log  "Started the Queryable State Proxy Server
>> @ ...".  Any one with this issue...
>>
>>
>>
>> 2019-04-24 15:54:26,296 INFO  
>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>   - -Dtaskmanager.numberOfTaskSlots=1
>>
>> 2019-04-24 15:54:26,296 INFO  
>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>   - --configDir
>>
>> 2019-04-24 15:54:26,296 INFO  
>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>   - /usr/local/flink/conf
>>
>> 2019-04-24 15:54:26,296 INFO  
>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>   -  Classpath:
>> /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:
>> */usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar*
>> :/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::
>>
>> 2019-04-24 15:54:26,296 INFO  
>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>   -
>> 
>>
>> 2019-04-24 15:54:26,298 INFO  
>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>   - Registered UNIX signal handlers for [TERM, HUP, INT]
>>
>> 2019-04-24 15:54:26,300 INFO  
>> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>>   - Maximum number of open file descriptors is 65536.
>>
>> 2019-04-24 15:54:26,305 INFO
>> org.apache.flink.configuration.GlobalConfiguration- Loading
>> configuration property: state.backend.fs.checkpointdir,
>> hdfs:///flink-checkpoints_k8s_test/prod
>>
>> 2
>>
>>
>>
>>
>> --
Best,
Guowei

Re: QueryableState startup regression in 1.8.0 ( migration from 1.7.2 )

2019-04-24 Thread Vishal Santoshi 
Any one ?

On Wed, Apr 24, 2019 at 12:02 PM Vishal Santoshi 
wrote:

> Hello folks,
>
>  Following
> https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/queryable_state.html#querying-state
>  .
> for setting up the Queryable Server and proxy, I have my classpath ( the
> lib directory ) that has the  required jar, But I do not see the mentioned
> log and of course am not able to set up the QS server/Proxy . This has
> worked on 1.7.2 and I think I have everything as advised, see the logs
> below. I do not  see this log  "Started the Queryable State Proxy Server
> @ ...".  Any one with this issue...
>
>
>
> 2019-04-24 15:54:26,296 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   - -Dtaskmanager.numberOfTaskSlots=1
>
> 2019-04-24 15:54:26,296 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   - --configDir
>
> 2019-04-24 15:54:26,296 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   - /usr/local/flink/conf
>
> 2019-04-24 15:54:26,296 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   -  Classpath:
> /usr/local/flink/lib/flink-metrics-prometheus-1.8.0.jar:
> */usr/local/flink/lib/flink-queryable-state-runtime_2.11-1.8.0.jar*
> :/usr/local/flink/lib/hadoop.jar:/usr/local/flink/lib/jobs.jar:/usr/local/flink/lib/log4j-1.2.17.jar:/usr/local/flink/lib/slf4j-log4j12-1.7.15.jar:/usr/local/flink/lib/flink-dist_2.11-1.8.0.jar:::
>
> 2019-04-24 15:54:26,296 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   -
> 
>
> 2019-04-24 15:54:26,298 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   - Registered UNIX signal handlers for [TERM, HUP, INT]
>
> 2019-04-24 15:54:26,300 INFO  
> org.apache.flink.runtime.taskexecutor.TaskManagerRunner
>   - Maximum number of open file descriptors is 65536.
>
> 2019-04-24 15:54:26,305 INFO
> org.apache.flink.configuration.GlobalConfiguration- Loading
> configuration property: state.backend.fs.checkpointdir,
> hdfs:///flink-checkpoints_k8s_test/prod
>
> 2
>
>
>
>
>