Obtaining Flink Conf in User Code

Hi all, Is it possible to obtain the Flink configuration in the user code? I've tried the Configuration parameter in the open method of rich functions and StreamExecutionEnvironment.getConfig().getGlobalJobParameters()--both do not give the configs in the flink configuration. Best, Mason

Re: Triggers for windowed aggregations in Table API

Thanks Guowei and Caizhi. As Guowei noted, I am using Table API and it seems that it does not support triggers at the moment. Is there a plan to support custom triggers in Table API/SQL too? Also, if I follow Guowei's suggestion, should I use DataStream for other parts of the aggregate computation

pyflink table to datastream

I have a question about how the conversion from Table API to Datastream API actually works under the covers. If I have a Table API operation that creates a random id, like: SELECT id, CAST(UUID() AS VARCHAR) as random_id FROM table ...then I convert this table to a datastream with t_env.to_retr

RE: [External] Re: Flink on Kubernetes

OK – I will increase the value to something higher and see how it does in recovering. Thank you for your help! ___ Julian Cardarelli CEO T (800) 961-1549 ejul...@thentia.com LinkedIn DISCLAIMER ​ ​Neither Thentia Corporation, nor its directors, officers, shareholders, representatives, employee

Re: Cleaning old incremental checkpoint files

Hi Robin, It's not easy to clean incremental checkpoints as different job instances have different checkpoint sub-directory (due to different job id). You could analysis your checkpoint metadata to see what files are still useful in older checkpoint directory. BTW, I also think of a possible s

Re: [Question] Basic Python examples.wordcount on local FlinkRunner

Thanks Dian, that seemed to do the trick. I built a Docker Image simply using: FROM apache/beam_python3.8_sdk:2.31.0 COPY flink_data/input.txt . I specified nothing in the Pipeline options other than: "--runner=FlinkRunner", "--flink_master=localhost:8081", "--environment_type=DOCKER", "--enviro

ApacheCon is just 3 weeks away!

[You are receiving this email because you are subscribed to the user list of one or more Apache project.] Dear Apache enthusiast, ApacheCon is our annual convention, featuring content related to our many software projects. This year, it will be held on September 21-23. Registration is free t

Re: Deploying Flink on Kubernetes with fractional CPU and different limits and requests

Hi Alexis, Thanks for sharing more thoughts about resource configuration. Your suggestions make a lot of sense to me. I believe it could also help others especially for those who are more familiar with K8s and tend to use pod template as far as possible. I have created a ticket for this feature[1

RE: Deploying Flink on Kubernetes with fractional CPU and different limits and requests

Hi Yang, I understand the issue, and yes, if Flink memory must be specified in the configuration anyway, it’s probably better to leave memory configuration in the templates empty. For the CPU case I still think the template’s requests/limits should have priority if they are specified. The fact

Re: FLINK-14316 happens on version 1.13.2

Hi Yun, Thanks alot. I am running a test, and facing the "Job Leader lost leadership..." issue, and also the checkpointing timeout at the same time,, not sure whether those 2 things related to each other. regarding your question: 1. GC looks ok. 2. seems like once the "Job Leader lost leadership...