Scaling in stateful application

2023-12-15 Thread rania duni
Hello all! I have enabled flink Kubernetes operator's scaler and I have a stateful application. In the yaml file I have set the savepoint directory and the upgrade mode is *savepoint. *What configurations should be enabled to ensure a safe upgrade with savepoints during scaling operations?

inputQueueSize metric

2023-12-01 Thread rania duni
Regarding the inputQueueSize metric. What does it express after all? Observing the inputQueueSize and inPoolUsage metrics in Prometheus, I found that when the inPoolUsage is high, the inputQueueSize is low. Thisobservation does not match the description of the inputQueueSize. Therefore, does this

Buffer metric

2023-11-26 Thread rania duni
Hello again! I found this metric in the docs *inputQueueSize*, and this is the description *"The real size of queued input buffers in bytes. The size for local input channels is always `0` since the local channel takes records directly from the output queue". *I did a prom query for this metric.

Pending records

2023-11-26 Thread rania duni
Hello! I want to get the pending records of a task. What is the best approach to get the unprocessed records of a task?

Operator ids

2023-11-25 Thread rania duni
Hello! I would like to know how can I get the operator ids of a running job. I know how can I get the task ids but I want the operator ids! I couldn’t find something to the REST API docs. Thank you.

Pending Metrics

2023-09-25 Thread rania duni
Hello! Is there any metric that refers to the queued records, for each operator? Thanks!

[no subject]

2023-05-23 Thread rania duni
Hello! I have deployed the flink kubernetes operator 1.4.0 on minikube and I enabled the autoscaler. However, I get this error "2023-05-22 14:54:25,494 o.a.f.k.o.a.m.ScalingMetrics [ERROR][default/example] Cannot compute true processing/output rate without busyTimeMsPerSecond " on logs of the

[no subject]

2023-05-23 Thread rania duni
Hello! I have deployed the flink kubernetes operator 1.4.0 on minikube and I enabled the autoscaler. However, I get this error "2023-05-22 14:54:25,494 o.a.f.k.o.a.m.ScalingMetrics [ERROR][default/example] Cannot compute true processing/output rate without busyTimeMsPerSecond " on logs of the

Autoscaler

2023-05-01 Thread rania duni
Hello! How can I enable the operator autoscaler and not the job autoscaler ? I don't understand if the default behavior is job scaling or operator scaling, I am a little confused with the documentation. Thank you!

Apache Flink Kubernetes Operator 1.4.0

2023-04-27 Thread rania duni
Hello! As a newcomer to Flink and Kubernetes, I am seeking detailed instructions to help me properly configure and deploy this Flink example ( https://github.com/apache/flink-kubernetes-operator/tree/main/examples/autoscaling) on a Minikube environment. Can you give me specific configurations and