Hi,
I am doing performance test for my pipeline keeping FSStateBackend, I have
observed frequent Full GC's after processing 20M records.
When I did memory analysis using MAT, it showed that the many objects
maintained by Flink state are live.
Flink keeps the state in memory even after checkpoint
Hi,
FSStateBackend operates completely on-heap and only snapshots for checkpoints
go against the file system. This is why the backend is typically faster for
small states, but can become problematic for larger states. If your state
exceeds a certain size, you should strongly consider to use Roc
Hi Stephan,
Thank you for the clarification.
Yes with RocksDB I don't see Full GC happening, also I am using Flink 1.2.0
version and I have set the statebackend in flink-conf.yaml file to rocksdb,
so by default does this do asynchronous checkpointing or I have to specify
it at the job level ?
Re
amlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_insta
>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers
; To start a new topic under Apache Flink User Mailing List archive.,
>>> email [hidden email]
>>> To unsubscribe from Apache Flink User Mailing List archive., click here.
>>> NAML
>>> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/templa
t;
>>>> P.S Using RocksDB is not causing Full GC at all.
>>>>
>>>> Regards,
>>>> Vinay Patil
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> If you reply to this email, your
