Thanks David for the tips. We have been running Flink with no performance degradation observed in EMR (which is EBS attached) for more than 1 year therefore we believe the same performance can be applied in Kubernetes.
On Sat, Sep 11, 2021 at 3:13 AM David Morávek <d...@apache.org> wrote: > OT: Beware that even if you manage to solve this, EBS is replicated > network storage, therefore rocksdb performance will be affected > significantly. > > Best, > D. > > On Fri 10. 9. 2021 at 16:19, tao xiao <xiaotao...@gmail.com> wrote: > >> The use case we have is to store the RocksDB sst files in EBS. The EC2 >> instance type (m5) we use doesn't provide local disk storage therefore EBS >> is the only option to store the local sst file. >> >> On Fri, Sep 10, 2021 at 7:10 PM Yang Wang <danrtsey...@gmail.com> wrote: >> >>> I am afraid Flink could not support creating dedicated PVC for each >>> TaskManager pod now. >>> But I think it might be a reasonable requirement. >>> >>> Could you please share why you need to mount a persistent volume claim >>> per TaskManager? >>> AFAIK, the TaskManager will be deleted once it fails. You expect the PVC >>> to also be deleted. Right? >>> >>> >>> Best, >>> Yang >>> >>> Xiaolong Wang <xiaolong.w...@smartnews.com> 于2021年9月10日周五 下午2:37写道： >>> >>>> Hi, >>>> >>>> I'm facing a tough question. I want to start a Flink Native >>>> Kubernetes job with each of the task manager pod mounted with an aws-ebs >>>> PVC. >>>> >>>> The first thought is to use the pod-template file to do this, but it >>>> soon went to a dead end. Since the pod-template on each of the task manager >>>> pod is the same, how can I mount different PVCs ? >>>> >>>> This issue is quite puzzling, will you please help me ? >>>> >>>> Thanks in advance ! >>>> >>> >> >> -- >> Regards, >> Tao >> > -- Regards, Tao
