Re: Size of state for any known production use case
hey Marta, Thank you! This is going to be pretty useful, let me go over these and get back to you if I have any questions. RK On Thu, Feb 13, 2020 at 6:21 AM Marta Paes Moreira wrote: > Hi, Reva. > > If you are looking for the maximum known state size, I believe Alibaba is > using Flink at the largest scale in production [1]. > > There are also other examples of variable scale scattered across Flink > Forward talks [2]. In particular, this Netflix talk [3] should be > interesting to you. > > Marta > > [1] > https://www.itnextsummit.com/wp-content/uploads/2019/11/Stephan_Ewen_Stream_Processing_Beyond_Streaming.pdf > (Slide > 3) > [2] https://www.youtube.com/channel/UCY8_lgiZLZErZPF47a2hXMA/videos > [3] https://www.youtube.com/watch?v=2C44mUPlx5o > > On Wed, Feb 12, 2020 at 10:42 PM RKandoji wrote: > >> Hi Team, >> >> I've done a POC using Flink and planning to give a presentation about my >> learnings and share the benefits of using Flink. >> >> I understand that companies are using Flink to handle Tera Bytes of >> state, but it would be great if you could point me to any reference of a >> company using Flink production for a known amount of state. Or any other >> related links where I can get these details? >> >> Basically I want to provide the known maximum limit of state that can be >> stored. This is needed because my use case requires performing stream joins >> on unbounded data (although data is unbounded, its not going to be super >> huge like 10TB) >> >> >> Thanks, >> Reva >> >
Re: Size of state for any known production use case
Hi, Reva. If you are looking for the maximum known state size, I believe Alibaba is using Flink at the largest scale in production [1]. There are also other examples of variable scale scattered across Flink Forward talks [2]. In particular, this Netflix talk [3] should be interesting to you. Marta [1] https://www.itnextsummit.com/wp-content/uploads/2019/11/Stephan_Ewen_Stream_Processing_Beyond_Streaming.pdf (Slide 3) [2] https://www.youtube.com/channel/UCY8_lgiZLZErZPF47a2hXMA/videos [3] https://www.youtube.com/watch?v=2C44mUPlx5o On Wed, Feb 12, 2020 at 10:42 PM RKandoji wrote: > Hi Team, > > I've done a POC using Flink and planning to give a presentation about my > learnings and share the benefits of using Flink. > > I understand that companies are using Flink to handle Tera Bytes of state, > but it would be great if you could point me to any reference of a company > using Flink production for a known amount of state. Or any other related > links where I can get these details? > > Basically I want to provide the known maximum limit of state that can be > stored. This is needed because my use case requires performing stream joins > on unbounded data (although data is unbounded, its not going to be super > huge like 10TB) > > > Thanks, > Reva >
Size of state for any known production use case
Hi Team, I've done a POC using Flink and planning to give a presentation about my learnings and share the benefits of using Flink. I understand that companies are using Flink to handle Tera Bytes of state, but it would be great if you could point me to any reference of a company using Flink production for a known amount of state. Or any other related links where I can get these details? Basically I want to provide the known maximum limit of state that can be stored. This is needed because my use case requires performing stream joins on unbounded data (although data is unbounded, its not going to be super huge like 10TB) Thanks, Reva
