Re: Tiered storage

2024-03-08 Thread Greg Harris
Hi Lukas, Thanks for your interest in Tiered Storage! Disclaimer: I work/have worked for both of the vendors mentioned. There are two different Tiered Storage implementations: * Confluent's closed-source implementation [1] * Apache Kafka's open-source implementation [2] You ca

Tiered storage

2024-03-08 Thread Lukas Zimmerman
Hey! I came across the Tiered Storage feature in Kafka and found this feature quite exciting! It seems like it could help with dealing with the retention of large amounts of data in Kafka topics. I have several questions: 1/ Is it still in early access? When will it be considered ready for

RE: questions regarding the tiered storage feature

2023-10-18 Thread miltan
Partner Manager. -Original Message- From: Christophe Kalenzaga [mailto:christophe.kalenz...@gmail.com] Sent: 17 October 2023 20:09 To: users@kafka.apache.org Subject: questions regarding the tiered storage feature Hi, Congratulation for the release of kafka 3.6 that contains a feature I was

RE: questions regarding the tiered storage feature

2023-10-18 Thread miltan
Manager. -Original Message- From: Josep Prat [mailto:josep.p...@aiven.io.INVALID] Sent: 17 October 2023 20:38 To: Users Subject: Re: questions regarding the tiered storage feature Hi Christophe, We highly encourage the community to try Tiered Storage in their staging environments

RE: questions regarding the tiered storage feature

2023-10-18 Thread miltan
Manager. -Original Message- From: Josep Prat [mailto:josep.p...@aiven.io.INVALID] Sent: 17 October 2023 21:18 To: Users Subject: Re: questions regarding the tiered storage feature Hi Christophe, I'll check and let you know when we plan to have Azure support. I'd also encour

RE: questions regarding the tiered storage feature

2023-10-18 Thread miltan
Manager. -Original Message- From: Christophe Kalenzaga [mailto:christophe.kalenz...@gmail.com] Sent: 17 October 2023 21:10 To: users@kafka.apache.org Subject: Re: questions regarding the tiered storage feature Thanks for the answer, Joseph. Aiven's implementation was already in my

RE: questions regarding the tiered storage feature

2023-10-18 Thread miltan
Manager. -Original Message- From: Christophe Kalenzaga [mailto:christophe.kalenz...@gmail.com] Sent: 17 October 2023 23:24 To: users@kafka.apache.org Subject: Re: questions regarding the tiered storage feature thanks Le mar. 17 oct. 2023 à 17:48, Josep Prat a écrit : > Hi Christo

Re: questions regarding the tiered storage feature

2023-10-17 Thread Christophe Kalenzaga
dIn's Azure implementation, but the latter is not maintained. So, if > > the tests are successful (and we have a better idea about the release > date > > of the production ready version of tiered storage), we'll either rely on > a > > more reliable azure implementatio

Re: questions regarding the tiered storage feature

2023-10-17 Thread Josep Prat
arks, I guess we'll use Aiven's S3 implementation and > LinkedIn's Azure implementation, but the latter is not maintained. So, if > the tests are successful (and we have a better idea about the release date > of the production ready version of tiered storage), we'll either rely

Re: questions regarding the tiered storage feature

2023-10-17 Thread Christophe Kalenzaga
I guess we'll use Aiven's S3 implementation and LinkedIn's Azure implementation, but the latter is not maintained. So, if the tests are successful (and we have a better idea about the release date of the production ready version of tiered storage), we'll either rely on a more relia

Re: questions regarding the tiered storage feature

2023-10-17 Thread Josep Prat
Hi Christophe, We highly encourage the community to try Tiered Storage in their staging environments and report any deficiencies they might encounter. That being said, at Aiven to develop our S3 and GCS plugin [1] we did some extensive testing and we will be enabling it to our service offering

questions regarding the tiered storage feature

2023-10-17 Thread Christophe Kalenzaga
Hi, Congratulation for the release of kafka 3.6 that contains a feature I was waiting for many years! Speaking of the devil, the tiered storage is currently in early access. When do you expect it to be production ready? I know it's not an easy question since the early access was just rel