Hi Gowtham,
On the face of it, it sounds like you are planning to use Cassandra for
a queue-like application, which is a well documented anti-pattern. If
that's not the case, can you please show the table schema and some
example queries?
Cheers,
Bowen
On 17/02/2024 08:44, Gowtham S wrote:
Dear Cassandra Community,
I am reaching out to seek your valuable feedback and insights on a
proposed solution we are considering for managing Kafka outages using
Cassandra.
At our organization, we heavily rely on Kafka for real-time data
processing and messaging. However, like any technology, Kafka is
susceptible to occasional outages which can disrupt our operations and
impact our services. To mitigate the impact of such outages and ensure
continuity, we are exploring the possibility of leveraging Cassandra
as a backup solution.
Our proposed approach involves storing messages in Cassandra during
Kafka outages. Subsequently, we plan to implement a scheduler that
will read from Cassandra and attempt to write these messages back into
Kafka once it is operational again.
We believe that by adopting this strategy, we can achieve the
following benefits:
1.
Improved Fault Tolerance: By having a backup mechanism in place,
we can reduce the risk of data loss and ensure continuity of
operations during Kafka outages.
2.
Enhanced Reliability: Cassandra's distributed architecture and
built-in replication features make it well-suited for storing data
reliably, even in the face of failures.
3.
Scalability: Both Cassandra and Kafka are designed to scale
horizontally, allowing us to handle increased loads seamlessly.
Before proceeding further with this approach, we would greatly
appreciate any feedback, suggestions, or concerns from the community.
Specifically, we are interested in hearing about:
* Potential challenges or drawbacks of using Cassandra as a backup
solution for Kafka outages.
* Best practices or recommendations for implementing such a backup
mechanism effectively.
* Any alternative approaches or technologies that we should consider?
Your expertise and insights are invaluable to us, and we are eager to
learn from your experiences and perspectives. Please feel free to
share your thoughts or reach out to us with any questions or
clarifications.
Thank you for taking the time to consider our proposal, and we look
forward to hearing from you soon.
Thanks and regards,
Gowtham S
