Hi there,
Is there any design docs or on going efforts there?
Thanks,
Chen
Hi Chen,
AFAIK, there currently isn’t any FLIP / JIRA / work currently for a
Cassandra state backend. I think it’ll definitely by interesting to have
one in Flink.
Regards,
Gordon
On July 25, 2016 at 10:24:32 AM, Chen Qin (qinnc...@gmail.com) wrote:
Hi there,
Is there any design docs or on g
Hi,
I thought there was a Jira for that but I looked and couldn't find it. If
you'd like you can create one and we can discuss the design. Do you have
any ideas yet?
The tricky things I see in this are:
- Knowing which data is the current data. This will require some kind of
timestamps or increas
Hi Aljoscha,
Cool! I created a JIRA for this.
https://issues.apache.org/jira/browse/FLINK-4266
Some comments inline.
Chen
On Mon, Jul 25, 2016 at 2:41 AM, Aljoscha Krettek
wrote:
> Hi,
> I thought there was a Jira for that but I looked and couldn't find it. If
> you'd like you can create one a
Hi,
thanks for opening the Jira issue. I'll continue the discussion here
instead of in the Jira, I hope that's OK.
That last paragraph of yours is the most interesting. We will have to adapt
the way that checkpoints are stored to accommodate state backends that
store state in some external system,
Aljoscha
Sorry about late reply.
David and I drafted a design doc with some diagrams. We may not work on it
immediately, but we thought it would be valuable to share our thoughts and
hear feedbacks.
https://docs.google.com/document/d/1diHQyOPZVxgmnmYfiTa6glLf-FlFjSHcL8J3YR2xLdk/edit#heading=h.12
Hi,
thanks for sharing the design doc, these are valuable ideas.
We might have to revisit the specifics once the re-sharding/key-group
changes are in Flink and once you actually want to start working on this.
Cheers,
Aljoscha
On Sat, 6 Aug 2016 at 07:32 Chen Qin wrote:
> Aljoscha
>
> Sorry abo
Hi,
I have done something similar in the past for storing state in sharded
MySql databases. We used this for a while for state size scaling reasons
but have switched to RocksDB later and therefore this statebackend has been
removed from Flink to cut some maintenance costs.
You can find the initia
Aljoscha,
Sure thing, will do after key/group feature in place when we got bandwith :)
Gyula,
That's where we started, many terms are copied over(logical timestamp,
compaction, lazy restore). we have to use Cassandra which offer less in
transaction and consistency to gain availability and cross
Chen Qin created FLINK-4266:
---
Summary: Cassandra Statebackend
Key: FLINK-4266
URL: https://issues.apache.org/jira/browse/FLINK-4266
Project: Flink
Issue Type: New Feature
Components
10 matches
Mail list logo
