There's no requirement for the partition key to contain the date/time
for a TWCS table. The important thing is data need to be written to the
table in chronological order (i.e. do not use the "USING TIMESTAMP" in
the CQL queries) and the same TTL is used for all partitions. TWCS was
introduced many years ago to replace DTCS. Could it be that you have had
some bad past experiences with DTCS?
I previously mentioned "date in the partition key" only because you
currently are querying by table names, which would almost certainly
contain a coarse date/time, and the equivalent in a TWCS table would
need to have that coarse date/time from the table name in the partition
key instead. If you don't need the ability to query by date/time, you
can have partition key(s) without any date/time in them.
On 07/12/2023 09:08, Sébastien Rebecchi wrote:
Thanks Bowen, I also thought about using TTL and TWCS, but in my past
experience with Cassandra I have had a lot of issues with data models
using TTL and creating many tombstones. I was probably not using the
right compaction at that time, but this experiences has a great impact
on me and I would say they made me very cautious about TTL,
even today, several years after ^^
Anyway, in the current 2 cases I can not put a date as partition key
alone.
In the first one I have a pair of columns acting as partition key, one
is a customer id and the other is a date (more precisely a second
"block" of time to group all events of the same second in the same
partition, to pre-group data). The customer id is mandatory in the
partition key in order not to have too wide partitions, and I never
have cases where I need to fetch cross-customer data. Is TWCS still
suited? As I read from doc, you don't need to have a time window as
partition key alone, as long as many partitions will die approx in the
same time it could work fine cause sometimes entire SSTables will be
deleted during compaction rather than rewritten in disk.
As for my second use case that creates many tables on demand, I can
not have the time window in the PK, cause I think it would lead to
degraded read performance, I have to put a visitor id in the PK and a
timestamp as CK in order to fetch all event of a visitor inside a time
window in 1 query. I could probably put a time window as PK (e.g. a
second block like the 1st use case, and then perform a small number of
queries to merge pre-results client-side) and in that case TTL+TWCS
would probably apply, it remains the same question as above.
Thanks for your time :)
Sébastien.
Le mer. 6 déc. 2023 à 15:46, Bowen Song via user
<user@cassandra.apache.org> a écrit :
There are many different ways to avoid or minimise the chance of
schema disagreements, the easiest way is to always send DDL
queries to the same node in the cluster. This is very easy to
implement and avoids schema disagreements at the cost of creating
a single point of failure for DDL queries. More sophisticated
methods also exist, such as locking and centralised schema
modification, and you should consider which one is more suitable
for your use case. Ignoring the schema disagreements problem is
not recommended, as this is not a tested state for the cluster,
you are likely to run into some known and unknown (and possibly
severe) issues later.
The system_schema.columns table will almost certainly have more
tombstones created than the number of tables deleted, unless each
deleted table had only one column. I doubt creating and deleting 8
tables per day will be a problem, but I would recommend you find a
way to test it before doing that on a production system, because I
don't know anyone else is using Cassandra in this way.
From the surface, it does sound like TWCS with the date in in the
partition key may fit your use case better than creating and
deleting tables every day.
On 06/12/2023 08:26, Sébastien Rebecchi wrote:
Hello Jeff, Bowen
Thanks for your answer.
Now I understand that there is a bug in Cassandra that can not
handle concurrent schema modifications, I was not aware of that
severity, I thought that temporary schema mismatches were
eventually resolved smartly, by a kind of "merge" mechanism.
For my use cases, keyspaces and tables are created "on-demand",
when receiving exceptions for invalid KS or table on insert (then
the KS and table are created and the insert is retried). I can
not afford to centralize schema modifications in a bottleneck,
but I can afford the data inconsistencies, waiting for the fix in
Cassandra.
I'm more worried about tombstones in system tables, I assume that
8 tombstones per day (or even more, but in the order of no more
than some dozens) is reasonable, can you confirm (or invalidate)
that please?
Sébastien.
Le mer. 6 déc. 2023 à 03:00, Bowen Song via user
<user@cassandra.apache.org> a écrit :
The same table name with two different CF IDs is not just
"temporary schema disagreements", it's much worse than that.
This breaks the eventual consistency guarantee, and leads to
silent data corruption. It's silently happening in the
background, and you don't realise it until you suddenly do,
and then everything seems to blow up at the same time. You
need to sort this out ASAP.
On 05/12/2023 19:57, Sébastien Rebecchi wrote:
Hi Bowen,
Thanks for your answer.
I was thinking of extreme use cases, but as far as I am
concerned I can deal with creation and deletion of 2 tables
every 6 hours for a keyspace. So it lets around 8 folders of
deleted tables per day - sometimes more cause I can see
sometimes 2 folders created for a same table name, with 2
different ids, caused by temporary schema disagreements I guess.
Basically it means 20 years before the KS folder has 65K
subfolders, so I would say I have time to think of
redesigning the data model ^^
Nevertheless, does it sound too much in terms of thombstones
in the systems tables (with the default GC grace period of
10 days)?
Sébastien.
Le mar. 5 déc. 2023, 12:19, Bowen Song via user
<user@cassandra.apache.org> a écrit :
Please rethink your use case. Create and delete tables
concurrently often lead to schema disagreement. Even
doing so on a single node sequentially will lead to a
large number of tombstones in the system tables.
On 04/12/2023 19:55, Sébastien Rebecchi wrote:
Thank you Dipan.
Do you know if there is a good reason for Cassandra to
let tables folder even when there is no snapshot?
I'm thinking of use cases where there is the need to
create and delete small tables at a high rate. You
could quickly end with more than 65K (limit of ext4)
subdirectories in the KS directory, while 99.9.. % of
them are residual of deleted tables.
That looks quite dirty from Cassandra to not clean its
own "garbage" by itself, and quite dangerous for the
end user to have to do it alone, don't you think so?
Thanks,
Sébastien.
Le lun. 4 déc. 2023, 11:28, Dipan Shah
<dipan....@hotmail.com> a écrit :
Hello Sebastien,
There are no inbuilt tools that will automatically
remove folders of deleted tables.
Thanks,
Dipan Shah
------------------------------------------------------------------------
*From:* Sébastien Rebecchi <srebec...@kameleoon.com>
*Sent:* 04 December 2023 13:54
*To:* user@cassandra.apache.org
<user@cassandra.apache.org>
*Subject:* Remove folders of deleted tables
Hello,
When we delete a table with Cassandra, it lets the
folder of that table on file system, even if there
is no snapshot (auto snapshots disabled).
So we end with the empty folder {data
folder}/{keyspace name}/{table name-table id}
containing only 1 subfolder, backups, which is
itself empty.
Is there a way to automatically remove folders of
deleted tables?
Sébastien.
