Hello community, I have a Cassandra server on 3.11.13 on SLESS 12.5. I have noticed in the logs the following line: Datacenter A org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find table for cfId d8c1bea0-82ed-11ee-8ac8-1513e17b60b1. If a table was just created, this is likely due to the schema not being fully propagated. Please wait for schema agreement on table creation. Datacenter B org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find table for cfId 0fedabd0-11f7-11ea-9450-e3ff59b2496b. If a table was just created, this is likely due to the schema not being fully propagated. Please wait for schema agreement on table creation.
This error results in failure of all streaming tasks. I have checked the sstables directories and I see that: In Datacenter A the sstables directory is: <table-name>-0fedabd0-11f7-11ea-9450-e3ff59b2496b In Datacenter B the sstables directory are: <table-name>-0fedabd011f711ea9450e3ff59b2496b <table-name>- d8c1bea082ed11ee8ac81513e17b60b1 In this datacenter although the <table-name>- d8c1bea082ed11ee8ac81513e17b60b1 dir is more recent it is empty and all sstables are stored under <table-name>-0fedabd011f711ea9450e3ff59b2496b I have also checked the system_schema.tables in all Cassandra nodes and I see that for the specific table the ID is consistent across all nodes and it is: d8c1bea0-82ed-11ee-8ac8-1513e17b60b1 So it seems that the schema is a bit mess in all my datacenters. I am not really interested to understand how it ended up in this status but more on how to recover. Both datacenters seem to have this inconsistency between the id stored system_schema.tables and the one used in the sstables directory. Do you have any proposal on how to recover? I have thought of renaming the dir from <table-name>-0fedabd011f711ea9450e3ff59b2496b to <table-name>- d8c1bea082ed11ee8ac81513e17b60b1 but it does not look safe and I would not want to risk my data since this is a production system. Thank you in advance. BR Michail Kotsiouros
