[
https://issues.apache.org/jira/browse/CASSANDRA-19018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816783#comment-17816783
]
Caleb Rackliffe edited comment on CASSANDRA-19018 at 2/12/24 8:24 PM:
----------------------------------------------------------------------
Alright, posting here, although there's been some Slack discussion. Queries on
indexed static columns break RFP, and here's a simple reproduction, whittled
down from a prior Harry test...
{noformat}
@Test
public void testPartialUpdatesStaticOnly()
{
CLUSTER.schemaChange(withKeyspace("CREATE TABLE %s.partial_updates (k
int, c int, s int static, b int, PRIMARY KEY (k, c)) WITH read_repair =
'NONE'"));
CLUSTER.schemaChange(withKeyspace("CREATE INDEX ON
%s.partial_updates(s) USING 'sai'"));
SAIUtil.waitForIndexQueryable(CLUSTER, KEYSPACE);
// insert a split row
CLUSTER.get(1).executeInternal(withKeyspace("INSERT INTO
%s.partial_updates(k, s) VALUES (0, 2) USING TIMESTAMP 100"));
CLUSTER.get(2).executeInternal(withKeyspace("INSERT INTO
%s.partial_updates(k, c, s, b) VALUES (0, 0, 1, 2) USING TIMESTAMP 10"));
//CLUSTER.get(1).nodetool("repair", KEYSPACE); <-- un-comment and
everything is fine, of course
String select = withKeyspace("SELECT * FROM %s.partial_updates WHERE s
= 2");
Object[][] initialRows = CLUSTER.coordinator(1).execute(select,
ConsistencyLevel.ALL);
assertRows(initialRows, row(0, 0, 2, 2));
}
{noformat}
Long story short, when RFP has a legitimate static row match from one replica,
but no clusterings, and the other replica has no static row match, but does
have clusterings that COULD BECOME matches together w/ the static row from the
first node, RFP does NOT fetch the entire partition like it's supposed to from
the silent node. This means the only clusterings in the final result will be
the ones we might get from the first node. (In my example above, there are
none, of course.) I've got this test passing by simply fetching the whole
partition from the silent replica when the static row isn't fully resolved by
the row merger/merge listener, but there may be a more efficient way.
was (Author: maedhroz):
Alright, posting here, although there's been some Slack discussion. Queries on
indexed static columns break RFP, and here's a simple reproduction, whittled
down from a prior Harry test...
{noformat}
@Test
public void testPartialUpdatesStaticOnly()
{
CLUSTER.schemaChange(withKeyspace("CREATE TABLE %s.partial_updates (k
int, c int, s int static, b int, PRIMARY KEY (k, c)) WITH read_repair =
'NONE'"));
CLUSTER.schemaChange(withKeyspace("CREATE INDEX ON
%s.partial_updates(s) USING 'sai'"));
SAIUtil.waitForIndexQueryable(CLUSTER, KEYSPACE);
// insert a split row
CLUSTER.get(1).executeInternal(withKeyspace("INSERT INTO
%s.partial_updates(k, s) VALUES (0, 2) USING TIMESTAMP 100"));
CLUSTER.get(2).executeInternal(withKeyspace("INSERT INTO
%s.partial_updates(k, c, s, b) VALUES (0, 0, 1, 2) USING TIMESTAMP 10"));
//CLUSTER.get(1).nodetool("repair", KEYSPACE); <-- un-comment and
everything is fine, of course
String select = withKeyspace("SELECT * FROM %s.partial_updates WHERE s
= 2");
Object[][] initialRows = CLUSTER.coordinator(1).execute(select,
ConsistencyLevel.ALL);
assertRows(initialRows, row(0, 0, 2, 2));
}
{noformat}
Long story short, when RFP has a legitimate static row match from one replica,
but no clusterings, and the other replica has no static row match, but does
have clusterings that COULD BECOME matches together w/ the static row from the
first node, RFP does NOT fetch the entire partition like it's supposed to from
the silent node. This means the only clusterings in the final result will be
the ones we might get from the first node. (In my example above, there are
none, of course.) I've got this test passing by simply fetching the whole
partition from the silent replica when the static row isn't fully resolved by
the row merger/merge listener.
> An SAI-specific mechanism to ensure consistency isn't violated for
> multi-column (i.e. AND) queries at CL > ONE
> --------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-19018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19018
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Feature/SAI
> Reporter: Caleb Rackliffe
> Assignee: Caleb Rackliffe
> Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary-1.html, ci_summary.html,
> result_details.tar-1.gz, result_details.tar.gz
>
> Time Spent: 8h 50m
> Remaining Estimate: 0h
>
> CASSANDRA-19007 is going to be where we add a guardrail around
> filtering/index queries that use intersection/AND over partially updated
> non-key columns. (ex. Restricting one clustering column and one normal column
> does not cause a consistency problem, as primary keys cannot be partially
> updated.) This issue exists to attempt to fix this specifically for SAI in
> 5.0.x, as Accord will (last I checked) not be available until the 5.1 release.
> The SAI-specific version of the originally reported issue is this:
> {noformat}
> try (Cluster cluster = init(Cluster.build(2).withConfig(config ->
> config.with(GOSSIP).with(NETWORK)).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (k int
> PRIMARY KEY, a int, b int)"));
> cluster.schemaChange(withKeyspace("CREATE INDEX ON %s.t(a) USING
> 'sai'"));
> cluster.schemaChange(withKeyspace("CREATE INDEX ON %s.t(b) USING
> 'sai'"));
> // insert a split row
> cluster.get(1).executeInternal(withKeyspace("INSERT INTO %s.t(k,
> a) VALUES (0, 1)"));
> cluster.get(2).executeInternal(withKeyspace("INSERT INTO %s.t(k,
> b) VALUES (0, 2)"));
> // Uncomment this line and test succeeds w/ partial writes
> completed...
> //cluster.get(1).nodetoolResult("repair",
> KEYSPACE).asserts().success();
> String select = withKeyspace("SELECT * FROM %s.t WHERE a = 1 AND
> b = 2");
> Object[][] initialRows = cluster.coordinator(1).execute(select,
> ConsistencyLevel.ALL);
> assertRows(initialRows, row(0, 1, 2)); // not found!!
> }
> {noformat}
> To make a long story short, the local SAI indexes are hiding local partial
> matches from the coordinator that would combine there to form full matches.
> Simple non-index filtering queries also suffer from this problem, but they
> hide the partial matches in a different way. I'll outline a possible solution
> for this in the comments that takes advantage of replica filtering protection
> and the repaired/unrepaired datasets...and attempts to minimize the amount of
> extra row data sent to the coordinator.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
