date:20180627

[jira] [Comment Edited] (CASSANDRA-14393) Incorrect view updates

2018-06-27 Thread Alexander Ivakov (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525917#comment-16525917
 ] 

Alexander Ivakov edited comment on CASSANDRA-14393 at 6/28/18 5:52 AM:
---

The above appears to be a design feature implemented in CASSANDRA-11500. See 
point c) in:

{{view row exists if any of following is true:

a. base row pk has live livenessInfo(timestamp) and base row pk satifies view's 
filter conditions if any.
b. or one of base row columns selected in view has live timestamp (via update) 
and base row pk satifies view's filter conditions if any. this is handled by 
existing mechanism of liveness and tombstone since all info are included in 
view row
c. or one of base row columns not selected in view has live timestamp (via 
update) and base row pk satifies view's filter conditions if any. Those 
unselected columns' timestamp/ttl/cell-deletion info are not currently stored 
on view row.}}

from here: 
https://issues.apache.org/jira/browse/CASSANDRA-11500?focusedCommentId=16082241=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16082241

When a base table row is updated, the corresponding view row has its (primary 
key) liveness info updated to that of the update, including ttl and local 
deletion time. This leads to the behaviour you see here if the update had a 
longer ttl than the actual row - the (view) row is marked with the ttl of the 
update and remains alive for that length of time (even if the data in the row 
is already expired). Note, this only affects the view, the base table row 
liveness is not altered (only INSERTs can set the row liveness on tables). 
Hence, if you query the table, no results are returned, while querying the view 
returns a row with empty (expired) data cells.

Note also that DELETE does not do the above, ie. does not update the row 
liveness and ttl of the MV row. 

Perhaps [~jasonstack] can comment?


was (Author: alex.ivakov):
The above appears to be a design feature implemented in CASSANDRA-11500. See 
point c) in:

{{view row exists if any of following is true:

a. base row pk has live livenessInfo(timestamp) and base row pk satifies view's 
filter conditions if any.
b. or one of base row columns selected in view has live timestamp (via update) 
and base row pk satifies view's filter conditions if any. this is handled by 
existing mechanism of liveness and tombstone since all info are included in 
view row
c. or one of base row columns not selected in view has live timestamp (via 
update) and base row pk satifies view's filter conditions if any. Those 
unselected columns' timestamp/ttl/cell-deletion info are not currently stored 
on view row.}}

from here: 
https://issues.apache.org/jira/browse/CASSANDRA-11500?focusedCommentId=16082241=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16082241

When a base table row is updated, the corresponding view row has its (primary 
key) liveness info updated to that of the update, including ttl and local 
deletion time. This leads to the behaviour you see here if the update had a 
longer ttl than the actual row - the row is marked with the ttl of the update 
and remains alive for that length of time (even if the data in the row is 
already expired). Note, this only affects the view, the base table row liveness 
is not altered (only INSERTs can set the row liveness on tables). Hence, if you 
query the table, no results are returned, while querying the view returns a row 
with empty (expired) data cells.

Note also that DELETE does not do the above, ie. does not update the row 
liveness and ttl of the MV row. 

Perhaps [~jasonstack] can comment?

> Incorrect view updates
> --
>
> Key: CASSANDRA-14393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14393
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
>Reporter: Duarte Nunes
>Priority: Major
>  Labels: materializedviews
>
> Consider the following:
> {noformat}
> create table t (p int, c int, v1 int, v2 int, primary key(p, c));
> create materialized view mv as select p, c, v1 from t 
> where p is not null and c is not null primary key (c, p);
> insert into t (p, c, v1, v2) VALUES(1, 1, 1, 1) using ttl 5;
> update t using ttl 1000 set v2 = 1 where p = 1 and c = 1;
> delete v2 from t where p = 1 and c = 1;
> // Wait 5 seconds
> select * from mv;
> c | p | v1
> ---+---+--
> 1 | 1 | null{noformat}
> The view row should be dead after 5 seconds, but it is not.
> This is because the liveness info calculated when deleting v2 is based on the 
> base table update liveness info, which has the timestamp of the first insert 
> statement. That liveness info is shadowed by the liveness info created in the 
> update, which has a higher timestamp.



--
This

[jira] [Commented] (CASSANDRA-14393) Incorrect view updates

2018-06-27 Thread Alexander Ivakov (JIRA)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-14393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525917#comment-16525917
 ] 

Alexander Ivakov commented on CASSANDRA-14393:
--

The above appears to be a design feature implemented in CASSANDRA-11500. See 
point c) in:

{{view row exists if any of following is true:

a. base row pk has live livenessInfo(timestamp) and base row pk satifies view's 
filter conditions if any.
b. or one of base row columns selected in view has live timestamp (via update) 
and base row pk satifies view's filter conditions if any. this is handled by 
existing mechanism of liveness and tombstone since all info are included in 
view row
c. or one of base row columns not selected in view has live timestamp (via 
update) and base row pk satifies view's filter conditions if any. Those 
unselected columns' timestamp/ttl/cell-deletion info are not currently stored 
on view row.}}

from here: 
https://issues.apache.org/jira/browse/CASSANDRA-11500?focusedCommentId=16082241=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16082241

When a base table row is updated, the corresponding view row has its (primary 
key) liveness info updated to that of the update, including ttl and local 
deletion time. This leads to the behaviour you see here if the update had a 
longer ttl than the actual row - the row is marked with the ttl of the update 
and remains alive for that length of time (even if the data in the row is 
already expired). Note, this only affects the view, the base table row liveness 
is not altered (only INSERTs can set the row liveness on tables). Hence, if you 
query the table, no results are returned, while querying the view returns a row 
with empty (expired) data cells.

Note also that DELETE does not do the above, ie. does not update the row 
liveness and ttl of the MV row. 

Perhaps [~jasonstack] can comment?

> Incorrect view updates
> --
>
> Key: CASSANDRA-14393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14393
> Project: Cassandra
>  Issue Type: Bug
>  Components: Materialized Views
>Reporter: Duarte Nunes
>Priority: Major
>  Labels: materializedviews
>
> Consider the following:
> {noformat}
> create table t (p int, c int, v1 int, v2 int, primary key(p, c));
> create materialized view mv as select p, c, v1 from t 
> where p is not null and c is not null primary key (c, p);
> insert into t (p, c, v1, v2) VALUES(1, 1, 1, 1) using ttl 5;
> update t using ttl 1000 set v2 = 1 where p = 1 and c = 1;
> delete v2 from t where p = 1 and c = 1;
> // Wait 5 seconds
> select * from mv;
> c | p | v1
> ---+---+--
> 1 | 1 | null{noformat}
> The view row should be dead after 5 seconds, but it is not.
> This is because the liveness info calculated when deleting v2 is based on the 
> base table update liveness info, which has the timestamp of the first insert 
> statement. That liveness info is shadowed by the liveness info created in the 
> update, which has a higher timestamp.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-11559) Enhance node representation

2018-06-27 Thread Alex Lourie (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-11559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414721#comment-16414721
]

Alex Lourie edited comment on CASSANDRA-11559 at 6/28/18 4:55 AM:
--

I've done some work on this ticket and it's available at
[https://github.com/apache/cassandra/compare/trunk...alourie:CASSANDRA-11559#files_bucket].
Few pointers:
* I've refactored InetAddressAndPort class into VirtualEndpoint class across
all the codebase (this will be responsible for the majority of the code
changes).
* I've added a UUID field to hold the hostID value of the endpoint and added
additional methods for working with it.
* I've reworked the TokenMetadata to hold structures other than maps for
UUID-host references and they would no longer be needed, i.e. keeping just a
set of endpoints is enough to hold both address data and the hostID data and to
also look up hosts by IDs or the vice versa.
* I've reworked the SystemKeyspace to also acknowledge the hostIDs where
significant (in local data/peer data storing/fetching), and also only create
new local id if requested (in most cases only when the node is created for the
first time, but also useful for tests that require initiating multiple "nodes"
on the same machine)
I've added a field in DatabaseDescriptor to mark that SystemKeyspace is ready
to be read. This is required for many unit tests that set up clusters "on the
fly" and for further endpoint information discovery during the test run.
* I've updated required unit tests to properly utilise the new object and
initialise and clean others as required.
* I've updated the code in some other locations to incorporate this change,
which does make it simpler on many occasions.

The current state is everything _seems_ to be working and the unit tests pass
([https://circleci.com/gh/alourie/cassandra/123])

The complication that comes out of this work is with building unit tests - the
host ID would now be kept in multiple structures:
* a VirtualEndpoint object when instantiated.
* SystemKeyspace.localHost (queries the DB)
* SystemKeyspace.peersInfo (queries the DB)
* TokenMetadata lists (such as allEndpoints, tokenMap, etc)
* Gossip.instance.endpointState maps (the specific endpoint is added including
the uuid)
* FBUtilities also keeps local reference once fetched

As a result, when creating tests, one needs to update or clear the
hostID-related information in all relevant places, otherwise, tests would fail
with really confusing messages (in most cases because in some thread an
endpoint comparison will happen and UUIDs won't match), such as "no seeds
found", "host cannot be contacted" or various kinds of timeouts and NPEs.
Additionally, when SystemKeyspace is ready to be read within a test flow, a
DatabaseDescriptor.canReadSystemKeyspace field will need to be set to true so
that the UUID would be fetched from SystemKeyspace.

Additionally, at the moment we are keeping EndpointState separately from this
object (in Gossip). Considering that now this VirtualEndpoint can include
basically any information about the endpoint, it may as well incorporate its
own state, and then all handling of the network/state information about an
endpoint will be in one place. Supposedly this should simplify things further
and allow clearing a lot of code.

[~aweisberg] - you have done the previous move away from InetAddress
representation to InetAddressAndPort, which this current patch changes
considerably. I'd love your feedback on this.

Any and all feedback is very welcome.

was (Author: alourie):
I've done some work on this ticket and it's available at
[https://github.com/apache/cassandra/compare/trunk...alourie:CASSANDRA-11559#files_bucket].
Few pointers:
* I've refactored InetAddressAndPort class into VirtualEndpoint class across
all the codebase (this will be responsible for the majority of the code
changes).
* I've added a UUID field to hold the hostID value of the endpoint and added
additional methods for working with it.
* I've reworked the TokenMetadata to hold structures other than maps for
UUID-host references and they would no longer be needed, i.e. keeping just a
set of endpoints is enough to hold both address data and the hostID data and to
also look up hosts by IDs or the vice versa.
* I've reworked the SystemKeyspace to also acknowledge the hostIDs where
significant (in local data/peer data storing/fetching), and also only create
new local id if requested (in most cases only when the node is created for the
first time, but also useful for tests that require initiating multiple "nodes"
on the same machine)
I've added a field in DatabaseDescriptor to mark that SystemKeyspace is ready
to be read. This is required for many unit tests that set up clusters "on the
fly" and for further endpoint information discovery during the test run.
* I've updated

43 matches

Mail list logo