[jira] [Commented] (CASSANDRA-5501) Missing data on SELECT on secondary index
[ https://issues.apache.org/jira/browse/CASSANDRA-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13643037#comment-13643037 ] Marco Matarazzo commented on CASSANDRA-5501: Alas, we had to update (and add) data in rows and now the query is correctly returning everything. I didn't know about the "TRACING ON" command, it's a great tool :-) I'm going to still post the result of the command, hoping that, apart from the result being correct, it will give you some clues anyway. cqlsh:goh_master> select agent_id,name,station_id,trading from agents where station_id='1110129'; agent_id | name | station_id | trading --+++- 6c8efeb6-7209-11e2-890a-aacc0216 | Jenny Olifield |1110129 |null b653d8c4-5fca-11e2-bd3a-aacc0216 |mammolo |1110129 |null cf9df102-7394-11e2-890a-aacc0216 | riolla |1110129 |null f9c0278b-aa5f-11e2-a860-aacc0216 | Terrinon |1110129 |null cf03e58b-6d6a-11e2-890a-aacc0216 | Fichte |1110129 |null 7e5d9601-70b5-11e2-a512-aacc0200 | miao |1110129 |null 8e50ab8c-63e7-11e2-8c38-aacc0201 | Reaper |1110129 | 0 bba46192-6c63-11e2-8c38-aacc0201 | crafter1 |1110129 |null 5521cda0-7394-11e2-890a-aacc0216 | olea |1110129 |null dc413373-6b06-11e2-8943-aacc0216 | Darren Matar |1110129 | 1 0e7074ac-64bd-11e2-8c38-aacc0201 | Tom Bishop |1110129 |null 02238149-717a-11e2-890a-aacc0216 | Capt. Andrew |1110129 |null d4e5a014-72ac-11e2-890a-aacc0216 | pluto |1110129 |null 2a483b11-70b5-11e2-8c38-aacc0201 | alexey |1110129 |null 1b462f09-65f3-4148-a1a6-536b52b3bcfa | Andrea Len |1110129 | 1 9a96615a-7a72-11e2-a513-aacc0216 | padme |1110129 |null 58670d03-70b6-11e2-8c38-aacc0201 | trilly |1110129 |null Tracing session: 1bd92f60-ae93-11e2-a990-2f5b109ee83c activity | timestamp| source | source_elapsed ---+--+--+ execute_cql3_query | 19:02:42,519 | 10.10.30.169 | 0 Message received from /10.10.30.169 | 19:02:42,516 | 10.10.30.170 | 21 Executing indexed scan for [min(-9223372036854775808), min(-9223372036854775808)] | 19:02:42,518 | 10.10.30.170 | 1866 Executing single-partition query on agents.agents_station_id | 19:02:42,518 | 10.10.30.170 | 2244 Parsing statement | 19:02:42,519 | 10.10.30.169 | 42 Acquiring sstable references | 19:02:42,519 | 10.10.30.170 | 2787 Peparing statement | 19:02:42,519 | 10.10.30.169 |122 Merging memtable contents | 19:02:42,519 | 10.10.30.170 | 3107 Determining replicas to query | 19:02:42,519 | 10.10.30.169 |216 Key cache hit for sstable 620 | 19:02:42,520 | 10.10.30.170 | 3806 Merging data from memtables and 1 sstables | 19:02:42,520 | 10.10.30.170 | 4135 Read 17 live cells and 0 tombstoned | 19:02:42,521 | 10.10.30.170 | 4826 Sending message to /10.10.30.170 | 19:02:42,522 | 10.10.30.169 | 2761 Executing single-partition query on agents | 19:02:42,522 | 10.10.30.170 | 5490 Acquiring sstable references | 19:02:42,522 | 10.10.30.170 | 5782 Merging memtable contents | 19:02:42,522 | 10.10.30.170 | 6062 Key cache hit for sstable 443 | 19:02:42,523 | 10.10.30.170 | 6350 Merging data from memtables and 1 sstables | 19:02:42,523 | 10.10.30.170 | 6628 Read 1 live cells and 3 tombstoned | 19:02:42,523 | 10.10.30.170 | 6991
[jira] [Commented] (CASSANDRA-5501) Missing data on SELECT on secondary index
[ https://issues.apache.org/jira/browse/CASSANDRA-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13643012#comment-13643012 ] Sylvain Lebresne commented on CASSANDRA-5501: - Marco, could you try to run the query the tracing enabled? I.e. by setting 'TRACING ON' in cqlsh before running the query. Not sure he'll help, but it can't hurt I suppose. It is definitively weird that the cli and cqlsh would differ here, as they should end up with the same call internally basically. > Missing data on SELECT on secondary index > -- > > Key: CASSANDRA-5501 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5501 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.2.4 > Environment: linux ubuntu 12.04 >Reporter: Marco Matarazzo > Attachments: query_log.txt > > > We have a 3 nodes cluster, and a keyspace with RF = 3. > From cassandra-cli everything is fine (we actually never use it, I just > launched it for a check in this particular case). > [default@goh_master] get agents where station_id = ascii(1110129); > --- > RowKey: 6c8efeb6-7209-11e2-890a-aacc0216 > => (column=, value=, timestamp=1364580868176000) > => (column=character_points, value=, timestamp=136103068689) > => (column=component_id, value=0, timestamp=1364580868176000) > => (column=corporation_id, value=3efc729e-7209-11e2-890a-aacc0216, > timestamp=136103068689) > => (column=entity_id, value=0, timestamp=1364580868176000) > => (column=manufacturing, value=, timestamp=136103068689) > => (column=model, value=55, timestamp=136103068689) > => (column=name, value=Jenny Olifield, timestamp=136103068689) > => (column=name_check, value=jenny_olifield, timestamp=136103068689) > => (column=station_id, value=1110129, timestamp=1364580868176000) > => (column=stats_intellect, value=8, timestamp=136103068689) > => (column=stats_reflexes, value=8, timestamp=136103068689) > => (column=stats_stamina, value=7, timestamp=136103068689) > => (column=stats_technology, value=7, timestamp=136103068689) > => (column=trading, value=, timestamp=136103068689) > --- > RowKey: dc413373-6b06-11e2-8943-aacc0216 > => (column=, value=, timestamp=136656818522) > => (column=character_points, value=100, timestamp=1364580381651000) > => (column=component_id, value=, timestamp=1364580381651000) > => (column=corporation_id, value=574934cc-6b06-11e2-a512-aacc0200, > timestamp=1364580381651000) > => (column=entity_id, value=0, timestamp=1364580381651000) > => (column=manufacturing, value=, timestamp=1364580381651000) > => (column=model, value=500018, timestamp=1364580381651000) > => (column=name, value=Darren Matar, timestamp=1364580381651000) > => (column=name_check, value=darren_matar, timestamp=1364580381651000) > => (column=station_id, value=1110129, timestamp=1364580381651000) > => (column=stats_intellect, value=10, timestamp=1364580381651000) > => (column=stats_reflexes, value=10, timestamp=1364580381651000) > => (column=stats_stamina, value=10, timestamp=1364580381651000) > => (column=stats_technology, value=10, timestamp=1364580381651000) > => (column=trading, value=1, timestamp=136656818522) > --- > RowKey: 0e7074ac-64bd-11e2-8c38-aacc0201 > => (column=, value=, timestamp=1364828039093000) > => (column=character_points, value=, timestamp=136103068676) > => (column=component_id, value=0, timestamp=1364828039093000) > => (column=corporation_id, value=e398294e-64bc-11e2-8c38-aacc0201, > timestamp=136103068676) > => (column=entity_id, value=0, timestamp=1364828039093000) > => (column=manufacturing, value=1, timestamp=1362517535613000) > => (column=model, value=58, timestamp=136103068676) > => (column=name, value=Tom Bishop, timestamp=136103068676) > => (column=name_check, value=tom_bishop, timestamp=136103068676) > => (column=station_id, value=1110129, timestamp=1364828039093000) > => (column=stats_intellect, value=9, timestamp=136103068676) > => (column=stats_reflexes, value=7, timestamp=136103068676) > => (column=stats_stamina, value=5, timestamp=136103068676) > => (column=stats_technology, value=9, timestamp=136103068676) > => (column=trading, value=, timestamp=136103068676) > --- > RowKey: 1b462f09-65f3-4148-a1a6-536b52b3bcfa > => (column=, value=, timestamp=1366568185096000) > => (column=character_points, value=100, timestamp=1364580381537000) > => (column=component_id, value=, timestamp=1364580381537000) > => (column=corporation_id, value=1d2a8803-d139-4b50-85eb-92cb1082de2e, > timestamp=1364580381537000) > => (column=entity_id, value=0, timestamp=1364580381537000) > => (column=manufacturing, value=, timestamp=1364580381537000) > => (column=model, v
[jira] [Commented] (CASSANDRA-5501) Missing data on SELECT on secondary index
[
https://issues.apache.org/jira/browse/CASSANDRA-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642659#comment-13642659
]
Marco Matarazzo commented on CASSANDRA-5501:
CREATE TABLE agents (
agent_id ascii PRIMARY KEY,
character_points ascii,
component_id ascii,
corporation_id ascii,
entity_id ascii,
manufacturing ascii,
model ascii,
name ascii,
name_check ascii,
station_id ascii,
stats_intellect ascii,
stats_reflexes ascii,
stats_stamina ascii,
stats_technology ascii,
trading ascii
) WITH
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};
CREATE INDEX agents_corporation_id ON agents (corporation_id);
CREATE INDEX agents_entity_id ON agents (entity_id);
CREATE INDEX agents_name_idx_1 ON agents (name);
CREATE INDEX agents_name_check ON agents (name_check);
CREATE INDEX agents_station_id ON agents (station_id);
CREATE INDEX agents_trading ON agents (trading);
> Missing data on SELECT on secondary index
> --
>
> Key: CASSANDRA-5501
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5501
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.2.4
> Environment: linux ubuntu 12.04
>Reporter: Marco Matarazzo
> Attachments: query_log.txt
>
>
> We have a 3 nodes cluster, and a keyspace with RF = 3.
> From cassandra-cli everything is fine (we actually never use it, I just
> launched it for a check in this particular case).
> [default@goh_master] get agents where station_id = ascii(1110129);
> ---
> RowKey: 6c8efeb6-7209-11e2-890a-aacc0216
> => (column=, value=, timestamp=1364580868176000)
> => (column=character_points, value=, timestamp=136103068689)
> => (column=component_id, value=0, timestamp=1364580868176000)
> => (column=corporation_id, value=3efc729e-7209-11e2-890a-aacc0216,
> timestamp=136103068689)
> => (column=entity_id, value=0, timestamp=1364580868176000)
> => (column=manufacturing, value=, timestamp=136103068689)
> => (column=model, value=55, timestamp=136103068689)
> => (column=name, value=Jenny Olifield, timestamp=136103068689)
> => (column=name_check, value=jenny_olifield, timestamp=136103068689)
> => (column=station_id, value=1110129, timestamp=1364580868176000)
> => (column=stats_intellect, value=8, timestamp=136103068689)
> => (column=stats_reflexes, value=8, timestamp=136103068689)
> => (column=stats_stamina, value=7, timestamp=136103068689)
> => (column=stats_technology, value=7, timestamp=136103068689)
> => (column=trading, value=, timestamp=136103068689)
> ---
> RowKey: dc413373-6b06-11e2-8943-aacc0216
> => (column=, value=, timestamp=136656818522)
> => (column=character_points, value=100, timestamp=1364580381651000)
> => (column=component_id, value=, timestamp=1364580381651000)
> => (column=corporation_id, value=574934cc-6b06-11e2-a512-aacc0200,
> timestamp=1364580381651000)
> => (column=entity_id, value=0, timestamp=1364580381651000)
> => (column=manufacturing, value=, timestamp=1364580381651000)
> => (column=model, value=500018, timestamp=1364580381651000)
> => (column=name, value=Darren Matar, timestamp=1364580381651000)
> => (column=name_check, value=darren_matar, timestamp=1364580381651000)
> => (column=station_id, value=1110129, timestamp=1364580381651000)
> => (column=stats_intellect, value=10, timestamp=1364580381651000)
> => (column=stats_reflexes, value=10, timestamp=1364580381651000)
> => (column=stats_stamina, value=10, timestamp=1364580381651000)
> => (column=stats_technology, value=10, timestamp=1364580381651000)
> => (column=trading, value=1, timestamp=136656818522)
> ---
> RowKey: 0e7074ac-64bd-11e2-8c38-aacc0201
> => (column=, value=, timestamp=1364828039093000)
> => (column=character_points, value=, timestamp=136103068676)
> => (column=component_id, value=0, timestamp=1364828039093000)
> => (column=corporation_id, value=e398294e-64bc-11e2-8c38-aacc0201,
> timestamp=136103068676)
> => (column=entity_id, value=0, timestamp=1364828039093000)
> => (column=manufacturing, value=1, timestamp=1362517535613000)
> => (column=model, value=58, timestamp=136103068676)
> => (column=name, value=Tom Bishop, timestamp=136103068676)
> => (column=name_check, value=tom_bishop, timestamp=136103068676)
> => (column=station_id, value=1110129, timestamp=1364828039093000)
> => (column=stats_intellect, value=9, timestamp=136103068676)
> => (column=stats_reflexes, value=7, timestam
[jira] [Commented] (CASSANDRA-5501) Missing data on SELECT on secondary index
[ https://issues.apache.org/jira/browse/CASSANDRA-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642487#comment-13642487 ] Jonathan Ellis commented on CASSANDRA-5501: --- Can you post your schema? > Missing data on SELECT on secondary index > -- > > Key: CASSANDRA-5501 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5501 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.2.4 > Environment: linux ubuntu 12.04 >Reporter: Marco Matarazzo > Attachments: query_log.txt > > > We have a 3 nodes cluster, and a keyspace with RF = 3. > From cassandra-cli everything is fine (we actually never use it, I just > launched it for a check in this particular case). > [default@goh_master] get agents where station_id = ascii(1110129); > --- > RowKey: 6c8efeb6-7209-11e2-890a-aacc0216 > => (column=, value=, timestamp=1364580868176000) > => (column=character_points, value=, timestamp=136103068689) > => (column=component_id, value=0, timestamp=1364580868176000) > => (column=corporation_id, value=3efc729e-7209-11e2-890a-aacc0216, > timestamp=136103068689) > => (column=entity_id, value=0, timestamp=1364580868176000) > => (column=manufacturing, value=, timestamp=136103068689) > => (column=model, value=55, timestamp=136103068689) > => (column=name, value=Jenny Olifield, timestamp=136103068689) > => (column=name_check, value=jenny_olifield, timestamp=136103068689) > => (column=station_id, value=1110129, timestamp=1364580868176000) > => (column=stats_intellect, value=8, timestamp=136103068689) > => (column=stats_reflexes, value=8, timestamp=136103068689) > => (column=stats_stamina, value=7, timestamp=136103068689) > => (column=stats_technology, value=7, timestamp=136103068689) > => (column=trading, value=, timestamp=136103068689) > --- > RowKey: dc413373-6b06-11e2-8943-aacc0216 > => (column=, value=, timestamp=136656818522) > => (column=character_points, value=100, timestamp=1364580381651000) > => (column=component_id, value=, timestamp=1364580381651000) > => (column=corporation_id, value=574934cc-6b06-11e2-a512-aacc0200, > timestamp=1364580381651000) > => (column=entity_id, value=0, timestamp=1364580381651000) > => (column=manufacturing, value=, timestamp=1364580381651000) > => (column=model, value=500018, timestamp=1364580381651000) > => (column=name, value=Darren Matar, timestamp=1364580381651000) > => (column=name_check, value=darren_matar, timestamp=1364580381651000) > => (column=station_id, value=1110129, timestamp=1364580381651000) > => (column=stats_intellect, value=10, timestamp=1364580381651000) > => (column=stats_reflexes, value=10, timestamp=1364580381651000) > => (column=stats_stamina, value=10, timestamp=1364580381651000) > => (column=stats_technology, value=10, timestamp=1364580381651000) > => (column=trading, value=1, timestamp=136656818522) > --- > RowKey: 0e7074ac-64bd-11e2-8c38-aacc0201 > => (column=, value=, timestamp=1364828039093000) > => (column=character_points, value=, timestamp=136103068676) > => (column=component_id, value=0, timestamp=1364828039093000) > => (column=corporation_id, value=e398294e-64bc-11e2-8c38-aacc0201, > timestamp=136103068676) > => (column=entity_id, value=0, timestamp=1364828039093000) > => (column=manufacturing, value=1, timestamp=1362517535613000) > => (column=model, value=58, timestamp=136103068676) > => (column=name, value=Tom Bishop, timestamp=136103068676) > => (column=name_check, value=tom_bishop, timestamp=136103068676) > => (column=station_id, value=1110129, timestamp=1364828039093000) > => (column=stats_intellect, value=9, timestamp=136103068676) > => (column=stats_reflexes, value=7, timestamp=136103068676) > => (column=stats_stamina, value=5, timestamp=136103068676) > => (column=stats_technology, value=9, timestamp=136103068676) > => (column=trading, value=, timestamp=136103068676) > --- > RowKey: 1b462f09-65f3-4148-a1a6-536b52b3bcfa > => (column=, value=, timestamp=1366568185096000) > => (column=character_points, value=100, timestamp=1364580381537000) > => (column=component_id, value=, timestamp=1364580381537000) > => (column=corporation_id, value=1d2a8803-d139-4b50-85eb-92cb1082de2e, > timestamp=1364580381537000) > => (column=entity_id, value=0, timestamp=1364580381537000) > => (column=manufacturing, value=, timestamp=1364580381537000) > => (column=model, value=53, timestamp=1364580381537000) > => (column=name, value=Andrea Len, timestamp=1364580381537000) > => (column=name_check, value=andrea_len, timestamp=1364580381537000) > => (column=station_id, value=1110129, timestamp=1364580381537000) > => (column=stats_intellect, value=10,
[jira] [Commented] (CASSANDRA-5501) Missing data on SELECT on secondary index
[ https://issues.apache.org/jira/browse/CASSANDRA-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637869#comment-13637869 ] Marco Matarazzo commented on CASSANDRA-5501: I don't know how the internal mechanism of Cassandra works, so this could absolutely be no related, but the two columns that are not coming out have a more recent timestamp as it can be seen from cassandra-cli, and this CF has two sstables on disk, ...-435 and ...-436 . Trying to rebuild the index with nodetool and with log level lowered to trace, it seems to me that -436 get skipped due to bloom filter, and i find it strange during an index rebuild phase. I repeat, I have just a very vague sense of how the internal mechanism works, so it could be perfectly fine and I could be completely wrong, that's why I'm writing this as a comment :-) Sensitive (to me) parts of the logs follows: INFO [RMI TCP Connection(6)-194.116.72.170] 2013-04-22 11:21:31,491 ColumnFamilyStore.java (line 552) User Requested secondary index re-build for goh_master/agents indexes INFO [RMI TCP Connection(6)-194.116.72.170] 2013-04-22 11:21:31,491 SecondaryIndexManager.java (line 136) Submitting index build of [agents_station_id] for data in SSTableReader(path='/var/lib/cassandra/data/goh_master/agents/goh_master-agents-ib-435-Data.db'), SSTableReader(path='/var/lib/cassandra/data/goh_master/agents/goh_master-agents-ib-436-Data.db') DEBUG [CompactionExecutor:4] 2013-04-22 11:21:31,493 Table.java (line 412) Indexing row a4a0c7d4-a1e9-11e2-a860-aacc0216 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,493 CollationController.java (line 233) collectAllData TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,493 SSTableReader.java (line 748) Cache hit for KeyCacheKey(/var/lib/cassandra/data/goh_master/agents/goh_master-agents-ib-435, 61346130633764342d613165392d313165322d613836302d616163633030303030323136) -> org.apache.cassandra.db.RowIndexEntry@39b0595c TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,493 SliceQueryFilter.java (line 161) collecting 0 of 1: :false:0@136604087967 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,493 SliceQueryFilter.java (line 161) collecting 1 of 1: component_id:false:3@136560356360 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,494 SliceQueryFilter.java (line 161) collecting 2 of 1: corporation_id:false:36@136560356360 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,494 SliceQueryFilter.java (line 161) collecting 3 of 1: entity_id:false:1@136604087967 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,494 SliceQueryFilter.java (line 161) collecting 4 of 1: name:false:6@136560356360 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,494 SliceQueryFilter.java (line 161) collecting 5 of 1: name_check:false:6@136560356360 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,495 SliceQueryFilter.java (line 161) collecting 6 of 1: station_id:false:36@136604087967 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,495 SliceQueryFilter.java (line 161) collecting 7 of 1: stats_intellect:false:1@136560356360 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,495 SliceQueryFilter.java (line 161) collecting 8 of 1: stats_reflexes:false:1@136560356360 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,495 SliceQueryFilter.java (line 161) collecting 9 of 1: stats_stamina:false:1@136560356360 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,495 SliceQueryFilter.java (line 161) collecting 10 of 1: stats_technology:false:1@136560356360 DEBUG [CompactionExecutor:4] 2013-04-22 11:21:31,496 Table.java (line 412) Indexing row 5315b187-bc33-4747-997a-dcc7bd60d768 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,496 CollationController.java (line 233) collectAllData TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,496 SSTableReader.java (line 748) Cache hit for KeyCacheKey(/var/lib/cassandra/data/goh_master/agents/goh_master-agents-ib-435, 35333135623138372d62632d343734372d393937612d646363376264363064373638) -> org.apache.cassandra.db.RowIndexEntry@6ee09a07 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,496 SliceQueryFilter.java (line 161) collecting 0 of 1: :false:0@1366568185181000 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,496 SliceQueryFilter.java (line 161) collecting 1 of 1: character_points:false:3@1364580381572000 [...lots of these repeated until...] DEBUG [CompactionExecutor:4] 2013-04-22 11:21:31,642 Table.java (line 412) Indexing row 7b594dc0-a901-11e2-80b4-aacc0201 TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,642 CollationController.java (line 233) collectAllData TRACE [CompactionExecutor:4] 2013-04-22 11:21:31,642 SSTableReader.java (line 748) Cache hit for KeyCacheKey(/var/lib/cassandra/data/goh_master/agents/goh_master-agents-ib-436, 37623539346463302d613930312d313165322d383062342d61616363
