[jira] [Commented] (CASSANDRA-10347) Bulk Loader API could not tolerate even node failure
[ https://issues.apache.org/jira/browse/CASSANDRA-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939369#comment-14939369 ] Piotr Kołaczkowski commented on CASSANDRA-10347: +1 to having this function in BulkLoader API. That would let us improve bulk loading from Spark. > Bulk Loader API could not tolerate even node failure > > > Key: CASSANDRA-10347 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10347 > Project: Cassandra > Issue Type: Bug >Reporter: Shenghua Wan >Assignee: Paulo Motta > Fix For: 2.1.x, 2.2.x, 3.0.x > > Attachments: AbstractBulkRecordWriter.java > > > When user uses CqlBulkOutputFormat, it tries to stream to all the nodes in > the token range, which includes the dead nodes. Therefore, the stream failed. > There was a design in C* API to allow stream() method to have a list of > ignore hosts, but it was not utilized. > The empty-argument stream() method is called in all existing versions of C*, > i.e. > in v2.0.11, > https://github.com/apache/cassandra/blob/cassandra-2.0.11/src/java/org/apache/cassandra/hadoop/AbstractBulkRecordWriter.java#L122 > in v2.1.5, > https://github.com/apache/cassandra/blob/cassandra-2.1.5/src/java/org/apache/cassandra/hadoop/AbstractBulkRecordWriter.java#L122 > and current trunk branch > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java#L241 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10421) Potential issue with LogTransaction as it only checks in a single directory for files
[
https://issues.apache.org/jira/browse/CASSANDRA-10421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939363#comment-14939363
]
Marcus Eriksson commented on CASSANDRA-10421:
-
Maybe we could lazily create the LogFiles when we do trackNew? And support
multiple LogFiles if we get files in separate directories
> Potential issue with LogTransaction as it only checks in a single directory
> for files
> -
>
> Key: CASSANDRA-10421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10421
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Stefania
>
> When creating a new LogTransaction we try to create the new logfile in the
> same directory as the one we are writing to, but as we use
> {{[directories.getDirectoryForNewSSTables()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java#L125]}}
> this might end up in "any" of the configured data directories. If it does,
> we will not be able to clean up leftovers as we check for files in the same
> directory as the logfile was created:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogRecord.java#L163
> cc [~Stefania]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10428) select with exact date is not working
[
https://issues.apache.org/jira/browse/CASSANDRA-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939351#comment-14939351
]
Chandran Anjur Narasimhan commented on CASSANDRA-10428:
---
It is not working with time_format set in the ~/.cassandra/cqlshrc
{noformat}
NCHAN-M-D0LZ:bin nchan$ more ~/.cassandra/cqlshrc
[ui]
time_format=%Y-%m-%d %H:%M:%S.%f%z
NCHAN-M-D0LZ:bin nchan$
NCHAN-M-D0LZ:bin nchan$ ./cqlsh
Connected to CCC Multi-Region Cassandra Cluster at 171.71.189.11:9042.
[cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
cqlsh> use ccc;
cqlsh:ccc> select submissionid, ezid, name, time, state, status,
translated_criteria_status from ez_task_result where
submissionid='760dd154670811e58c04005056bb6ff0' and
ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
time>='2015-09-29 20:54:23-0700';
submissionid | ezid | name
| time | state | status |
translated_criteria_status
--+--+--++---+-+
760dd154670811e58c04005056bb6ff0 | 760dd6de670811e594fc005056bb6ff0 |
run-sanities | 2015-09-29 20:54:23.f-0700 | EXECUTING | IN_PROGRESS |
run-sanities started
(1 rows)
cqlsh:ccc> select submissionid, ezid, name, time, state, status,
translated_criteria_status from ez_task_result where
submissionid='760dd154670811e58c04005056bb6ff0' and
ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
time='2015-09-29 20:54:23-0700';
submissionid | ezid | name | time | analyzed_index_root | analyzed_log_path |
clientid | end_time | jenkins_path | log_file_path | path_available |
path_to_task | required_for_overall_status | start_time | state | status |
translated_criteria_status | type
--+--+--+--+-+---+--+--+--+---++--+-++---+++--
(0 rows)
{noformat}
> select with exact date is not working
> -
>
> Key: CASSANDRA-10428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10428
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: OSX 10.10.2
>Reporter: Chandran Anjur Narasimhan
> Labels: cqlsh
> Fix For: 2.1.x
>
>
> Query with >= timestamp works. But the exact timestamp value is not working.
> {noformat}
> NCHAN-M-D0LZ:bin nchan$ ./cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at :.
> [cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh>
> {noformat}
> {panel:title=Schema|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> desc COLUMNFAMILY ez_task_result ;
> CREATE TABLE ccc.ez_task_result (
> submissionid text,
> ezid text,
> name text,
> time timestamp,
> analyzed_index_root text,
> ...
> ...
> PRIMARY KEY (submissionid, ezid, name, time)
> {panel}
> {panel:title=Working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> select submissionid, ezid, name, time, state, status,
> translated_criteria_status from ez_task_result where
> submissionid='760dd154670811e58c04005056bb6ff0' and
> ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
> time>='2015-09-29 20:54:23-0700';
> submissionid | ezid | name
> | time | state | status |
> translated_criteria_status
> --+--+--+--+---+-+
> 760dd154670811e58c04005056bb6ff0 | 760dd6de670811e594fc005056bb6ff0 |
> run-sanities | 2015-09-29 20:54:23-0700 | EXECUTING | IN_PROGRESS |
> run-sanities started
> (1 rows)
> cqlsh:ccc>
> {panel}
> {panel:title=Not
> working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> select submissionid, ezid, name, time, state, status,
> translated_criteria_status from ez_task_result where
> submissionid='760dd154670811e58c04005056bb6ff0' and
> ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
> time='2015-09-29 20:54:23-0700';
> submissionid | ezid | name | time | analyzed_index_root | analyzed_log_path
> | clientid | end_time | jenkins_path | log_file_path | path_available |
> path_to_task | required_for_overall_status | start_time | state | status |
> translated_criteria_status
[jira] [Commented] (CASSANDRA-10231) Null status entries on nodes that crash during decommission of a different node
[ https://issues.apache.org/jira/browse/CASSANDRA-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939288#comment-14939288 ] Stefania commented on CASSANDRA-10231: -- Of course, I totally forgot this was originally a MV test. It's not worth it, I can run the dtest a few more times, plus I will also try and write a new dtest for 10298. Let's keep the focus on 3.0 for this ticket. > Null status entries on nodes that crash during decommission of a different > node > --- > > Key: CASSANDRA-10231 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10231 > Project: Cassandra > Issue Type: Bug >Reporter: Joel Knighton >Assignee: Stefania > Fix For: 3.0.0 rc2 > > Attachments: n1.log, n2.log, n3.log, n4.log, n5.log > > > This issue is reproducible through a Jepsen test of materialized views that > crashes and decommissions nodes throughout the test. > In a 5 node cluster, if a node crashes at a certain point (unknown) during > the decommission of a different node, it may start with a null entry for the > decommissioned node like so: > DN 10.0.0.5 ? 256 ? null rack1 > This entry does not get updated/cleared by gossip. This entry is removed upon > a restart of the affected node. > This issue is further detailed in ticket > [10068|https://issues.apache.org/jira/browse/CASSANDRA-10068]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10231) Null status entries on nodes that crash during decommission of a different node
[ https://issues.apache.org/jira/browse/CASSANDRA-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939281#comment-14939281 ] Joel Knighton edited comment on CASSANDRA-10231 at 10/1/15 3:53 AM: I can try to reproduce on 2.1 or 2.2 with Jepsen, but there would be a delay to write the test. The current test that (used to) reproduce uses materialized views, so it is 3.0 and up only. was (Author: jkni): I can try to reproduce on 2.1 or 2.2 with Jepsen, but there would be a delay to write the test. The current test that (used to) reproduce uses materialized views, so is 3.0 and up only. > Null status entries on nodes that crash during decommission of a different > node > --- > > Key: CASSANDRA-10231 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10231 > Project: Cassandra > Issue Type: Bug >Reporter: Joel Knighton >Assignee: Stefania > Fix For: 3.0.0 rc2 > > Attachments: n1.log, n2.log, n3.log, n4.log, n5.log > > > This issue is reproducible through a Jepsen test of materialized views that > crashes and decommissions nodes throughout the test. > In a 5 node cluster, if a node crashes at a certain point (unknown) during > the decommission of a different node, it may start with a null entry for the > decommissioned node like so: > DN 10.0.0.5 ? 256 ? null rack1 > This entry does not get updated/cleared by gossip. This entry is removed upon > a restart of the affected node. > This issue is further detailed in ticket > [10068|https://issues.apache.org/jira/browse/CASSANDRA-10068]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10231) Null status entries on nodes that crash during decommission of a different node
[ https://issues.apache.org/jira/browse/CASSANDRA-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939281#comment-14939281 ] Joel Knighton commented on CASSANDRA-10231: --- I can try to reproduce on 2.1 or 2.2 with Jepsen, but there would be a delay to write the test. The current test that (used to) reproduce uses materialized views, so is 3.0 and up only. > Null status entries on nodes that crash during decommission of a different > node > --- > > Key: CASSANDRA-10231 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10231 > Project: Cassandra > Issue Type: Bug >Reporter: Joel Knighton >Assignee: Stefania > Fix For: 3.0.0 rc2 > > Attachments: n1.log, n2.log, n3.log, n4.log, n5.log > > > This issue is reproducible through a Jepsen test of materialized views that > crashes and decommissions nodes throughout the test. > In a 5 node cluster, if a node crashes at a certain point (unknown) during > the decommission of a different node, it may start with a null entry for the > decommissioned node like so: > DN 10.0.0.5 ? 256 ? null rack1 > This entry does not get updated/cleared by gossip. This entry is removed upon > a restart of the affected node. > This issue is further detailed in ticket > [10068|https://issues.apache.org/jira/browse/CASSANDRA-10068]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10231) Null status entries on nodes that crash during decommission of a different node
[ https://issues.apache.org/jira/browse/CASSANDRA-10231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939224#comment-14939224 ] Stefania commented on CASSANDRA-10231: -- I've removed the additional GOSSIP entry log message from the proper branch but I've left the old branch with such entry in the [10231-3.0-debug|https://github.com/stef1927/cassandra/commits/10231-3.0-debug] branch. I've tried reproducing the issue for 2.1 or 2.2 but couldn't, it seems CASSANDRA-10298 is a slightly different issue after all (GOSSIP major update with status LEFT but missing tokens and host id). Are you able to confirm with Jepsen if 2.1 and 2.2 are affected or not? I should also add the CI links for 3.0: http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10231-3.0-testall/ http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-10231-3.0-dtest/ > Null status entries on nodes that crash during decommission of a different > node > --- > > Key: CASSANDRA-10231 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10231 > Project: Cassandra > Issue Type: Bug >Reporter: Joel Knighton >Assignee: Stefania > Fix For: 3.0.0 rc2 > > Attachments: n1.log, n2.log, n3.log, n4.log, n5.log > > > This issue is reproducible through a Jepsen test of materialized views that > crashes and decommissions nodes throughout the test. > In a 5 node cluster, if a node crashes at a certain point (unknown) during > the decommission of a different node, it may start with a null entry for the > decommissioned node like so: > DN 10.0.0.5 ? 256 ? null rack1 > This entry does not get updated/cleared by gossip. This entry is removed upon > a restart of the affected node. > This issue is further detailed in ticket > [10068|https://issues.apache.org/jira/browse/CASSANDRA-10068]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager
[
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938177#comment-14938177
]
J.P. Eiti Kimura edited comment on CASSANDRA-10233 at 10/1/15 3:29 AM:
---
Guys I code a little patch for the problem. Looks like it is affecting the
hinted hand off scheduler.
{code}
cqlsh:system> SELECT target_id,hint_id,message_version FROM hints LIMIT 5;
target_id| hint_id |
message_version
--+--+-
| 2f5e0320-62d3-11e5-877e-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fad152f0-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fad152f1-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fb69e970-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fb69e971-662a-11e5-89ed-77558ae77cc8 |
8
(5 rows)
{code}
We have an empty target_id in our hints table. That is causing the problem.
Can you check my patch?
was (Author: eitikimura):
Guys I code a little patch for the problem. Looks like it is affecting the
hinted hand off scheduler.
{code}
cqlsh:system> SELECT target_id,hint_id,message_version FROM hints LIMIT 5;
target_id| hint_id |
message_version
--+--+-
|
2f5e0320-62d3-11e5-877e-77558ae77cc8 | 8
72888e32-dae5-41cd-a033-3c5871a3e045 | fad152f0-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fad152f1-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fb69e970-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fb69e971-662a-11e5-89ed-77558ae77cc8 |
8
(5 rows)
{code}
We have an empty target_id in our hints table. That is causing the problem.
Can you check my patch?
> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Andrew Hust
> Attachments: cassandra-2.1.8-10233.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO [CompactionExecutor:1381] 2015-08-31 18:31:55,506
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20)
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
> ]
> INFO [CompactionExecutor:1381] 2015-08-31 18:31:55,599
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1
> sstables to
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,]
> to level=0. 1,544,495 bytes to 1,544,495 (~100% of original) in 93ms =
> 15.838121MB/s. 0 total partitions merged to 4. Partition merge counts were
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 -
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
> at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410)
> ~[na:1.7.0_79]
> at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_79]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> [na:1.7.0_79]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> [na:1.7.0_79]
> at
> java.util.c
[jira] [Comment Edited] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager
[
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938177#comment-14938177
]
J.P. Eiti Kimura edited comment on CASSANDRA-10233 at 10/1/15 3:29 AM:
---
Guys I code a little patch for the problem. Looks like it is affecting the
hinted hand off scheduler.
{code}
cqlsh:system> SELECT target_id,hint_id,message_version FROM hints LIMIT 5;
target_id| hint_id |
message_version
--+--+-
| 2f5e0320-62d3-11e5-877e-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fad152f0-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fad152f1-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fb69e970-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fb69e971-662a-11e5-89ed-77558ae77cc8 |
8
(5 rows)
{code}
We have an empty target_id in our hints table. That is causing the problem.
Can you check my patch?
was (Author: eitikimura):
Guys I code a little patch for the problem. Looks like it is affecting the
hinted hand off scheduler.
{code}
cqlsh:system> SELECT target_id,hint_id,message_version FROM hints LIMIT 5;
target_id| hint_id |
message_version
--+--+-
| 2f5e0320-62d3-11e5-877e-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fad152f0-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fad152f1-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fb69e970-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fb69e971-662a-11e5-89ed-77558ae77cc8 |
8
(5 rows)
{code}
We have an empty target_id in our hints table. That is causing the problem.
Can you check my patch?
> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Andrew Hust
> Attachments: cassandra-2.1.8-10233.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO [CompactionExecutor:1381] 2015-08-31 18:31:55,506
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20)
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
> ]
> INFO [CompactionExecutor:1381] 2015-08-31 18:31:55,599
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1
> sstables to
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,]
> to level=0. 1,544,495 bytes to 1,544,495 (~100% of original) in 93ms =
> 15.838121MB/s. 0 total partitions merged to 4. Partition merge counts were
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 -
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
> at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410)
> ~[na:1.7.0_79]
> at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_79]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> [na:1.7.0_79]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> [na:1.7.0_79]
> at
> java.util.concur
[jira] [Updated] (CASSANDRA-7276) Include keyspace and table names in logs where possible
[
https://issues.apache.org/jira/browse/CASSANDRA-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
J.P. Eiti Kimura updated CASSANDRA-7276:
Attachment: cassandra-2.1.9-7276-v2.txt
[~pauloricardomg] can you help reviewing this new patch v2?
I'm now using the MDC from slf4j, I refectored some parts to add the clean MDC
option according to the links you provide me.
I did a better organization trying to figure out the best places to add the log
entries.
> Include keyspace and table names in logs where possible
> ---
>
> Key: CASSANDRA-7276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7276
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Nitzan Volman
>Priority: Minor
> Labels: bootcamp, lhf
> Fix For: 2.1.x
>
> Attachments: 2.1-CASSANDRA-7276-v1.txt,
> cassandra-2.1-7276-compaction.txt, cassandra-2.1-7276.txt,
> cassandra-2.1.9-7276-v2.txt, cassandra-2.1.9-7276.txt
>
>
> Most error messages and stacktraces give you no clue as to what keyspace or
> table was causing the problem. For example:
> {noformat}
> ERROR [MutationStage:61648] 2014-05-20 12:05:45,145 CassandraDaemon.java
> (line 198) Exception in thread Thread[MutationStage:61648,5,main]
> java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Unknown Source)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:63)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:72)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:98)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:35)
> at
> edu.stanford.ppl.concurrent.SnapTreeMap$1.compareTo(SnapTreeMap.java:538)
> at
> edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1108)
> at
> edu.stanford.ppl.concurrent.SnapTreeMap.updateUnderRoot(SnapTreeMap.java:1059)
> at edu.stanford.ppl.concurrent.SnapTreeMap.update(SnapTreeMap.java:1023)
> at
> edu.stanford.ppl.concurrent.SnapTreeMap.putIfAbsent(SnapTreeMap.java:985)
> at
> org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:328)
> at
> org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDelta(AtomicSortedColumns.java:200)
> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:226)
> at org.apache.cassandra.db.Memtable.put(Memtable.java:173)
> at
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:893)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:368)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:333)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:206)
> at
> org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:56)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {noformat}
> We should try to include info on the keyspace and column family in the error
> messages or logs whenever possible. This includes reads, writes,
> compactions, flushes, repairs, and probably more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10298) Replaced dead node stayed in gossip forever
[
https://issues.apache.org/jira/browse/CASSANDRA-10298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939214#comment-14939214
]
Stefania commented on CASSANDRA-10298:
--
Sorry I wasn't very clear: the patch needs to be reviewed, then a committer
will be able to commit it.
However, I would like to reproduce this problem in a dtest first, could you
give us the exact steps on how to reproduce it? Did you follow [these
instructions|http://docs.datastax.com/en/cassandra/2.1/cassandra/operations/ops_replace_node_t.html]
or deviated in any way?
In fact from a closer look, this is a slightly different problem than
CASSANDRA-10231 because there we are processing new states ({{applyNewStates}})
whereas here we have received a major update ({{handleMajorStateChange}}).
> Replaced dead node stayed in gossip forever
> ---
>
> Key: CASSANDRA-10298
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10298
> Project: Cassandra
> Issue Type: Bug
>Reporter: Dikang Gu
>Assignee: Dikang Gu
> Fix For: 2.1.x
>
> Attachments: CASSANDRA-10298.patch
>
>
> The dead node stayed in the nodetool status,
> DN 10.210.165.55379.76 GB 256 ? null
> And in the log, it throws NPE when trying to remove it.
> {code}
> 2015-09-10_06:41:22.92453 ERROR 06:41:22 Exception in thread
> Thread[GossipStage:1,5,main]
> 2015-09-10_06:41:22.92454 java.lang.NullPointerException: null
> 2015-09-10_06:41:22.92455 at
> org.apache.cassandra.utils.UUIDGen.decompose(UUIDGen.java:100)
> 2015-09-10_06:41:22.92455 at
> org.apache.cassandra.db.HintedHandOffManager.deleteHintsForEndpoint(HintedHandOffManager.java:201)
>
> 2015-09-10_06:41:22.92455 at
> org.apache.cassandra.service.StorageService.excise(StorageService.java:1886)
> 2015-09-10_06:41:22.92455 at
> org.apache.cassandra.service.StorageService.excise(StorageService.java:1902)
> 2015-09-10_06:41:22.92456 at
> org.apache.cassandra.service.StorageService.handleStateLeft(StorageService.java:1805)
> 2015-09-10_06:41:22.92457 at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1473)
>
> 2015-09-10_06:41:22.92457 at
> org.apache.cassandra.service.StorageService.onJoin(StorageService.java:2099)
> 2015-09-10_06:41:22.92457 at
> org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1009)
> 2015-09-10_06:41:22.92458 at
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1085)
> 2015-09-10_06:41:22.92458 at
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)
>
> 2015-09-10_06:41:22.92458 at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
> 2015-09-10_06:41:22.92459 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_45]
> 2015-09-10_06:41:22.92460 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ~[na:1.7.0_45]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7276) Include keyspace and table names in logs where possible
[
https://issues.apache.org/jira/browse/CASSANDRA-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939160#comment-14939160
]
J.P. Eiti Kimura commented on CASSANDRA-7276:
-
Ops, sorry for the MDC from log4j, the weird part is that is compiling well in
my eclipse ide. I'll fix that.
You said: "There were also some warnings when applying the patch:"
How can I simulate this apply to check this warnings? is there a link with more
explanations or a process to perform before submit the patch? I can figure it
out before and you don't need to spend time checking this kind of thing.
All right, let me check these links, I'll post a refactored version later.
Thanks for the review [~pauloricardomg]!
> Include keyspace and table names in logs where possible
> ---
>
> Key: CASSANDRA-7276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7276
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Nitzan Volman
>Priority: Minor
> Labels: bootcamp, lhf
> Fix For: 2.1.x
>
> Attachments: 2.1-CASSANDRA-7276-v1.txt,
> cassandra-2.1-7276-compaction.txt, cassandra-2.1-7276.txt,
> cassandra-2.1.9-7276.txt
>
>
> Most error messages and stacktraces give you no clue as to what keyspace or
> table was causing the problem. For example:
> {noformat}
> ERROR [MutationStage:61648] 2014-05-20 12:05:45,145 CassandraDaemon.java
> (line 198) Exception in thread Thread[MutationStage:61648,5,main]
> java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Unknown Source)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:63)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:72)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:98)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:35)
> at
> edu.stanford.ppl.concurrent.SnapTreeMap$1.compareTo(SnapTreeMap.java:538)
> at
> edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1108)
> at
> edu.stanford.ppl.concurrent.SnapTreeMap.updateUnderRoot(SnapTreeMap.java:1059)
> at edu.stanford.ppl.concurrent.SnapTreeMap.update(SnapTreeMap.java:1023)
> at
> edu.stanford.ppl.concurrent.SnapTreeMap.putIfAbsent(SnapTreeMap.java:985)
> at
> org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:328)
> at
> org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDelta(AtomicSortedColumns.java:200)
> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:226)
> at org.apache.cassandra.db.Memtable.put(Memtable.java:173)
> at
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:893)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:368)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:333)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:206)
> at
> org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:56)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {noformat}
> We should try to include info on the keyspace and column family in the error
> messages or logs whenever possible. This includes reads, writes,
> compactions, flushes, repairs, and probably more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10428) select with exact date is not working
[
https://issues.apache.org/jira/browse/CASSANDRA-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939157#comment-14939157
]
Stefania edited comment on CASSANDRA-10428 at 10/1/15 1:42 AM:
---
We could add microseconds but it's not as trivial as changing the default
format.
On 2.1 HEAD, after setting {{DEFAULT_TIME_FORMAT}} to {{'%Y-%m-%d
%H:%M:%S.%f%z'}}, I tested a bit with this simplified schema:
{code}
CREATE TABLE ks.test (
id text,
time timestamp,
val text,
PRIMARY KEY (id, time)
)
{code}
With these inserts:
{code}
INSERT into test (id, time, val) VALUES ( '1', '2015-09-29 20:54:23-0700',
'abc');
INSERT into test (id, time, val) VALUES ( '1', '2015-09-29 20:54:24.100-0700',
'abc');
INSERT into test (id, time, val) VALUES ( '1', '2015-09-29 20:54:24.200',
'abc');
{code}
here is what's displayed:
{code}
SELECT * from test where id='1' and time >= '2015-09-29 20:54:24';
id | time| val
+-+-
1 | 2015-09-29 12:54:24.00+ | abc
1 | 2015-09-30 03:54:23.00+ | abc
1 | 2015-09-30 03:54:24.00+ | abc
{code}
We cannot support micros at present and we fix the TZ to UTC (CASSANDRA-1000).
In _formatting.py_, {{format_value_timestamp}} calls {{calendar.timegm()}}
which looses sub-second precision and converts to UTC. We also use a helper
method from the driver, {{datetime_from_timestamp}} which currently expects
seconds. So we'd have to manually add the microseconds before formatting the
time.
Given that users already can change {{time_format}} we should probably add
microseconds.
was (Author: stefania):
We could add microseconds but it's not as trivial as changing the format.
On 2.1 HEAD, after setting {{DEFAULT_TIME_FORMAT}} to {{'%Y-%m-%d
%H:%M:%S.%f%z'}}, I tested a bit with this simplified schema:
{code}
CREATE TABLE ks.test (
id text,
time timestamp,
val text,
PRIMARY KEY (id, time)
)
{code}
With these inserts:
{code}
INSERT into test (id, time, val) VALUES ( '1', '2015-09-29 20:54:23-0700',
'abc');
INSERT into test (id, time, val) VALUES ( '1', '2015-09-29 20:54:24.100-0700',
'abc');
INSERT into test (id, time, val) VALUES ( '1', '2015-09-29 20:54:24.200',
'abc');
{code}
here is what's displayed:
{code}
SELECT * from test where id='1' and time >= '2015-09-29 20:54:24';
id | time| val
+-+-
1 | 2015-09-29 12:54:24.00+ | abc
1 | 2015-09-30 03:54:23.00+ | abc
1 | 2015-09-30 03:54:24.00+ | abc
{code}
We cannot support micros at present and we fix the TZ to UTC (CASSANDRA-1000).
In _formatting.py_, {{format_value_timestamp}} calls {{calendar.timegm()}}
which looses sub-second precision and converts to UTC. We also use a helper
method from the driver, {{datetime_from_timestamp}} which currently expects
seconds. So we'd have to manually add the microseconds before formatting the
time.
Given that users already can change {{time_format}} we should probably add
microseconds.
> select with exact date is not working
> -
>
> Key: CASSANDRA-10428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10428
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: OSX 10.10.2
>Reporter: Chandran Anjur Narasimhan
> Labels: cqlsh
> Fix For: 2.1.x
>
>
> Query with >= timestamp works. But the exact timestamp value is not working.
> {noformat}
> NCHAN-M-D0LZ:bin nchan$ ./cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at :.
> [cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh>
> {noformat}
> {panel:title=Schema|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> desc COLUMNFAMILY ez_task_result ;
> CREATE TABLE ccc.ez_task_result (
> submissionid text,
> ezid text,
> name text,
> time timestamp,
> analyzed_index_root text,
> ...
> ...
> PRIMARY KEY (submissionid, ezid, name, time)
> {panel}
> {panel:title=Working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> select submissionid, ezid, name, time, state, status,
> translated_criteria_status from ez_task_result where
> submissionid='760dd154670811e58c04005056bb6ff0' and
> ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
> time>='2015-09-29 20:54:23-0700';
> submissionid | ezid | name
> | time | state | status |
> translated_criteria_status
> --+--+--+--+---+-+
[jira] [Commented] (CASSANDRA-10428) select with exact date is not working
[
https://issues.apache.org/jira/browse/CASSANDRA-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939157#comment-14939157
]
Stefania commented on CASSANDRA-10428:
--
We could add microseconds but it's not as trivial as changing the format.
On 2.1 HEAD, after setting {{DEFAULT_TIME_FORMAT}} to {{'%Y-%m-%d
%H:%M:%S.%f%z'}}, I tested a bit with this simplified schema:
{code}
CREATE TABLE ks.test (
id text,
time timestamp,
val text,
PRIMARY KEY (id, time)
)
{code}
With these inserts:
{code}
INSERT into test (id, time, val) VALUES ( '1', '2015-09-29 20:54:23-0700',
'abc');
INSERT into test (id, time, val) VALUES ( '1', '2015-09-29 20:54:24.100-0700',
'abc');
INSERT into test (id, time, val) VALUES ( '1', '2015-09-29 20:54:24.200',
'abc');
{code}
here is what's displayed:
{code}
SELECT * from test where id='1' and time >= '2015-09-29 20:54:24';
id | time| val
+-+-
1 | 2015-09-29 12:54:24.00+ | abc
1 | 2015-09-30 03:54:23.00+ | abc
1 | 2015-09-30 03:54:24.00+ | abc
{code}
We cannot support micros at present and we fix the TZ to UTC (CASSANDRA-1000).
In _formatting.py_, {{format_value_timestamp}} calls {{calendar.timegm()}}
which looses sub-second precision and converts to UTC. We also use a helper
method from the driver, {{datetime_from_timestamp}} which currently expects
seconds. So we'd have to manually add the microseconds before formatting the
time.
Given that users already can change {{time_format}} we should probably add
microseconds.
> select with exact date is not working
> -
>
> Key: CASSANDRA-10428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10428
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: OSX 10.10.2
>Reporter: Chandran Anjur Narasimhan
> Labels: cqlsh
> Fix For: 2.1.x
>
>
> Query with >= timestamp works. But the exact timestamp value is not working.
> {noformat}
> NCHAN-M-D0LZ:bin nchan$ ./cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at :.
> [cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh>
> {noformat}
> {panel:title=Schema|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> desc COLUMNFAMILY ez_task_result ;
> CREATE TABLE ccc.ez_task_result (
> submissionid text,
> ezid text,
> name text,
> time timestamp,
> analyzed_index_root text,
> ...
> ...
> PRIMARY KEY (submissionid, ezid, name, time)
> {panel}
> {panel:title=Working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> select submissionid, ezid, name, time, state, status,
> translated_criteria_status from ez_task_result where
> submissionid='760dd154670811e58c04005056bb6ff0' and
> ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
> time>='2015-09-29 20:54:23-0700';
> submissionid | ezid | name
> | time | state | status |
> translated_criteria_status
> --+--+--+--+---+-+
> 760dd154670811e58c04005056bb6ff0 | 760dd6de670811e594fc005056bb6ff0 |
> run-sanities | 2015-09-29 20:54:23-0700 | EXECUTING | IN_PROGRESS |
> run-sanities started
> (1 rows)
> cqlsh:ccc>
> {panel}
> {panel:title=Not
> working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> select submissionid, ezid, name, time, state, status,
> translated_criteria_status from ez_task_result where
> submissionid='760dd154670811e58c04005056bb6ff0' and
> ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
> time='2015-09-29 20:54:23-0700';
> submissionid | ezid | name | time | analyzed_index_root | analyzed_log_path
> | clientid | end_time | jenkins_path | log_file_path | path_available |
> path_to_task | required_for_overall_status | start_time | state | status |
> translated_criteria_status | type
> --+--+--+--+-+---+--+--+--+---++--+-++---+++--
> (0 rows)
> cqlsh:ccc>
> {panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10429) Flush schema tables after local schema change
[
https://issues.apache.org/jira/browse/CASSANDRA-10429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939151#comment-14939151
]
Aleksey Yeschenko commented on CASSANDRA-10429:
---
The patch is good as is, but I would prefer instead to remove the overload of
{{SchemaKeyspace::mergeSchema}} with the boolean doFlush altogether, and put
the {{cassandra.test.flush_local_schema_changes}} flag check into the
single-argument {{SchemaKeyspace::mergeSchema}}. No good reason to expose the
unsafe method version.
> Flush schema tables after local schema change
> -
>
> Key: CASSANDRA-10429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10429
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Minor
> Fix For: 3.0.0 rc2
>
>
> In CASSANDRA-7327, we disabled the normal flush of system schema tables after
> "local" schema changes to improve the runtime of unit tests. However, there
> are some cases where this flush is necessary for schema durability. For
> example, if a custom secondary index needs to make schema changes as part of
> it's creation, this is desirable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10289) Fix cqlshlib tests
[ https://issues.apache.org/jira/browse/CASSANDRA-10289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939150#comment-14939150 ] Stefania commented on CASSANDRA-10289: -- It seems on CASSCI we only run these tests on trunk at the moment. However some of these issues probably also affect 2.1+. [~mambocab] do you think we should set-up jobs and backport the patch to 2.1+? > Fix cqlshlib tests > -- > > Key: CASSANDRA-10289 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10289 > Project: Cassandra > Issue Type: Bug > Components: Tests >Reporter: Jim Witschey >Assignee: Jim Witschey > Labels: cqlsh > Fix For: 3.0.0 rc2 > > Attachments: trunk-10289.diff > > > The cqlsh tests in trunk haven't been running for a while: > http://cassci.datastax.com/view/All_Jobs/job/trunk_cqlshlib/423/testReport/ > This looks like the driver errors that happened because of CASSANDRA-6717. > Not sure why it's happening now; the driver installation looks normal to me > on those jobs. [~mshuler]? > There were also some changes to cqlsh itself that also broke the test > harness, but I believe those are fixed here: > https://github.com/mambocab/cassandra/tree/fix-cqlsh-tests > Once the tests are running successfully on CassCI, I'll test my patch and > mark as patch available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9766) Bootstrap outgoing streaming speeds are much slower than during repair
[ https://issues.apache.org/jira/browse/CASSANDRA-9766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939147#comment-14939147 ] Yuki Morishita commented on CASSANDRA-9766: --- Both does the same for compressed SSTables. Bootstrap can send whole SSTable which is much larger than receiving buffer, while repair sends only part of SSTable that is different. > Bootstrap outgoing streaming speeds are much slower than during repair > -- > > Key: CASSANDRA-9766 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9766 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Cassandra 2.1.2. more details in the pdf attached >Reporter: Alexei K >Assignee: Yuki Morishita >Priority: Minor > Fix For: 2.1.x > > Attachments: problem.pdf > > > I have a cluster in Amazon cloud , its described in detail in the attachment. > What I've noticed is that we during bootstrap we never go above 12MB/sec > transmission speeds and also those speeds flat line almost like we're hitting > some sort of a limit ( this remains true for other tests that I've ran) > however during the repair we see much higher,variable sending rates. I've > provided network charts in the attachment as well . Is there an explanation > for this? Is something wrong with my configuration, or is it a possible bug? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9766) Bootstrap outgoing streaming speeds are much slower than during repair
[ https://issues.apache.org/jira/browse/CASSANDRA-9766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939129#comment-14939129 ] Jonathan Ellis commented on CASSANDRA-9766: --- Is that actually a different path for bootstrap than repair, or do you think the reported behavior is inaccurate? > Bootstrap outgoing streaming speeds are much slower than during repair > -- > > Key: CASSANDRA-9766 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9766 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Cassandra 2.1.2. more details in the pdf attached >Reporter: Alexei K >Assignee: Yuki Morishita >Priority: Minor > Fix For: 2.1.x > > Attachments: problem.pdf > > > I have a cluster in Amazon cloud , its described in detail in the attachment. > What I've noticed is that we during bootstrap we never go above 12MB/sec > transmission speeds and also those speeds flat line almost like we're hitting > some sort of a limit ( this remains true for other tests that I've ran) > however during the repair we see much higher,variable sending rates. I've > provided network charts in the attachment as well . Is there an explanation > for this? Is something wrong with my configuration, or is it a possible bug? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10326) Performance is worse in 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939123#comment-14939123 ] Jonathan Ellis commented on CASSANDRA-10326: bq. I would recommend that - until we can assess the GC behavioural changes further, we revert CASSANDRA-7486 +1 > Performance is worse in 3.0 > --- > > Key: CASSANDRA-10326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10326 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Priority: Critical > Fix For: 3.0.x > > > Performance is generally turning out to be worse after 8099, despite a number > of unrelated performance enhancements being delivered. This isn't entirely > unexpected, given a great deal of time was spent optimising the old code, > however things appear worse than we had hoped. > My expectation was that workloads making extensive use of CQL constructs > would be faster post-8099, however the latest tests performed with very large > CQL rows, including use of collections, still exhibit performance below that > of 2.1 and 2.2. > Eventually, as the dataset size grows large enough and the locality of access > is just right, the reduction in size of our dataset will yield a window > during which some users will perform better due simply to improved page cache > hit rates. We seem to see this in some of the tests. However we should be at > least as fast (and really faster) off the bat. > The following are some large partition benchmark results, with as many as 40K > rows per partition, running LCS. There are a number of parameters we can > modify to see how behaviour changes and under what scenarios we might still > be faster, but the picture painted isn't brilliant, and is consistent, so we > should really try and figure out what's up before GA. > [trades-with-flags (collections), > blade11b|http://cstar.datastax.com/graph?stats=f0a17292-5a13-11e5-847a-42010af0688f&metric=op_rate&operation=1_user&smoothing=1&show_aggregates=true&xmin=0&xmax=4387.02&ymin=0&ymax=122951.4] > [trades-with-flags (collections), > blade11|http://cstar.datastax.com/graph?stats=e250-5a13-11e5-ae0d-42010af0688f&metric=op_rate&operation=1_user&smoothing=1&show_aggregates=true&xmin=0&xmax=4424.75&ymin=0&ymax=130158.6] > [trades (no collections), > blade11|http://cstar.datastax.com/graph?stats=9b7da48e-570c-11e5-90fe-42010af0688f&metric=op_rate&operation=1_user&smoothing=1&show_aggregates=true&xmin=0&xmax=2682.46&ymin=0&ymax=142547.9] > [~slebresne]: will you have time to look into this before GA? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10326) Performance is worse in 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939123#comment-14939123 ] Jonathan Ellis edited comment on CASSANDRA-10326 at 10/1/15 12:58 AM: -- bq. I would recommend that - until we can assess the GC behavioural changes further, we revert CASSANDRA-7486 +1. I think the consensus we came to on chat was to revert but leave the G1 settings commented out so a user with a large heap can easily enable it. was (Author: jbellis): bq. I would recommend that - until we can assess the GC behavioural changes further, we revert CASSANDRA-7486 +1 > Performance is worse in 3.0 > --- > > Key: CASSANDRA-10326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10326 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Priority: Critical > Fix For: 3.0.x > > > Performance is generally turning out to be worse after 8099, despite a number > of unrelated performance enhancements being delivered. This isn't entirely > unexpected, given a great deal of time was spent optimising the old code, > however things appear worse than we had hoped. > My expectation was that workloads making extensive use of CQL constructs > would be faster post-8099, however the latest tests performed with very large > CQL rows, including use of collections, still exhibit performance below that > of 2.1 and 2.2. > Eventually, as the dataset size grows large enough and the locality of access > is just right, the reduction in size of our dataset will yield a window > during which some users will perform better due simply to improved page cache > hit rates. We seem to see this in some of the tests. However we should be at > least as fast (and really faster) off the bat. > The following are some large partition benchmark results, with as many as 40K > rows per partition, running LCS. There are a number of parameters we can > modify to see how behaviour changes and under what scenarios we might still > be faster, but the picture painted isn't brilliant, and is consistent, so we > should really try and figure out what's up before GA. > [trades-with-flags (collections), > blade11b|http://cstar.datastax.com/graph?stats=f0a17292-5a13-11e5-847a-42010af0688f&metric=op_rate&operation=1_user&smoothing=1&show_aggregates=true&xmin=0&xmax=4387.02&ymin=0&ymax=122951.4] > [trades-with-flags (collections), > blade11|http://cstar.datastax.com/graph?stats=e250-5a13-11e5-ae0d-42010af0688f&metric=op_rate&operation=1_user&smoothing=1&show_aggregates=true&xmin=0&xmax=4424.75&ymin=0&ymax=130158.6] > [trades (no collections), > blade11|http://cstar.datastax.com/graph?stats=9b7da48e-570c-11e5-90fe-42010af0688f&metric=op_rate&operation=1_user&smoothing=1&show_aggregates=true&xmin=0&xmax=2682.46&ymin=0&ymax=142547.9] > [~slebresne]: will you have time to look into this before GA? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10382) nodetool info doesn't show the correct DC and RACK
[
https://issues.apache.org/jira/browse/CASSANDRA-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939113#comment-14939113
]
Stefania edited comment on CASSANDRA-10382 at 10/1/15 12:42 AM:
Latest change LGTM, [~JoshuaMcKenzie] could you commit it please?
was (Author: stefania):
Latest change LGTM.
> nodetool info doesn't show the correct DC and RACK
> --
>
> Key: CASSANDRA-10382
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10382
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 2.2.1
> GossipingPropertyFileSnitch
>Reporter: Ruggero Marchei
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 2.1.10
>
>
> When running *nodetool info* cassandra returns UNKNOWN_DC and UNKNOWN_RACK:
> {code}
> # nodetool info
> ID : b94f9ca0-f886-4111-a471-02f295573f37
> Gossip active : true
> Thrift active : true
> Native Transport active: true
> Load : 44.97 MB
> Generation No : 1442913138
> Uptime (seconds) : 5386
> Heap Memory (MB) : 429.07 / 3972.00
> Off Heap Memory (MB) : 0.08
> Data Center: UNKNOWN_DC
> Rack : UNKNOWN_RACK
> Exceptions : 1
> Key Cache : entries 642, size 58.16 KB, capacity 100 MB, 5580
> hits, 8320 requests, 0.671 recent hit rate, 14400 save period in seconds
> Row Cache : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0
> requests, NaN recent hit rate, 0 save period in seconds
> Counter Cache : entries 0, size 0 bytes, capacity 50 MB, 0 hits, 0
> requests, NaN recent hit rate, 7200 save period in seconds
> Token : (invoke with -T/--tokens to see all 256 tokens)
> {code}
> Correct DCs and RACKs are returned by *nodetool status* and *nodetool
> gossipinfo* commands:
> {code}
> # nodetool gossipinfo|grep -E 'RACK|DC'
> DC:POZ
> RACK:RACK30
> DC:POZ
> RACK:RACK30
> DC:SJC
> RACK:RACK68
> DC:POZ
> RACK:RACK30
> DC:SJC
> RACK:RACK62
> DC:SJC
> RACK:RACK62
> {code}
> {code}
> # nodetool status|grep Datacenter
> Datacenter: SJC
> Datacenter: POZ
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10382) nodetool info doesn't show the correct DC and RACK
[
https://issues.apache.org/jira/browse/CASSANDRA-10382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939113#comment-14939113
]
Stefania commented on CASSANDRA-10382:
--
Latest change LGTM.
> nodetool info doesn't show the correct DC and RACK
> --
>
> Key: CASSANDRA-10382
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10382
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 2.2.1
> GossipingPropertyFileSnitch
>Reporter: Ruggero Marchei
>Assignee: Carl Yeksigian
>Priority: Minor
> Fix For: 2.1.10
>
>
> When running *nodetool info* cassandra returns UNKNOWN_DC and UNKNOWN_RACK:
> {code}
> # nodetool info
> ID : b94f9ca0-f886-4111-a471-02f295573f37
> Gossip active : true
> Thrift active : true
> Native Transport active: true
> Load : 44.97 MB
> Generation No : 1442913138
> Uptime (seconds) : 5386
> Heap Memory (MB) : 429.07 / 3972.00
> Off Heap Memory (MB) : 0.08
> Data Center: UNKNOWN_DC
> Rack : UNKNOWN_RACK
> Exceptions : 1
> Key Cache : entries 642, size 58.16 KB, capacity 100 MB, 5580
> hits, 8320 requests, 0.671 recent hit rate, 14400 save period in seconds
> Row Cache : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0
> requests, NaN recent hit rate, 0 save period in seconds
> Counter Cache : entries 0, size 0 bytes, capacity 50 MB, 0 hits, 0
> requests, NaN recent hit rate, 7200 save period in seconds
> Token : (invoke with -T/--tokens to see all 256 tokens)
> {code}
> Correct DCs and RACKs are returned by *nodetool status* and *nodetool
> gossipinfo* commands:
> {code}
> # nodetool gossipinfo|grep -E 'RACK|DC'
> DC:POZ
> RACK:RACK30
> DC:POZ
> RACK:RACK30
> DC:SJC
> RACK:RACK68
> DC:POZ
> RACK:RACK30
> DC:SJC
> RACK:RACK62
> DC:SJC
> RACK:RACK62
> {code}
> {code}
> # nodetool status|grep Datacenter
> Datacenter: SJC
> Datacenter: POZ
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10289) Fix cqlshlib tests
[ https://issues.apache.org/jira/browse/CASSANDRA-10289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939112#comment-14939112 ] Stefania commented on CASSANDRA-10289: -- It looks excellent, +1, thanks! > Fix cqlshlib tests > -- > > Key: CASSANDRA-10289 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10289 > Project: Cassandra > Issue Type: Bug > Components: Tests >Reporter: Jim Witschey >Assignee: Jim Witschey > Labels: cqlsh > Fix For: 3.0.0 rc2 > > Attachments: trunk-10289.diff > > > The cqlsh tests in trunk haven't been running for a while: > http://cassci.datastax.com/view/All_Jobs/job/trunk_cqlshlib/423/testReport/ > This looks like the driver errors that happened because of CASSANDRA-6717. > Not sure why it's happening now; the driver installation looks normal to me > on those jobs. [~mshuler]? > There were also some changes to cqlsh itself that also broke the test > harness, but I believe those are fixed here: > https://github.com/mambocab/cassandra/tree/fix-cqlsh-tests > Once the tests are running successfully on CassCI, I'll test my patch and > mark as patch available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10052) Misleading down-node push notifications when rpc_address is shared
[
https://issues.apache.org/jira/browse/CASSANDRA-10052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939109#comment-14939109
]
Stefania commented on CASSANDRA-10052:
--
[~thobbs] and [~omichallat]: thank you so much for your work on this!
> Misleading down-node push notifications when rpc_address is shared
> --
>
> Key: CASSANDRA-10052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10052
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sharvanath Pathak
>Assignee: Stefania
> Labels: client-impacting
> Fix For: 3.0.0 rc2, 2.1.10, 2.2.2
>
>
> When a node goes down, the other nodes learn that through the gossip.
> And I do see the log from (Gossiper.java):
> {code}
> private void markDead(InetAddress addr, EndpointState localState)
>{
>if (logger.isTraceEnabled())
>logger.trace("marking as down {}", addr);
>localState.markDead();
>liveEndpoints.remove(addr);
>unreachableEndpoints.put(addr, System.nanoTime());
>logger.info("InetAddress {} is now DOWN", addr);
>for (IEndpointStateChangeSubscriber subscriber : subscribers)
>subscriber.onDead(addr, localState);
>if (logger.isTraceEnabled())
>logger.trace("Notified " + subscribers);
>}
> {code}
> Saying: "InetAddress 192.168.101.1 is now Down", in the Cassandra's system
> log.
> Now on all the other nodes the client side (java driver) says, " Cannot
> connect to any host, scheduling retry in 1000 milliseconds". They eventually
> do reconnect but some queries fail during this intermediate period.
> To me it seems like when the server pushes the nodeDown event, it call the
> getRpcAddress(endpoint), and thus sends localhost as the argument in the
> nodeDown event.
> As in org.apache.cassandra.transport.Server.java
> {code}
> public void onDown(InetAddress endpoint)
>{
>
> server.connectionTracker.send(Event.StatusChange.nodeDown(getRpcAddress(endpoint),
> server.socket.getPort()));
>}
> {code}
> the getRpcAddress returns localhost for any endpoint if the cassandra.yaml is
> using localhost as the configuration for rpc_address (which by the way is the
> default).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10421) Potential issue with LogTransaction as it only checks in a single directory for files
[
https://issues.apache.org/jira/browse/CASSANDRA-10421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939108#comment-14939108
]
Stefania commented on CASSANDRA-10421:
--
If we can have a transaction involving sstables located in different folders,
or the txn log can end up in a totally different folder as it seems to be the
case, then {{LogRecord.getExistingFiles()}} definitely needs to be fixed.
However, we then have a bigger problem in listing temporary files, in that
{{LogAwareFileLister}} assumes the txn log is located in the folder it is
listing so we might miss temporary files when listing. We would have to either
search for txn logs in multiple folders or pin them to a dedicated folder or
enhance {{LogAwareFileLister}} to list all the sstable possible folders.
[~benedict] WDYT?
> Potential issue with LogTransaction as it only checks in a single directory
> for files
> -
>
> Key: CASSANDRA-10421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10421
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Stefania
>
> When creating a new LogTransaction we try to create the new logfile in the
> same directory as the one we are writing to, but as we use
> {{[directories.getDirectoryForNewSSTables()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java#L125]}}
> this might end up in "any" of the configured data directories. If it does,
> we will not be able to clean up leftovers as we check for files in the same
> directory as the logfile was created:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogRecord.java#L163
> cc [~Stefania]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-10421) Potential issue with LogTransaction as it only checks in a single directory for files
[
https://issues.apache.org/jira/browse/CASSANDRA-10421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefania reassigned CASSANDRA-10421:
Assignee: Stefania
> Potential issue with LogTransaction as it only checks in a single directory
> for files
> -
>
> Key: CASSANDRA-10421
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10421
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Stefania
>
> When creating a new LogTransaction we try to create the new logfile in the
> same directory as the one we are writing to, but as we use
> {{[directories.getDirectoryForNewSSTables()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java#L125]}}
> this might end up in "any" of the configured data directories. If it does,
> we will not be able to clean up leftovers as we check for files in the same
> directory as the logfile was created:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogRecord.java#L163
> cc [~Stefania]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10341) Streaming does not guarantee cache invalidation
[ https://issues.apache.org/jira/browse/CASSANDRA-10341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939071#comment-14939071 ] Paulo Motta commented on CASSANDRA-10341: - bq. we can constuct ranges of partition keys and Iterate over keys after adding SSTables in row cache and check if key is in ranges, we can drop cache. nice, thanks! I guess key range invalidation is the way to go, since keeping a copy of keys in memory could be costly for large streams. > Streaming does not guarantee cache invalidation > --- > > Key: CASSANDRA-10341 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10341 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Paulo Motta > > Looking at the code, we attempt to invalidate the row cache for any rows we > receive via streaming, however we invalidate them immediately, before the new > data is available. So, if it is requested (which is likely if it is "hot") in > the interval, it will be re-cached and not invalidated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7276) Include keyspace and table names in logs where possible
[
https://issues.apache.org/jira/browse/CASSANDRA-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939065#comment-14939065
]
Paulo Motta commented on CASSANDRA-7276:
[~eitikimura] some comments:
I'm not able to compile:
{noformat}
[javac]
/home/paulo/workspace/cassandra/src/java/org/apache/cassandra/utils/LogUtils.java:23:
error: package org.apache.log4j does not exist
[javac] import org.apache.log4j.MDC;
[javac]
{noformat}
There were also some warnings when applying the patch:
{noformat}
cassandra-2.1.9-7276.txt:54: trailing whitespace.
cassandra-2.1.9-7276.txt:141: trailing whitespace.
cassandra-2.1.9-7276.txt:219: trailing whitespace.
cassandra-2.1.9-7276.txt:222: trailing whitespace.
for (ColumnFamily columnFamily : getColumnFamilies())
cassandra-2.1.9-7276.txt:228: trailing whitespace.
warning: squelched 17 whitespace errors
warning: 22 lines add whitespace errors.
{noformat}
Also, I just noticed there might be some inconsistencies with MDC and [managed
threads|http://logback.qos.ch/manual/mdc.html#managedThreads], so it would be
nice to verify that all mdc statements that you put will be run on the same
thread as the logging statements, as well as cleanup with MDC.clear or
MDC.remove after the operation is finished so we don't risk propagating garbage
to other threads (you may find more information about thread locality of MDC on
this
[post|http://blog.trifork.com/2013/06/06/adding-user-info-to-log-entries-in-a-multi-user-app-using-mapped-diagnostic-context/]).
> Include keyspace and table names in logs where possible
> ---
>
> Key: CASSANDRA-7276
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7276
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Nitzan Volman
>Priority: Minor
> Labels: bootcamp, lhf
> Fix For: 2.1.x
>
> Attachments: 2.1-CASSANDRA-7276-v1.txt,
> cassandra-2.1-7276-compaction.txt, cassandra-2.1-7276.txt,
> cassandra-2.1.9-7276.txt
>
>
> Most error messages and stacktraces give you no clue as to what keyspace or
> table was causing the problem. For example:
> {noformat}
> ERROR [MutationStage:61648] 2014-05-20 12:05:45,145 CassandraDaemon.java
> (line 198) Exception in thread Thread[MutationStage:61648,5,main]
> java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Unknown Source)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:63)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:72)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:98)
> at
> org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:35)
> at
> edu.stanford.ppl.concurrent.SnapTreeMap$1.compareTo(SnapTreeMap.java:538)
> at
> edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1108)
> at
> edu.stanford.ppl.concurrent.SnapTreeMap.updateUnderRoot(SnapTreeMap.java:1059)
> at edu.stanford.ppl.concurrent.SnapTreeMap.update(SnapTreeMap.java:1023)
> at
> edu.stanford.ppl.concurrent.SnapTreeMap.putIfAbsent(SnapTreeMap.java:985)
> at
> org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:328)
> at
> org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDelta(AtomicSortedColumns.java:200)
> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:226)
> at org.apache.cassandra.db.Memtable.put(Memtable.java:173)
> at
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:893)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:368)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:333)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:206)
> at
> org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:56)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {noformat}
> We should try to include info on the keyspace and column family in the error
> messages or logs whenever possible. This includes reads, writes,
> compactions, flushes, repairs, and probably more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10429) Flush schema tables after local schema change
[ https://issues.apache.org/jira/browse/CASSANDRA-10429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939059#comment-14939059 ] Tyler Hobbs commented on CASSANDRA-10429: - Patch for 3.0 (to keep the changes conservative) with pending CI runs: * [branch|https://github.com/thobbs/cassandra/tree/CASSANDRA-10429] * [testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10429-testall/] * [dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10429-dtest/] and trunk: * [branch|https://github.com/thobbs/cassandra/tree/CASSANDRA-10429-trunk] * [testall|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10429-trunk-testall/] * [dtest|http://cassci.datastax.com/view/Dev/view/thobbs/job/thobbs-CASSANDRA-10429-trunk-dtest/] The patch adds a system property specifically to disable flushing after local schema changes. This defaults to flushing, but the unit test config defaults to not flushing in order to keep unit test run times short. > Flush schema tables after local schema change > - > > Key: CASSANDRA-10429 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10429 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Tyler Hobbs >Assignee: Tyler Hobbs >Priority: Minor > Fix For: 3.0.0 rc2 > > > In CASSANDRA-7327, we disabled the normal flush of system schema tables after > "local" schema changes to improve the runtime of unit tests. However, there > are some cases where this flush is necessary for schema durability. For > example, if a custom secondary index needs to make schema changes as part of > it's creation, this is desirable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10429) Flush schema tables after local schema change
Tyler Hobbs created CASSANDRA-10429: --- Summary: Flush schema tables after local schema change Key: CASSANDRA-10429 URL: https://issues.apache.org/jira/browse/CASSANDRA-10429 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Tyler Hobbs Assignee: Tyler Hobbs Priority: Minor Fix For: 3.0.0 rc2 In CASSANDRA-7327, we disabled the normal flush of system schema tables after "local" schema changes to improve the runtime of unit tests. However, there are some cases where this flush is necessary for schema durability. For example, if a custom secondary index needs to make schema changes as part of it's creation, this is desirable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10341) Streaming does not guarantee cache invalidation
[ https://issues.apache.org/jira/browse/CASSANDRA-10341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939040#comment-14939040 ] Yuki Morishita commented on CASSANDRA-10341: We don't need to read SSTables again, since we already obtain partition key when reading from network. So we can construct set of partition keys to invalidate, or if those are too many, we can constuct ranges of partition keys. Iterate over keys after adding SSTables in row cache and check if key is in ranges, we can drop cache. > Streaming does not guarantee cache invalidation > --- > > Key: CASSANDRA-10341 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10341 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Paulo Motta > > Looking at the code, we attempt to invalidate the row cache for any rows we > receive via streaming, however we invalidate them immediately, before the new > data is available. So, if it is requested (which is likely if it is "hot") in > the interval, it will be re-cached and not invalidated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10341) Streaming does not guarantee cache invalidation
[ https://issues.apache.org/jira/browse/CASSANDRA-10341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938982#comment-14938982 ] Paulo Motta commented on CASSANDRA-10341: - I thought of 3 approaches to solve this problem: 1. After all sstables of a given table are received and added to CFS, iterate over them and invalidate the keys. Upside: ease of implementation. Drawback: for large streams, this would mean reading all data from disk again at a later point in time, which could add some overhead, like affecting the page cache. 2. Instead of adding sstables to CFS after all sstables of a given table are transferred, add each individual sstable to CFS when it is received, and iterate over them to invalidate keys. Upside: better than previous approach, since for moderate-sized sstables would probably still be cached soon after it is received. Drawback: Failed stream sessions would add data to the live system, even though they did not complete successfully. 3. Implement a dependency invalidation mechanism on the caching service, so we would pre-invalidate keys as they arrive, and complete the invalidation after the stream session is complete (or drop the invalidation). Upside: we would not need to re-read data to invalidate the keys. Drawback: add complexity to caching service. [~benedict] [~yukim] any thoughts on what would be the preferable approach? > Streaming does not guarantee cache invalidation > --- > > Key: CASSANDRA-10341 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10341 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Paulo Motta > > Looking at the code, we attempt to invalidate the row cache for any rows we > receive via streaming, however we invalidate them immediately, before the new > data is available. So, if it is requested (which is likely if it is "hot") in > the interval, it will be re-cached and not invalidated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10428) select with exact date is not working
[
https://issues.apache.org/jira/browse/CASSANDRA-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938956#comment-14938956
]
Tyler Hobbs commented on CASSANDRA-10428:
-
cqlsh's default timestamp display format omits milliseconds and microseconds,
so you won't see those in your result rows. You can change this through the
cqlsh config file like so:
{noformat}
[ui]
time_format = %Y-%m-%d %H:%M:%S.%f%z
{noformat}
That should allow you to see the full timestamp and fetch it exactly. Let me
know if this fixes the problem for you.
On a related note, perhaps we should change the default to include this? WDYT
[~Stefania]?
> select with exact date is not working
> -
>
> Key: CASSANDRA-10428
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10428
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: OSX 10.10.2
>Reporter: Chandran Anjur Narasimhan
> Labels: cqlsh
> Fix For: 2.1.x
>
>
> Query with >= timestamp works. But the exact timestamp value is not working.
> {noformat}
> NCHAN-M-D0LZ:bin nchan$ ./cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at :.
> [cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh>
> {noformat}
> {panel:title=Schema|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> desc COLUMNFAMILY ez_task_result ;
> CREATE TABLE ccc.ez_task_result (
> submissionid text,
> ezid text,
> name text,
> time timestamp,
> analyzed_index_root text,
> ...
> ...
> PRIMARY KEY (submissionid, ezid, name, time)
> {panel}
> {panel:title=Working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> select submissionid, ezid, name, time, state, status,
> translated_criteria_status from ez_task_result where
> submissionid='760dd154670811e58c04005056bb6ff0' and
> ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
> time>='2015-09-29 20:54:23-0700';
> submissionid | ezid | name
> | time | state | status |
> translated_criteria_status
> --+--+--+--+---+-+
> 760dd154670811e58c04005056bb6ff0 | 760dd6de670811e594fc005056bb6ff0 |
> run-sanities | 2015-09-29 20:54:23-0700 | EXECUTING | IN_PROGRESS |
> run-sanities started
> (1 rows)
> cqlsh:ccc>
> {panel}
> {panel:title=Not
> working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
> cqlsh:ccc> select submissionid, ezid, name, time, state, status,
> translated_criteria_status from ez_task_result where
> submissionid='760dd154670811e58c04005056bb6ff0' and
> ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
> time='2015-09-29 20:54:23-0700';
> submissionid | ezid | name | time | analyzed_index_root | analyzed_log_path
> | clientid | end_time | jenkins_path | log_file_path | path_available |
> path_to_task | required_for_overall_status | start_time | state | status |
> translated_criteria_status | type
> --+--+--+--+-+---+--+--+--+---++--+-++---+++--
> (0 rows)
> cqlsh:ccc>
> {panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10428) select with exact date is not working
[
https://issues.apache.org/jira/browse/CASSANDRA-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chandran Anjur Narasimhan updated CASSANDRA-10428:
--
Description:
Query with >= timestamp works. But the exact timestamp value is not working.
{noformat}
NCHAN-M-D0LZ:bin nchan$ ./cqlsh
Connected to CCC Multi-Region Cassandra Cluster at :.
[cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
cqlsh>
{noformat}
{panel:title=Schema|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
cqlsh:ccc> desc COLUMNFAMILY ez_task_result ;
CREATE TABLE ccc.ez_task_result (
submissionid text,
ezid text,
name text,
time timestamp,
analyzed_index_root text,
...
...
PRIMARY KEY (submissionid, ezid, name, time)
{panel}
{panel:title=Working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
cqlsh:ccc> select submissionid, ezid, name, time, state, status,
translated_criteria_status from ez_task_result where
submissionid='760dd154670811e58c04005056bb6ff0' and
ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
time>='2015-09-29 20:54:23-0700';
submissionid | ezid | name
| time | state | status |
translated_criteria_status
--+--+--+--+---+-+
760dd154670811e58c04005056bb6ff0 | 760dd6de670811e594fc005056bb6ff0 |
run-sanities | 2015-09-29 20:54:23-0700 | EXECUTING | IN_PROGRESS |
run-sanities started
(1 rows)
cqlsh:ccc>
{panel}
{panel:title=Not
working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
cqlsh:ccc> select submissionid, ezid, name, time, state, status,
translated_criteria_status from ez_task_result where
submissionid='760dd154670811e58c04005056bb6ff0' and
ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
time='2015-09-29 20:54:23-0700';
submissionid | ezid | name | time | analyzed_index_root | analyzed_log_path |
clientid | end_time | jenkins_path | log_file_path | path_available |
path_to_task | required_for_overall_status | start_time | state | status |
translated_criteria_status | type
--+--+--+--+-+---+--+--+--+---++--+-++---+++--
(0 rows)
cqlsh:ccc>
{panel}
was:
Query with >= timestamp works. But the exact timestamp value is not working.
{quote}
NCHAN-M-D0LZ:bin nchan$ ./cqlsh
Connected to CCC Multi-Region Cassandra Cluster at :.
[cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
cqlsh>
{quote}
:
{panel:title=Working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
cqlsh:ccc> select submissionid, ezid, name, time, state, status,
translated_criteria_status from ez_task_result where
submissionid='760dd154670811e58c04005056bb6ff0' and
ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
time>='2015-09-29 20:54:23-0700';
submissionid | ezid | name
| time | state | status |
translated_criteria_status
--+--+--+--+---+-+
760dd154670811e58c04005056bb6ff0 | 760dd6de670811e594fc005056bb6ff0 |
run-sanities | 2015-09-29 20:54:23-0700 | EXECUTING | IN_PROGRESS |
run-sanities started
(1 rows)
cqlsh:ccc>
{panel}
{panel:title=Not
working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
cqlsh:ccc> select submissionid, ezid, name, time, state, status,
translated_criteria_status from ez_task_result where
submissionid='760dd154670811e58c04005056bb6ff0' and
ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
time='2015-09-29 20:54:23-0700';
submissionid | ezid | name | time | analyzed_index_root | analyzed_log_path |
clientid | end_time | jenkins_path | log_file_path | path_available |
path_to_task | required_for_overall_status | start_time | state | status |
translated_criteria_status | type
--+--+--+--+-+---+--+--+--+---++--+-++---+++--
(0 rows)
cqlsh:ccc>
{panel}
> select with exact date is not working
> -
>
> Key: CASSANDRA-10428
> URL: https://issues
[jira] [Created] (CASSANDRA-10428) select with exact date is not working
Chandran Anjur Narasimhan created CASSANDRA-10428:
-
Summary: select with exact date is not working
Key: CASSANDRA-10428
URL: https://issues.apache.org/jira/browse/CASSANDRA-10428
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: OSX 10.10.2
Reporter: Chandran Anjur Narasimhan
Fix For: 2.1.x
Query with >= timestamp works. But the exact timestamp value is not working.
{quote}
NCHAN-M-D0LZ:bin nchan$ ./cqlsh
Connected to CCC Multi-Region Cassandra Cluster at :.
[cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
Use HELP for help.
cqlsh>
{quote}
:
{panel:title=Working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
cqlsh:ccc> select submissionid, ezid, name, time, state, status,
translated_criteria_status from ez_task_result where
submissionid='760dd154670811e58c04005056bb6ff0' and
ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
time>='2015-09-29 20:54:23-0700';
submissionid | ezid | name
| time | state | status |
translated_criteria_status
--+--+--+--+---+-+
760dd154670811e58c04005056bb6ff0 | 760dd6de670811e594fc005056bb6ff0 |
run-sanities | 2015-09-29 20:54:23-0700 | EXECUTING | IN_PROGRESS |
run-sanities started
(1 rows)
cqlsh:ccc>
{panel}
{panel:title=Not
working|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1|bgColor=#CE}
cqlsh:ccc> select submissionid, ezid, name, time, state, status,
translated_criteria_status from ez_task_result where
submissionid='760dd154670811e58c04005056bb6ff0' and
ezid='760dd6de670811e594fc005056bb6ff0' and name='run-sanities' and
time='2015-09-29 20:54:23-0700';
submissionid | ezid | name | time | analyzed_index_root | analyzed_log_path |
clientid | end_time | jenkins_path | log_file_path | path_available |
path_to_task | required_for_overall_status | start_time | state | status |
translated_criteria_status | type
--+--+--+--+-+---+--+--+--+---++--+-++---+++--
(0 rows)
cqlsh:ccc>
{panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9774) fix sstableverify dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938875#comment-14938875
]
Jim Witschey commented on CASSANDRA-9774:
-
Thanks for sorting this out, Sylvain. Style-wise, your changes to the dtests
look good; you have a +1 from me after review.
> fix sstableverify dtest
> ---
>
> Key: CASSANDRA-9774
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9774
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.0.0 rc2
>
>
> One of our dtests for {{sstableverify}}
> ({{offline_tools_test.py:TestOfflineTools.sstableverify_test}}) is failing
> hard on trunk ([cassci
> history|http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/offline_tools_test/TestOfflineTools/sstableverify_test/history/])
> The way the test works is by deleting an SSTable, then running
> {{sstableverify}} on its table. In earlier versions, it successfully detects
> this problem and outputs that it "was not released before the reference was
> garbage collected". The test no longer finds this string in the output;
> looking through the output of the test, it doesn't look like it reports any
> problems at all.
> EDIT: After digging into the C* source a bit, I may have misattributed the
> problem to {{sstableverify}}; this could be a more general memory management
> problem, as the error text expected in the dtest is emitted by part of the
> {{Ref}} implementation:
> https://github.com/apache/cassandra/blob/075ff5000ced24b42f3b540815cae471bee4049d/src/java/org/apache/cassandra/utils/concurrent/Ref.java#L187
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10416) Windows dtest 3.0: clustering_order_test dtest fails
[ https://issues.apache.org/jira/browse/CASSANDRA-10416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938874#comment-14938874 ] Adam Holmberg commented on CASSANDRA-10416: --- Glad to hear it's sorted out. > Windows dtest 3.0: clustering_order_test dtest fails > > > Key: CASSANDRA-10416 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10416 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jim Witschey >Assignee: Yuki Morishita > Fix For: 3.0.0 rc2 > > > [schema_metadata_test.TestSchemaMetadata.clustering_order_test|http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/72/testReport/schema_metadata_test/TestSchemaMetadata/clustering_order_test/] > fails on cassandra-3.0 under Windows -- the test expects a table's metadata > object, represented as a CQL query, to indicate its ordering. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-10416) Windows dtest 3.0: clustering_order_test dtest fails
[ https://issues.apache.org/jira/browse/CASSANDRA-10416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Witschey resolved CASSANDRA-10416. -- Resolution: Fixed > Windows dtest 3.0: clustering_order_test dtest fails > > > Key: CASSANDRA-10416 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10416 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jim Witschey >Assignee: Yuki Morishita > Fix For: 3.0.0 rc2 > > > [schema_metadata_test.TestSchemaMetadata.clustering_order_test|http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/72/testReport/schema_metadata_test/TestSchemaMetadata/clustering_order_test/] > fails on cassandra-3.0 under Windows -- the test expects a table's metadata > object, represented as a CQL query, to indicate its ordering. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10417) Windows dtest 3.0: creating_and_dropping_table_with_2ary_indexes_test
[ https://issues.apache.org/jira/browse/CASSANDRA-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938863#comment-14938863 ] Jim Witschey commented on CASSANDRA-10417: -- That was the problem, thanks: https://issues.apache.org/jira/browse/CASSANDRA-10416?focusedCommentId=14938859&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14938859 > Windows dtest 3.0: creating_and_dropping_table_with_2ary_indexes_test > - > > Key: CASSANDRA-10417 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10417 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jim Witschey >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > [schema_metadata_test.TestSchemaMetadata.creating_and_dropping_table_with_2ary_indexes_test|http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/72/testReport/schema_metadata_test/TestSchemaMetadata/creating_and_dropping_table_with_2ary_indexes_test/] > fails on cassandra-3.0 under Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10416) Windows dtest 3.0: clustering_order_test dtest fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938859#comment-14938859
]
Jim Witschey commented on CASSANDRA-10416:
--
Sorry to bring you in on this prematurely, Adam. We were pulling but not
reinstalling:
{code}
cd C:/Users/jenkins/python-driver
git pull
{code}
I changed it to this on a scratch job on CassCI:
{code}
cd C:/Users/jenkins/python-driver
git pull
echo y | pip uninstall cassandra-driver
pip install -e .
{code}
(The existing install is done with {{-e}} so we won't use {{--upgrade}}.)
My scratch job ran clean and addressed the issues here and on CASSANDRA-10417:
http://cassci.datastax.com/view/All_Jobs/job/mambocab-scratch_cassandra-3.0_dtest_win32/lastCompletedBuild/console
Since that succeeded, I've switched over the other windows dtest jobs to
uninstall and reinstall.
> Windows dtest 3.0: clustering_order_test dtest fails
>
>
> Key: CASSANDRA-10416
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10416
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
> Fix For: 3.0.0 rc2
>
>
> [schema_metadata_test.TestSchemaMetadata.clustering_order_test|http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/72/testReport/schema_metadata_test/TestSchemaMetadata/clustering_order_test/]
> fails on cassandra-3.0 under Windows -- the test expects a table's metadata
> object, represented as a CQL query, to indicate its ordering.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10360) UnsupportedOperationException when compacting system.size_estimates after 2.1 -> 2.2 -> 3.0 upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-10360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Hust updated CASSANDRA-10360:
Description:
When upgrading from 2.1 -> 2.2 -> 3.0 the system.size_estimates table will get
stuck in a compaction loop throwing:
{code}
java.lang.UnsupportedOperationException
at
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.readNext(UnfilteredDeserializer.java:382)
at
org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.computeNext(SSTableSimpleIterator.java:147)
at
org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.computeNext(SSTableSimpleIterator.java:96)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:100)
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:30)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
at
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at
org.apache.cassandra.utils.MergeIterator$TrivialOneToOne.computeNext(MergeIterator.java:460)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at
org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:503)
at
org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:363)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at
org.apache.cassandra.db.rows.WrappingUnfilteredRowIterator.hasNext(WrappingUnfilteredRowIterator.java:80)
at
org.apache.cassandra.db.rows.AlteringUnfilteredRowIterator.hasNext(AlteringUnfilteredRowIterator.java:72)
at
org.apache.cassandra.db.rows.UnfilteredRowIterator.isEmpty(UnfilteredRowIterator.java:100)
at
org.apache.cassandra.db.partitions.PurgingPartitionIterator.hasNext(PurgingPartitionIterator.java:72)
at
org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:223)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:177)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
at
org.apache.cassandra.db.compaction.CompactionManager$8.runMayThrow(CompactionManager.java:539)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
It will only occur when upgrading thru 2.2. Going from 2.1 -> 3.0 will not
surface the issue. It can be reproduced with the following (note -- changing
jdks will need to be altered if you aren't on OSX)
{code}
#!/bin/sh
echo "using java7 for cassandra-2.1 instance"
export JAVA_HOME=$(/usr/libexec/java_home -v 1.7)
ccm stop
ccm remove upgrade_nodes
ccm create -n 1 -v git:cassandra-2.1 upgrade_nodes
ccm start
ccm node1 stress write n=500K -rate threads=4 -mode native cql3
sleep 10
for cver in 3.0
do
echo "Draining all nodes"
ccm node1 nodetool drain
ccm stop
echo "switching to java 8"
export JAVA_HOME=$(/usr/libexec/java_home -v 1.8)
echo "Upgrading to git:cassandra-$cver"
ccm setdir -v git:cassandra-$cver
ccm start
echo "Sleeping to all version migrations"
sleep 30
echo "Upgrading sstables"
ccm node1 nodetool upgradesstables
ccm node1 nodetool upgradesstables system
ccm node1 nodetool compact system
ccm node1 stress write n=500K -rate threads=4 -mode native cql3
sleep 10
done
echo "***"
echo "instead of creating churn to cause compaction naturally forcing
compaction of system keyspace"
echo "***"
ccm node1 nodetool compact system
ccm stop
{code}
The example uses {{nodetool compact system}} but it will also occur with
{{nodetool upgradesstables system}}. I'm puzzled by that since the script runs
{{upgradesstables}} on each iteration. Is the system keyspace not effected by
[jira] [Comment Edited] (CASSANDRA-10360) UnsupportedOperationException when compacting system.size_estimates after 2.1 -> 2.2 -> 3.0 upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-10360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937461#comment-14937461
]
Andrew Hust edited comment on CASSANDRA-10360 at 9/30/15 8:21 PM:
--
I have now reproduced this issue when upgrading from 2.1 to 3.0 when doing a
rolling upgrade with cstar_perf. Attached site_estimates sstables and system
log from one of the nodes. I'll work on getting a isolated script to reproduce.
UPDATE: The above shell script has been updated to show failure from 2.1 ->
3.0. It just needed more activity on the stress operation to trigger the
issue. Change line {{for cver in 3.0}} to {{for cver in 2.2 3.0}} for the
original upgrade path.
was (Author: nutbunnies):
I have now reproduced this issue when upgrading from 2.1 to 3.0 when doing a
rolling upgrade with cstar_perf. Attached site_estimates sstables and system
log from one of the nodes. I'll work on getting a isolated script to reproduce.
> UnsupportedOperationException when compacting system.size_estimates after 2.1
> -> 2.2 -> 3.0 upgrade
> ---
>
> Key: CASSANDRA-10360
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10360
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andrew Hust
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.0.0 rc2
>
> Attachments: size_estimates.tar.bz2, system.log.bz2
>
>
> When upgrading from 2.1 -> 2.2 -> 3.0 the system.size_estimates table will
> get stuck in a compaction loop throwing:
> {code}
> java.lang.UnsupportedOperationException
> at
> org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.readNext(UnfilteredDeserializer.java:382)
> at
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.computeNext(SSTableSimpleIterator.java:147)
> at
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.computeNext(SSTableSimpleIterator.java:96)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:100)
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:30)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.utils.MergeIterator$TrivialOneToOne.computeNext(MergeIterator.java:460)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:503)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:363)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.rows.WrappingUnfilteredRowIterator.hasNext(WrappingUnfilteredRowIterator.java:80)
> at
> org.apache.cassandra.db.rows.AlteringUnfilteredRowIterator.hasNext(AlteringUnfilteredRowIterator.java:72)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterator.isEmpty(UnfilteredRowIterator.java:100)
> at
> org.apache.cassandra.db.partitions.PurgingPartitionIterator.hasNext(PurgingPartitionIterator.java:72)
> at
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:223)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:177)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
> at
> org.apache.cassandra.db.compaction.CompactionManager$8.runMayThrow(CompactionManager.java:539)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE
[jira] [Updated] (CASSANDRA-10360) UnsupportedOperationException when compacting system.size_estimates after 2.1 -> 2.2 -> 3.0 upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-10360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Hust updated CASSANDRA-10360:
Description:
When upgrading from 2.1 -> 2.2 -> 3.0 the system.size_estimates table will get
stuck in a compaction loop throwing:
{code}
java.lang.UnsupportedOperationException
at
org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.readNext(UnfilteredDeserializer.java:382)
at
org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.computeNext(SSTableSimpleIterator.java:147)
at
org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.computeNext(SSTableSimpleIterator.java:96)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:100)
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:30)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
at
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at
org.apache.cassandra.utils.MergeIterator$TrivialOneToOne.computeNext(MergeIterator.java:460)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at
org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:503)
at
org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:363)
at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
at
org.apache.cassandra.db.rows.WrappingUnfilteredRowIterator.hasNext(WrappingUnfilteredRowIterator.java:80)
at
org.apache.cassandra.db.rows.AlteringUnfilteredRowIterator.hasNext(AlteringUnfilteredRowIterator.java:72)
at
org.apache.cassandra.db.rows.UnfilteredRowIterator.isEmpty(UnfilteredRowIterator.java:100)
at
org.apache.cassandra.db.partitions.PurgingPartitionIterator.hasNext(PurgingPartitionIterator.java:72)
at
org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:223)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:177)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:78)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
at
org.apache.cassandra.db.compaction.CompactionManager$8.runMayThrow(CompactionManager.java:539)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
It will only occur when upgrading thru 2.2. Going from 2.1 -> 3.0 will not
surface the issue. It can be reproduced with the following (note -- changing
jdks will need to be altered if you aren't on OSX)
{code}
#!/bin/sh
echo "using java7 for cassandra-2.1 instance"
export JAVA_HOME=$(/usr/libexec/java_home -v 1.7)
ccm stop
ccm remove upgrade_nodes
ccm create -n 1 -v git:cassandra-2.1 upgrade_nodes
ccm start
ccm node1 stress write n=500K -rate threads=4 -mode native cql3
sleep 10
for cver in 2.2 3.0
do
echo "Draining all nodes"
ccm node1 nodetool drain
ccm stop
echo "switching to java 8"
export JAVA_HOME=$(/usr/libexec/java_home -v 1.8)
echo "Upgrading to git:cassandra-$cver"
ccm setdir -v git:cassandra-$cver
ccm start
echo "Sleeping to all version migrations"
sleep 30
echo "Upgrading sstables"
ccm node1 nodetool upgradesstables
ccm node1 nodetool upgradesstables system
ccm node1 nodetool compact system
ccm node1 stress write n=500K -rate threads=4 -mode native cql3
sleep 10
done
echo "***"
echo "instead of creating churn to cause compaction naturally forcing
compaction of system keyspace"
echo "***"
ccm node1 nodetool compact system
ccm stop
{code}
The example uses {{nodetool compact system}} but it will also occur with
{{nodetool upgradesstables system}}. I'm puzzled by that since the script runs
{{upgradesstables}} on each iteration. Is the system keyspace not effected
[jira] [Commented] (CASSANDRA-10323) Add more MaterializedView metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938771#comment-14938771
]
Carl Yeksigian commented on CASSANDRA-10323:
Overall +1. I pushed [a
branch|https://github.com/carlyeks/cassandra/tree/review/10323/3.0] with just a
few nits; if you could take a look at them [~cnlwsu].
* Removed the overload of {{mutateMV}} since we haven't hit a stable release
with MV in yet.
* Don't initialize the {{viewLockAcquire}} and {{viewRead}} metrics if this is
a view. I was expecting those values to be updated when I was looking at the
metrics; probably makes sense to just not have them at all for views instead of
having the unused metrics.
* Formatting miscellany
> Add more MaterializedView metrics
> -
>
> Key: CASSANDRA-10323
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10323
> Project: Cassandra
> Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: Chris Lohfink
> Fix For: 3.0.0 rc2
>
> Attachments: trunk-10323-v2.txt, trunk-10323.txt
>
>
> We need to add more metrics to help understand where time is spent in
> materialized view writes. We currently track the ratio of async base -> view
> mutations that fail.
> We should also add
> * The amount of time spent waiting for the partition lock (contention)
> * The amount of time spent reading data
> Any others?
> [~carlyeks] [~jkni]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-10427) compactionstats 'completed' field not updating
[
https://issues.apache.org/jira/browse/CASSANDRA-10427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson reassigned CASSANDRA-10427:
---
Assignee: Marcus Eriksson
> compactionstats 'completed' field not updating
> --
>
> Key: CASSANDRA-10427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10427
> Project: Cassandra
> Issue Type: Bug
>Reporter: T Jake Luciani
>Assignee: Marcus Eriksson
>Priority: Critical
> Fix For: 3.0.0 rc2
>
>
> nodetool compactionstats is no longer showing any change to completed field
> or progress.
> Can be reproduced with:
> * Start cassandra
> * Run stress write
> * run nodetool flush followed by nodetool compactionstats
> {code}
> ➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool flush
> ➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
> pending tasks: 1
> id compaction typekeyspace
> table completed totalunit progress
>75609a20-67a8-11e5-8003-c909ff7fd15eCompaction keyspace1
> standard1 0 350132121 bytes 0.00%
> Active compaction remaining time : 0h00m20s
> ➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
> pending tasks: 1
> id compaction typekeyspace
> table completed totalunit progress
>75609a20-67a8-11e5-8003-c909ff7fd15eCompaction keyspace1
> standard1 0 350132121 bytes 0.00%
> Active compaction remaining time : 0h00m20s
> ➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
> pending tasks: 1
> id compaction typekeyspace
> table completed totalunit progress
>75609a20-67a8-11e5-8003-c909ff7fd15eCompaction keyspace1
> standard1 0 350132121 bytes 0.00%
> Active compaction remaining time : 0h00m20s
> ➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
> pending tasks: 1
> id compaction typekeyspace
> table completed totalunit progress
>75609a20-67a8-11e5-8003-c909ff7fd15eCompaction keyspace1
> standard1 0 350132121 bytes 0.00%
> Active compaction remaining time : 0h00m20s
> ➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
> pending tasks: 0
> ➜ cassandra git:(cassandra-3.0) ✗
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938731#comment-14938731 ] Jonathan Shook edited comment on CASSANDRA-10403 at 9/30/15 7:45 PM: - To simplify, implementing CASSANDRA-10425 is effectively the same as reverting for the systems that we have commonly tested for, while allowing a likely better starting point for those that we have field experience with G1. was (Author: jshook): To simplify, implementing CASSANDRA-10425 is effectively the same as reverting for the system that we have tested for, while allowing a likely better starting point for those that we have field experience with G1. > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Start metrics reporter before bootstrap
Repository: cassandra
Updated Branches:
refs/heads/trunk 5d3855990 -> bd46463fb
Start metrics reporter before bootstrap
Patch by Tommy Stendahl; reviewed by jmckenzie for CASSANDRA-9594
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bd46463f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bd46463f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bd46463f
Branch: refs/heads/trunk
Commit: bd46463fbb7d6b0998c837450ce61df13eda041d
Parents: 5d38559
Author: Tommy Stendahl
Authored: Wed Sep 30 15:43:43 2015 -0400
Committer: Joshua McKenzie
Committed: Wed Sep 30 15:43:43 2015 -0400
--
.../cassandra/service/CassandraDaemon.java | 63 +++-
1 file changed, 35 insertions(+), 28 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/bd46463f/src/java/org/apache/cassandra/service/CassandraDaemon.java
--
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index f9ee9e8..41510ea 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -321,20 +321,6 @@ public class CassandraDaemon
SystemKeyspace.finishStartup();
-// start server internals
-StorageService.instance.registerDaemon(this);
-try
-{
-StorageService.instance.initServer();
-}
-catch (ConfigurationException e)
-{
-System.err.println(e.getMessage() + "\nFatal configuration error;
unable to start server. See log for stacktrace.");
-exitOrFail(1, "Fatal configuration error", e);
-}
-
-Mx4jTool.maybeLoad();
-
// Metrics
String metricsReporterConfigFile =
System.getProperty("cassandra.metricsReporterConfigFile");
if (metricsReporterConfigFile != null)
@@ -351,6 +337,20 @@ public class CassandraDaemon
}
}
+// start server internals
+StorageService.instance.registerDaemon(this);
+try
+{
+StorageService.instance.initServer();
+}
+catch (ConfigurationException e)
+{
+System.err.println(e.getMessage() + "\nFatal configuration error;
unable to start server. See log for stacktrace.");
+exitOrFail(1, "Fatal configuration error", e);
+}
+
+Mx4jTool.maybeLoad();
+
if
(!FBUtilities.getBroadcastAddress().equals(InetAddress.getLoopbackAddress()))
waitForGossipToSettle();
@@ -533,9 +533,12 @@ public class CassandraDaemon
//Allow the server to start even if the bean can't be
registered
}
-try {
+try
+{
DatabaseDescriptor.forceStaticInitialization();
-} catch (ExceptionInInitializerError e) {
+}
+catch (ExceptionInInitializerError e)
+{
throw e.getCause();
}
@@ -608,7 +611,8 @@ public class CassandraDaemon
stop();
destroy();
// completely shut down cassandra
-if(!runManaged) {
+if(!runManaged)
+{
System.exit(0);
}
}
@@ -668,21 +672,24 @@ public class CassandraDaemon
instance.activate();
}
-private void exitOrFail(int code, String message) {
+private void exitOrFail(int code, String message)
+{
exitOrFail(code, message, null);
}
-private void exitOrFail(int code, String message, Throwable cause) {
-if(runManaged) {
-RuntimeException t = cause!=null ? new
RuntimeException(message, cause) : new RuntimeException(message);
-throw t;
-}
-else {
-logger.error(message, cause);
-System.exit(code);
-}
-
+private void exitOrFail(int code, String message, Throwable cause)
+{
+if (runManaged)
+{
+RuntimeException t = cause!=null ? new RuntimeException(message,
cause) : new RuntimeException(message);
+throw t;
}
+else
+{
+logger.error(message, cause);
+System.exit(code);
+}
+}
static class NativeAccess implements NativeAccessMBean
{
[jira] [Commented] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938731#comment-14938731 ] Jonathan Shook commented on CASSANDRA-10403: To simplify, implementing CASSANDRA-10425 is effectively the same as reverting for the system that we have tested for, while allowing a likely better starting point for those that we have field experience with G1. > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10427) compactionstats 'completed' field not updating
[
https://issues.apache.org/jira/browse/CASSANDRA-10427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938343#comment-14938343
]
T Jake Luciani commented on CASSANDRA-10427:
Note the compactions do complete in the background but there is no user
feedback on progress
> compactionstats 'completed' field not updating
> --
>
> Key: CASSANDRA-10427
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10427
> Project: Cassandra
> Issue Type: Bug
>Reporter: T Jake Luciani
>Priority: Critical
> Fix For: 3.0.0 rc2
>
>
> nodetool compactionstats is no longer showing any change to completed field
> or progress.
> Can be reproduced with:
> * Start cassandra
> * Run stress write
> * run nodetool flush followed by nodetool compactionstats
> {code}
> ➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool flush
> ➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
> pending tasks: 1
> id compaction typekeyspace
> table completed totalunit progress
>75609a20-67a8-11e5-8003-c909ff7fd15eCompaction keyspace1
> standard1 0 350132121 bytes 0.00%
> Active compaction remaining time : 0h00m20s
> ➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
> pending tasks: 1
> id compaction typekeyspace
> table completed totalunit progress
>75609a20-67a8-11e5-8003-c909ff7fd15eCompaction keyspace1
> standard1 0 350132121 bytes 0.00%
> Active compaction remaining time : 0h00m20s
> ➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
> pending tasks: 1
> id compaction typekeyspace
> table completed totalunit progress
>75609a20-67a8-11e5-8003-c909ff7fd15eCompaction keyspace1
> standard1 0 350132121 bytes 0.00%
> Active compaction remaining time : 0h00m20s
> ➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
> pending tasks: 1
> id compaction typekeyspace
> table completed totalunit progress
>75609a20-67a8-11e5-8003-c909ff7fd15eCompaction keyspace1
> standard1 0 350132121 bytes 0.00%
> Active compaction remaining time : 0h00m20s
> ➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
> pending tasks: 0
> ➜ cassandra git:(cassandra-3.0) ✗
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938336#comment-14938336 ] Jonathan Shook edited comment on CASSANDRA-10403 at 9/30/15 7:28 PM: - [~JoshuaMcKenzie] I understand and appreciate the need to control scoping effort for 3.0 planning. bq. Shouldn't the read/write workload distribution also play into that? Yes, but there is a mostly orthogonal effect to the nuances of the workload mix which has to do with the vertical scalability of GC when the system is more fully utilized. This is visible along the sizing spectrum. Run the same workload and try to scale the heap proportionally over the memory (1/4 or whatever) and you will likely see CMS suffer no matter what. This is slightly conjectural, but easily verifiable with some effort. bq. the idea of having a default that's optimal for everyone is unrealistic I think we are converging on a common perspective on this. [~slebresne] bq. 3.2 will come only 2 months after 3.0 My preference would be to have the CASSANDRA-10425 out of the gate, but this still would require some testing effort for safety. The reason being that 3.0 represents a reframing of performance expectations, and after that, any changes to default, even for larger memory systems constitute a bigger chance of surprise. Do we have a chance to learn about sizing from surveys, etc before the runway ends for 3.0? If we could get something like CASSANDRA-10425 in place, it would cover both bases. was (Author: jshook): [~JoshuaMcKenzie] I understand and appreciate the need to control scoping effort for 3.0 planning. bq. Shouldn't the read/write workload distribution also play into that? Yes, but there is a mostly orthogonal effect to the nuances of the workload mix which has to do with the vertical scalability of GC when the system. This is visible along the sizing spectrum. Run the same workload and try to scale the heap proportionally over the memory (1/4 or whatever) and you will likely see CMS suffer no matter what. This is slightly conjectural, but easily verifiable with some effort. bq. the idea of having a default that's optimal for everyone is unrealistic I think we are converging on a common perspective on this. [~slebresne] bq. 3.2 will come only 2 months after 3.0 My preference would be to have the CASSANDRA-10425 out of the gate, but this still would require some testing effort for safety. The reason being that 3.0 represents a reframing of performance expectations, and after that, any changes to default, even for larger memory systems constitute a bigger chance of surprise. Do we have a chance to learn about sizing from surveys, etc before the runway ends for 3.0? If we could get something like CASSANDRA-10425 in place, it would cover both bases. > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938336#comment-14938336 ] Jonathan Shook commented on CASSANDRA-10403: [~JoshuaMcKenzie] I understand and appreciate the need to control scoping effort for 3.0 planning. bq. Shouldn't the read/write workload distribution also play into that? Yes, but there is a mostly orthogonal effect to the nuances of the workload mix which has to do with the vertical scalability of GC when the system. This is visible along the sizing spectrum. Run the same workload and try to scale the heap proportionally over the memory (1/4 or whatever) and you will likely see CMS suffer no matter what. This is slightly conjectural, but easily verifiable with some effort. bq. the idea of having a default that's optimal for everyone is unrealistic I think we are converging on a common perspective on this. [~slebresne] bq. 3.2 will come only 2 months after 3.0 My preference would be to have the CASSANDRA-10425 out of the gate, but this still would require some testing effort for safety. The reason being that 3.0 represents a reframing of performance expectations, and after that, any changes to default, even for larger memory systems constitute a bigger chance of surprise. Do we have a chance to learn about sizing from surveys, etc before the runway ends for 3.0? If we could get something like CASSANDRA-10425 in place, it would cover both bases. > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10427) compactionstats 'completed' field not updating
T Jake Luciani created CASSANDRA-10427:
--
Summary: compactionstats 'completed' field not updating
Key: CASSANDRA-10427
URL: https://issues.apache.org/jira/browse/CASSANDRA-10427
Project: Cassandra
Issue Type: Bug
Reporter: T Jake Luciani
Priority: Critical
Fix For: 3.0.0 rc2
nodetool compactionstats is no longer showing any change to completed field or
progress.
Can be reproduced with:
* Start cassandra
* Run stress write
* run nodetool flush followed by nodetool compactionstats
{code}
➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool flush
➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
pending tasks: 1
id compaction typekeyspace
table completed totalunit progress
75609a20-67a8-11e5-8003-c909ff7fd15eCompaction keyspace1
standard1 0 350132121 bytes 0.00%
Active compaction remaining time : 0h00m20s
➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
pending tasks: 1
id compaction typekeyspace
table completed totalunit progress
75609a20-67a8-11e5-8003-c909ff7fd15eCompaction keyspace1
standard1 0 350132121 bytes 0.00%
Active compaction remaining time : 0h00m20s
➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
pending tasks: 1
id compaction typekeyspace
table completed totalunit progress
75609a20-67a8-11e5-8003-c909ff7fd15eCompaction keyspace1
standard1 0 350132121 bytes 0.00%
Active compaction remaining time : 0h00m20s
➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
pending tasks: 1
id compaction typekeyspace
table completed totalunit progress
75609a20-67a8-11e5-8003-c909ff7fd15eCompaction keyspace1
standard1 0 350132121 bytes 0.00%
Active compaction remaining time : 0h00m20s
➜ cassandra git:(cassandra-3.0) ✗ ./bin/nodetool compactionstats
pending tasks: 0
➜ cassandra git:(cassandra-3.0) ✗
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938330#comment-14938330 ] Aleksey Yeschenko commented on CASSANDRA-10403: --- I agree that we should revert in 3.0.0 (while making it trivial to enable), and revisit for 3.2. > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-10426) When trying to connect from the python driver, bus error is occurs.
[ https://issues.apache.org/jira/browse/CASSANDRA-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-10426. -- Resolution: Invalid Reproduced In: 2.1.9, 2.1.7 (was: 2.1.7, 2.1.9) >From your description, this sounds like a problem with the python driver if >anything, and the python driver is not maintained here, you want to check >their [JIRA|https://datastax-oss.atlassian.net/projects/PYTHON/summary]. > When trying to connect from the python driver, bus error is occurs. > --- > > Key: CASSANDRA-10426 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10426 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Odroid XU4, both ubuntu 15 or ubuntu 14. >Reporter: Rajendra Adhikari > Fix For: 2.1.x > > > I am working with cassandra on odroid-xu4 with its Ubuntu-14 image. (I also > tried Ubuntu-15 image. > (http://odroid.com/dokuwiki/doku.php?id=en:xu3_release_linux_ubuntu) > I installed the latest cassandra and its python driver. However trying to > connect to it from the cassandra python-driver gives a serios bus error and > makes the python quit. Not even raises an exception. > from cassandra.cluster import Cluster > k = Cluster(['127.0.0.1']) > k.connect() > #crashes the python. Bus error appears on the command prompt/terminal > Looks like some incompatibility between things. I confirm that I can > successfully cqlsh to the cassandra. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938295#comment-14938295 ] Sylvain Lebresne commented on CASSANDRA-10403: -- My 2 cents here is: let's revert to CMS for 3.0 (while tweaking our startup script so it's easier to switch setting as suggested above, that's a good idea and it should be trivial) since that's been our default forever and it's clear we haven't experimented enough yet to be 100% sure that G1 is a clearly better default and so we should err on the side of conservatism. We absolutely should continue to experiment on this and refine our defaults and we now have CASSANDRA-10425 for that. But as far as 3.0 is concerned, I would venture that using our available resource for more testing (and things like testing performance of clusters during upgrade to 3.0 to take just one example) would be a better choice that spending it on more experimentation on this. Even if G1 turns out to be an overall better default, 3.2 will come only 2 months after 3.0. > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10426) When trying to connect from the python driver, bus error is occurs.
Rajendra Adhikari created CASSANDRA-10426: - Summary: When trying to connect from the python driver, bus error is occurs. Key: CASSANDRA-10426 URL: https://issues.apache.org/jira/browse/CASSANDRA-10426 Project: Cassandra Issue Type: Bug Components: Core Environment: Odroid XU4, both ubuntu 15 or ubuntu 14. Reporter: Rajendra Adhikari Fix For: 2.1.x I am working with cassandra on odroid-xu4 with its Ubuntu-14 image. (I also tried Ubuntu-15 image. (http://odroid.com/dokuwiki/doku.php?id=en:xu3_release_linux_ubuntu) I installed the latest cassandra and its python driver. However trying to connect to it from the cassandra python-driver gives a serios bus error and makes the python quit. Not even raises an exception. from cassandra.cluster import Cluster k = Cluster(['127.0.0.1']) k.connect() #crashes the python. Bus error appears on the command prompt/terminal Looks like some incompatibility between things. I confirm that I can successfully cqlsh to the cassandra. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10347) Bulk Loader API could not tolerate even node failure
[ https://issues.apache.org/jira/browse/CASSANDRA-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938283#comment-14938283 ] Carl Yeksigian commented on CASSANDRA-10347: Makes sense. I'm +1 on the patch as-is. > Bulk Loader API could not tolerate even node failure > > > Key: CASSANDRA-10347 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10347 > Project: Cassandra > Issue Type: Bug >Reporter: Shenghua Wan >Assignee: Paulo Motta > Fix For: 2.1.x, 2.2.x, 3.0.x > > Attachments: AbstractBulkRecordWriter.java > > > When user uses CqlBulkOutputFormat, it tries to stream to all the nodes in > the token range, which includes the dead nodes. Therefore, the stream failed. > There was a design in C* API to allow stream() method to have a list of > ignore hosts, but it was not utilized. > The empty-argument stream() method is called in all existing versions of C*, > i.e. > in v2.0.11, > https://github.com/apache/cassandra/blob/cassandra-2.0.11/src/java/org/apache/cassandra/hadoop/AbstractBulkRecordWriter.java#L122 > in v2.1.5, > https://github.com/apache/cassandra/blob/cassandra-2.1.5/src/java/org/apache/cassandra/hadoop/AbstractBulkRecordWriter.java#L122 > and current trunk branch > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java#L241 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938282#comment-14938282 ] Joshua McKenzie commented on CASSANDRA-10403: - bq. I'd prefer not to make too many assumptions about confirmation or (human) memory bias on this. We will not get off this carousel without actual data. Fair point, and I agree on the fact that all this needs further exploration. Unfortunately we have neither infinite time nor resources to get ready for 3.0, so there's a reduction in scope as to what this ticket's trying to solve. bq. if we simply align the GC settings to the type of hardware that they work well for. Shouldn't the read/write workload distribution also play into that? So after being a PITA and devil's advocate on this ticket, the end perspective I come down to is: there's a bunch of different workloads and a bunch of different hardware that C* runs on, and the idea of having a default that's optimal for everyone is unrealistic. It may very well be that G1 is a better "good enough" default for most distributions, large heap or no, and that's the conversation on IRC that led Jonathan to his comment on the other ticket to go with it. [JEP 248|http://openjdk.java.net/jeps/248] seems to imply that Oracle thinks that's the case. Within the scope of this ticket, pending the 100x results (if they're inline with 10x implications), I'd be comfortable using this ticket as an opportunity to add back the 2.2 settings commented out with some extra context for the workloads we expect those settings to excel on and keeping G1 the default. [~pauloricardomg]: could you elaborate on some of the pain points you ran into with an 8G heap and G1? > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/3] cassandra git commit: Save space in WriteCallbackInfo (CASSANDRA-9833)
Save space in WriteCallbackInfo (CASSANDRA-9833)
Performs decision to save hint mutation eagerly, and
saves only the mutation itself
patch by benedict; reviewed by branimir for CASSANDRA-9833
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/40f4daa3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/40f4daa3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/40f4daa3
Branch: refs/heads/trunk
Commit: 40f4daa36769d4aba4f1d7aa291546b44cad8f77
Parents: e3d5844
Author: Benedict Elliott Smith
Authored: Thu Jul 16 18:46:35 2015 +0100
Committer: Benedict Elliott Smith
Committed: Wed Sep 30 19:56:49 2015 +0100
--
.../apache/cassandra/net/WriteCallbackInfo.java | 36 +
test/unit/org/apache/cassandra/MockSchema.java | 2 +-
.../cassandra/net/WriteCallbackInfoTest.java| 79
3 files changed, 102 insertions(+), 15 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/40f4daa3/src/java/org/apache/cassandra/net/WriteCallbackInfo.java
--
diff --git a/src/java/org/apache/cassandra/net/WriteCallbackInfo.java
b/src/java/org/apache/cassandra/net/WriteCallbackInfo.java
index 582298c..bf7cc3a 100644
--- a/src/java/org/apache/cassandra/net/WriteCallbackInfo.java
+++ b/src/java/org/apache/cassandra/net/WriteCallbackInfo.java
@@ -28,9 +28,8 @@ import org.apache.cassandra.service.paxos.Commit;
public class WriteCallbackInfo extends CallbackInfo
{
-private final MessageOut sentMessage;
-private final ConsistencyLevel consistencyLevel;
-private final boolean allowHints;
+// either a Mutation, or a Paxos Commit (MessageOut)
+private final Object mutation;
public WriteCallbackInfo(InetAddress target,
IAsyncCallback callback,
@@ -41,23 +40,32 @@ public class WriteCallbackInfo extends CallbackInfo
{
super(target, callback, serializer, true);
assert message != null;
-this.sentMessage = message;
-this.consistencyLevel = consistencyLevel;
-this.allowHints = allowHints;
+this.mutation = shouldHint(allowHints, message, consistencyLevel);
}
-Mutation mutation()
+public boolean shouldHint()
{
-return sentMessage.verb == MessagingService.Verb.PAXOS_COMMIT
- ? ((Commit) sentMessage.payload).makeMutation()
- : (Mutation) sentMessage.payload;
+return mutation != null && StorageProxy.shouldHint(target);
}
-public boolean shouldHint()
+public Mutation mutation()
+{
+return getMutation(mutation);
+}
+
+private static Mutation getMutation(Object object)
+{
+assert object instanceof Commit || object instanceof Mutation : object;
+return object instanceof Commit ? ((Commit) object).makeMutation()
+: (Mutation) object;
+}
+
+private static Object shouldHint(boolean allowHints, MessageOut
sentMessage, ConsistencyLevel consistencyLevel)
{
return allowHints
-&& sentMessage.verb != MessagingService.Verb.COUNTER_MUTATION
-&& consistencyLevel != ConsistencyLevel.ANY
-&& StorageProxy.shouldHint(target);
+ && sentMessage.verb != MessagingService.Verb.COUNTER_MUTATION
+ && consistencyLevel != ConsistencyLevel.ANY
+ ? sentMessage.payload : null;
}
+
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/40f4daa3/test/unit/org/apache/cassandra/MockSchema.java
--
diff --git a/test/unit/org/apache/cassandra/MockSchema.java
b/test/unit/org/apache/cassandra/MockSchema.java
index 6b50e49..a406290 100644
--- a/test/unit/org/apache/cassandra/MockSchema.java
+++ b/test/unit/org/apache/cassandra/MockSchema.java
@@ -142,7 +142,7 @@ public class MockSchema
return new ColumnFamilyStore(ks, cfname, 0, metadata, new
Directories(metadata), false, false);
}
-private static CFMetaData newCFMetaData(String ksname, String cfname)
+public static CFMetaData newCFMetaData(String ksname, String cfname)
{
CFMetaData metadata = CFMetaData.Builder.create(ksname, cfname)
.addPartitionKey("key",
UTF8Type.instance)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/40f4daa3/test/unit/org/apache/cassandra/net/WriteCallbackInfoTest.java
--
diff --git a/test/unit/org/apache/cassandra/net/WriteCallbackInfoTest.java
b/test/unit/org/apache/cassandra/net/WriteCallbackInfoTest.java
new file mode 100644
index 000..ac726d5
--- /dev/null
+++ b/tes
[3/3] cassandra git commit: Merge branch 'cassandra-3.0' into trunk
Merge branch 'cassandra-3.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5d385599 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5d385599 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5d385599 Branch: refs/heads/trunk Commit: 5d38559908cfd8b5ecbad03a1cedb355d7856cee Parents: b06e703 40f4daa Author: Benedict Elliott Smith Authored: Wed Sep 30 19:57:02 2015 +0100 Committer: Benedict Elliott Smith Committed: Wed Sep 30 19:57:02 2015 +0100 -- .../apache/cassandra/net/WriteCallbackInfo.java | 36 + test/unit/org/apache/cassandra/MockSchema.java | 2 +- .../cassandra/net/WriteCallbackInfoTest.java| 79 3 files changed, 102 insertions(+), 15 deletions(-) --
[1/3] cassandra git commit: Save space in WriteCallbackInfo (CASSANDRA-9833)
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 e3d58448b -> 40f4daa36
refs/heads/trunk b06e703d7 -> 5d3855990
Save space in WriteCallbackInfo (CASSANDRA-9833)
Performs decision to save hint mutation eagerly, and
saves only the mutation itself
patch by benedict; reviewed by branimir for CASSANDRA-9833
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/40f4daa3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/40f4daa3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/40f4daa3
Branch: refs/heads/cassandra-3.0
Commit: 40f4daa36769d4aba4f1d7aa291546b44cad8f77
Parents: e3d5844
Author: Benedict Elliott Smith
Authored: Thu Jul 16 18:46:35 2015 +0100
Committer: Benedict Elliott Smith
Committed: Wed Sep 30 19:56:49 2015 +0100
--
.../apache/cassandra/net/WriteCallbackInfo.java | 36 +
test/unit/org/apache/cassandra/MockSchema.java | 2 +-
.../cassandra/net/WriteCallbackInfoTest.java| 79
3 files changed, 102 insertions(+), 15 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/40f4daa3/src/java/org/apache/cassandra/net/WriteCallbackInfo.java
--
diff --git a/src/java/org/apache/cassandra/net/WriteCallbackInfo.java
b/src/java/org/apache/cassandra/net/WriteCallbackInfo.java
index 582298c..bf7cc3a 100644
--- a/src/java/org/apache/cassandra/net/WriteCallbackInfo.java
+++ b/src/java/org/apache/cassandra/net/WriteCallbackInfo.java
@@ -28,9 +28,8 @@ import org.apache.cassandra.service.paxos.Commit;
public class WriteCallbackInfo extends CallbackInfo
{
-private final MessageOut sentMessage;
-private final ConsistencyLevel consistencyLevel;
-private final boolean allowHints;
+// either a Mutation, or a Paxos Commit (MessageOut)
+private final Object mutation;
public WriteCallbackInfo(InetAddress target,
IAsyncCallback callback,
@@ -41,23 +40,32 @@ public class WriteCallbackInfo extends CallbackInfo
{
super(target, callback, serializer, true);
assert message != null;
-this.sentMessage = message;
-this.consistencyLevel = consistencyLevel;
-this.allowHints = allowHints;
+this.mutation = shouldHint(allowHints, message, consistencyLevel);
}
-Mutation mutation()
+public boolean shouldHint()
{
-return sentMessage.verb == MessagingService.Verb.PAXOS_COMMIT
- ? ((Commit) sentMessage.payload).makeMutation()
- : (Mutation) sentMessage.payload;
+return mutation != null && StorageProxy.shouldHint(target);
}
-public boolean shouldHint()
+public Mutation mutation()
+{
+return getMutation(mutation);
+}
+
+private static Mutation getMutation(Object object)
+{
+assert object instanceof Commit || object instanceof Mutation : object;
+return object instanceof Commit ? ((Commit) object).makeMutation()
+: (Mutation) object;
+}
+
+private static Object shouldHint(boolean allowHints, MessageOut
sentMessage, ConsistencyLevel consistencyLevel)
{
return allowHints
-&& sentMessage.verb != MessagingService.Verb.COUNTER_MUTATION
-&& consistencyLevel != ConsistencyLevel.ANY
-&& StorageProxy.shouldHint(target);
+ && sentMessage.verb != MessagingService.Verb.COUNTER_MUTATION
+ && consistencyLevel != ConsistencyLevel.ANY
+ ? sentMessage.payload : null;
}
+
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/40f4daa3/test/unit/org/apache/cassandra/MockSchema.java
--
diff --git a/test/unit/org/apache/cassandra/MockSchema.java
b/test/unit/org/apache/cassandra/MockSchema.java
index 6b50e49..a406290 100644
--- a/test/unit/org/apache/cassandra/MockSchema.java
+++ b/test/unit/org/apache/cassandra/MockSchema.java
@@ -142,7 +142,7 @@ public class MockSchema
return new ColumnFamilyStore(ks, cfname, 0, metadata, new
Directories(metadata), false, false);
}
-private static CFMetaData newCFMetaData(String ksname, String cfname)
+public static CFMetaData newCFMetaData(String ksname, String cfname)
{
CFMetaData metadata = CFMetaData.Builder.create(ksname, cfname)
.addPartitionKey("key",
UTF8Type.instance)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/40f4daa3/test/unit/org/apache/cassandra/net/WriteCallbackInfoTest.java
--
diff --git a/test/unit/org/apache/cassandra/net/WriteCallbackInfo
[jira] [Comment Edited] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938199#comment-14938199 ] Ryan McGuire edited comment on CASSANDRA-10403 at 9/30/15 6:54 PM: --- We enabled the use of different stress branches, so we should be able to run it now. I'll restart the job that benedict ran: http://cstar.datastax.com/tests/id/01b714d8-5f32-11e5-a4e4-42010af0688f I'll use [~benedict]'s stress branch from: https://github.com/belliottsmith/cassandra/tree/stress-report-interval EDIT: test started: http://cstar.datastax.com/tests/id/ac36457a-67a0-11e5-8666-42010af0688f was (Author: enigmacurry): We enabled the use of different stress branches, so we should be able to run it now. I'll restart the job that benedict ran: http://cstar.datastax.com/tests/id/01b714d8-5f32-11e5-a4e4-42010af0688f I'll use [~benedict]'s stress branch from: https://github.com/belliottsmith/cassandra/tree/stress-report-interval > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10357) mmap file boundary selection is broken for some large files
[ https://issues.apache.org/jira/browse/CASSANDRA-10357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938267#comment-14938267 ] Benedict commented on CASSANDRA-10357: -- Committed as c37562e345c24720c55428a8644191df68319812 (with magic number extracted to const) > mmap file boundary selection is broken for some large files > > > Key: CASSANDRA-10357 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10357 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Benedict > Fix For: 2.1.10 > > > If an early open interval occurs to close to an mmap boundary, the boundary > can be lost. Patch available > [here|https://github.com/belliottsmith/cassandra/tree/mmap-boundaries]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10347) Bulk Loader API could not tolerate even node failure
[
https://issues.apache.org/jira/browse/CASSANDRA-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938264#comment-14938264
]
Paulo Motta commented on CASSANDRA-10347:
-
bq. These options should be moved into {{ConfigHelper}} instead of
{{AbstractBulkOutputFormat}}
The reason why the options are in AbstractBulkOutputFormat is that they're
exclusive to *BulkOutputFormat (see CqlBulkOutputFormat, which also has
exclusive options), so they're available via CqlBulkOutputFormat or
BulkOutputFormat. On the 3.0 branch the option is only available on
CqlBulkOutputFormat, along with other exclusive cql bulk output format options.
> Bulk Loader API could not tolerate even node failure
>
>
> Key: CASSANDRA-10347
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10347
> Project: Cassandra
> Issue Type: Bug
>Reporter: Shenghua Wan
>Assignee: Paulo Motta
> Fix For: 2.1.x, 2.2.x, 3.0.x
>
> Attachments: AbstractBulkRecordWriter.java
>
>
> When user uses CqlBulkOutputFormat, it tries to stream to all the nodes in
> the token range, which includes the dead nodes. Therefore, the stream failed.
> There was a design in C* API to allow stream() method to have a list of
> ignore hosts, but it was not utilized.
> The empty-argument stream() method is called in all existing versions of C*,
> i.e.
> in v2.0.11,
> https://github.com/apache/cassandra/blob/cassandra-2.0.11/src/java/org/apache/cassandra/hadoop/AbstractBulkRecordWriter.java#L122
> in v2.1.5,
> https://github.com/apache/cassandra/blob/cassandra-2.1.5/src/java/org/apache/cassandra/hadoop/AbstractBulkRecordWriter.java#L122
> and current trunk branch
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java#L241
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[15/16] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e3d58448 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e3d58448 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e3d58448 Branch: refs/heads/trunk Commit: e3d58448b6cadef7be060bb35996cd86d9ec9d59 Parents: a039b7d 25de92e Author: Benedict Elliott Smith Authored: Wed Sep 30 19:47:39 2015 +0100 Committer: Benedict Elliott Smith Committed: Wed Sep 30 19:47:39 2015 +0100 -- --
[08/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/util/ByteBufferDataInput.java
--
diff --cc src/java/org/apache/cassandra/io/util/ByteBufferDataInput.java
index bf926e9,000..2f00687
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/io/util/ByteBufferDataInput.java
+++ b/src/java/org/apache/cassandra/io/util/ByteBufferDataInput.java
@@@ -1,171 -1,0 +1,177 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.io.util;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+
+import org.apache.cassandra.utils.ByteBufferUtil;
+
+public class ByteBufferDataInput extends AbstractDataInput implements
FileDataInput, DataInput
+{
+private final ByteBuffer buffer;
+private final String filename;
+private final long segmentOffset;
+private int position;
+
+public ByteBufferDataInput(ByteBuffer buffer, String filename, long
segmentOffset, int position)
+{
+assert buffer != null;
+this.buffer = buffer;
+this.filename = filename;
+this.segmentOffset = segmentOffset;
+this.position = position;
+}
+
+// Only use when we know the seek in within the mapped segment. Throws an
+// IOException otherwise.
+public void seek(long pos) throws IOException
+{
+long inSegmentPos = pos - segmentOffset;
- if (inSegmentPos < 0 || inSegmentPos > buffer.capacity())
++if (!contains(pos))
+throw new IOException(String.format("Seek position %d is not
within mmap segment (seg offs: %d, length: %d)", pos, segmentOffset,
buffer.capacity()));
+
+position = (int) inSegmentPos;
+}
+
++public boolean contains(long pos)
++{
++long inSegmentPos = pos - segmentOffset;
++return inSegmentPos >= 0 && inSegmentPos < buffer.capacity();
++}
++
+public long getFilePointer()
+{
+return segmentOffset + position;
+}
+
+public long getPosition()
+{
+return segmentOffset + position;
+}
+
+public long getPositionLimit()
+{
+return segmentOffset + buffer.capacity();
+}
+
+@Override
+public boolean markSupported()
+{
+return false;
+}
+
+public void reset(FileMark mark) throws IOException
+{
+assert mark instanceof MappedFileDataInputMark;
+position = ((MappedFileDataInputMark) mark).position;
+}
+
+public FileMark mark()
+{
+return new MappedFileDataInputMark(position);
+}
+
+public long bytesPastMark(FileMark mark)
+{
+assert mark instanceof MappedFileDataInputMark;
+assert position >= ((MappedFileDataInputMark) mark).position;
+return position - ((MappedFileDataInputMark) mark).position;
+}
+
+public boolean isEOF() throws IOException
+{
+return position == buffer.capacity();
+}
+
+public long bytesRemaining() throws IOException
+{
+return buffer.capacity() - position;
+}
+
+public String getPath()
+{
+return filename;
+}
+
+public int read() throws IOException
+{
+if (isEOF())
+return -1;
+return buffer.get(position++) & 0xFF;
+}
+
+/**
+ * Does the same thing as readFully do but without copying
data (thread safe)
+ * @param length length of the bytes to read
+ * @return buffer with portion of file content
+ * @throws IOException on any fail of I/O operation
+ */
+public ByteBuffer readBytes(int length) throws IOException
+{
+int remaining = buffer.remaining() - position;
+if (length > remaining)
+throw new IOException(String.format("mmap segment underflow;
remaining is %d but %d requested",
+remaining, length));
+
+if (length == 0)
+return ByteBufferUtil.EMPTY_BYTE_BUFFER;
+
+ByteBuffer bytes = buffer.duplicate();
+bytes.position(buffer.position() + positi
[05/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/util/ByteBufferDataInput.java
--
diff --cc src/java/org/apache/cassandra/io/util/ByteBufferDataInput.java
index bf926e9,000..2f00687
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/io/util/ByteBufferDataInput.java
+++ b/src/java/org/apache/cassandra/io/util/ByteBufferDataInput.java
@@@ -1,171 -1,0 +1,177 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.io.util;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+
+import org.apache.cassandra.utils.ByteBufferUtil;
+
+public class ByteBufferDataInput extends AbstractDataInput implements
FileDataInput, DataInput
+{
+private final ByteBuffer buffer;
+private final String filename;
+private final long segmentOffset;
+private int position;
+
+public ByteBufferDataInput(ByteBuffer buffer, String filename, long
segmentOffset, int position)
+{
+assert buffer != null;
+this.buffer = buffer;
+this.filename = filename;
+this.segmentOffset = segmentOffset;
+this.position = position;
+}
+
+// Only use when we know the seek in within the mapped segment. Throws an
+// IOException otherwise.
+public void seek(long pos) throws IOException
+{
+long inSegmentPos = pos - segmentOffset;
- if (inSegmentPos < 0 || inSegmentPos > buffer.capacity())
++if (!contains(pos))
+throw new IOException(String.format("Seek position %d is not
within mmap segment (seg offs: %d, length: %d)", pos, segmentOffset,
buffer.capacity()));
+
+position = (int) inSegmentPos;
+}
+
++public boolean contains(long pos)
++{
++long inSegmentPos = pos - segmentOffset;
++return inSegmentPos >= 0 && inSegmentPos < buffer.capacity();
++}
++
+public long getFilePointer()
+{
+return segmentOffset + position;
+}
+
+public long getPosition()
+{
+return segmentOffset + position;
+}
+
+public long getPositionLimit()
+{
+return segmentOffset + buffer.capacity();
+}
+
+@Override
+public boolean markSupported()
+{
+return false;
+}
+
+public void reset(FileMark mark) throws IOException
+{
+assert mark instanceof MappedFileDataInputMark;
+position = ((MappedFileDataInputMark) mark).position;
+}
+
+public FileMark mark()
+{
+return new MappedFileDataInputMark(position);
+}
+
+public long bytesPastMark(FileMark mark)
+{
+assert mark instanceof MappedFileDataInputMark;
+assert position >= ((MappedFileDataInputMark) mark).position;
+return position - ((MappedFileDataInputMark) mark).position;
+}
+
+public boolean isEOF() throws IOException
+{
+return position == buffer.capacity();
+}
+
+public long bytesRemaining() throws IOException
+{
+return buffer.capacity() - position;
+}
+
+public String getPath()
+{
+return filename;
+}
+
+public int read() throws IOException
+{
+if (isEOF())
+return -1;
+return buffer.get(position++) & 0xFF;
+}
+
+/**
+ * Does the same thing as readFully do but without copying
data (thread safe)
+ * @param length length of the bytes to read
+ * @return buffer with portion of file content
+ * @throws IOException on any fail of I/O operation
+ */
+public ByteBuffer readBytes(int length) throws IOException
+{
+int remaining = buffer.remaining() - position;
+if (length > remaining)
+throw new IOException(String.format("mmap segment underflow;
remaining is %d but %d requested",
+remaining, length));
+
+if (length == 0)
+return ByteBufferUtil.EMPTY_BYTE_BUFFER;
+
+ByteBuffer bytes = buffer.duplicate();
+bytes.position(buffer.position() + positi
[09/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
index 87891ae,000..2b65792
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
@@@ -1,2287 -1,0 +1,2304 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.io.sstable.format;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+import java.util.*;
+import java.util.concurrent.*;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.base.Predicate;
+import com.google.common.collect.Iterables;
+import com.google.common.collect.Iterators;
+import com.google.common.collect.Ordering;
+import com.google.common.primitives.Longs;
+import com.google.common.util.concurrent.RateLimiter;
+
+import com.clearspring.analytics.stream.cardinality.CardinalityMergeException;
+import com.clearspring.analytics.stream.cardinality.HyperLogLogPlus;
+import com.clearspring.analytics.stream.cardinality.ICardinality;
+import com.codahale.metrics.Counter;
+import org.apache.cassandra.cache.CachingOptions;
+import org.apache.cassandra.cache.InstrumentingCache;
+import org.apache.cassandra.cache.KeyCacheKey;
+import org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor;
+import org.apache.cassandra.concurrent.ScheduledExecutors;
+import org.apache.cassandra.config.*;
+import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.columniterator.OnDiskAtomIterator;
+import org.apache.cassandra.db.commitlog.ReplayPosition;
+import org.apache.cassandra.db.composites.CellName;
+import org.apache.cassandra.db.filter.ColumnSlice;
+import org.apache.cassandra.db.index.SecondaryIndex;
+import org.apache.cassandra.db.lifecycle.Tracker;
+import org.apache.cassandra.dht.*;
+import org.apache.cassandra.io.FSError;
++import org.apache.cassandra.io.FSReadError;
+import org.apache.cassandra.io.compress.CompressionMetadata;
+import org.apache.cassandra.io.sstable.*;
+import org.apache.cassandra.io.sstable.metadata.*;
+import org.apache.cassandra.io.util.*;
+import org.apache.cassandra.metrics.RestorableMeter;
+import org.apache.cassandra.metrics.StorageMetrics;
+import org.apache.cassandra.service.ActiveRepairService;
+import org.apache.cassandra.service.CacheService;
+import org.apache.cassandra.service.StorageService;
+import org.apache.cassandra.utils.*;
+import org.apache.cassandra.utils.concurrent.OpOrder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.cassandra.utils.concurrent.Ref;
+import org.apache.cassandra.utils.concurrent.SelfRefCounted;
+
+import static
org.apache.cassandra.db.Directories.SECONDARY_INDEX_NAME_SEPARATOR;
+
+/**
+ * An SSTableReader can be constructed in a number of places, but typically
is either
+ * read from disk at startup, or constructed from a flushed memtable, or
after compaction
+ * to replace some existing sstables. However once created, an sstablereader
may also be modified.
+ *
+ * A reader's OpenReason describes its current stage in its lifecycle, as
follows:
+ *
+ *
+ * {@code
+ * NORMAL
+ * From: None=> Reader has been read from disk, either at
startup or from a flushed memtable
+ * EARLY => Reader is the final result of a compaction
+ * MOVED_START => Reader WAS being compacted, but this failed and
it has been restored to NORMAL status
+ *
+ * EARLY
+ * From: None=> Reader is a compaction replacement that is
either incomplete and has been opened
+ *to represent its partial result status, or has
been finished but the compaction
+ *it is a part of has not yet completed fully
+ * EARLY => Same as from None, only it is not the first
t
[13/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Conflicts:
src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
src/java/org/apache/cassandra/io/util/MmappedSegmentedFile.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/25de92e3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/25de92e3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/25de92e3
Branch: refs/heads/trunk
Commit: 25de92e321604626d6c098233082904832c07814
Parents: 7452b20 c37562e
Author: Benedict Elliott Smith
Authored: Wed Sep 30 19:47:14 2015 +0100
Committer: Benedict Elliott Smith
Committed: Wed Sep 30 19:47:14 2015 +0100
--
.../io/sstable/AbstractSSTableSimpleWriter.java | 2 +
.../cassandra/io/sstable/CQLSSTableWriter.java | 15 +
.../io/sstable/SSTableSimpleUnsortedWriter.java | 6 +
.../io/sstable/SSTableSimpleWriter.java | 13 +
.../io/sstable/format/SSTableReader.java| 17 +
.../cassandra/io/util/ByteBufferDataInput.java | 8 +-
.../cassandra/io/util/MmappedSegmentedFile.java | 275
.../apache/cassandra/io/util/SegmentedFile.java | 1 +
.../sstable/LongSegmentedFileBoundaryTest.java | 324 +++
9 files changed, 605 insertions(+), 56 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
index 4181ed0,c364171..8873f88
--- a/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
@@@ -370,20 -376,22 +380,25 @@@ public class CQLSSTableWriter implement
}
}
+ CFMetaData metadata()
+ {
+ return schema;
+ }
+
/**
- * Adds the specified column family to the specified keyspace.
+ * Creates the keyspace with the specified table.
*
- * @param ksm the keyspace meta data
- * @param cfm the column family meta data
+ * @param the table the table that must be created.
*/
-private static void addTableToKeyspace(KSMetaData ksm, CFMetaData cfm)
+private static void createKeyspaceWithTable(CFMetaData table)
{
-ksm = KSMetaData.cloneWith(ksm,
Iterables.concat(ksm.cfMetaData().values(), Collections.singleton(cfm)));
-Schema.instance.load(cfm);
-Schema.instance.setKeyspaceDefinition(ksm);
+KSMetaData ksm;
+ksm = KSMetaData.newKeyspace(table.ksName,
+
AbstractReplicationStrategy.getClass("org.apache.cassandra.locator.SimpleStrategy"),
+
ImmutableMap.of("replication_factor", "1"),
+ true,
+ Collections.singleton(table));
+Schema.instance.load(ksm);
}
/**
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
--
diff --cc
src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
index 4bb75bc,25ec354..534e77b
--- a/src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
@@@ -143,7 -140,13 +143,13 @@@ public class SSTableSimpleUnsortedWrite
return previous;
}
+ public Descriptor getCurrentDescriptor()
+ {
+ // can be implemented, but isn't necessary
+ throw new UnsupportedOperationException();
+ }
+
-protected ColumnFamily createColumnFamily() throws IOException
+protected ColumnFamily createColumnFamily()
{
return ArrayBackedSortedColumns.factory.create(metadata);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
index 2601d6d,23da501..f81e57d
--- a/src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
+++ b/src/java/org/apache/cassand
[11/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/util/ByteBufferDataInput.java
--
diff --cc src/java/org/apache/cassandra/io/util/ByteBufferDataInput.java
index bf926e9,000..2f00687
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/io/util/ByteBufferDataInput.java
+++ b/src/java/org/apache/cassandra/io/util/ByteBufferDataInput.java
@@@ -1,171 -1,0 +1,177 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.io.util;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+
+import org.apache.cassandra.utils.ByteBufferUtil;
+
+public class ByteBufferDataInput extends AbstractDataInput implements
FileDataInput, DataInput
+{
+private final ByteBuffer buffer;
+private final String filename;
+private final long segmentOffset;
+private int position;
+
+public ByteBufferDataInput(ByteBuffer buffer, String filename, long
segmentOffset, int position)
+{
+assert buffer != null;
+this.buffer = buffer;
+this.filename = filename;
+this.segmentOffset = segmentOffset;
+this.position = position;
+}
+
+// Only use when we know the seek in within the mapped segment. Throws an
+// IOException otherwise.
+public void seek(long pos) throws IOException
+{
+long inSegmentPos = pos - segmentOffset;
- if (inSegmentPos < 0 || inSegmentPos > buffer.capacity())
++if (!contains(pos))
+throw new IOException(String.format("Seek position %d is not
within mmap segment (seg offs: %d, length: %d)", pos, segmentOffset,
buffer.capacity()));
+
+position = (int) inSegmentPos;
+}
+
++public boolean contains(long pos)
++{
++long inSegmentPos = pos - segmentOffset;
++return inSegmentPos >= 0 && inSegmentPos < buffer.capacity();
++}
++
+public long getFilePointer()
+{
+return segmentOffset + position;
+}
+
+public long getPosition()
+{
+return segmentOffset + position;
+}
+
+public long getPositionLimit()
+{
+return segmentOffset + buffer.capacity();
+}
+
+@Override
+public boolean markSupported()
+{
+return false;
+}
+
+public void reset(FileMark mark) throws IOException
+{
+assert mark instanceof MappedFileDataInputMark;
+position = ((MappedFileDataInputMark) mark).position;
+}
+
+public FileMark mark()
+{
+return new MappedFileDataInputMark(position);
+}
+
+public long bytesPastMark(FileMark mark)
+{
+assert mark instanceof MappedFileDataInputMark;
+assert position >= ((MappedFileDataInputMark) mark).position;
+return position - ((MappedFileDataInputMark) mark).position;
+}
+
+public boolean isEOF() throws IOException
+{
+return position == buffer.capacity();
+}
+
+public long bytesRemaining() throws IOException
+{
+return buffer.capacity() - position;
+}
+
+public String getPath()
+{
+return filename;
+}
+
+public int read() throws IOException
+{
+if (isEOF())
+return -1;
+return buffer.get(position++) & 0xFF;
+}
+
+/**
+ * Does the same thing as readFully do but without copying
data (thread safe)
+ * @param length length of the bytes to read
+ * @return buffer with portion of file content
+ * @throws IOException on any fail of I/O operation
+ */
+public ByteBuffer readBytes(int length) throws IOException
+{
+int remaining = buffer.remaining() - position;
+if (length > remaining)
+throw new IOException(String.format("mmap segment underflow;
remaining is %d but %d requested",
+remaining, length));
+
+if (length == 0)
+return ByteBufferUtil.EMPTY_BYTE_BUFFER;
+
+ByteBuffer bytes = buffer.duplicate();
+bytes.position(buffer.position() + positi
[12/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
index 87891ae,000..2b65792
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
@@@ -1,2287 -1,0 +1,2304 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.io.sstable.format;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+import java.util.*;
+import java.util.concurrent.*;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.base.Predicate;
+import com.google.common.collect.Iterables;
+import com.google.common.collect.Iterators;
+import com.google.common.collect.Ordering;
+import com.google.common.primitives.Longs;
+import com.google.common.util.concurrent.RateLimiter;
+
+import com.clearspring.analytics.stream.cardinality.CardinalityMergeException;
+import com.clearspring.analytics.stream.cardinality.HyperLogLogPlus;
+import com.clearspring.analytics.stream.cardinality.ICardinality;
+import com.codahale.metrics.Counter;
+import org.apache.cassandra.cache.CachingOptions;
+import org.apache.cassandra.cache.InstrumentingCache;
+import org.apache.cassandra.cache.KeyCacheKey;
+import org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor;
+import org.apache.cassandra.concurrent.ScheduledExecutors;
+import org.apache.cassandra.config.*;
+import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.columniterator.OnDiskAtomIterator;
+import org.apache.cassandra.db.commitlog.ReplayPosition;
+import org.apache.cassandra.db.composites.CellName;
+import org.apache.cassandra.db.filter.ColumnSlice;
+import org.apache.cassandra.db.index.SecondaryIndex;
+import org.apache.cassandra.db.lifecycle.Tracker;
+import org.apache.cassandra.dht.*;
+import org.apache.cassandra.io.FSError;
++import org.apache.cassandra.io.FSReadError;
+import org.apache.cassandra.io.compress.CompressionMetadata;
+import org.apache.cassandra.io.sstable.*;
+import org.apache.cassandra.io.sstable.metadata.*;
+import org.apache.cassandra.io.util.*;
+import org.apache.cassandra.metrics.RestorableMeter;
+import org.apache.cassandra.metrics.StorageMetrics;
+import org.apache.cassandra.service.ActiveRepairService;
+import org.apache.cassandra.service.CacheService;
+import org.apache.cassandra.service.StorageService;
+import org.apache.cassandra.utils.*;
+import org.apache.cassandra.utils.concurrent.OpOrder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.cassandra.utils.concurrent.Ref;
+import org.apache.cassandra.utils.concurrent.SelfRefCounted;
+
+import static
org.apache.cassandra.db.Directories.SECONDARY_INDEX_NAME_SEPARATOR;
+
+/**
+ * An SSTableReader can be constructed in a number of places, but typically
is either
+ * read from disk at startup, or constructed from a flushed memtable, or
after compaction
+ * to replace some existing sstables. However once created, an sstablereader
may also be modified.
+ *
+ * A reader's OpenReason describes its current stage in its lifecycle, as
follows:
+ *
+ *
+ * {@code
+ * NORMAL
+ * From: None=> Reader has been read from disk, either at
startup or from a flushed memtable
+ * EARLY => Reader is the final result of a compaction
+ * MOVED_START => Reader WAS being compacted, but this failed and
it has been restored to NORMAL status
+ *
+ * EARLY
+ * From: None=> Reader is a compaction replacement that is
either incomplete and has been opened
+ *to represent its partial result status, or has
been finished but the compaction
+ *it is a part of has not yet completed fully
+ * EARLY => Same as from None, only it is not the first
t
[07/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Conflicts:
src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
src/java/org/apache/cassandra/io/util/MmappedSegmentedFile.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/25de92e3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/25de92e3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/25de92e3
Branch: refs/heads/cassandra-3.0
Commit: 25de92e321604626d6c098233082904832c07814
Parents: 7452b20 c37562e
Author: Benedict Elliott Smith
Authored: Wed Sep 30 19:47:14 2015 +0100
Committer: Benedict Elliott Smith
Committed: Wed Sep 30 19:47:14 2015 +0100
--
.../io/sstable/AbstractSSTableSimpleWriter.java | 2 +
.../cassandra/io/sstable/CQLSSTableWriter.java | 15 +
.../io/sstable/SSTableSimpleUnsortedWriter.java | 6 +
.../io/sstable/SSTableSimpleWriter.java | 13 +
.../io/sstable/format/SSTableReader.java| 17 +
.../cassandra/io/util/ByteBufferDataInput.java | 8 +-
.../cassandra/io/util/MmappedSegmentedFile.java | 275
.../apache/cassandra/io/util/SegmentedFile.java | 1 +
.../sstable/LongSegmentedFileBoundaryTest.java | 324 +++
9 files changed, 605 insertions(+), 56 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
index 4181ed0,c364171..8873f88
--- a/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
@@@ -370,20 -376,22 +380,25 @@@ public class CQLSSTableWriter implement
}
}
+ CFMetaData metadata()
+ {
+ return schema;
+ }
+
/**
- * Adds the specified column family to the specified keyspace.
+ * Creates the keyspace with the specified table.
*
- * @param ksm the keyspace meta data
- * @param cfm the column family meta data
+ * @param the table the table that must be created.
*/
-private static void addTableToKeyspace(KSMetaData ksm, CFMetaData cfm)
+private static void createKeyspaceWithTable(CFMetaData table)
{
-ksm = KSMetaData.cloneWith(ksm,
Iterables.concat(ksm.cfMetaData().values(), Collections.singleton(cfm)));
-Schema.instance.load(cfm);
-Schema.instance.setKeyspaceDefinition(ksm);
+KSMetaData ksm;
+ksm = KSMetaData.newKeyspace(table.ksName,
+
AbstractReplicationStrategy.getClass("org.apache.cassandra.locator.SimpleStrategy"),
+
ImmutableMap.of("replication_factor", "1"),
+ true,
+ Collections.singleton(table));
+Schema.instance.load(ksm);
}
/**
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
--
diff --cc
src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
index 4bb75bc,25ec354..534e77b
--- a/src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
@@@ -143,7 -140,13 +143,13 @@@ public class SSTableSimpleUnsortedWrite
return previous;
}
+ public Descriptor getCurrentDescriptor()
+ {
+ // can be implemented, but isn't necessary
+ throw new UnsupportedOperationException();
+ }
+
-protected ColumnFamily createColumnFamily() throws IOException
+protected ColumnFamily createColumnFamily()
{
return ArrayBackedSortedColumns.factory.create(metadata);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
index 2601d6d,23da501..f81e57d
--- a/src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
+++ b/src/java/org/apache
[03/16] cassandra git commit: Fix Mmapped File Boundaries
Fix Mmapped File Boundaries
This patch fixes two bugs with mmap segment boundary
tracking, and introduces automated correction of
this bug on startup
patch by benedict; reviewed by tjake for CASSANDRA-10357
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c37562e3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c37562e3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c37562e3
Branch: refs/heads/cassandra-3.0
Commit: c37562e345c24720c55428a8644191df68319812
Parents: f6cab37
Author: Benedict Elliott Smith
Authored: Wed Sep 16 18:09:32 2015 +0100
Committer: Benedict Elliott Smith
Committed: Wed Sep 30 19:45:49 2015 +0100
--
.../io/sstable/AbstractSSTableSimpleWriter.java | 2 +
.../cassandra/io/sstable/CQLSSTableWriter.java | 15 +
.../cassandra/io/sstable/SSTableReader.java | 34 +-
.../io/sstable/SSTableSimpleUnsortedWriter.java | 6 +
.../io/sstable/SSTableSimpleWriter.java | 12 +
.../cassandra/io/util/MappedFileDataInput.java | 8 +-
.../cassandra/io/util/MmappedSegmentedFile.java | 270 +---
.../apache/cassandra/io/util/SegmentedFile.java | 1 +
.../sstable/LongSegmentedFileBoundaryTest.java | 322 +++
9 files changed, 601 insertions(+), 69 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c37562e3/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
--
diff --git
a/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
b/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
index 165a4b2..557c3de 100644
--- a/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
@@ -200,4 +200,6 @@ public abstract class AbstractSSTableSimpleWriter
implements Closeable
protected abstract void writeRow(DecoratedKey key, ColumnFamily
columnFamily) throws IOException;
protected abstract ColumnFamily getColumnFamily() throws IOException;
+
+public abstract Descriptor getCurrentDescriptor();
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c37562e3/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
b/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
index b211a90..c364171 100644
--- a/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
@@ -271,6 +271,16 @@ public class CQLSSTableWriter implements Closeable
writer.close();
}
+public Descriptor getCurrentDescriptor()
+{
+return writer.getCurrentDescriptor();
+}
+
+public CFMetaData getCFMetaData()
+{
+return writer.metadata;
+}
+
/**
* A Builder for a CQLSSTableWriter object.
*/
@@ -366,6 +376,11 @@ public class CQLSSTableWriter implements Closeable
}
}
+CFMetaData metadata()
+{
+return schema;
+}
+
/**
* Adds the specified column family to the specified keyspace.
*
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c37562e3/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index 0f307b0..84add6f 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -17,13 +17,7 @@
*/
package org.apache.cassandra.io.sstable;
-import java.io.BufferedInputStream;
-import java.io.DataInputStream;
-import java.io.File;
-import java.io.FileInputStream;
-import java.io.FileNotFoundException;
-import java.io.FileOutputStream;
-import java.io.IOException;
+import java.io.*;
import java.nio.ByteBuffer;
import java.util.*;
import java.util.concurrent.*;
@@ -70,20 +64,14 @@ import org.apache.cassandra.dht.LocalPartitioner;
import org.apache.cassandra.dht.Range;
import org.apache.cassandra.dht.Token;
import org.apache.cassandra.io.FSError;
+import org.apache.cassandra.io.FSReadError;
import org.apache.cassandra.io.compress.CompressionMetadata;
import org.apache.cassandra.io.sstable.metadata.CompactionMetadata;
import org.apache.cassandra.io.sstable.metadata.MetadataComponent;
import org.apache.cassandra.io.sstable.metadata.MetadataType;
import org.apache.cassandra.io.sstable.metadata.StatsMetadata;
import org.
[jira] [Updated] (CASSANDRA-10214) Enable index selection to be overridden on a per query basis
[
https://issues.apache.org/jira/browse/CASSANDRA-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-10214:
-
Fix Version/s: (was: 3.0.0 rc2)
3.x
> Enable index selection to be overridden on a per query basis
>
>
> Key: CASSANDRA-10214
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10214
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Fix For: 3.x
>
>
> (Broken out of CASSANDRA-10124)
> We could add a {{USING INDEX }} clause to {{SELECT}} syntax to
> force the choice of index and bypass the usual index selection mechanism.
> {code}
> CREATE TABLE ks.t1(k int, v1 int, v2 int, PRIMARY KEY (k));
> CREATE INDEX v1_idx ON ks.t1(v1);
> CREATE INDEX v2_idx ON ks.t1(v2);
> CREATE CUSTOM INDEX v1_v2_idx ON ks.t1(v1, v2) USING
> 'com.foo.bar.CustomMultiColumnIndex';
> # Override internal index selection mechanism
> SELECT * FROM ks.t1 WHERE v1=0 AND v2=0 USING INDEX v1_idx;
> SELECT * FROM ks.t1 WHERE v1=0 AND v2=0 USING INDEX v2_idx;
> SELECT * FROM ks.t1 WHERE v1=0 AND v2=0 USING INDEX v1_v2_idx;
> {code}
> This is in some ways similar to [index
> hinting|http://docs.oracle.com/cd/B19306_01/server.102/b14211/hintsref.htm#CHDJDIAH]
> in Oracle.
> edit: fixed typo's (missing INDEX in the USING clauses)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[01/16] cassandra git commit: Fix Mmapped File Boundaries
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 f6cab37d5 -> c37562e34
refs/heads/cassandra-2.2 7452b2050 -> 25de92e32
refs/heads/cassandra-3.0 a039b7df5 -> e3d58448b
refs/heads/trunk 092281fee -> b06e703d7
Fix Mmapped File Boundaries
This patch fixes two bugs with mmap segment boundary
tracking, and introduces automated correction of
this bug on startup
patch by benedict; reviewed by tjake for CASSANDRA-10357
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c37562e3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c37562e3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c37562e3
Branch: refs/heads/cassandra-2.1
Commit: c37562e345c24720c55428a8644191df68319812
Parents: f6cab37
Author: Benedict Elliott Smith
Authored: Wed Sep 16 18:09:32 2015 +0100
Committer: Benedict Elliott Smith
Committed: Wed Sep 30 19:45:49 2015 +0100
--
.../io/sstable/AbstractSSTableSimpleWriter.java | 2 +
.../cassandra/io/sstable/CQLSSTableWriter.java | 15 +
.../cassandra/io/sstable/SSTableReader.java | 34 +-
.../io/sstable/SSTableSimpleUnsortedWriter.java | 6 +
.../io/sstable/SSTableSimpleWriter.java | 12 +
.../cassandra/io/util/MappedFileDataInput.java | 8 +-
.../cassandra/io/util/MmappedSegmentedFile.java | 270 +---
.../apache/cassandra/io/util/SegmentedFile.java | 1 +
.../sstable/LongSegmentedFileBoundaryTest.java | 322 +++
9 files changed, 601 insertions(+), 69 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c37562e3/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
--
diff --git
a/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
b/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
index 165a4b2..557c3de 100644
--- a/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
@@ -200,4 +200,6 @@ public abstract class AbstractSSTableSimpleWriter
implements Closeable
protected abstract void writeRow(DecoratedKey key, ColumnFamily
columnFamily) throws IOException;
protected abstract ColumnFamily getColumnFamily() throws IOException;
+
+public abstract Descriptor getCurrentDescriptor();
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c37562e3/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
b/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
index b211a90..c364171 100644
--- a/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
@@ -271,6 +271,16 @@ public class CQLSSTableWriter implements Closeable
writer.close();
}
+public Descriptor getCurrentDescriptor()
+{
+return writer.getCurrentDescriptor();
+}
+
+public CFMetaData getCFMetaData()
+{
+return writer.metadata;
+}
+
/**
* A Builder for a CQLSSTableWriter object.
*/
@@ -366,6 +376,11 @@ public class CQLSSTableWriter implements Closeable
}
}
+CFMetaData metadata()
+{
+return schema;
+}
+
/**
* Adds the specified column family to the specified keyspace.
*
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c37562e3/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index 0f307b0..84add6f 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -17,13 +17,7 @@
*/
package org.apache.cassandra.io.sstable;
-import java.io.BufferedInputStream;
-import java.io.DataInputStream;
-import java.io.File;
-import java.io.FileInputStream;
-import java.io.FileNotFoundException;
-import java.io.FileOutputStream;
-import java.io.IOException;
+import java.io.*;
import java.nio.ByteBuffer;
import java.util.*;
import java.util.concurrent.*;
@@ -70,20 +64,14 @@ import org.apache.cassandra.dht.LocalPartitioner;
import org.apache.cassandra.dht.Range;
import org.apache.cassandra.dht.Token;
import org.apache.cassandra.io.FSError;
+import org.apache.cassandra.io.FSReadError;
import org.apache.cassandra.io.compress.CompressionMetadata;
import org.apache.cassandra.io.sstable.me
[02/16] cassandra git commit: Fix Mmapped File Boundaries
Fix Mmapped File Boundaries
This patch fixes two bugs with mmap segment boundary
tracking, and introduces automated correction of
this bug on startup
patch by benedict; reviewed by tjake for CASSANDRA-10357
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c37562e3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c37562e3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c37562e3
Branch: refs/heads/cassandra-2.2
Commit: c37562e345c24720c55428a8644191df68319812
Parents: f6cab37
Author: Benedict Elliott Smith
Authored: Wed Sep 16 18:09:32 2015 +0100
Committer: Benedict Elliott Smith
Committed: Wed Sep 30 19:45:49 2015 +0100
--
.../io/sstable/AbstractSSTableSimpleWriter.java | 2 +
.../cassandra/io/sstable/CQLSSTableWriter.java | 15 +
.../cassandra/io/sstable/SSTableReader.java | 34 +-
.../io/sstable/SSTableSimpleUnsortedWriter.java | 6 +
.../io/sstable/SSTableSimpleWriter.java | 12 +
.../cassandra/io/util/MappedFileDataInput.java | 8 +-
.../cassandra/io/util/MmappedSegmentedFile.java | 270 +---
.../apache/cassandra/io/util/SegmentedFile.java | 1 +
.../sstable/LongSegmentedFileBoundaryTest.java | 322 +++
9 files changed, 601 insertions(+), 69 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c37562e3/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
--
diff --git
a/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
b/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
index 165a4b2..557c3de 100644
--- a/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
@@ -200,4 +200,6 @@ public abstract class AbstractSSTableSimpleWriter
implements Closeable
protected abstract void writeRow(DecoratedKey key, ColumnFamily
columnFamily) throws IOException;
protected abstract ColumnFamily getColumnFamily() throws IOException;
+
+public abstract Descriptor getCurrentDescriptor();
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c37562e3/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
b/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
index b211a90..c364171 100644
--- a/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
@@ -271,6 +271,16 @@ public class CQLSSTableWriter implements Closeable
writer.close();
}
+public Descriptor getCurrentDescriptor()
+{
+return writer.getCurrentDescriptor();
+}
+
+public CFMetaData getCFMetaData()
+{
+return writer.metadata;
+}
+
/**
* A Builder for a CQLSSTableWriter object.
*/
@@ -366,6 +376,11 @@ public class CQLSSTableWriter implements Closeable
}
}
+CFMetaData metadata()
+{
+return schema;
+}
+
/**
* Adds the specified column family to the specified keyspace.
*
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c37562e3/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index 0f307b0..84add6f 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -17,13 +17,7 @@
*/
package org.apache.cassandra.io.sstable;
-import java.io.BufferedInputStream;
-import java.io.DataInputStream;
-import java.io.File;
-import java.io.FileInputStream;
-import java.io.FileNotFoundException;
-import java.io.FileOutputStream;
-import java.io.IOException;
+import java.io.*;
import java.nio.ByteBuffer;
import java.util.*;
import java.util.concurrent.*;
@@ -70,20 +64,14 @@ import org.apache.cassandra.dht.LocalPartitioner;
import org.apache.cassandra.dht.Range;
import org.apache.cassandra.dht.Token;
import org.apache.cassandra.io.FSError;
+import org.apache.cassandra.io.FSReadError;
import org.apache.cassandra.io.compress.CompressionMetadata;
import org.apache.cassandra.io.sstable.metadata.CompactionMetadata;
import org.apache.cassandra.io.sstable.metadata.MetadataComponent;
import org.apache.cassandra.io.sstable.metadata.MetadataType;
import org.apache.cassandra.io.sstable.metadata.StatsMetadata;
import org.
[10/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Conflicts:
src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
src/java/org/apache/cassandra/io/util/MmappedSegmentedFile.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/25de92e3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/25de92e3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/25de92e3
Branch: refs/heads/cassandra-2.2
Commit: 25de92e321604626d6c098233082904832c07814
Parents: 7452b20 c37562e
Author: Benedict Elliott Smith
Authored: Wed Sep 30 19:47:14 2015 +0100
Committer: Benedict Elliott Smith
Committed: Wed Sep 30 19:47:14 2015 +0100
--
.../io/sstable/AbstractSSTableSimpleWriter.java | 2 +
.../cassandra/io/sstable/CQLSSTableWriter.java | 15 +
.../io/sstable/SSTableSimpleUnsortedWriter.java | 6 +
.../io/sstable/SSTableSimpleWriter.java | 13 +
.../io/sstable/format/SSTableReader.java| 17 +
.../cassandra/io/util/ByteBufferDataInput.java | 8 +-
.../cassandra/io/util/MmappedSegmentedFile.java | 275
.../apache/cassandra/io/util/SegmentedFile.java | 1 +
.../sstable/LongSegmentedFileBoundaryTest.java | 324 +++
9 files changed, 605 insertions(+), 56 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
index 4181ed0,c364171..8873f88
--- a/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
@@@ -370,20 -376,22 +380,25 @@@ public class CQLSSTableWriter implement
}
}
+ CFMetaData metadata()
+ {
+ return schema;
+ }
+
/**
- * Adds the specified column family to the specified keyspace.
+ * Creates the keyspace with the specified table.
*
- * @param ksm the keyspace meta data
- * @param cfm the column family meta data
+ * @param the table the table that must be created.
*/
-private static void addTableToKeyspace(KSMetaData ksm, CFMetaData cfm)
+private static void createKeyspaceWithTable(CFMetaData table)
{
-ksm = KSMetaData.cloneWith(ksm,
Iterables.concat(ksm.cfMetaData().values(), Collections.singleton(cfm)));
-Schema.instance.load(cfm);
-Schema.instance.setKeyspaceDefinition(ksm);
+KSMetaData ksm;
+ksm = KSMetaData.newKeyspace(table.ksName,
+
AbstractReplicationStrategy.getClass("org.apache.cassandra.locator.SimpleStrategy"),
+
ImmutableMap.of("replication_factor", "1"),
+ true,
+ Collections.singleton(table));
+Schema.instance.load(ksm);
}
/**
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
--
diff --cc
src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
index 4bb75bc,25ec354..534e77b
--- a/src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableSimpleUnsortedWriter.java
@@@ -143,7 -140,13 +143,13 @@@ public class SSTableSimpleUnsortedWrite
return previous;
}
+ public Descriptor getCurrentDescriptor()
+ {
+ // can be implemented, but isn't necessary
+ throw new UnsupportedOperationException();
+ }
+
-protected ColumnFamily createColumnFamily() throws IOException
+protected ColumnFamily createColumnFamily()
{
return ArrayBackedSortedColumns.factory.create(metadata);
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
index 2601d6d,23da501..f81e57d
--- a/src/java/org/apache/cassandra/io/sstable/SSTableSimpleWriter.java
+++ b/src/java/org/apache
[16/16] cassandra git commit: Merge branch 'cassandra-3.0' into trunk
Merge branch 'cassandra-3.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b06e703d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b06e703d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b06e703d Branch: refs/heads/trunk Commit: b06e703d750dff27809eabebfb1fb2531dd4e0fe Parents: 092281f e3d5844 Author: Benedict Elliott Smith Authored: Wed Sep 30 19:47:52 2015 +0100 Committer: Benedict Elliott Smith Committed: Wed Sep 30 19:47:52 2015 +0100 -- --
[04/16] cassandra git commit: Fix Mmapped File Boundaries
Fix Mmapped File Boundaries
This patch fixes two bugs with mmap segment boundary
tracking, and introduces automated correction of
this bug on startup
patch by benedict; reviewed by tjake for CASSANDRA-10357
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c37562e3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c37562e3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c37562e3
Branch: refs/heads/trunk
Commit: c37562e345c24720c55428a8644191df68319812
Parents: f6cab37
Author: Benedict Elliott Smith
Authored: Wed Sep 16 18:09:32 2015 +0100
Committer: Benedict Elliott Smith
Committed: Wed Sep 30 19:45:49 2015 +0100
--
.../io/sstable/AbstractSSTableSimpleWriter.java | 2 +
.../cassandra/io/sstable/CQLSSTableWriter.java | 15 +
.../cassandra/io/sstable/SSTableReader.java | 34 +-
.../io/sstable/SSTableSimpleUnsortedWriter.java | 6 +
.../io/sstable/SSTableSimpleWriter.java | 12 +
.../cassandra/io/util/MappedFileDataInput.java | 8 +-
.../cassandra/io/util/MmappedSegmentedFile.java | 270 +---
.../apache/cassandra/io/util/SegmentedFile.java | 1 +
.../sstable/LongSegmentedFileBoundaryTest.java | 322 +++
9 files changed, 601 insertions(+), 69 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c37562e3/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
--
diff --git
a/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
b/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
index 165a4b2..557c3de 100644
--- a/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/AbstractSSTableSimpleWriter.java
@@ -200,4 +200,6 @@ public abstract class AbstractSSTableSimpleWriter
implements Closeable
protected abstract void writeRow(DecoratedKey key, ColumnFamily
columnFamily) throws IOException;
protected abstract ColumnFamily getColumnFamily() throws IOException;
+
+public abstract Descriptor getCurrentDescriptor();
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c37562e3/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
b/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
index b211a90..c364171 100644
--- a/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/CQLSSTableWriter.java
@@ -271,6 +271,16 @@ public class CQLSSTableWriter implements Closeable
writer.close();
}
+public Descriptor getCurrentDescriptor()
+{
+return writer.getCurrentDescriptor();
+}
+
+public CFMetaData getCFMetaData()
+{
+return writer.metadata;
+}
+
/**
* A Builder for a CQLSSTableWriter object.
*/
@@ -366,6 +376,11 @@ public class CQLSSTableWriter implements Closeable
}
}
+CFMetaData metadata()
+{
+return schema;
+}
+
/**
* Adds the specified column family to the specified keyspace.
*
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c37562e3/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index 0f307b0..84add6f 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -17,13 +17,7 @@
*/
package org.apache.cassandra.io.sstable;
-import java.io.BufferedInputStream;
-import java.io.DataInputStream;
-import java.io.File;
-import java.io.FileInputStream;
-import java.io.FileNotFoundException;
-import java.io.FileOutputStream;
-import java.io.IOException;
+import java.io.*;
import java.nio.ByteBuffer;
import java.util.*;
import java.util.concurrent.*;
@@ -70,20 +64,14 @@ import org.apache.cassandra.dht.LocalPartitioner;
import org.apache.cassandra.dht.Range;
import org.apache.cassandra.dht.Token;
import org.apache.cassandra.io.FSError;
+import org.apache.cassandra.io.FSReadError;
import org.apache.cassandra.io.compress.CompressionMetadata;
import org.apache.cassandra.io.sstable.metadata.CompactionMetadata;
import org.apache.cassandra.io.sstable.metadata.MetadataComponent;
import org.apache.cassandra.io.sstable.metadata.MetadataType;
import org.apache.cassandra.io.sstable.metadata.StatsMetadata;
import org.apache.c
[14/16] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e3d58448 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e3d58448 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e3d58448 Branch: refs/heads/cassandra-3.0 Commit: e3d58448b6cadef7be060bb35996cd86d9ec9d59 Parents: a039b7d 25de92e Author: Benedict Elliott Smith Authored: Wed Sep 30 19:47:39 2015 +0100 Committer: Benedict Elliott Smith Committed: Wed Sep 30 19:47:39 2015 +0100 -- --
[06/16] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/25de92e3/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
index 87891ae,000..2b65792
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java
@@@ -1,2287 -1,0 +1,2304 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.io.sstable.format;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+import java.util.*;
+import java.util.concurrent.*;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.base.Predicate;
+import com.google.common.collect.Iterables;
+import com.google.common.collect.Iterators;
+import com.google.common.collect.Ordering;
+import com.google.common.primitives.Longs;
+import com.google.common.util.concurrent.RateLimiter;
+
+import com.clearspring.analytics.stream.cardinality.CardinalityMergeException;
+import com.clearspring.analytics.stream.cardinality.HyperLogLogPlus;
+import com.clearspring.analytics.stream.cardinality.ICardinality;
+import com.codahale.metrics.Counter;
+import org.apache.cassandra.cache.CachingOptions;
+import org.apache.cassandra.cache.InstrumentingCache;
+import org.apache.cassandra.cache.KeyCacheKey;
+import org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor;
+import org.apache.cassandra.concurrent.ScheduledExecutors;
+import org.apache.cassandra.config.*;
+import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.columniterator.OnDiskAtomIterator;
+import org.apache.cassandra.db.commitlog.ReplayPosition;
+import org.apache.cassandra.db.composites.CellName;
+import org.apache.cassandra.db.filter.ColumnSlice;
+import org.apache.cassandra.db.index.SecondaryIndex;
+import org.apache.cassandra.db.lifecycle.Tracker;
+import org.apache.cassandra.dht.*;
+import org.apache.cassandra.io.FSError;
++import org.apache.cassandra.io.FSReadError;
+import org.apache.cassandra.io.compress.CompressionMetadata;
+import org.apache.cassandra.io.sstable.*;
+import org.apache.cassandra.io.sstable.metadata.*;
+import org.apache.cassandra.io.util.*;
+import org.apache.cassandra.metrics.RestorableMeter;
+import org.apache.cassandra.metrics.StorageMetrics;
+import org.apache.cassandra.service.ActiveRepairService;
+import org.apache.cassandra.service.CacheService;
+import org.apache.cassandra.service.StorageService;
+import org.apache.cassandra.utils.*;
+import org.apache.cassandra.utils.concurrent.OpOrder;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.apache.cassandra.utils.concurrent.Ref;
+import org.apache.cassandra.utils.concurrent.SelfRefCounted;
+
+import static
org.apache.cassandra.db.Directories.SECONDARY_INDEX_NAME_SEPARATOR;
+
+/**
+ * An SSTableReader can be constructed in a number of places, but typically
is either
+ * read from disk at startup, or constructed from a flushed memtable, or
after compaction
+ * to replace some existing sstables. However once created, an sstablereader
may also be modified.
+ *
+ * A reader's OpenReason describes its current stage in its lifecycle, as
follows:
+ *
+ *
+ * {@code
+ * NORMAL
+ * From: None=> Reader has been read from disk, either at
startup or from a flushed memtable
+ * EARLY => Reader is the final result of a compaction
+ * MOVED_START => Reader WAS being compacted, but this failed and
it has been restored to NORMAL status
+ *
+ * EARLY
+ * From: None=> Reader is a compaction replacement that is
either incomplete and has been opened
+ *to represent its partial result status, or has
been finished but the compaction
+ *it is a part of has not yet completed fully
+ * EARLY => Same as from None, only it is not the first
t
[jira] [Updated] (CASSANDRA-9774) fix sstableverify dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-9774:
Reviewer: Branimir Lambov
> fix sstableverify dtest
> ---
>
> Key: CASSANDRA-9774
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9774
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.0.0 rc2
>
>
> One of our dtests for {{sstableverify}}
> ({{offline_tools_test.py:TestOfflineTools.sstableverify_test}}) is failing
> hard on trunk ([cassci
> history|http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/offline_tools_test/TestOfflineTools/sstableverify_test/history/])
> The way the test works is by deleting an SSTable, then running
> {{sstableverify}} on its table. In earlier versions, it successfully detects
> this problem and outputs that it "was not released before the reference was
> garbage collected". The test no longer finds this string in the output;
> looking through the output of the test, it doesn't look like it reports any
> problems at all.
> EDIT: After digging into the C* source a bit, I may have misattributed the
> problem to {{sstableverify}}; this could be a more general memory management
> problem, as the error text expected in the dtest is emitted by part of the
> {{Ref}} implementation:
> https://github.com/apache/cassandra/blob/075ff5000ced24b42f3b540815cae471bee4049d/src/java/org/apache/cassandra/utils/concurrent/Ref.java#L187
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-9774) fix sstableverify dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne reassigned CASSANDRA-9774:
---
Assignee: Sylvain Lebresne (was: Jeff Jirsa)
> fix sstableverify dtest
> ---
>
> Key: CASSANDRA-9774
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9774
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Sylvain Lebresne
>Priority: Blocker
> Fix For: 3.0.0 rc2
>
>
> One of our dtests for {{sstableverify}}
> ({{offline_tools_test.py:TestOfflineTools.sstableverify_test}}) is failing
> hard on trunk ([cassci
> history|http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/offline_tools_test/TestOfflineTools/sstableverify_test/history/])
> The way the test works is by deleting an SSTable, then running
> {{sstableverify}} on its table. In earlier versions, it successfully detects
> this problem and outputs that it "was not released before the reference was
> garbage collected". The test no longer finds this string in the output;
> looking through the output of the test, it doesn't look like it reports any
> problems at all.
> EDIT: After digging into the C* source a bit, I may have misattributed the
> problem to {{sstableverify}}; this could be a more general memory management
> problem, as the error text expected in the dtest is emitted by part of the
> {{Ref}} implementation:
> https://github.com/apache/cassandra/blob/075ff5000ced24b42f3b540815cae471bee4049d/src/java/org/apache/cassandra/utils/concurrent/Ref.java#L187
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9774) fix sstableverify dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938242#comment-14938242
]
Sylvain Lebresne commented on CASSANDRA-9774:
-
I had a look at this dtest and I'm pretty convinced it's the test that is
broken and I've pushed changes (of the dtest) to fix it
[here|https://github.com/pcmanus/cassandra-dtest/commits/9774]. The break down
of the problems is:
# I'm not really sure why the test is checking the log for "was not released
before the reference was garbage collected", there is no particular reason to
check for it at that particular place (what could make sense is to have all the
offline test check for errors in the log, which would catch this in particular,
but that's a different issue). But if anything, getting that message is a bug,
not a feature, so not getting it for 3.0 is good and certainly not something
that "blocks" RC2. So anyway, I've removed that check from the test. I'll note
that the fact this does happen in 2.2 warrants a look, and I'll finish tracking
that down tomorrow and create a proper ticket for it (but again, it's a 2.2
issue).
# The whole part about the test deleting a sstable and expecting sstableverify
to detect it is bogus since sstableverify is not able to detect such thing. In
fact, the comment in the test saying that it "ensure the missing table is
found" is inconsistent with the fact the test expects a return of {{0}} from
the next call. So anyway, I think we can/should remove that part of the test.
# we seems to send a slightly different message in 2.2 and 3.0 when a corrupted
sstable is found, so even with the changes above the test doesn't pass on 3.0.
And as the message is imo better in 3.0, I've just made the test last assertion
be version dependent.
Overall, I don't think there is a problem with sstableverify itself and the
attached dtest branch makes the test pass.
> fix sstableverify dtest
> ---
>
> Key: CASSANDRA-9774
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9774
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jim Witschey
>Assignee: Jeff Jirsa
>Priority: Blocker
> Fix For: 3.0.0 rc2
>
>
> One of our dtests for {{sstableverify}}
> ({{offline_tools_test.py:TestOfflineTools.sstableverify_test}}) is failing
> hard on trunk ([cassci
> history|http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/offline_tools_test/TestOfflineTools/sstableverify_test/history/])
> The way the test works is by deleting an SSTable, then running
> {{sstableverify}} on its table. In earlier versions, it successfully detects
> this problem and outputs that it "was not released before the reference was
> garbage collected". The test no longer finds this string in the output;
> looking through the output of the test, it doesn't look like it reports any
> problems at all.
> EDIT: After digging into the C* source a bit, I may have misattributed the
> problem to {{sstableverify}}; this could be a more general memory management
> problem, as the error text expected in the dtest is emitted by part of the
> {{Ref}} implementation:
> https://github.com/apache/cassandra/blob/075ff5000ced24b42f3b540815cae471bee4049d/src/java/org/apache/cassandra/utils/concurrent/Ref.java#L187
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938237#comment-14938237 ] Jonathan Shook commented on CASSANDRA-10403: I created CASSANDRA-10425 to discuss the per-size defaults. > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10425) Autoselect GC settings depending on system memory
[ https://issues.apache.org/jira/browse/CASSANDRA-10425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938236#comment-14938236 ] Jonathan Shook commented on CASSANDRA-10425: Consider adding some weightings for different levels of buffer-cache sensitivity in workload. > Autoselect GC settings depending on system memory > - > > Key: CASSANDRA-10425 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10425 > Project: Cassandra > Issue Type: Bug > Components: Config, Core >Reporter: Jonathan Shook > > 1) Make GC modular within cassandra-env > 2) For systems with 32GB or less of ram, use the classic CMS with the > established default settings. > 3) For systems with 48GB or more of ram, use 1/2 or up to 32GB of heap with > G1, whichever is lower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-10425) Autoselect GC settings depending on system memory
Jonathan Shook created CASSANDRA-10425: -- Summary: Autoselect GC settings depending on system memory Key: CASSANDRA-10425 URL: https://issues.apache.org/jira/browse/CASSANDRA-10425 Project: Cassandra Issue Type: Bug Components: Config, Core Reporter: Jonathan Shook 1) Make GC modular within cassandra-env 2) For systems with 32GB or less of ram, use the classic CMS with the established default settings. 3) For systems with 48GB or more of ram, use 1/2 or up to 32GB of heap with G1, whichever is lower. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10416) Windows dtest 3.0: clustering_order_test dtest fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938226#comment-14938226
]
Adam Holmberg commented on CASSANDRA-10416:
---
Neither of these errors give any indication of what version of the driver is in
use. I believe {{cassandra-test}} was up-to-date as of the cassandra-3.0.0-rc1
release.
Can you enlighten me on how the driver is pulled/installed/path'd in windows?
Is there some way we can confirm the version in use matches the latest on
{{cassandra-test}}?
> Windows dtest 3.0: clustering_order_test dtest fails
>
>
> Key: CASSANDRA-10416
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10416
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
> Fix For: 3.0.0 rc2
>
>
> [schema_metadata_test.TestSchemaMetadata.clustering_order_test|http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/72/testReport/schema_metadata_test/TestSchemaMetadata/clustering_order_test/]
> fails on cassandra-3.0 under Windows -- the test expects a table's metadata
> object, represented as a CQL query, to indicate its ordering.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938223#comment-14938223 ] Jonathan Shook commented on CASSANDRA-10403: [~JoshuaMcKenzie] I'd prefer not to make too many assumptions about confirmation or (human) memory bias on this. We will not get off this carousel without actual data. However, to the degree that you are right about it, it should encourage us to explore further, not less. CMS's pain in those cases has much to do with its inability to scale with hardware sizing and concurrency trends, which we seem to be working really hard to disregard. Until someone puts together a view of current and emerging system parameters, we really don't have the data that we need to set a default. I posit that the general case system is much bigger in practice that in the past. I also posit that on those systems, G1 is an obviously better default than CMS. So, we are likely going to get some data on 1) what the hardware data looks like in the field and 2) whether or not we can demonstrate the CMS improvements with larger memory that we've seen with *actual workloads* on *current system profiles*. I'm simply eager to see more data at this point. This is a bit out of scope of the ticket, but it is important. If we were able to set a default depending on the available memory, there would not be a single default. Trying to scale GC bandwidth up on bigger metal with CMS is arguably more painful than trying to make G1 useable with lower memory. However, we don't have to make that bargain as either-or. We can have the best of both, if we simply align the GC settings to the type of hardware that they work well for. I'll create another ticket for that. > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10401) json2sstable fails with NPE
[
https://issues.apache.org/jira/browse/CASSANDRA-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938221#comment-14938221
]
Jim Witschey commented on CASSANDRA-10401:
--
D'oh, didn't notice that -- if I understand correctly, the {{-c}} takes the
name of a table, not a SSTable directory. [~jpoblete] Give that a try?
> json2sstable fails with NPE
> ---
>
> Key: CASSANDRA-10401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10401
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Cassandra 2.1.8.621
>Reporter: Jose Martinez Poblete
>
> We have the following table...
> {noformat}
> CREATE TABLE keyspace_name.table_name (
> col1 text,
> col2 text,
> col3 text,
> col4 text,
> PRIMARY KEY ((col1, col2), col3)
> ) WITH CLUSTERING ORDER BY (col3 ASC)
> {noformat}
> And the following json in a file created from sstable2json tool
> {noformat}
> [
> {"key": "This is col1:This is col2,
> "cells": [["This is col3:","",1443217787319002],
>["This is col3:"col4","This is col4",1443217787319002]]}
> ]
> {noformat}
> Let's say we deleted that record form the DB and wanted to bring it back
> If we try to create an sstable from this data in a json file named
> test_file.json, we get a NPE
> {noformat}
> -bash-4.1$ json2sstable -K elp -c table_name-3264cbe063c211e5bc34e746786b7b29
> test_file.json
> /var/lib/cassandra/data/keyspace_name/table_name-3264cbe063c211e5bc34e746786b7b29/keyspace_name-table_name-ka-1-Data.db
> Importing 1 keys...
> java.lang.NullPointerException
> at
> org.apache.cassandra.tools.SSTableImport.getKeyValidator(SSTableImport.java:442)
> at
> org.apache.cassandra.tools.SSTableImport.importUnsorted(SSTableImport.java:316)
> at
> org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:287)
> at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:514)
> ERROR: null
> -bash-4.1$
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938199#comment-14938199 ] Ryan McGuire commented on CASSANDRA-10403: -- We enabled the use of different stress branches, so we should be able to run it now. I'll restart the job that benedict ran: http://cstar.datastax.com/tests/id/01b714d8-5f32-11e5-a4e4-42010af0688f I'll use [~benedict]'s stress branch from: https://github.com/belliottsmith/cassandra/tree/stress-report-interval > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938181#comment-14938181 ] Joshua McKenzie edited comment on CASSANDRA-10403 at 9/30/15 6:09 PM: -- -The debate here isn't between "Do we consider G1 or disregard G1 entirely as the general case", but rather "Do we use CMS or G1 as our default GC for 3.0, keeping the settings for the alternative available in our configuration files for users to swap between easily".- Edit: missed the "general case" phrase. To that point, I haven't heard of actual data showing what the general case is for C* usage is in the wild regarding heap size. I would expect that large heap deployments will be disproportionately reflected in the memories of people in the field since CMS causes pain at that point. [~enigmacurry]: did we ever get that 100x test run to complete successfully? Before we determine whether Paulo should go fully down this rabbit-hole pre 3.0 releasing, I'd like that data point since we invested some effort into getting that running already. Edit: was (Author: joshuamckenzie): The debate here isn't between "Do we consider G1 or disregard G1 entirely", but rather "Do we use CMS or G1 as our default GC for 3.0, keeping the settings for the alternative available in our configuration files for users to swap between easily". [~enigmacurry]: did we ever get that 100x test run to complete successfully? Before we determine whether Paulo should go fully down this rabbit-hole pre 3.0 releasing, I'd like that data point since we invested some effort into getting that running already. > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10416) Windows dtest 3.0: clustering_order_test dtest fails
[
https://issues.apache.org/jira/browse/CASSANDRA-10416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938188#comment-14938188
]
Jim Witschey commented on CASSANDRA-10416:
--
[~aholmber] Same as my comment on CASSANDRA-10417 -- does anything need to
change with our use of the {{cassandra-test}} branch on Windows?
> Windows dtest 3.0: clustering_order_test dtest fails
>
>
> Key: CASSANDRA-10416
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10416
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Yuki Morishita
> Fix For: 3.0.0 rc2
>
>
> [schema_metadata_test.TestSchemaMetadata.clustering_order_test|http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/72/testReport/schema_metadata_test/TestSchemaMetadata/clustering_order_test/]
> fails on cassandra-3.0 under Windows -- the test expects a table's metadata
> object, represented as a CQL query, to indicate its ordering.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9594) metrics reporter doesn't start until after a bootstrap
[
https://issues.apache.org/jira/browse/CASSANDRA-9594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938189#comment-14938189
]
Yuki Morishita commented on CASSANDRA-9594:
---
I think it's totally fine to swap order.
Metrics reporter just publishes whatever metrics available at the time of
reporting.
(Maybe we can swap Mx4J set up before initServer also for the same reason).
> metrics reporter doesn't start until after a bootstrap
> --
>
> Key: CASSANDRA-9594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9594
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Eric Evans
>Assignee: Tommy Stendahl
>Priority: Minor
> Labels: lhf
> Attachments: 9594.txt
>
>
> In {{o.a.c.service.CassandraDaemon#setup}}, the metrics reporter is started
> immediately after the invocation of
> {{o.a.c.service.StorageService#initServer}}, which for a bootstrapping node
> may block for a considerable period of time. If the metrics reporter is your
> only source of visibility, then you are blind until the bootstrap completes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10417) Windows dtest 3.0: creating_and_dropping_table_with_2ary_indexes_test
[ https://issues.apache.org/jira/browse/CASSANDRA-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938191#comment-14938191 ] Paulo Motta commented on CASSANDRA-10417: - Does it reinstall or only pulls? I installed from the cassandra-test branch on my machine and it worked well (before the upgrade there was the same error as in ci). > Windows dtest 3.0: creating_and_dropping_table_with_2ary_indexes_test > - > > Key: CASSANDRA-10417 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10417 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jim Witschey >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > [schema_metadata_test.TestSchemaMetadata.creating_and_dropping_table_with_2ary_indexes_test|http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/72/testReport/schema_metadata_test/TestSchemaMetadata/creating_and_dropping_table_with_2ary_indexes_test/] > fails on cassandra-3.0 under Windows. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10417) Windows dtest 3.0: creating_and_dropping_table_with_2ary_indexes_test
[
https://issues.apache.org/jira/browse/CASSANDRA-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938184#comment-14938184
]
Jim Witschey commented on CASSANDRA-10417:
--
The Windows test nodes pull from [the {{cassandra-test}}
branch|https://github.com/datastax/python-driver/commits/cassandra-test] on
every run. [~aholmber] Does {{cassandra-test}} need an update, or do we need to
start pulling from a different branch?
> Windows dtest 3.0: creating_and_dropping_table_with_2ary_indexes_test
> -
>
> Key: CASSANDRA-10417
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10417
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Paulo Motta
> Fix For: 3.0.0 rc2
>
>
> [schema_metadata_test.TestSchemaMetadata.creating_and_dropping_table_with_2ary_indexes_test|http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/72/testReport/schema_metadata_test/TestSchemaMetadata/creating_and_dropping_table_with_2ary_indexes_test/]
> fails on cassandra-3.0 under Windows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10403) Consider reverting to CMS GC on 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938181#comment-14938181 ] Joshua McKenzie commented on CASSANDRA-10403: - The debate here isn't between "Do we consider G1 or disregard G1 entirely", but rather "Do we use CMS or G1 as our default GC for 3.0, keeping the settings for the alternative available in our configuration files for users to swap between easily". [~enigmacurry]: did we ever get that 100x test run to complete successfully? Before we determine whether Paulo should go fully down this rabbit-hole pre 3.0 releasing, I'd like that data point since we invested some effort into getting that running already. > Consider reverting to CMS GC on 3.0 > --- > > Key: CASSANDRA-10403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10403 > Project: Cassandra > Issue Type: Improvement > Components: Config >Reporter: Joshua McKenzie >Assignee: Paulo Motta > Fix For: 3.0.0 rc2 > > > Reference discussion on CASSANDRA-7486. > For smaller heap sizes G1 appears to have some throughput/latency issues when > compared to CMS. With our default max heap size at 8G on 3.0, there's a > strong argument to be made for having CMS as the default for the 3.0 release. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager
[
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14938177#comment-14938177
]
J.P. Eiti Kimura commented on CASSANDRA-10233:
--
Guys I code a little patch for the problem. Looks like it is affecting the
hinted hand off scheduler.
{code}
cqlsh:system> SELECT target_id,hint_id,message_version FROM hints LIMIT 5;
target_id| hint_id |
message_version
--+--+-
|
2f5e0320-62d3-11e5-877e-77558ae77cc8 | 8
72888e32-dae5-41cd-a033-3c5871a3e045 | fad152f0-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fad152f1-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fb69e970-662a-11e5-89ed-77558ae77cc8 |
8
72888e32-dae5-41cd-a033-3c5871a3e045 | fb69e971-662a-11e5-89ed-77558ae77cc8 |
8
(5 rows)
{code}
We have an empty target_id in our hints table. That is causing the problem.
Can you check my patch?
> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Andrew Hust
> Attachments: cassandra-2.1.8-10233.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO [CompactionExecutor:1381] 2015-08-31 18:31:55,506
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20)
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
> ]
> INFO [CompactionExecutor:1381] 2015-08-31 18:31:55,599
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1
> sstables to
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,]
> to level=0. 1,544,495 bytes to 1,544,495 (~100% of original) in 93ms =
> 15.838121MB/s. 0 total partitions merged to 4. Partition merge counts were
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 -
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
> at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410)
> ~[na:1.7.0_79]
> at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_79]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> [na:1.7.0_79]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> [na:1.7.0_79]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [na:1.7.0_79]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_79]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_79]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10233) IndexOutOfBoundsException in HintedHandOffManager
[
https://issues.apache.org/jira/browse/CASSANDRA-10233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
J.P. Eiti Kimura updated CASSANDRA-10233:
-
Attachment: cassandra-2.1.8-10233.txt
patch attached
> IndexOutOfBoundsException in HintedHandOffManager
> -
>
> Key: CASSANDRA-10233
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10233
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.2.0
>Reporter: Omri Iluz
>Assignee: Andrew Hust
> Attachments: cassandra-2.1.8-10233.txt
>
>
> After upgrading our cluster to 2.2.0, the following error started showing
> exectly every 10 minutes on every server in the cluster:
> {noformat}
> INFO [CompactionExecutor:1381] 2015-08-31 18:31:55,506
> CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20)
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0,
> ]
> INFO [CompactionExecutor:1381] 2015-08-31 18:31:55,599
> CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1
> sstables to
> [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,]
> to level=0. 1,544,495 bytes to 1,544,495 (~100% of original) in 93ms =
> 15.838121MB/s. 0 total partitions merged to 4. Partition merge counts were
> {1:4, }
> ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 -
> Exception in thread Thread[HintedHandoff:1,1,main]
> java.lang.IndexOutOfBoundsException: null
> at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
> at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410)
> ~[na:1.7.0_79]
> at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> ~[apache-cassandra-2.2.0.jar:2.2.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_79]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> [na:1.7.0_79]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> [na:1.7.0_79]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [na:1.7.0_79]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_79]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_79]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10416) Windows dtest 3.0: clustering_order_test dtest fails
[ https://issues.apache.org/jira/browse/CASSANDRA-10416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14937966#comment-14937966 ] Paulo Motta commented on CASSANDRA-10416: - Similar to CASSANDRA-10417, this is fixed by upgrading the python driver on WIndows test nodes. > Windows dtest 3.0: clustering_order_test dtest fails > > > Key: CASSANDRA-10416 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10416 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jim Witschey >Assignee: Yuki Morishita > Fix For: 3.0.0 rc2 > > > [schema_metadata_test.TestSchemaMetadata.clustering_order_test|http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/72/testReport/schema_metadata_test/TestSchemaMetadata/clustering_order_test/] > fails on cassandra-3.0 under Windows -- the test expects a table's metadata > object, represented as a CQL query, to indicate its ordering. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
