[jira] [Commented] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623252#comment-14623252
]
Jeff Jirsa commented on CASSANDRA-9686:
---
Is this related to CASSANDRA-9774 ?
> FSReadError and LEAK DETECTED after upgrading
> -
>
> Key: CASSANDRA-9686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
>Reporter: Andreas Schnitzerling
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x
>
> Attachments: cassandra.bat, cassandra.yaml,
> compactions_in_progress.zip, sstable_activity.zip, system.log
>
>
> After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and
> LEAK DETECTED on start. Deleting the listed files, the failure goes away.
> {code:title=system.log}
> ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554
> DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
> org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file
> with 0 chunks encountered: java.io.DataInputStream@1c42271
> at
> org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:117)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> ~[na:1.7.0_55]
> at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> [na:1.7.0_55]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> [na:1.7.0_55]
> at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
> Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
> java.io.DataInputStream@1c42271
> at
> org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> ... 15 common frames omitted
> ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
> was not released before the reference was garbage collected
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9774) fix sstableverify dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623251#comment-14623251
]
Jeff Jirsa commented on CASSANDRA-9774:
---
Is this related to CASSANDRA-9686 ?
> fix sstableverify dtest
> ---
>
> Key: CASSANDRA-9774
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9774
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Jim Witschey
>Priority: Blocker
> Fix For: 3.0.x
>
>
> One of our dtests for {{sstableverify}}
> ({{offline_tools_test.py:TestOfflineTools.sstableverify_test}}) is failing
> hard on trunk ([cassci
> history|http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/offline_tools_test/TestOfflineTools/sstableverify_test/history/])
> The way the test works is by deleting an SSTable, then running
> {{sstableverify}} on its table. In earlier versions, it successfully detects
> this problem and outputs that it "was not released before the reference was
> garbage collected". The test no longer finds this string in the output;
> looking through the output of the test, it doesn't look like it reports any
> problems at all.
> EDIT: After digging into the C* source a bit, I may have misattributed the
> problem to {{sstableverify}}; this could be a more general memory management
> problem, as the error text expected in the dtest is emitted by part of the
> {{Ref}} implementation:
> https://github.com/apache/cassandra/blob/075ff5000ced24b42f3b540815cae471bee4049d/src/java/org/apache/cassandra/utils/concurrent/Ref.java#L187
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623168#comment-14623168
]
Joshua McKenzie commented on CASSANDRA-6477:
Thus far, the implementation looks pretty solid. Fairly self-documenting
(pending some of the naming issues we discussed above). I have more code to go
through but have some more feedback.
Questions:
# Currently we're submitting the MaterializedViewBuilder to the standard
executor in CompactionManager. Is it logical to have these 2 operations share a
thread pool and resources?
# Why the removal of updatePKIndexes from Cells.java.reconcile?
Feedback:
h6. {{General}}
* Why are concurrent_batchlog_writes and concurrent_materialized_view_writes
both hard-coded to 32 and not-in the yaml?
h6. {{AlterTableStatement}}
* mv.included.isEmpty() as a check to see if all columns are included is
counter-intuitive. Add a helper function mv.included.containsAll() (mentioned
prior, may be addressed in subsequent commits)
* announceMigration: materializedViewDrops is never initialized or used
h6. {{CreateMaterializedViewStatement}}
* in CreateMaterializedViewStatement.announceMigration, while turning
ColumnIdentifier.Raw into ColumnIdentifier, we allow <= 1 non-pk column in a MV
partition key however the error message we log on multiple attempts reads
"Cannot include non-primary key column '%s' in materialized view partition
key". We should log that <= 1 are allowed instead. We should also document why
this restriction is in-place in the code.
* refactor out duplication w/building targetPartitionKeys and
targetPartitionColumns w/nonPKTarget
h6. {{DropMaterializedViewStatement}}
* In findMaterializedView, you don't need to iterate across all the members of
cfm.getMaterializedViews().values() as it's a Map, you can just check for
whether or not it contains a member at columnFamily() index.
h6. {{SSTableIterator}}
* l252. Was this addressing a bug you uncovered during development or an
accidental change? Your update is changing what we're comparing to
indexes.size() by pre-incrementing currentIndexIdx before said comparison.
h6. {{MaterializedViewBuilder}}
* MaterializedViewBuilder.getCompactionInfo isn't giving us particularly good
information about the # Tokens built vs. total. We discussed this offline -
needs some comments in the code as to why it's currently limited in this
fashion.
h6. {{MaterializedViewUtils}}
* Should probably add a comment explaining why baseNaturalEndpoints and
viewNaturalEndpoints always have the same # of entries, so .get on baseIdx is
safe. Otherwise it takes a lot of knowledge about the MV RF implementation to
understand it (thinking about future developers here)
h6. {{AbstractReadCommandBuilder}}
* In {{.makeColumnFilter()}} - Why the change to the temporary stack ptr for
CFMetaData?
* It's not immediately clear to me why you changed from the allColumnsBuilder
to the selectionBuilder - could use some clarification on that (for me here,
not necessarily comments on code)
h6. {{ColumnFamilyStore}}
* Pull contents of initRowCache into init() -> rather than breaking into 2
separate methods, just have the 1 renamed w/MVM init in it
h6. Nits:
* SingleColumnRestriction: unnecessary whitespace changes
* AlterTableStatement:
** extraneous whitespace in announceMigration
** tab in announceMigration modified to break 120 char
* CreateMaterializedViewStatement:
** unused imports
** Double whitespace between methods
* DropMaterializedViewStatement: "Cannot drop non existing" should be "Cannot
drop non-existent"
* CompactionManager: Need space after submitMaterializedViewBuilder method close
> Materialized Views (was: Global Indexes)
>
>
> Key: CASSANDRA-6477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
> Project: Cassandra
> Issue Type: New Feature
> Components: API, Core
>Reporter: Jonathan Ellis
>Assignee: Carl Yeksigian
> Labels: cql
> Fix For: 3.0 beta 1
>
> Attachments: test-view-data.sh, users.yaml
>
>
> Local indexes are suitable for low-cardinality data, where spreading the
> index across the cluster is a Good Thing. However, for high-cardinality
> data, local indexes require querying most nodes in the cluster even if only a
> handful of rows is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623160#comment-14623160
]
Stefania commented on CASSANDRA-9686:
-
CI results for 2.1:
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9686-2.1-testall/lastCompletedBuild/testReport/
http://cassci.datastax.com/view/Dev/view/stef1927/job/stef1927-9686-2.1-dtest/lastCompletedBuild/testReport/
There seems to be a few flacky dtests so I launched a second dtest build.
> FSReadError and LEAK DETECTED after upgrading
> -
>
> Key: CASSANDRA-9686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
>Reporter: Andreas Schnitzerling
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x
>
> Attachments: cassandra.bat, cassandra.yaml,
> compactions_in_progress.zip, sstable_activity.zip, system.log
>
>
> After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and
> LEAK DETECTED on start. Deleting the listed files, the failure goes away.
> {code:title=system.log}
> ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554
> DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
> org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file
> with 0 chunks encountered: java.io.DataInputStream@1c42271
> at
> org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:117)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> ~[na:1.7.0_55]
> at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> [na:1.7.0_55]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> [na:1.7.0_55]
> at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
> Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
> java.io.DataInputStream@1c42271
> at
> org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> ... 15 common frames omitted
> ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
> was not released before the reference was garbage collected
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9765) checkForEndpointCollision fails for legitimate collisions
[ https://issues.apache.org/jira/browse/CASSANDRA-9765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefania updated CASSANDRA-9765: Reviewer: Richard Low > checkForEndpointCollision fails for legitimate collisions > - > > Key: CASSANDRA-9765 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9765 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Richard Low >Assignee: Stefania > Fix For: 2.0.17 > > > Since CASSANDRA-7939, checkForEndpointCollision no longer catches a > legitimate collision. Without CASSANDRA-7939, wiping a node and starting it > again fails with 'A node with address %s already exists', but with it the > node happily enters joining state, potentially streaming from the wrong place > and violating consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9736) Add alter statement for MV
[
https://issues.apache.org/jira/browse/CASSANDRA-9736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-9736:
---
Reviewer: Joshua McKenzie
> Add alter statement for MV
> --
>
> Key: CASSANDRA-9736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9736
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Carl Yeksigian
>
> {{ALTER MV}} would allow us to drop columns in the base table without first
> dropping the materialized views, since we'd be able to later drop columns in
> the MV.
> Also, we should be able to add new columns to the MV; a new builder would
> have to run to copy the values for these additional columns.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9729) CQLSH exception - OverflowError: normalized days too large to fit in a C int
[ https://issues.apache.org/jira/browse/CASSANDRA-9729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622881#comment-14622881 ] Chandran Anjur Narasimhan commented on CASSANDRA-9729: -- Here is the output of the command: Please not that there are some junk values in the start_time row 16 & 20. Is this is the one causing the issue? cqlsh:ccc> select time, timestampAsBlob(time), start_time, timestampAsBlob(start_time) from task_result where submissionid='40f89a3d1f4711e5ac2b005056bb0e8b'; time | timestampAsBlob(time) | start_time | timestampAsBlob(start_time) --+---+---+- 2015-06-30 09:44:23-0700 |0x014e455aed9a | 2015-06-30 09:44:23-0700 | 0x014e455aea58 2015-06-30 09:44:58-0700 |0x014e455b737a | 2015-06-30 09:44:58-0700 | 0x014e455b7310 2015-06-30 09:45:15-0700 |0x014e455bb867 | 2015-06-30 09:45:15-0700 | 0x014e455bb578 2015-06-30 09:46:02-0700 |0x014e455c6de0 | 2015-06-30 09:45:15-0700 | 0x014e455bb578 2015-06-30 09:46:03-0700 |0x014e455c7342 | 2015-06-30 09:46:03-0700 | 0x014e455c70f8 2015-06-30 09:46:18-0700 |0x014e455caf72 | 2015-06-30 09:46:03-0700 | 0x014e455c70f8 2015-06-30 09:46:23-0700 |0x014e455cc180 | 2015-06-30 09:46:19-0700 | 0x014e455caf78 2015-06-30 09:48:24-0700 |0x014e455e9847 | 2015-06-30 09:48:23-0700 | 0x014e455e965c 2015-06-30 09:48:30-0700 |0x014e455eb0d4 | 2015-06-30 09:48:24-0700 | 0x014e455e9940 2015-06-30 09:48:34-0700 |0x014e455ec04c | 2015-06-30 09:48:24-0700 | 0x014e455e9940 2015-06-30 09:48:35-0700 |0x014e455ec3fb | 2015-06-30 09:48:24-0700 | 0x014e455e9940 2015-06-30 10:06:55-0700 |0x014e456f8bee | 2015-06-30 09:48:24-0700 | 0x014e455e9940 2015-06-30 10:06:56-0700 |0x014e456f8f96 | 2015-06-30 09:48:24-0700 | 0x014e455e9940 2015-06-30 10:11:00-0700 |0x014e45734b99 | 2015-06-30 09:48:24-0700 | 0x014e455e9940 2015-06-30 10:11:02-0700 |0x014e45735155 | 2015-06-30 09:48:24-0700 | 0x014e455e9940 2015-06-30 09:49:59-0700 |0x014e45600cf0 | 266574838-06-29 18:08:16-0800 | 0x74be3c152aa4ed80 2015-06-30 09:50:01-0700 |0x014e4560146a | 2015-06-30 09:48:57-0700 | 0x014e455f18a8 2015-06-30 09:50:01-0700 |0x014e4560162a | 2015-06-30 09:48:57-0700 | 0x014e455f18a8 2015-06-30 10:09:07-0700 |0x014e4571928b | 2015-06-30 09:48:57-0700 | 0x014e455f18a8 2015-06-30 09:48:54-0700 |0x014e455f0d23 | 266574820-06-28 18:08:16-0800 | 0x74be3b90e6700980 2015-06-30 09:48:55-0700 |0x014e455f1113 | 2015-06-30 09:48:39-0700 | 0x014e455ed258 2015-06-30 09:48:55-0700 |0x014e455f12aa | 2015-06-30 09:48:39-0700 | 0x014e455ed258 2015-06-30 10:09:02-0700 |0x014e45717d12 | 2015-06-30 09:48:39-0700 | 0x014e455ed258 2015-06-30 10:09:03-0700 |0x014e45718124 | 2015-06-30 09:48:39-0700 | 0x014e455ed258 2015-06-30 10:09:07-0700 |0x014e45718f62 | 2015-06-30 09:48:39-0700 | 0x014e455ed258 (25 rows) cqlsh:ccc> > CQLSH exception - OverflowError: normalized days too large to fit in a C int > > > Key: CASSANDRA-9729 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9729 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: OSX 10.10.2 >Reporter: Chandran Anjur Narasimhan > Labels: cqlsh > > Running a select command using CQLSH 2.1.5, 2.1.7 throws exception. This > works nicely in 2.0.14 version. > Environment: > > JAVA - 1.8 > Python - 2.7.6 > Cassandra Server - 2.1.7 > CQLSH - 5.0.1 > Logs: > == > CQLSH - cassandra 2.0.14 - working with no issues > - > NCHAN-M-D0LZ:apache nchan$ cd apache-cassandra-2.0.14/ > NCHAN-M-D0LZ:apache-cassandra-2.0.14 nchan$ bin/cqlsh > Connected to CCC Multi-Region Cassandra Cluster at :9160. > [cqlsh 4.1.1 | Cassandra 2.1.7 | CQL spec 3.1.1 | Thrift protocol 19.39.0] > Use HELP for help. > cqlsh> use ccc; > cqlsh:ccc> select count(*) from task_result where > submissionid='40f89a3d1f4711e5ac2b005056bb0e8b'; > count > --- > 25 > (1 rows) > cqlsh:ccc> select * from task_result where > submissionid='40f89a3d1f4711e5ac2b00505
[jira] [Assigned] (CASSANDRA-9384) Update jBCrypt dependency to version 0.4
[ https://issues.apache.org/jira/browse/CASSANDRA-9384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marko Denda reassigned CASSANDRA-9384: -- Assignee: Marko Denda > Update jBCrypt dependency to version 0.4 > > > Key: CASSANDRA-9384 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9384 > Project: Cassandra > Issue Type: Bug >Reporter: Sam Tunnicliffe >Assignee: Marko Denda > Fix For: 2.1.x, 2.0.x, 2.2.x > > > https://bugzilla.mindrot.org/show_bug.cgi?id=2097 > Although the bug tracker lists it as NEW/OPEN, the release notes for 0.4 > indicate that this is now fixed, so we should update. > Thanks to [~Bereng] for identifying the issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9670) Cannot run CQL scripts on Windows AND having error Ubuntu Linux
[
https://issues.apache.org/jira/browse/CASSANDRA-9670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622846#comment-14622846
]
Sanjay Patel commented on CASSANDRA-9670:
-
Philip, any workaround for this ?
Thanks
> Cannot run CQL scripts on Windows AND having error Ubuntu Linux
> ---
>
> Key: CASSANDRA-9670
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9670
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: DataStax Community Edition
> on Windows 7, 64 Bit and Ubuntu
>Reporter: Sanjay Patel
>Assignee: Philip Thompson
> Labels: cqlsh
> Fix For: 2.1.x
>
> Attachments: cities.cql, germany_cities.cql, germany_cities.cql,
> india_cities.csv, india_states.csv, sp_setup.cql
>
>
> After installation of 2.1.6 and 2.1.7 it is not possible to execute cql
> scripts, which were earlier executed on windows + Linux environment
> successfully.
> I have tried to install Python 2 latest version and try to execute, but
> having same error.
> Attaching cities.cql for reference.
> ---
> {code}
> cqlsh> source 'shoppoint_setup.cql' ;
> shoppoint_setup.cql:16:InvalidRequest: code=2200 [Invalid query]
> message="Keyspace 'shopping' does not exist"
> shoppoint_setup.cql:647:'ascii' codec can't decode byte 0xc3 in position 57:
> ordinal not in range(128)
> cities.cql:9:'ascii' codec can't decode byte 0xc3 in position 51: ordinal not
> in range(128)
> cities.cql:14:
> Error starting import process:
> cities.cql:14:Can't pickle : it's not found as thread.lock
> cities.cql:14:can only join a started process
> cities.cql:16:
> Error starting import process:
> cities.cql:16:Can't pickle : it's not found as thread.lock
> cities.cql:16:can only join a started process
> Traceback (most recent call last):
> File "", line 1, in
> File "I:\programm\python2710\lib\multiprocessing\forking.py", line 380, in
> main
> prepare(preparation_data)
> File "I:\programm\python2710\lib\multiprocessing\forking.py", line 489, in
> prepare
> Traceback (most recent call last):
> File "", line 1, in
> file, path_name, etc = imp.find_module(main_name, dirs)
> ImportError: No module named cqlsh
> File "I:\programm\python2710\lib\multiprocessing\forking.py", line 380, in
> main
> prepare(preparation_data)
> File "I:\programm\python2710\lib\multiprocessing\forking.py", line 489, in
> prepare
> file, path_name, etc = imp.find_module(main_name, dirs)
> ImportError: No module named cqlsh
> shoppoint_setup.cql:663:'ascii' codec can't decode byte 0xc3 in position 18:
> ordinal not in range(128)
> ipcache.cql:28:ServerError: message="java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: java.io.FileNotFoundException:
> I:\var\lib\cassandra\data\syste
> m\schema_columns-296e9c049bec3085827dc17d3df2122a\system-schema_columns-ka-300-Data.db
> (The process cannot access the file because it is being used by another
> process)">
> ccavn_bulkupdate.cql:75:ServerError: message="java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> java.lang.RuntimeException: java.io.FileNotFoundException:
> I:\var\lib\cassandra\d
> ata\system\schema_columns-296e9c049bec3085827dc17d3df2122a\system-schema_columns-tmplink-ka-339-Data.db
> (The process cannot access the file because it is being used by another
> process)">
> shoppoint_setup.cql:680:'ascii' codec can't decode byte 0xe2 in position 14:
> ordinal not in range(128){code}
> -
> In one of Ubuntu development environment we have similar errors.
> -
> {code}
> shoppoint_setup.cql:647:'ascii' codec can't decode byte 0xc3 in position 57:
> ordinal not in range(128)
> cities.cql:9:'ascii' codec can't decode byte 0xc3 in position 51: ordinal not
> in range(128)
> (corresponding line) COPY cities (city,country_code,state,isactive) FROM
> 'testdata/india_cities.csv' ;
> [19:53:18] j.basu: shoppoint_setup.cql:663:'ascii' codec can't decode byte
> 0xc3 in position 18: ordinal not in range(128)
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622833#comment-14622833 ] Alan Boudreault commented on CASSANDRA-6477: Using Carl's branch 6477-rebase > Materialized Views (was: Global Indexes) > > > Key: CASSANDRA-6477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Jonathan Ellis >Assignee: Carl Yeksigian > Labels: cql > Fix For: 3.0 beta 1 > > Attachments: test-view-data.sh, users.yaml > > > Local indexes are suitable for low-cardinality data, where spreading the > index across the cluster is a Good Thing. However, for high-cardinality > data, local indexes require querying most nodes in the cluster even if only a > handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9666) Provide an alternative to DTCS
[ https://issues.apache.org/jira/browse/CASSANDRA-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622822#comment-14622822 ] Robbie Strickland commented on CASSANDRA-9666: -- I'd like to second the changes [~krummas] suggested at the very least, as I agree that it's a saner scheme than tiers. DTCS basically never stops compacting under "normal" conditions, and for most use cases there's little real benefit in compacting older data into larger sstables. > Provide an alternative to DTCS > -- > > Key: CASSANDRA-9666 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9666 > Project: Cassandra > Issue Type: Improvement >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Fix For: 2.1.x, 2.2.x > > > DTCS is great for time series data, but it comes with caveats that make it > difficult to use in production (typical operator behaviors such as bootstrap, > removenode, and repair have MAJOR caveats as they relate to > max_sstable_age_days, and hints/read repair break the selection algorithm). > I'm proposing an alternative, TimeWindowCompactionStrategy, that sacrifices > the tiered nature of DTCS in order to address some of DTCS' operational > shortcomings. I believe it is necessary to propose an alternative rather than > simply adjusting DTCS, because it fundamentally removes the tiered nature in > order to remove the parameter max_sstable_age_days - the result is very very > different, even if it is heavily inspired by DTCS. > Specifically, rather than creating a number of windows of ever increasing > sizes, this strategy allows an operator to choose the window size, compact > with STCS within the first window of that size, and aggressive compact down > to a single sstable once that window is no longer current. The window size is > a combination of unit (minutes, hours, days) and size (1, etc), such that an > operator can expect all data using a block of that size to be compacted > together (that is, if your unit is hours, and size is 6, you will create > roughly 4 sstables per day, each one containing roughly 6 hours of data). > The result addresses a number of the problems with > DateTieredCompactionStrategy: > - At the present time, DTCS’s first window is compacted using an unusual > selection criteria, which prefers files with earlier timestamps, but ignores > sizes. In TimeWindowCompactionStrategy, the first window data will be > compacted with the well tested, fast, reliable STCS. All STCS options can be > passed to TimeWindowCompactionStrategy to configure the first window’s > compaction behavior. > - HintedHandoff may put old data in new sstables, but it will have little > impact other than slightly reduced efficiency (sstables will cover a wider > range, but the old timestamps will not impact sstable selection criteria > during compaction) > - ReadRepair may put old data in new sstables, but it will have little impact > other than slightly reduced efficiency (sstables will cover a wider range, > but the old timestamps will not impact sstable selection criteria during > compaction) > - Small, old sstables resulting from streams of any kind will be swiftly and > aggressively compacted with the other sstables matching their similar > maxTimestamp, without causing sstables in neighboring windows to grow in size. > - The configuration options are explicit and straightforward - the tuning > parameters leave little room for error. The window is set in common, easily > understandable terms such as “12 hours”, “1 Day”, “30 days”. The > minute/hour/day options are granular enough for users keeping data for hours, > and users keeping data for years. > - There is no explicitly configurable max sstable age, though sstables will > naturally stop compacting once new data is written in that window. > - Streaming operations can create sstables with old timestamps, and they'll > naturally be joined together with sstables in the same time bucket. This is > true for bootstrap/repair/sstableloader/removenode. > - It remains true that if old data and new data is written into the memtable > at the same time, the resulting sstables will be treated as if they were new > sstables, however, that no longer negatively impacts the compaction > strategy’s selection criteria for older windows. > Patch provided for both 2.1 ( > https://github.com/jeffjirsa/cassandra/commits/twcs-2.1 ) and 2.2 ( > https://github.com/jeffjirsa/cassandra/commits/twcs ) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-6477: --- Attachment: users.yaml > Materialized Views (was: Global Indexes) > > > Key: CASSANDRA-6477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Jonathan Ellis >Assignee: Carl Yeksigian > Labels: cql > Fix For: 3.0 beta 1 > > Attachments: test-view-data.sh, users.yaml > > > Local indexes are suitable for low-cardinality data, where spreading the > index across the cluster is a Good Thing. However, for high-cardinality > data, local indexes require querying most nodes in the cluster even if only a > handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
[
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622809#comment-14622809
]
Alan Boudreault commented on CASSANDRA-6477:
Okay, I've been working on these comparisons but haven't been able to provide
useful results due to an issue I hit. I am doing my benchmarks on ec2 with a
cluster of 3 nodes. Basically, I can get realistic and useful results with C*
stock (no MV) and C* with a Secondary Index ( between 7 and 85000 op/s).
When it comes to testing C* with 1 MV, I got many many WriteTimeoutExceptions
which results in a performance of 100 operations per second. I have been able
to reproduce that 100 op/s locally using a 3 nodes cluster. The issue doesn't
seem to be present when using a single node cluster.
I've profiled one of the node and it looks like most of the time is spend in
io.netty.channel.epoll.EpollEventLoop.epollWait() (like 75% of the time).
Here's a yourkit snapshot of the first node of the cluster.
http://dl.alanb.ca/CassandraDaemon-cluster-3-nodes-2015-07-10.snapshot.zip
I've attached my users.yaml profile that I am using for testing: [^users.yaml]
Here's the materialized view creation statement:
{code}
CREATE MATERIALIZED VIEW perftesting.users_by_first_name AS SELECT * FROM
perftesting.users PRIMARY KEY (first_name);
{code}
Here's the stress command I've been using:
{code}
cassandra-stress user profile=/path/to/users.yaml ops\(insert=1\) n=500
no-warmup -pop seq=1..200M no-wrap -rate threads=200 -node
127.0.0.1,127.0.0.2,127.0.0.3
{code}
Let me know if I am doing anything wrong or if I can provide anything else to
help. I'll provide the benchmarks as soon as I have a workaround for this issue.
> Materialized Views (was: Global Indexes)
>
>
> Key: CASSANDRA-6477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
> Project: Cassandra
> Issue Type: New Feature
> Components: API, Core
>Reporter: Jonathan Ellis
>Assignee: Carl Yeksigian
> Labels: cql
> Fix For: 3.0 beta 1
>
> Attachments: test-view-data.sh
>
>
> Local indexes are suitable for low-cardinality data, where spreading the
> index across the cluster is a Good Thing. However, for high-cardinality
> data, local indexes require querying most nodes in the cluster even if only a
> handful of rows is returned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: Remove dead iSchemaKeyspace/LegacySchemaMigrator code
Repository: cassandra
Updated Branches:
refs/heads/trunk 16044a6f4 -> c734cb8b6
Remove dead iSchemaKeyspace/LegacySchemaMigrator code
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c734cb8b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c734cb8b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c734cb8b
Branch: refs/heads/trunk
Commit: c734cb8b60c9bc96303d0cf5b77a7eabec5a49e4
Parents: 16044a6
Author: Aleksey Yeschenko
Authored: Fri Jul 10 21:50:16 2015 +0300
Committer: Aleksey Yeschenko
Committed: Fri Jul 10 22:16:46 2015 +0300
--
.../org/apache/cassandra/db/SystemKeyspace.java | 1 -
.../cassandra/schema/LegacySchemaMigrator.java | 7 +-
.../apache/cassandra/schema/SchemaKeyspace.java | 89 ++--
.../schema/LegacySchemaMigratorTest.java| 2 -
4 files changed, 6 insertions(+), 93 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c734cb8b/src/java/org/apache/cassandra/db/SystemKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index f0c91d6..e8247a3 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -301,7 +301,6 @@ public final class SystemKeyspace
+ "default_time_to_live int,"
+ "default_validator text,"
+ "dropped_columns map,"
-+ "dropped_columns_types map,"
+ "gc_grace_seconds int,"
+ "is_dense boolean,"
+ "key_validator text,"
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c734cb8b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
--
diff --git a/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
index 4748820..996b5ff 100644
--- a/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
+++ b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
@@ -330,12 +330,7 @@ public final class LegacySchemaMigrator
cfm.bloomFilterFpChance(cfm.getBloomFilterFpChance());
if (tableRow.has("dropped_columns"))
-{
-Map types = tableRow.has("dropped_columns_types")
- ?
tableRow.getMap("dropped_columns_types", UTF8Type.instance, UTF8Type.instance)
- : Collections.emptyMap();
-addDroppedColumns(cfm, tableRow.getMap("dropped_columns",
UTF8Type.instance, LongType.instance), types);
-}
+addDroppedColumns(cfm, tableRow.getMap("dropped_columns",
UTF8Type.instance, LongType.instance), Collections.emptyMap());
cfm.triggers(createTriggersFromTriggerRows(triggerRows));
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c734cb8b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
index 5aad59f..8411104 100644
--- a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
+++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
@@ -1010,18 +1010,11 @@ public final class SchemaKeyspace
// We don't really use the default validator but as we have it for
backward compatibility, we use it to know if it's a counter table
AbstractType defaultValidator =
TypeParser.parse(result.getString("default_validator"));
-boolean isCounter = defaultValidator instanceof CounterColumnType;
+boolean isCounter = defaultValidator instanceof CounterColumnType;
UUID cfId = result.getUUID("cf_id");
boolean isCQLTable = !isSuper && !isDense && isCompound;
-boolean isStaticCompactTable = !isDense && !isCompound;
-
-// Internally, compact tables have a specific layout, see
CompactTables. But when upgrading from
-// previous versions, they may not have the expected schema, so detect
if we need to upgrade and do
-// it in createColumnsFromColumnRows.
-// We can remove this once we don't support upgrade from versions <
3.0.
-boolean needsUpgrade = !isCQLTable &&
checkNeedsUpgrade(serializedColumnDefinitions, isSuper, isStaticCompactTable);
List columnDefs =
createColumnsFromColumnRows(serializedColumnDefinitions,
ksName,
@@ -1029,12 +1022,7 @@ public final class SchemaKeyspace
[jira] [Commented] (CASSANDRA-9729) CQLSH exception - OverflowError: normalized days too large to fit in a C int
[
https://issues.apache.org/jira/browse/CASSANDRA-9729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622777#comment-14622777
]
Adam Holmberg commented on CASSANDRA-9729:
--
[~reach.nchan] that confirms that we are working with timestamp types, which is
what I suspected. Perhaps we could determine if you have incorrectly-encoded
timestamps as follows:
{code}
select time, timestampAsBlob(time), start_time, timestampAsBlob(start_time),
from task_result where submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
{code}
Running this in Cassandra 2.0 cqlsh should show us if the raw values are scaled
incorrectly.
If scaling is the issue, we'll need to determine if this workaround is safe and
worth putting in the new driver/cqlsh. How you handle this in the application
tier depends on what clients you have in use.
Please let me know what you find out.
> CQLSH exception - OverflowError: normalized days too large to fit in a C int
>
>
> Key: CASSANDRA-9729
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9729
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: OSX 10.10.2
>Reporter: Chandran Anjur Narasimhan
> Labels: cqlsh
>
> Running a select command using CQLSH 2.1.5, 2.1.7 throws exception. This
> works nicely in 2.0.14 version.
> Environment:
>
> JAVA - 1.8
> Python - 2.7.6
> Cassandra Server - 2.1.7
> CQLSH - 5.0.1
> Logs:
> ==
> CQLSH - cassandra 2.0.14 - working with no issues
> -
> NCHAN-M-D0LZ:apache nchan$ cd apache-cassandra-2.0.14/
> NCHAN-M-D0LZ:apache-cassandra-2.0.14 nchan$ bin/cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at :9160.
> [cqlsh 4.1.1 | Cassandra 2.1.7 | CQL spec 3.1.1 | Thrift protocol 19.39.0]
> Use HELP for help.
> cqlsh> use ccc;
> cqlsh:ccc> select count(*) from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> count
> ---
> 25
> (1 rows)
> cqlsh:ccc> select * from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> < i get all the 25 values>
> CQLSH - cassandra 2.1.5 - python exception
> -
> NCHAN-M-D0LZ:apache-cassandra-2.1.5 nchan$ bin/cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at :9042.
> [cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh> use ccc;
> cqlsh:ccc> select count(*) from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> count
> ---
> 25
> (1 rows)
> cqlsh:ccc> select * from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> Traceback (most recent call last):
> File "bin/cqlsh", line 1001, in perform_simple_statement
> rows = self.session.execute(statement, trace=self.tracing_enabled)
> File
> "/Users/nchan/Programs/apache/apache-cassandra-2.1.5/bin/../lib/cassandra-driver-internal-only-2.5.0.zip/cassandra-driver-2.5.0/cassandra/cluster.py",
> line 1404, in execute
> result = future.result(timeout)
> File
> "/Users/nchan/Programs/apache/apache-cassandra-2.1.5/bin/../lib/cassandra-driver-internal-only-2.5.0.zip/cassandra-driver-2.5.0/cassandra/cluster.py",
> line 2974, in result
> raise self._final_exception
> OverflowError: normalized days too large to fit in a C int
> cqlsh:ccc>
> CQLSH - cassandra 2.1.7 - python exception
> -
> NCHAN-M-D0LZ:apache-cassandra-2.1.7 nchan$ bin/cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at 171.71.189.11:9042.
> [cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh> use ccc;
> cqlsh:ccc> select count(*) from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> count
> ---
> 25
> (1 rows)
> cqlsh:ccc> select * from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> Traceback (most recent call last):
> File "bin/cqlsh", line 1041, in perform_simple_statement
> rows = self.session.execute(statement, trace=self.tracing_enabled)
> File
> "/Users/nchan/Programs/apache/apache-cassandra-2.1.7/bin/../lib/cassandra-driver-internal-only-2.5.1.zip/cassandra-driver-2.5.1/cassandra/cluster.py",
> line 1405, in execute
> result = future.result(timeout)
> File
> "/Users/nchan/Programs/apache/apache-cassandra-2.1.7/bin/../lib/cassandra-driver-internal-only-2.5.1.zip/cassandra-driver-2.5.1/cassandra/cluster.py",
> line 2976, in result
> raise self._final_exception
> OverflowError: normalized days too large to fit in a C int
> cqlsh:ccc>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9771) Revert CASSANDRA-9542 (allow native functions in UDA)
[
https://issues.apache.org/jira/browse/CASSANDRA-9771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622713#comment-14622713
]
Aleksey Yeschenko commented on CASSANDRA-9771:
--
Aside from the typo in {{CQL.textile}}, the revert LGTM. So long as cassci is
happy.
> Revert CASSANDRA-9542 (allow native functions in UDA)
> -
>
> Key: CASSANDRA-9771
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9771
> Project: Cassandra
> Issue Type: Task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Blocker
> Fix For: 2.2.0, 3.0.0 rc1
>
>
> As [noted in this
> comment|https://issues.apache.org/jira/browse/CASSANDRA-9542?focusedCommentId=14620414&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14620414]
> of CASSANDRA-9542, we should revert it.
> Setting priority to blocker, since once 9542 gets into 2.2.0, we cannot
> revert it.
> Will provide a patch soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9777) If you have a ~/.cqlshrc and a ~/.cassandra/cqlshrc, cqlsh will overwrite the latter with the former
Jon Moses created CASSANDRA-9777:
Summary: If you have a ~/.cqlshrc and a ~/.cassandra/cqlshrc,
cqlsh will overwrite the latter with the former
Key: CASSANDRA-9777
URL: https://issues.apache.org/jira/browse/CASSANDRA-9777
Project: Cassandra
Issue Type: Bug
Reporter: Jon Moses
If you have a .cqlshrc file, and a ~/.cassandra/cqlshrc file, when you run
`cqlsh`, it will overwrite the latter with the former.
https://github.com/apache/cassandra/blob/trunk/bin/cqlsh#L202
If the 'new' path exists (~/.cassandra/cqlsh), cqlsh should either WARN or just
leave the files alone.
{noformat}
~$ cat .cqlshrc
[authentication]
~$ cat .cassandra/cqlshrc
[connection]
~$ cqlsh
~$ cat .cqlshrc
cat: .cqlshrc: No such file or directory
~$ cat .cassandra/cqlshrc
[authentication]
~$
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9771) Revert CASSANDRA-9542 (allow native functions in UDA)
[ https://issues.apache.org/jira/browse/CASSANDRA-9771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622707#comment-14622707 ] Robert Stupp commented on CASSANDRA-9771: - Reverted CASSANDRA-9542. [Branch for 2.2|https://github.com/snazy/cassandra/tree/9771-revert-9542-2.2] [Branch for trunk|https://github.com/snazy/cassandra/tree/9771-revert-9542-trunk] [2.2 testall|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-9772-revert-9542-2.2-testall/] [2.2 dtests|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-9772-revert-9542-2.2-dtest/] [trunk testall|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-9772-revert-9542-trunk-testall/] [trunk dtests|http://cassci.datastax.com/view/Dev/view/snazy/job/snazy-9772-revert-9542-trunk-dtest/] (cassci should start soon) > Revert CASSANDRA-9542 (allow native functions in UDA) > - > > Key: CASSANDRA-9771 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9771 > Project: Cassandra > Issue Type: Task >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Blocker > Fix For: 2.2.0 > > > As [noted in this > comment|https://issues.apache.org/jira/browse/CASSANDRA-9542?focusedCommentId=14620414&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14620414] > of CASSANDRA-9542, we should revert it. > Setting priority to blocker, since once 9542 gets into 2.2.0, we cannot > revert it. > Will provide a patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9448) Metrics should use up to date nomenclature
[
https://issues.apache.org/jira/browse/CASSANDRA-9448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622693#comment-14622693
]
Yuki Morishita commented on CASSANDRA-9448:
---
[~stefania_alborghetti] Thanks! So it is possible to have both old and new name
for metrics.
Even though we cannot mark old ones as {{@Deprecated}} for technical reason,
this is enough for transition.
I created [pull request|https://github.com/stef1927/cassandra/pull/1] for your
branch to use new names for {{ColumnFamilyStoreMBean}}.
If that looks good, I will commit the change.
> Metrics should use up to date nomenclature
> --
>
> Key: CASSANDRA-9448
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9448
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Sam Tunnicliffe
>Assignee: Stefania
> Labels: docs-impacting, jmx
> Fix For: 3.0 beta 1
>
>
> There are a number of exposed metrics that currently are named using the old
> nomenclature of columnfamily and rows (meaning partitions).
> It would be good to audit all metrics and update any names to match what they
> actually represent; we should probably do that in a single sweep to avoid a
> confusing mixture of old and new terminology.
> As we'd need to do this in a major release, I've initially set the fixver for
> 3.0 beta1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9729) CQLSH exception - OverflowError: normalized days too large to fit in a C int
[
https://issues.apache.org/jira/browse/CASSANDRA-9729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622651#comment-14622651
]
Chandran Anjur Narasimhan commented on CASSANDRA-9729:
--
Here is the schema for this CF:
qlsh:ccc> desc COLUMNFAMILY ez_task_result;
CREATE TABLE ccc.task_result (
submissionid text,
ezid text,
name text,
time timestamp,
analyzed_index_root text,
analyzed_log_path text,
clientid text,
end_time timestamp,
jenkins_path text,
log_file_path text,
path_available boolean,
path_to_task text,
required_for_overall_status boolean,
start_time timestamp,
state text,
status text,
translated_criteria_status text,
type text,
PRIMARY KEY (submissionid, ezid, name, time)
) WITH CLUSTERING ORDER BY (ezid ASC, name ASC, time ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = 'Stores results of each task'
AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.1
AND speculative_retry = '99.0PERCENTILE';
CREATE INDEX ez_task_result_task_type ON ccc.ez_task_result (type);
let me know if you need more info
> CQLSH exception - OverflowError: normalized days too large to fit in a C int
>
>
> Key: CASSANDRA-9729
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9729
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: OSX 10.10.2
>Reporter: Chandran Anjur Narasimhan
> Labels: cqlsh
>
> Running a select command using CQLSH 2.1.5, 2.1.7 throws exception. This
> works nicely in 2.0.14 version.
> Environment:
>
> JAVA - 1.8
> Python - 2.7.6
> Cassandra Server - 2.1.7
> CQLSH - 5.0.1
> Logs:
> ==
> CQLSH - cassandra 2.0.14 - working with no issues
> -
> NCHAN-M-D0LZ:apache nchan$ cd apache-cassandra-2.0.14/
> NCHAN-M-D0LZ:apache-cassandra-2.0.14 nchan$ bin/cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at :9160.
> [cqlsh 4.1.1 | Cassandra 2.1.7 | CQL spec 3.1.1 | Thrift protocol 19.39.0]
> Use HELP for help.
> cqlsh> use ccc;
> cqlsh:ccc> select count(*) from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> count
> ---
> 25
> (1 rows)
> cqlsh:ccc> select * from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> < i get all the 25 values>
> CQLSH - cassandra 2.1.5 - python exception
> -
> NCHAN-M-D0LZ:apache-cassandra-2.1.5 nchan$ bin/cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at :9042.
> [cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh> use ccc;
> cqlsh:ccc> select count(*) from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> count
> ---
> 25
> (1 rows)
> cqlsh:ccc> select * from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> Traceback (most recent call last):
> File "bin/cqlsh", line 1001, in perform_simple_statement
> rows = self.session.execute(statement, trace=self.tracing_enabled)
> File
> "/Users/nchan/Programs/apache/apache-cassandra-2.1.5/bin/../lib/cassandra-driver-internal-only-2.5.0.zip/cassandra-driver-2.5.0/cassandra/cluster.py",
> line 1404, in execute
> result = future.result(timeout)
> File
> "/Users/nchan/Programs/apache/apache-cassandra-2.1.5/bin/../lib/cassandra-driver-internal-only-2.5.0.zip/cassandra-driver-2.5.0/cassandra/cluster.py",
> line 2974, in result
> raise self._final_exception
> OverflowError: normalized days too large to fit in a C int
> cqlsh:ccc>
> CQLSH - cassandra 2.1.7 - python exception
> -
> NCHAN-M-D0LZ:apache-cassandra-2.1.7 nchan$ bin/cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at 171.71.189.11:9042.
> [cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh> use ccc;
> cqlsh:ccc> select count(*) from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> count
> ---
> 25
> (1 rows)
> cqlsh:ccc> select * from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> Traceback (most recent call last):
> File "bin/cqlsh", line 1041, in perform_simple_statement
> rows =
[jira] [Comment Edited] (CASSANDRA-9729) CQLSH exception - OverflowError: normalized days too large to fit in a C int
[
https://issues.apache.org/jira/browse/CASSANDRA-9729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622651#comment-14622651
]
Chandran Anjur Narasimhan edited comment on CASSANDRA-9729 at 7/10/15 5:49 PM:
---
Here is the schema for this CF:
qlsh:ccc> desc COLUMNFAMILY task_result;
CREATE TABLE ccc.task_result (
submissionid text,
ezid text,
name text,
time timestamp,
analyzed_index_root text,
analyzed_log_path text,
clientid text,
end_time timestamp,
jenkins_path text,
log_file_path text,
path_available boolean,
path_to_task text,
required_for_overall_status boolean,
start_time timestamp,
state text,
status text,
translated_criteria_status text,
type text,
PRIMARY KEY (submissionid, ezid, name, time)
) WITH CLUSTERING ORDER BY (ezid ASC, name ASC, time ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = 'Stores results of each task'
AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.1
AND speculative_retry = '99.0PERCENTILE';
CREATE INDEX ez_task_result_task_type ON ccc.ez_task_result (type);
let me know if you need more info
was (Author: reach.nchan):
Here is the schema for this CF:
qlsh:ccc> desc COLUMNFAMILY ez_task_result;
CREATE TABLE ccc.task_result (
submissionid text,
ezid text,
name text,
time timestamp,
analyzed_index_root text,
analyzed_log_path text,
clientid text,
end_time timestamp,
jenkins_path text,
log_file_path text,
path_available boolean,
path_to_task text,
required_for_overall_status boolean,
start_time timestamp,
state text,
status text,
translated_criteria_status text,
type text,
PRIMARY KEY (submissionid, ezid, name, time)
) WITH CLUSTERING ORDER BY (ezid ASC, name ASC, time ASC)
AND bloom_filter_fp_chance = 0.01
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = 'Stores results of each task'
AND compaction = {'min_threshold': '4', 'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32'}
AND compression = {'sstable_compression':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.1
AND speculative_retry = '99.0PERCENTILE';
CREATE INDEX ez_task_result_task_type ON ccc.ez_task_result (type);
let me know if you need more info
> CQLSH exception - OverflowError: normalized days too large to fit in a C int
>
>
> Key: CASSANDRA-9729
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9729
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: OSX 10.10.2
>Reporter: Chandran Anjur Narasimhan
> Labels: cqlsh
>
> Running a select command using CQLSH 2.1.5, 2.1.7 throws exception. This
> works nicely in 2.0.14 version.
> Environment:
>
> JAVA - 1.8
> Python - 2.7.6
> Cassandra Server - 2.1.7
> CQLSH - 5.0.1
> Logs:
> ==
> CQLSH - cassandra 2.0.14 - working with no issues
> -
> NCHAN-M-D0LZ:apache nchan$ cd apache-cassandra-2.0.14/
> NCHAN-M-D0LZ:apache-cassandra-2.0.14 nchan$ bin/cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at :9160.
> [cqlsh 4.1.1 | Cassandra 2.1.7 | CQL spec 3.1.1 | Thrift protocol 19.39.0]
> Use HELP for help.
> cqlsh> use ccc;
> cqlsh:ccc> select count(*) from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> count
> ---
> 25
> (1 rows)
> cqlsh:ccc> select * from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
> < i get all the 25 values>
> CQLSH - cassandra 2.1.5 - python exception
> -
> NCHAN-M-D0LZ:apache-cassandra-2.1.5 nchan$ bin/cqlsh
> Connected to CCC Multi-Region Cassandra Cluster at :9042.
> [cqlsh 5.0.1 | Cassandra 2.1.7 | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh> use ccc;
> cqlsh:ccc> select count(*) from task_result where
> submissionid='40f89a3d1f4711e5ac2b005056bb0e8b';
[jira] [Commented] (CASSANDRA-9712) Refactor CFMetaData
[
https://issues.apache.org/jira/browse/CASSANDRA-9712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622601#comment-14622601
]
Aleksey Yeschenko commented on CASSANDRA-9712:
--
cassci complains, but not about anything new
(http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-9712-dtest/,
http://cassci.datastax.com/view/Dev/view/iamaleksey/job/iamaleksey-9712-testall/).
Anyway, committed to trunk as {{16044a6f4c19a899172efc8b2d0ac3e4723d4c88}},
thanks.
> Refactor CFMetaData
> ---
>
> Key: CASSANDRA-9712
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9712
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 3.x
>
>
> As part of CASSANDRA-9425 and a follow-up to CASSANDRA-9665, and a
> pre-requisite for new schema change protocol, this ticket will do the
> following
> 1. Make the triggers {{HashMap}} immutable (new {{Triggers}} class)
> 2. Allow multiple 2i definitions per column in CFMetaData
> 3.
> 4. Rename and move {{config.CFMetaData}} to {{schema.TableMetadata}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: Make CFMetaData.triggers immutable
Repository: cassandra
Updated Branches:
refs/heads/trunk a827a3717 -> 16044a6f4
Make CFMetaData.triggers immutable
patch by Aleksey Yeschenko; reviewed by Robert Stupp for CASSANDRA-9712
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/16044a6f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/16044a6f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/16044a6f
Branch: refs/heads/trunk
Commit: 16044a6f4c19a899172efc8b2d0ac3e4723d4c88
Parents: a827a37
Author: Aleksey Yeschenko
Authored: Thu Jul 2 16:46:14 2015 +0300
Committer: Aleksey Yeschenko
Committed: Fri Jul 10 20:08:59 2015 +0300
--
.../org/apache/cassandra/config/CFMetaData.java | 31 +
.../cassandra/config/TriggerDefinition.java | 69 --
.../cql3/statements/CreateTriggerStatement.java | 22 +--
.../cql3/statements/DropTriggerStatement.java | 20 ++-
.../cassandra/schema/LegacySchemaMigrator.java | 13 +-
.../apache/cassandra/schema/SchemaKeyspace.java | 51 ---
.../cassandra/schema/TriggerMetadata.java | 72 ++
.../org/apache/cassandra/schema/Triggers.java | 137 +++
.../cassandra/thrift/ThriftConversion.java | 21 +--
.../cassandra/triggers/TriggerExecutor.java | 9 +-
.../cql3/validation/operations/CreateTest.java | 46 +++
.../schema/LegacySchemaMigratorTest.java| 8 +-
.../cassandra/triggers/TriggerExecutorTest.java | 24 ++--
.../cassandra/triggers/TriggersSchemaTest.java | 26 ++--
14 files changed, 348 insertions(+), 201 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/16044a6f/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java
b/src/java/org/apache/cassandra/config/CFMetaData.java
index f3c8bc1..53d2171 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -49,6 +49,7 @@ import org.apache.cassandra.io.compress.CompressionParameters;
import org.apache.cassandra.io.compress.LZ4Compressor;
import org.apache.cassandra.io.util.DataOutputPlus;
import org.apache.cassandra.schema.SchemaKeyspace;
+import org.apache.cassandra.schema.Triggers;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.UUIDGen;
@@ -190,8 +191,8 @@ public final class CFMetaData
private volatile int memtableFlushPeriod = 0;
private volatile int defaultTimeToLive = DEFAULT_DEFAULT_TIME_TO_LIVE;
private volatile SpeculativeRetry speculativeRetry =
DEFAULT_SPECULATIVE_RETRY;
-private volatile Map droppedColumns = new
HashMap();
-private volatile Map triggers = new HashMap<>();
+private volatile Map droppedColumns = new
HashMap<>();
+private volatile Triggers triggers = Triggers.none();
private volatile boolean isPurged = false;
/*
* All CQL3 columns definition are stored in the columnMetadata map.
@@ -237,7 +238,7 @@ public final class CFMetaData
public CFMetaData defaultTimeToLive(int prop) {defaultTimeToLive = prop;
return this;}
public CFMetaData speculativeRetry(SpeculativeRetry prop)
{speculativeRetry = prop; return this;}
public CFMetaData droppedColumns(Map
cols) {droppedColumns = cols; return this;}
-public CFMetaData triggers(Map prop) {triggers
= prop; return this;}
+public CFMetaData triggers(Triggers prop) {triggers = prop; return this;}
private CFMetaData(String keyspace,
String name,
@@ -352,7 +353,7 @@ public final class CFMetaData
return CFMetaData.Builder.create(keyspace,
name).addPartitionKey("key", BytesType.instance).build();
}
-public Map getTriggers()
+public Triggers getTriggers()
{
return triggers;
}
@@ -467,7 +468,7 @@ public final class CFMetaData
.speculativeRetry(oldCFMD.speculativeRetry)
.memtableFlushPeriod(oldCFMD.memtableFlushPeriod)
.droppedColumns(new HashMap<>(oldCFMD.droppedColumns))
- .triggers(new HashMap<>(oldCFMD.triggers));
+ .triggers(oldCFMD.triggers);
}
/**
@@ -1198,24 +1199,6 @@ public final class CFMetaData
return removed;
}
-public void addTriggerDefinition(TriggerDefinition def) throws
InvalidRequestException
-{
-if (containsTriggerDefinition(def))
-throw new InvalidRequestException(
-String.format("Cannot create trigger %s, a trigger with the
same name already exists", def.name));
-triggers.put(def.name, def);
-}
-
-public boolean containsT
[jira] [Updated] (CASSANDRA-9705) Simplify some of 8099's concrete implementations
[ https://issues.apache.org/jira/browse/CASSANDRA-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-9705: Reviewer: Benedict > Simplify some of 8099's concrete implementations > > > Key: CASSANDRA-9705 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9705 > Project: Cassandra > Issue Type: Sub-task >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 3.0 beta 1 > > > As mentioned in the ticket comments, some of the concrete implementations > (for Cell, Row, Clustering, PartitionUpdate, ...) of the initial patch for > CASSANDRA-8099 are more complex than they should be (the use of flyweight is > typically probably ill-fitted), which probably has performance consequence. > This ticket is to track the refactoring/simplifying those implementation > (mainly by removing the use of flyweights and simplifying accordingly). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9705) Simplify some of 8099's concrete implementations
[
https://issues.apache.org/jira/browse/CASSANDRA-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622556#comment-14622556
]
Sylvain Lebresne commented on CASSANDRA-9705:
-
Pushed a branch for this
[here|https://github.com/pcmanus/cassandra/commits/9705]. It removes all use of
flyweights, and basically rewrite all Row, Cell and Partition implementations.
The result is admittedly a lot simpler and less error prone. I suspect it's
also faster but haven't looked at that much yet.
The patch is actually not very small because the change of implementation
allowed a lot of related simplifications. Hopefully this won't be too
disruptive on other patches, but I'd still be happy if we can get that in as
fast as possible, if only because I'd rather not spend too much time fixing
unit tests and dtests that are fixed with this or easier to.
Concretely there is 3 commits:
* the first one update the implementations of {{ClusteringPrefix}} and its
subclasses. That one is pretty simple and self-contained.
* the second one is the main meat. It rewrites most of the rest and was
unfortunately much harder to split in smaller pieces.
* the last one if kind of a follow-up: we're currently using {{LivenessInfo}}
for both {{Row}} and {{Cell}}, but after the previous patches it's barely used
by {{Cell}}. So that 3rd patch make it be only for rows, which make things
cleaner/simpler anyways.
I plan on doing another review of the whole patch on Monday and to add comments
where they are missing so there may be a few minor updates then, but it's
basically ready for review otherwise.
> Simplify some of 8099's concrete implementations
>
>
> Key: CASSANDRA-9705
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9705
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0 beta 1
>
>
> As mentioned in the ticket comments, some of the concrete implementations
> (for Cell, Row, Clustering, PartitionUpdate, ...) of the initial patch for
> CASSANDRA-8099 are more complex than they should be (the use of flyweight is
> typically probably ill-fitted), which probably has performance consequence.
> This ticket is to track the refactoring/simplifying those implementation
> (mainly by removing the use of flyweights and simplifying accordingly).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9712) Refactor CFMetaData
[
https://issues.apache.org/jira/browse/CASSANDRA-9712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622555#comment-14622555
]
Robert Stupp commented on CASSANDRA-9712:
-
ship it (unless cassci complains)
> Refactor CFMetaData
> ---
>
> Key: CASSANDRA-9712
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9712
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 3.x
>
>
> As part of CASSANDRA-9425 and a follow-up to CASSANDRA-9665, and a
> pre-requisite for new schema change protocol, this ticket will do the
> following
> 1. Make the triggers {{HashMap}} immutable (new {{Triggers}} class)
> 2. Allow multiple 2i definitions per column in CFMetaData
> 3.
> 4. Rename and move {{config.CFMetaData}} to {{schema.TableMetadata}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9774) fix sstableverify dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622523#comment-14622523
]
Benedict commented on CASSANDRA-9774:
-
I'm afraid I'm not the best person to ask about that, as I'm not familiar with
these tools. All I can tell you is that whenever you see "was not released
before the reference was garbage collected" it is a bug, not a feature :)
> fix sstableverify dtest
> ---
>
> Key: CASSANDRA-9774
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9774
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Jim Witschey
>Priority: Blocker
> Fix For: 3.0.x
>
>
> One of our dtests for {{sstableverify}}
> ({{offline_tools_test.py:TestOfflineTools.sstableverify_test}}) is failing
> hard on trunk ([cassci
> history|http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/offline_tools_test/TestOfflineTools/sstableverify_test/history/])
> The way the test works is by deleting an SSTable, then running
> {{sstableverify}} on its table. In earlier versions, it successfully detects
> this problem and outputs that it "was not released before the reference was
> garbage collected". The test no longer finds this string in the output;
> looking through the output of the test, it doesn't look like it reports any
> problems at all.
> EDIT: After digging into the C* source a bit, I may have misattributed the
> problem to {{sstableverify}}; this could be a more general memory management
> problem, as the error text expected in the dtest is emitted by part of the
> {{Ref}} implementation:
> https://github.com/apache/cassandra/blob/075ff5000ced24b42f3b540815cae471bee4049d/src/java/org/apache/cassandra/utils/concurrent/Ref.java#L187
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9765) checkForEndpointCollision fails for legitimate collisions
[ https://issues.apache.org/jira/browse/CASSANDRA-9765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622507#comment-14622507 ] Richard Low commented on CASSANDRA-9765: Yes I can. > checkForEndpointCollision fails for legitimate collisions > - > > Key: CASSANDRA-9765 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9765 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Richard Low >Assignee: Stefania > Fix For: 2.0.17 > > > Since CASSANDRA-7939, checkForEndpointCollision no longer catches a > legitimate collision. Without CASSANDRA-7939, wiping a node and starting it > again fails with 'A node with address %s already exists', but with it the > node happily enters joining state, potentially streaming from the wrong place > and violating consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9765) checkForEndpointCollision fails for legitimate collisions
[ https://issues.apache.org/jira/browse/CASSANDRA-9765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622498#comment-14622498 ] Jonathan Ellis commented on CASSANDRA-9765: --- Do you have time to review, [~rlow]? > checkForEndpointCollision fails for legitimate collisions > - > > Key: CASSANDRA-9765 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9765 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Richard Low >Assignee: Stefania > Fix For: 2.0.17 > > > Since CASSANDRA-7939, checkForEndpointCollision no longer catches a > legitimate collision. Without CASSANDRA-7939, wiping a node and starting it > again fails with 'A node with address %s already exists', but with it the > node happily enters joining state, potentially streaming from the wrong place > and violating consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9774) fix sstableverify dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Witschey updated CASSANDRA-9774:
Summary: fix sstableverify dtest (was: sstableverify doesn't detect
missing sstables on trunk)
> fix sstableverify dtest
> ---
>
> Key: CASSANDRA-9774
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9774
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Jim Witschey
>Priority: Blocker
> Fix For: 3.0.x
>
>
> One of our dtests for {{sstableverify}}
> ({{offline_tools_test.py:TestOfflineTools.sstableverify_test}}) is failing
> hard on trunk ([cassci
> history|http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/offline_tools_test/TestOfflineTools/sstableverify_test/history/])
> The way the test works is by deleting an SSTable, then running
> {{sstableverify}} on its table. In earlier versions, it successfully detects
> this problem and outputs that it "was not released before the reference was
> garbage collected". The test no longer finds this string in the output;
> looking through the output of the test, it doesn't look like it reports any
> problems at all.
> EDIT: After digging into the C* source a bit, I may have misattributed the
> problem to {{sstableverify}}; this could be a more general memory management
> problem, as the error text expected in the dtest is emitted by part of the
> {{Ref}} implementation:
> https://github.com/apache/cassandra/blob/075ff5000ced24b42f3b540815cae471bee4049d/src/java/org/apache/cassandra/utils/concurrent/Ref.java#L187
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9774) sstableverify doesn't detect missing sstables on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Witschey updated CASSANDRA-9774:
Assignee: Jim Witschey
> sstableverify doesn't detect missing sstables on trunk
> --
>
> Key: CASSANDRA-9774
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9774
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Jim Witschey
>Priority: Blocker
> Fix For: 3.0.x
>
>
> One of our dtests for {{sstableverify}}
> ({{offline_tools_test.py:TestOfflineTools.sstableverify_test}}) is failing
> hard on trunk ([cassci
> history|http://cassci.datastax.com/view/trunk/job/trunk_dtest/lastCompletedBuild/testReport/offline_tools_test/TestOfflineTools/sstableverify_test/history/])
> The way the test works is by deleting an SSTable, then running
> {{sstableverify}} on its table. In earlier versions, it successfully detects
> this problem and outputs that it "was not released before the reference was
> garbage collected". The test no longer finds this string in the output;
> looking through the output of the test, it doesn't look like it reports any
> problems at all.
> EDIT: After digging into the C* source a bit, I may have misattributed the
> problem to {{sstableverify}}; this could be a more general memory management
> problem, as the error text expected in the dtest is emitted by part of the
> {{Ref}} implementation:
> https://github.com/apache/cassandra/blob/075ff5000ced24b42f3b540815cae471bee4049d/src/java/org/apache/cassandra/utils/concurrent/Ref.java#L187
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: Fix InsertUpdateIfConditionTest
Repository: cassandra
Updated Branches:
refs/heads/trunk 7d6c876ec -> a827a3717
Fix InsertUpdateIfConditionTest
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a827a371
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a827a371
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a827a371
Branch: refs/heads/trunk
Commit: a827a37171b14ec1446196733eb941e7da42a96a
Parents: 7d6c876
Author: Aleksey Yeschenko
Authored: Fri Jul 10 18:19:35 2015 +0300
Committer: Aleksey Yeschenko
Committed: Fri Jul 10 18:19:35 2015 +0300
--
.../operations/InsertUpdateIfConditionTest.java | 30
1 file changed, 24 insertions(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a827a371/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
b/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
index 19f85bf..a289df9 100644
---
a/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
+++
b/test/unit/org/apache/cassandra/cql3/validation/operations/InsertUpdateIfConditionTest.java
@@ -23,10 +23,11 @@ import org.junit.Test;
import org.apache.cassandra.cql3.CQLTester;
import org.apache.cassandra.exceptions.InvalidRequestException;
import org.apache.cassandra.exceptions.SyntaxException;
+import org.apache.cassandra.schema.SchemaKeyspace;
+import static java.lang.String.format;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
-import static org.junit.Assert.fail;
public class InsertUpdateIfConditionTest extends CQLTester
{
@@ -769,17 +770,26 @@ public class InsertUpdateIfConditionTest extends CQLTester
// create and confirm
schemaChange("CREATE KEYSPACE IF NOT EXISTS " + keyspace + " WITH
replication = { 'class':'SimpleStrategy', 'replication_factor':1} and
durable_writes = true ");
-assertRows(execute("select durable_writes from system.schema_keyspaces
where keyspace_name = ?", keyspace), row(true));
+assertRows(execute(format("select durable_writes from %s.%s where
keyspace_name = ?",
+ SchemaKeyspace.NAME,
+ SchemaKeyspace.KEYSPACES),
+ keyspace),
+ row(true));
// unsuccessful create since it's already there, confirm settings
don't change
schemaChange("CREATE KEYSPACE IF NOT EXISTS " + keyspace + " WITH
replication = {'class':'SimpleStrategy', 'replication_factor':1} and
durable_writes = false ");
-assertRows(execute("select durable_writes from system.schema_keyspaces
where keyspace_name = ?", keyspace), row(true));
+assertRows(execute(format("select durable_writes from %s.%s where
keyspace_name = ?",
+ SchemaKeyspace.NAME,
+ SchemaKeyspace.KEYSPACES),
+ keyspace),
+ row(true));
// drop and confirm
schemaChange("DROP KEYSPACE IF EXISTS " + keyspace);
-assertEmpty(execute("select * from system.schema_keyspaces where
keyspace_name = ?", keyspace));
+assertEmpty(execute(format("select * from %s.%s where keyspace_name =
?", SchemaKeyspace.NAME, SchemaKeyspace.KEYSPACES),
+keyspace));
}
@@ -854,7 +864,11 @@ public class InsertUpdateIfConditionTest extends CQLTester
// create and confirm
execute("CREATE TYPE IF NOT EXISTS mytype (somefield int)");
-assertRows(execute("SELECT type_name from system.schema_usertypes
where keyspace_name = ? and type_name = ?", KEYSPACE, "mytype"),
+assertRows(execute(format("SELECT type_name from %s.%s where
keyspace_name = ? and type_name = ?",
+ SchemaKeyspace.NAME,
+ SchemaKeyspace.TYPES),
+ KEYSPACE,
+ "mytype"),
row("mytype"));
// unsuccessful create since it 's already there
@@ -863,6 +877,10 @@ public class InsertUpdateIfConditionTest extends CQLTester
// drop and confirm
execute("DROP TYPE IF EXISTS mytype");
-assertEmpty(execute("SELECT type_name from system.schema_usertypes
where keyspace_name = ? and type_name = ?", KEYSPACE, "mytype"));
+assertEmpty(execute(format("SELECT type_name from %s.%s where
keyspace_name = ? and type_name = ?",
+ SchemaKeyspace.NAME,
[jira] [Commented] (CASSANDRA-9519) CASSANDRA-8448 Doesn't seem to be fixed
[
https://issues.apache.org/jira/browse/CASSANDRA-9519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622321#comment-14622321
]
Sylvain Lebresne commented on CASSANDRA-9519:
-
bq. My take from the stacktrace was that DES.sortByProximityWithScore was
calling super to AES.sortByProximity which then had a problem with the array
changing while being sorted.
Well, the whole point of {{sortByProximity}} is to sort the list input in
place, so if the caller changes the input behind our back, we have a very
serious problem (and a quick scan of the call site indicates we do no such
thing). In fact, your patch break {{sortByProximity}} plain and simple since it
make it sort a local copy of the list that nobody ever gets (and the list that
should be sorted isn't).
Besides, the error message strongly suggests the problem is with the comparison
method.
bq. It would be nice if we had a test that could reproduce this so we don't
have to play guessing games
It's actually not all that hard. I've push on [my
branch|https://github.com/pcmanus/cassandra/commits/9519] a test that on my box
fail pretty reliably without the patch but haven't failed with it
> CASSANDRA-8448 Doesn't seem to be fixed
> ---
>
> Key: CASSANDRA-9519
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9519
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jeremiah Jordan
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 9519.txt
>
>
> Still seeing the "Comparison method violates its general contract!" in 2.1.5
> {code}
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.TimSort.mergeHi(TimSort.java:895) ~[na:1.8.0_45]
> at java.util.TimSort.mergeAt(TimSort.java:512) ~[na:1.8.0_45]
> at java.util.TimSort.mergeCollapse(TimSort.java:437) ~[na:1.8.0_45]
> at java.util.TimSort.sort(TimSort.java:241) ~[na:1.8.0_45]
> at java.util.Arrays.sort(Arrays.java:1512) ~[na:1.8.0_45]
> at java.util.ArrayList.sort(ArrayList.java:1454) ~[na:1.8.0_45]
> at java.util.Collections.sort(Collections.java:175) ~[na:1.8.0_45]
> at
> org.apache.cassandra.locator.AbstractEndpointSnitch.sortByProximity(AbstractEndpointSnitch.java:49)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximityWithScore(DynamicEndpointSnitch.java:158)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximityWithBadness(DynamicEndpointSnitch.java:187)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.locator.DynamicEndpointSnitch.sortByProximity(DynamicEndpointSnitch.java:152)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.service.StorageProxy.getLiveSortedEndpoints(StorageProxy.java:1530)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.service.StorageProxy.getRangeSlice(StorageProxy.java:1688)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:256)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:209)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:63)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:238)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:260)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:272)
> ~[cassandra-all-2.1.5.469.jar:2.1.5.469]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6717) Modernize schema tables
[
https://issues.apache.org/jira/browse/CASSANDRA-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622277#comment-14622277
]
Sam Tunnicliffe commented on CASSANDRA-6717:
PR updating {{auth_test.py}} [here |
https://github.com/riptano/cassandra-dtest/pull/380]
> Modernize schema tables
> ---
>
> Key: CASSANDRA-6717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6717
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Aleksey Yeschenko
> Labels: client-impacting
> Fix For: 3.0 beta 1
>
>
> There is a few problems/improvements that can be done with the way we store
> schema:
> # CASSANDRA-4988: as explained on the ticket, storing the comparator is now
> redundant (or almost, we'd need to store whether the table is COMPACT or not
> too, which we don't currently is easy and probably a good idea anyway), it
> can be entirely reconstructed from the infos in schema_columns (the same is
> true of key_validator and subcomparator, and replacing default_validator by a
> COMPACT_VALUE column in all case is relatively simple). And storing the
> comparator as an opaque string broke concurrent updates of sub-part of said
> comparator (concurrent collection addition or altering 2 separate clustering
> columns typically) so it's really worth removing it.
> # CASSANDRA-4603: it's time to get rid of those ugly json maps. I'll note
> that schema_keyspaces is a problem due to its use of COMPACT STORAGE, but I
> think we should fix it once and for-all nonetheless (see below).
> # For CASSANDRA-6382 and to allow indexing both map keys and values at the
> same time, we'd need to be able to have more than one index definition for a
> given column.
> # There is a few mismatches in table options between the one stored in the
> schema and the one used when declaring/altering a table which would be nice
> to fix. The compaction, compression and replication maps are one already
> mentioned from CASSANDRA-4603, but also for some reason
> 'dclocal_read_repair_chance' in CQL is called just 'local_read_repair_chance'
> in the schema table, and 'min/max_compaction_threshold' are column families
> option in the schema but just compaction options for CQL (which makes more
> sense).
> None of those issues are major, and we could probably deal with them
> independently but it might be simpler to just fix them all in one shot so I
> wanted to sum them all up here. In particular, the fact that
> 'schema_keyspaces' uses COMPACT STORAGE is annoying (for the replication map,
> but it may limit future stuff too) which suggest we should migrate it to a
> new, non COMPACT table. And while that's arguably a detail, it wouldn't hurt
> to rename schema_columnfamilies to schema_tables for the years to come since
> that's the prefered vernacular for CQL.
> Overall, what I would suggest is to move all schema tables to a new keyspace,
> named 'schema' for instance (or 'system_schema' but I prefer the shorter
> version), and fix all the issues above at once. Since we currently don't
> exchange schema between nodes of different versions, all we'd need to do that
> is a one shot startup migration, and overall, I think it could be simpler for
> clients to deal with one clear migration than to have to handle minor
> individual changes all over the place. I also think it's somewhat cleaner
> conceptually to have schema tables in their own keyspace since they are
> replicated through a different mechanism than other system tables.
> If we do that, we could, for instance, migrate to the following schema tables
> (details up for discussion of course):
> {noformat}
> CREATE TYPE user_type (
> name text,
> column_names list,
> column_types list
> )
> CREATE TABLE keyspaces (
> name text PRIMARY KEY,
> durable_writes boolean,
> replication map,
> user_types map
> )
> CREATE TYPE trigger_definition (
> name text,
> options map
> )
> CREATE TABLE tables (
> keyspace text,
> name text,
> id uuid,
> table_type text, // COMPACT, CQL or SUPER
> dropped_columns map,
> triggers map,
> // options
> comment text,
> compaction map,
> compression map,
> read_repair_chance double,
> dclocal_read_repair_chance double,
> gc_grace_seconds int,
> caching text,
> rows_per_partition_to_cache text,
> default_time_to_live int,
> min_index_interval int,
> max_index_interval int,
> speculative_retry text,
> populate_io_cache_on_flush boolean,
> bloom_filter_fp_chance double
> memtable_flush_period_in_ms int,
> PRIMARY KEY (keyspace, name)
> )
> CREATE TYPE index_definition (
> name text,
> index_type text,
> options map
> )
> CREATE TABLE columns (
> keyspace text,
> table text,
> name text,
> kind text, // PARTITION_KEY, CL
[jira] [Commented] (CASSANDRA-9661) Endless compaction to a tiny, tombstoned SStable
[
https://issues.apache.org/jira/browse/CASSANDRA-9661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622269#comment-14622269
]
WeiFan commented on CASSANDRA-9661:
---
Yes, we first deployed the 2.1.5 released from official page and encountered
this problem. After that we patched the code from tag
"cassandra-2.1.5-tentative" (b4fae85578b1bd31d162be9cb58b03c0be9f853f)
> Endless compaction to a tiny, tombstoned SStable
>
>
> Key: CASSANDRA-9661
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9661
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: WeiFan
>Assignee: Yuki Morishita
> Labels: compaction, dtcs
>
> We deployed a 3-nodes cluster (with 2.1.5) which worked under stable write
> requests ( about 2k wps) to a CF with DTCS, a default TTL as 43200s and
> gc_grace as 21600s. The CF contained inserted only, complete time series
> data. We found cassandra will occasionally keep writing logs like this:
> INFO [CompactionExecutor:30551] 2015-06-26 18:10:06,195
> CompactionTask.java:270 - Compacted 1 sstables to
> [/home/cassandra/workdata/data/sen_vaas_test/nodestatus-f96c7c50155811e589f69752ac9b06c7/sen_vaas_test-nodestatus-ka-2516270,].
> 449 bytes to 449 (~100% of original) in 12ms = 0.035683MB/s. 4 total
> partitions merged to 4. Partition merge counts were {1:4, }
> INFO [CompactionExecutor:30551] 2015-06-26 18:10:06,241
> CompactionTask.java:140 - Compacting
> [SSTableReader(path='/home/cassandra/workdata/data/sen_vaas_test/nodestatus-f96c7c50155811e589f69752ac9b06c7/sen_vaas_test-nodestatus-ka-2516270-Data.db')]
> INFO [CompactionExecutor:30551] 2015-06-26 18:10:06,253
> CompactionTask.java:270 - Compacted 1 sstables to
> [/home/cassandra/workdata/data/sen_vaas_test/nodestatus-f96c7c50155811e589f69752ac9b06c7/sen_vaas_test-nodestatus-ka-2516271,].
> 449 bytes to 449 (~100% of original) in 12ms = 0.035683MB/s. 4 total
> partitions merged to 4. Partition merge counts were {1:4, }
> It seems that cassandra kept doing compacting to a single SStable, serveral
> times per second, and lasted for many hours. Tons of logs were thrown and one
> CPU core exhausted during this time. The endless compacting finally end when
> another compaction started with a group of SStables (including previous one).
> All of our 3 nodes have been hit by this problem, but occurred in different
> time.
> We could not figure out how the problematic SStable come up because the log
> has wrapped around.
> We have dumped the records in the SStable and found it has the oldest data in
> our CF (again, our data was time series), and all of the record in this
> SStable have bben expired for more than 18 hours (12 hrs TTL + 6 hrs gc) so
> they should be dropped. However, c* do nothing to this SStable but compacting
> it again and again, until more SStable were out-dated enough to be considered
> for compacting together with this one by DTCS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6717) Modernize schema tables
[
https://issues.apache.org/jira/browse/CASSANDRA-6717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1469#comment-1469
]
Aleksey Yeschenko commented on CASSANDRA-6717:
--
Committed to trunk as {{7d6c876ec9f8dd143046ff49b5d61066ad5206c1}}. Fixed the
dtests broken by the patch in cassandra-dtest commit
{{d31b56075623c19e5151400f83c8ea43f986d5ea}}, with the exception of
{{auth_test.py}}.
Dtests should really switch to using python-driver metadata API directly, and
not query the internal schema tables. Will ask someone to make those changes.
> Modernize schema tables
> ---
>
> Key: CASSANDRA-6717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6717
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Sylvain Lebresne
>Assignee: Aleksey Yeschenko
> Labels: client-impacting
> Fix For: 3.0 beta 1
>
>
> There is a few problems/improvements that can be done with the way we store
> schema:
> # CASSANDRA-4988: as explained on the ticket, storing the comparator is now
> redundant (or almost, we'd need to store whether the table is COMPACT or not
> too, which we don't currently is easy and probably a good idea anyway), it
> can be entirely reconstructed from the infos in schema_columns (the same is
> true of key_validator and subcomparator, and replacing default_validator by a
> COMPACT_VALUE column in all case is relatively simple). And storing the
> comparator as an opaque string broke concurrent updates of sub-part of said
> comparator (concurrent collection addition or altering 2 separate clustering
> columns typically) so it's really worth removing it.
> # CASSANDRA-4603: it's time to get rid of those ugly json maps. I'll note
> that schema_keyspaces is a problem due to its use of COMPACT STORAGE, but I
> think we should fix it once and for-all nonetheless (see below).
> # For CASSANDRA-6382 and to allow indexing both map keys and values at the
> same time, we'd need to be able to have more than one index definition for a
> given column.
> # There is a few mismatches in table options between the one stored in the
> schema and the one used when declaring/altering a table which would be nice
> to fix. The compaction, compression and replication maps are one already
> mentioned from CASSANDRA-4603, but also for some reason
> 'dclocal_read_repair_chance' in CQL is called just 'local_read_repair_chance'
> in the schema table, and 'min/max_compaction_threshold' are column families
> option in the schema but just compaction options for CQL (which makes more
> sense).
> None of those issues are major, and we could probably deal with them
> independently but it might be simpler to just fix them all in one shot so I
> wanted to sum them all up here. In particular, the fact that
> 'schema_keyspaces' uses COMPACT STORAGE is annoying (for the replication map,
> but it may limit future stuff too) which suggest we should migrate it to a
> new, non COMPACT table. And while that's arguably a detail, it wouldn't hurt
> to rename schema_columnfamilies to schema_tables for the years to come since
> that's the prefered vernacular for CQL.
> Overall, what I would suggest is to move all schema tables to a new keyspace,
> named 'schema' for instance (or 'system_schema' but I prefer the shorter
> version), and fix all the issues above at once. Since we currently don't
> exchange schema between nodes of different versions, all we'd need to do that
> is a one shot startup migration, and overall, I think it could be simpler for
> clients to deal with one clear migration than to have to handle minor
> individual changes all over the place. I also think it's somewhat cleaner
> conceptually to have schema tables in their own keyspace since they are
> replicated through a different mechanism than other system tables.
> If we do that, we could, for instance, migrate to the following schema tables
> (details up for discussion of course):
> {noformat}
> CREATE TYPE user_type (
> name text,
> column_names list,
> column_types list
> )
> CREATE TABLE keyspaces (
> name text PRIMARY KEY,
> durable_writes boolean,
> replication map,
> user_types map
> )
> CREATE TYPE trigger_definition (
> name text,
> options map
> )
> CREATE TABLE tables (
> keyspace text,
> name text,
> id uuid,
> table_type text, // COMPACT, CQL or SUPER
> dropped_columns map,
> triggers map,
> // options
> comment text,
> compaction map,
> compression map,
> read_repair_chance double,
> dclocal_read_repair_chance double,
> gc_grace_seconds int,
> caching text,
> rows_per_partition_to_cache text,
> default_time_to_live int,
> min_index_interval int,
> max_index_interval int,
> speculative_retry text,
> populate_io_cache_on_flush boolean,
> bloom_filter
[1/4] cassandra git commit: Move schema tables to the new system_schema keyspace
Repository: cassandra
Updated Branches:
refs/heads/trunk 81ba56163 -> 7d6c876ec
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6c876e/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 93f69a9..596e463 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2471,7 +2471,7 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
public int forceKeyspaceCleanup(String keyspaceName, String...
columnFamilies) throws IOException, ExecutionException, InterruptedException
{
-if (keyspaceName.equals(SystemKeyspace.NAME))
+if (Schema.isSystemKeyspace(keyspaceName))
throw new RuntimeException("Cleanup of the system keyspace is
neither necessary nor wise");
CompactionManager.AllSSTableOpStatus status =
CompactionManager.AllSSTableOpStatus.SUCCESSFUL;
@@ -2705,7 +2705,7 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
Map snapshotMap = new HashMap<>();
for (Keyspace keyspace : Keyspace.all())
{
-if (SystemKeyspace.NAME.equals(keyspace.getName()))
+if (Schema.isSystemKeyspace(keyspace.getName()))
continue;
for (ColumnFamilyStore cfStore : keyspace.getColumnFamilyStores())
@@ -2731,7 +2731,7 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
long total = 0;
for (Keyspace keyspace : Keyspace.all())
{
-if (SystemKeyspace.NAME.equals(keyspace.getName()))
+if (Schema.isSystemKeyspace(keyspace.getName()))
continue;
for (ColumnFamilyStore cfStore : keyspace.getColumnFamilyStores())
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6c876e/src/java/org/apache/cassandra/thrift/ThriftConversion.java
--
diff --git a/src/java/org/apache/cassandra/thrift/ThriftConversion.java
b/src/java/org/apache/cassandra/thrift/ThriftConversion.java
index f60ea48..3e0c8f4 100644
--- a/src/java/org/apache/cassandra/thrift/ThriftConversion.java
+++ b/src/java/org/apache/cassandra/thrift/ThriftConversion.java
@@ -40,9 +40,9 @@ import
org.apache.cassandra.locator.AbstractReplicationStrategy;
import org.apache.cassandra.locator.LocalStrategy;
import org.apache.cassandra.schema.KeyspaceMetadata;
import org.apache.cassandra.schema.KeyspaceParams;
+import org.apache.cassandra.schema.SchemaKeyspace;
import org.apache.cassandra.schema.Tables;
import org.apache.cassandra.serializers.MarshalException;
-import org.apache.cassandra.schema.LegacySchemaTables;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.UUIDGen;
@@ -414,7 +414,7 @@ public class ThriftConversion
cols.add(convertThriftCqlRow(row));
UntypedResultSet colsRows = UntypedResultSet.create(cols);
-return LegacySchemaTables.createTableFromTableRowAndColumnRows(cfRow,
colsRows);
+return SchemaKeyspace.createTableFromTableRowAndColumnRows(cfRow,
colsRows);
}
private static Map convertThriftCqlRow(CqlRow row)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6c876e/src/java/org/apache/cassandra/thrift/ThriftValidation.java
--
diff --git a/src/java/org/apache/cassandra/thrift/ThriftValidation.java
b/src/java/org/apache/cassandra/thrift/ThriftValidation.java
index dd5bf98..13c55aa 100644
--- a/src/java/org/apache/cassandra/thrift/ThriftValidation.java
+++ b/src/java/org/apache/cassandra/thrift/ThriftValidation.java
@@ -632,8 +632,8 @@ public class ThriftValidation
public static void validateKeyspaceNotSystem(String modifiedKeyspace)
throws org.apache.cassandra.exceptions.InvalidRequestException
{
-if (modifiedKeyspace.equalsIgnoreCase(SystemKeyspace.NAME))
-throw new
org.apache.cassandra.exceptions.InvalidRequestException("system keyspace is not
user-modifiable");
+if (Schema.isSystemKeyspace(modifiedKeyspace))
+throw new
org.apache.cassandra.exceptions.InvalidRequestException(String.format("%s
keyspace is not user-modifiable", modifiedKeyspace));
}
//public static IDiskAtomFilter asIFilter(SlicePredicate sp, CFMetaData
metadata, ByteBuffer superColumn)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6c876e/src/java/org/apache/cassandra/tools/nodetool/Cleanup.java
--
diff --git a/src/java/org/apache/cassandra/tools/nodetool/Cleanup.java
b/src/java/org/apache/cassandra/tools/nodetool/Cleanup.java
[2/4] cassandra git commit: Move schema tables to the new system_schema keyspace
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6c876e/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
--
diff --git a/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
new file mode 100644
index 000..0e40ed2
--- /dev/null
+++ b/src/java/org/apache/cassandra/schema/SchemaKeyspace.java
@@ -0,0 +1,1501 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.schema;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+import java.util.*;
+import java.util.concurrent.TimeUnit;
+import java.util.function.Function;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.MapDifference;
+import com.google.common.collect.Maps;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.cassandra.cache.CachingOptions;
+import org.apache.cassandra.config.*;
+import org.apache.cassandra.cql3.ColumnIdentifier;
+import org.apache.cassandra.cql3.QueryProcessor;
+import org.apache.cassandra.cql3.UntypedResultSet;
+import org.apache.cassandra.cql3.functions.*;
+import org.apache.cassandra.db.ClusteringComparator;
+import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.marshal.*;
+import org.apache.cassandra.db.partitions.*;
+import org.apache.cassandra.db.rows.*;
+import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.io.compress.CompressionParameters;
+import org.apache.cassandra.service.StorageService;
+import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.FBUtilities;
+import org.apache.cassandra.utils.concurrent.OpOrder;
+
+import static org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal;
+import static org.apache.cassandra.utils.FBUtilities.fromJsonMap;
+import static org.apache.cassandra.utils.FBUtilities.json;
+
+/**
+ * system_schema.* tables and methods for manipulating them.
+ */
+public final class SchemaKeyspace
+{
+private SchemaKeyspace()
+{
+}
+
+private static final Logger logger =
LoggerFactory.getLogger(SchemaKeyspace.class);
+
+public static final String NAME = "system_schema";
+
+public static final String KEYSPACES = "keyspaces";
+public static final String TABLES = "tables";
+public static final String COLUMNS = "columns";
+public static final String TRIGGERS = "triggers";
+public static final String TYPES = "types";
+public static final String FUNCTIONS = "functions";
+public static final String AGGREGATES = "aggregates";
+
+public static final List ALL =
+ImmutableList.of(KEYSPACES, TABLES, COLUMNS, TRIGGERS, TYPES,
FUNCTIONS, AGGREGATES);
+
+private static final CFMetaData Keyspaces =
+compile(KEYSPACES,
+"keyspace definitions",
+"CREATE TABLE %s ("
++ "keyspace_name text,"
++ "durable_writes boolean,"
++ "replication map,"
++ "PRIMARY KEY ((keyspace_name)))");
+
+private static final CFMetaData Tables =
+compile(TABLES,
+"table definitions",
+"CREATE TABLE %s ("
++ "keyspace_name text,"
++ "table_name text,"
++ "bloom_filter_fp_chance double,"
++ "caching text,"
++ "cf_id uuid," // post-2.1 UUID cfid
++ "comment text,"
++ "compaction_strategy_class text,"
++ "compaction_strategy_options text,"
++ "comparator text,"
++ "compression_parameters text,"
++ "default_time_to_live int,"
++ "default_validator text,"
++ "dropped_columns map,"
++ "dropped_columns_types map,"
++ "gc_grace_seconds int,"
++ "is_dense boolean,"
++ "key_validator text,"
++ "local_read_repair_chance double,"
+
[3/4] cassandra git commit: Move schema tables to the new system_schema keyspace
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6c876e/src/java/org/apache/cassandra/schema/LegacySchemaTables.java
--
diff --git a/src/java/org/apache/cassandra/schema/LegacySchemaTables.java
b/src/java/org/apache/cassandra/schema/LegacySchemaTables.java
deleted file mode 100644
index c8e163c..000
--- a/src/java/org/apache/cassandra/schema/LegacySchemaTables.java
+++ /dev/null
@@ -1,1502 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.cassandra.schema;
-
-import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.security.MessageDigest;
-import java.security.NoSuchAlgorithmException;
-import java.util.*;
-import java.util.concurrent.TimeUnit;
-import java.util.function.Function;
-
-import com.google.common.collect.MapDifference;
-import com.google.common.collect.Maps;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import org.apache.cassandra.cache.CachingOptions;
-import org.apache.cassandra.config.*;
-import org.apache.cassandra.cql3.*;
-import org.apache.cassandra.cql3.functions.AbstractFunction;
-import org.apache.cassandra.cql3.functions.FunctionName;
-import org.apache.cassandra.cql3.functions.UDFunction;
-import org.apache.cassandra.cql3.functions.UDAggregate;
-import org.apache.cassandra.db.*;
-import org.apache.cassandra.db.rows.*;
-import org.apache.cassandra.db.marshal.*;
-import org.apache.cassandra.db.partitions.*;
-import org.apache.cassandra.exceptions.ConfigurationException;
-import org.apache.cassandra.exceptions.InvalidRequestException;
-import org.apache.cassandra.io.compress.CompressionParameters;
-import org.apache.cassandra.service.StorageService;
-import org.apache.cassandra.utils.ByteBufferUtil;
-import org.apache.cassandra.utils.FBUtilities;
-import org.apache.cassandra.utils.concurrent.OpOrder;
-
-import static org.apache.cassandra.cql3.QueryProcessor.executeOnceInternal;
-import static org.apache.cassandra.utils.FBUtilities.fromJsonMap;
-import static org.apache.cassandra.utils.FBUtilities.json;
-
-/** system.schema_* tables used to store keyspace/table/type attributes prior
to C* 3.0 */
-public final class LegacySchemaTables
-{
-private LegacySchemaTables()
-{
-}
-
-private static final Logger logger =
LoggerFactory.getLogger(LegacySchemaTables.class);
-
-public static final String KEYSPACES = "schema_keyspaces";
-public static final String COLUMNFAMILIES = "schema_columnfamilies";
-public static final String COLUMNS = "schema_columns";
-public static final String TRIGGERS = "schema_triggers";
-public static final String USERTYPES = "schema_usertypes";
-public static final String FUNCTIONS = "schema_functions";
-public static final String AGGREGATES = "schema_aggregates";
-
-public static final List ALL = Arrays.asList(KEYSPACES,
COLUMNFAMILIES, COLUMNS, TRIGGERS, USERTYPES, FUNCTIONS, AGGREGATES);
-
-private static final CFMetaData Keyspaces =
-compile(KEYSPACES,
-"keyspace definitions",
-"CREATE TABLE %s ("
-+ "keyspace_name text,"
-+ "durable_writes boolean,"
-+ "strategy_class text,"
-+ "strategy_options text,"
-+ "PRIMARY KEY ((keyspace_name))) "
-+ "WITH COMPACT STORAGE");
-
-private static final CFMetaData Columnfamilies =
-compile(COLUMNFAMILIES,
-"table definitions",
-"CREATE TABLE %s ("
-+ "keyspace_name text,"
-+ "columnfamily_name text,"
-+ "bloom_filter_fp_chance double,"
-+ "caching text,"
-+ "cf_id uuid," // post-2.1 UUID cfid
-+ "comment text,"
-+ "compaction_strategy_class text,"
-+ "compaction_strategy_options text,"
-+ "comparator text,"
-+ "compression_parameters text,"
-+ "default_time_to_live int,"
-+ "default_validator text,"
-+ "dropped_columns map,"
-+ "dropped_columns_types map,"
-+ "gc_grace_se
[4/4] cassandra git commit: Move schema tables to the new system_schema keyspace
Move schema tables to the new system_schema keyspace patch by Aleksey Yeschenko; reviewed by Tyler Hobbs for CASSANDRA-6717 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7d6c876e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7d6c876e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7d6c876e Branch: refs/heads/trunk Commit: 7d6c876ec9f8dd143046ff49b5d61066ad5206c1 Parents: 81ba561 Author: Aleksey Yeschenko Authored: Thu Jul 9 21:42:52 2015 +0300 Committer: Aleksey Yeschenko Committed: Fri Jul 10 15:28:37 2015 +0300 -- NEWS.txt|4 + ...-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar | Bin 2154972 -> 0 bytes ...ra-driver-core-2.2.0-rc2-SNAPSHOT-shaded.jar | Bin 0 -> 2162223 bytes ...sandra-driver-internal-only-2.6.0c2.post.zip | Bin 198346 -> 0 bytes lib/cassandra-driver-internal-only-2.6.0c2.zip | Bin 0 -> 203206 bytes pylib/cqlshlib/cql3handling.py |4 +- .../org/apache/cassandra/config/CFMetaData.java |6 +- .../org/apache/cassandra/config/Schema.java | 29 +- .../cql3/statements/AlterKeyspaceStatement.java |3 +- .../db/DefinitionsUpdateVerbHandler.java|4 +- src/java/org/apache/cassandra/db/Keyspace.java |6 +- .../db/MigrationRequestVerbHandler.java |4 +- .../org/apache/cassandra/db/ReadCommand.java|3 +- .../org/apache/cassandra/db/SystemKeyspace.java | 172 +- .../io/sstable/format/SSTableReader.java|2 +- .../org/apache/cassandra/schema/Functions.java |7 + .../cassandra/schema/LegacySchemaMigrator.java | 804 ++ .../cassandra/schema/LegacySchemaTables.java| 1502 -- .../apache/cassandra/schema/SchemaKeyspace.java | 1501 + src/java/org/apache/cassandra/schema/Types.java | 12 + .../cassandra/service/CassandraDaemon.java |9 + .../apache/cassandra/service/ClientState.java |9 +- .../cassandra/service/MigrationManager.java | 71 +- .../apache/cassandra/service/MigrationTask.java |4 +- .../apache/cassandra/service/StorageProxy.java |2 +- .../cassandra/service/StorageService.java |6 +- .../cassandra/thrift/ThriftConversion.java |4 +- .../cassandra/thrift/ThriftValidation.java |4 +- .../cassandra/tools/nodetool/Cleanup.java |6 +- .../utils/NativeSSTableLoaderClient.java| 19 +- .../unit/org/apache/cassandra/SchemaLoader.java |4 +- .../apache/cassandra/config/CFMetaDataTest.java | 10 +- .../config/LegacySchemaTablesTest.java | 153 -- .../cql3/validation/entities/UFTest.java| 16 +- .../cql3/validation/operations/AlterTest.java | 24 +- .../operations/InsertUpdateIfConditionTest.java |6 +- .../org/apache/cassandra/schema/DefsTest.java |2 +- .../schema/LegacySchemaMigratorTest.java| 549 +++ .../cassandra/schema/SchemaKeyspaceTest.java| 153 ++ .../service/StorageServiceServerTest.java |5 +- 40 files changed, 3328 insertions(+), 1791 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6c876e/NEWS.txt -- diff --git a/NEWS.txt b/NEWS.txt index 54ed7c6..ce05b92 100644 --- a/NEWS.txt +++ b/NEWS.txt @@ -21,6 +21,10 @@ Upgrading _ New SSTable version 'la' with improved bloom-filter false-positive handling compared to previous version 'ka' used in 2.2 and 2.1. Running sstableupgrade is not necessary but recommended. + - Before upgrading to 3.0, make sure that your cluster is in complete agreement + (schema versions outputted by `nodetool describecluster` are all the same). + - Schema metadata is now stored in the new `system_schema` keyspace, and + legacy `system.schema_*` tables are now gone; see CASSANDRA-6717 for details. - Pig's CassandraStorage has been removed. Use CqlNativeStorage instead. - Hadoop BulkOutputFormat and BulkRecordWriter have been removed; use CqlBulkOutputFormat and CqlBulkRecordWriter instead. http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6c876e/lib/cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar -- diff --git a/lib/cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar b/lib/cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar deleted file mode 100644 index 7d971df..000 Binary files a/lib/cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar and /dev/null differ http://git-wip-us.apache.org/repos/asf/cassandra/blob/7d6c876e/lib/cassandra-driver-core-2.2.0-rc2-SNAPSHOT-shaded.jar -- dif
[jira] [Commented] (CASSANDRA-9717) TestCommitLog segment size dtests fail on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-9717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622145#comment-14622145
]
Branimir Lambov commented on CASSANDRA-9717:
I am sorry, the seed discussion only applies to compressed versions of the
test, which {{default_segment_size_test}} isn't.
I looked at the test and am wondering why it expects an overall size of 60mb
for two segments of 32mb each? It appears to error out as it sees 64, which is
correct and as expected.
60 (or maybe 62.5 as the test appears to allow for 12 or 13 segments) is
correct for {{small_segment_size_test}}.
The compressed version of this test, however, fails as the last segment of the
log is smaller. This is also expected-- compressed segments grow as data is
added; we probably need to special-case this.
> TestCommitLog segment size dtests fail on trunk
> ---
>
> Key: CASSANDRA-9717
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9717
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Branimir Lambov
>Priority: Blocker
> Fix For: 3.0 beta 1
>
>
> The test for the commit log segment size when the specified size is 32MB. It
> fails for me locally and on on cassci. ([cassci
> link|http://cassci.datastax.com/view/trunk/job/trunk_dtest/305/testReport/commitlog_test/TestCommitLog/default_segment_size_test/])
> The command to run the test by itself is {{CASSANDRA_VERSION=git:trunk
> nosetests commitlog_test.py:TestCommitLog.default_segment_size_test}}.
> EDIT: a similar test,
> {{commitlog_test.py:TestCommitLog.small_segment_size_test}}, also fails with
> a similar error.
> The solution here may just be to change the expected size or the acceptable
> error -- the result isn't far off. I'm happy to make the dtest change if
> that's the solution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621937#comment-14621937
]
Stefania commented on CASSANDRA-9686:
-
It's [done|https://github.com/stef1927/cassandra/commits/9686-2.1].
I will post the CI results once they are available.
> FSReadError and LEAK DETECTED after upgrading
> -
>
> Key: CASSANDRA-9686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
>Reporter: Andreas Schnitzerling
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x
>
> Attachments: cassandra.bat, cassandra.yaml,
> compactions_in_progress.zip, sstable_activity.zip, system.log
>
>
> After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and
> LEAK DETECTED on start. Deleting the listed files, the failure goes away.
> {code:title=system.log}
> ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554
> DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
> org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file
> with 0 chunks encountered: java.io.DataInputStream@1c42271
> at
> org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:117)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> ~[na:1.7.0_55]
> at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> [na:1.7.0_55]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> [na:1.7.0_55]
> at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
> Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
> java.io.DataInputStream@1c42271
> at
> org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> ... 15 common frames omitted
> ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
> was not released before the reference was garbage collected
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9765) checkForEndpointCollision fails for legitimate collisions
[ https://issues.apache.org/jira/browse/CASSANDRA-9765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621927#comment-14621927 ] Stefania commented on CASSANDRA-9765: - CI results: http://cassci.datastax.com/job/stef1927-9765-2.0-testall/lastSuccessfulBuild/ http://cassci.datastax.com/job/stef1927-9765-2.0-dtest/lastSuccessfulBuild/ > checkForEndpointCollision fails for legitimate collisions > - > > Key: CASSANDRA-9765 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9765 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Richard Low >Assignee: Stefania > Fix For: 2.0.17 > > > Since CASSANDRA-7939, checkForEndpointCollision no longer catches a > legitimate collision. Without CASSANDRA-7939, wiping a node and starting it > again fails with 'A node with address %s already exists', but with it the > node happily enters joining state, potentially streaming from the wrong place > and violating consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-9686:
---
Fix Version/s: 2.1.x
> FSReadError and LEAK DETECTED after upgrading
> -
>
> Key: CASSANDRA-9686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
>Reporter: Andreas Schnitzerling
>Assignee: Stefania
> Fix For: 2.1.x, 2.2.x
>
> Attachments: cassandra.bat, cassandra.yaml,
> compactions_in_progress.zip, sstable_activity.zip, system.log
>
>
> After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and
> LEAK DETECTED on start. Deleting the listed files, the failure goes away.
> {code:title=system.log}
> ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554
> DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
> org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file
> with 0 chunks encountered: java.io.DataInputStream@1c42271
> at
> org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:117)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> ~[na:1.7.0_55]
> at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> [na:1.7.0_55]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> [na:1.7.0_55]
> at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
> Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
> java.io.DataInputStream@1c42271
> at
> org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> ... 15 common frames omitted
> ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
> was not released before the reference was garbage collected
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
[
https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621869#comment-14621869
]
Marcus Eriksson commented on CASSANDRA-9686:
this looks good to me, +1
we should push this to 2.1 as well I think, could you backport (and squash etc)?
> FSReadError and LEAK DETECTED after upgrading
> -
>
> Key: CASSANDRA-9686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
>Reporter: Andreas Schnitzerling
>Assignee: Stefania
> Fix For: 2.2.x
>
> Attachments: cassandra.bat, cassandra.yaml,
> compactions_in_progress.zip, sstable_activity.zip, system.log
>
>
> After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and
> LEAK DETECTED on start. Deleting the listed files, the failure goes away.
> {code:title=system.log}
> ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554
> DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
> org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file
> with 0 chunks encountered: java.io.DataInputStream@1c42271
> at
> org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.(CompressionMetadata.java:117)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> ~[na:1.7.0_55]
> at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> [na:1.7.0_55]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> [na:1.7.0_55]
> at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
> Caused by: java.io.IOException: Compressed file with 0 chunks encountered:
> java.io.DataInputStream@1c42271
> at
> org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
> ~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> ... 15 common frames omitted
> ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK
> DETECTED: a reference
> (org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
> was not released before the reference was garbage collected
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
