[jira] [Commented] (CASSANDRA-8610) Allow IF EXISTS for UPDATE statements
[
https://issues.apache.org/jira/browse/CASSANDRA-8610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304868#comment-14304868
]
Sylvain Lebresne commented on CASSANDRA-8610:
-
I think we just forgot about it, no other non-obvious reason as far as I
remember.
> Allow IF EXISTS for UPDATE statements
> -
>
> Key: CASSANDRA-8610
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8610
> Project: Cassandra
> Issue Type: Improvement
> Components: API
> Environment: Cassandra 2.1.2
>Reporter: DOAN DuyHai
>Priority: Minor
> Attachments: 8610.patch
>
>
> While creating a hands-on exercice for Cassandra, I was facing a quite
> annoying limitation.
> Let's take this table:
> {code:sql}
> CREATE TABLE killrchat.chat_rooms(
> room_name text,
> creation_date timestamp,
> banner text,
> creator text,
> participants set,
> PRIMARY KEY(room_name));
> {code}
> Upon a new participant joining the room, to be concurrency-proof (avoiding
> mutating the participants set if the room is deleted concurrently), I would
> like to issue this query:
> {code:sql}
> UPDATE chat_rooms SET participants = participants + {'johnny'} WHERE
> room_name = 'games' IF EXISTS;
> {code}
> Unfortunately the clause IF EXISTS is not allowed for UPDATE statements.
> Similarly I tried
> {code:sql}
> UPDATE chat_rooms SET participants = participants + {'johnny'} WHERE
> room_name = 'games' IF room_name='games';
> {code}
> It doesn't work either, it is not allowed to use one column of the primary
> key as condition column for LWT (why ? mystery).
> So far, the only work-around I found is:
> {code:sql}
> UPDATE chat_rooms SET participants = participants + {'johnny'} WHERE
> room_name = 'games' IF name='games';
> {code}
> I added an extra column called *name* which is just the duplicate of the
> partition key *room_name*. It does work but is not very elegant.
> I believe there are legit use cases for UPDATE ... IF EXISTS;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8707) Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted
[ https://issues.apache.org/jira/browse/CASSANDRA-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304901#comment-14304901 ] Marcus Eriksson commented on CASSANDRA-8707: * class comments in RefCounted and Ref out of date (we don't have any RefCounted.Impl anymore for example) * make SegmentedFile.Cleanup abstract and remove the empty tidy implementation. There is an instance created in BufferedSegmentedFile, I think it would be clearer if the empty tidy() method is implemented there. * A few unused SharedClosable imports in the *SegmentedFile files Still waiting for a simplification of the class hierarchy in SSTableReader as it is very hard to follow (though, yes, it is perhaps simpler than the linked list approach). If it is as simple as it can be, I'm almost inclined to suggest that we should stop doing early opening of compaction results as I doubt the complexity is worth it, we have had so many issues with this since we released it (quite a few caused by me trying to fix other issues with it). > Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted > > > Key: CASSANDRA-8707 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8707 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 2.1.3 > > > There are still a few bugs with resource management, especially around > SSTableReader cleanup, esp. when intermixing with compaction. This migration > should help. We can simultaneously "simplify" the logic in SSTableReader to > not track the replacement chain, only to take a new reference to each of the > underlying resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (CASSANDRA-8308) Windows: Commitlog access violations on unit tests
[
https://issues.apache.org/jira/browse/CASSANDRA-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp reopened CASSANDRA-8308:
-
I'm really sorry to reopen this one again. My last patch accidentally made
trunk-utest-win failing.
Follow-up patch is here: https://gist.github.com/snazy/2494731cb7e2ed1e2479
> Windows: Commitlog access violations on unit tests
> --
>
> Key: CASSANDRA-8308
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8308
> Project: Cassandra
> Issue Type: Bug
>Reporter: Joshua McKenzie
>Assignee: Joshua McKenzie
>Priority: Minor
> Labels: Windows
> Fix For: 3.0
>
> Attachments: 8308-post-fix.txt, 8308_v1.txt, 8308_v2.txt, 8308_v3.txt
>
>
> We have four unit tests failing on trunk on Windows, all with
> FileSystemException's related to the SchemaLoader:
> {noformat}
> [junit] Test
> org.apache.cassandra.db.compaction.DateTieredCompactionStrategyTest FAILED
> [junit] Test org.apache.cassandra.cql3.ThriftCompatibilityTest FAILED
> [junit] Test org.apache.cassandra.io.sstable.SSTableRewriterTest FAILED
> [junit] Test org.apache.cassandra.repair.LocalSyncTaskTest FAILED
> {noformat}
> Example error:
> {noformat}
> [junit] Caused by: java.nio.file.FileSystemException:
> build\test\cassandra\commitlog;0\CommitLog-5-1415908745965.log: The process
> cannot access the file because it is being used by another process.
> [junit]
> [junit] at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
> [junit] at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> [junit] at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
> [junit] at
> sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
> [junit] at
> sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
> [junit] at java.nio.file.Files.delete(Files.java:1079)
> [junit] at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:125)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8728) Unit test support for native protocol
[
https://issues.apache.org/jira/browse/CASSANDRA-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304975#comment-14304975
]
Sylvain Lebresne commented on CASSANDRA-8728:
-
I'm fine having basic unit tests for the protocol, and I understand that when
developing new features for said protocol it's frustrating to have to wait for
drivers to catch up to actually test the feature, but I think we should also
balance how much efforts is worth putting in this.
Typically, emitting notifications might prove hard to test with a single node
unit test, and imo it's fine leaving that to dtests (as it's a multi-node thing
really). I'm also not entire sure that the concept of writing bytes manually in
the tests (your {{NativeFrame}}) is worth the effort (except maybe for testing
corrupted frames, but tbh it's not like the protocol is terribly resilient to
that so there is so much we can test on that front). As we already have code to
serialize and deserialize frames in the code, let's maybe start by simple unit
tests that validate that by serializing and deserializing a given message we
get back what we're supposed to.
Similarly, for concurrent access, it's probably not too hard to create a couple
ad-hoc concurrent tests, but I wouldn't add too much complexity for that when
it's already fairly heavily tested through drivers (by stress for instance).
> Unit test support for native protocol
> -
>
> Key: CASSANDRA-8728
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8728
> Project: Cassandra
> Issue Type: Improvement
> Components: Tests
>Reporter: Robert Stupp
>Priority: Minor
>
> Currently we do not have any chance to directly test the native protocol „on
> the wire“. Especially when coding new protocol features, it’s more a ”blind
> flight” and basically ”hope” that the code is correct.
> Purpose of this ticket is to provide unit test code that allows testing of
> native protocol as is. It’s purpose is not to test any CQL stuff as it.
> Since the native protocol is multiplexed and as such is intended to be used
> concurrently, unit test support should allow concurrent access to a single
> connection.
> Native protocol test code should work against a single node (for unit testing
> inside C* source tree) but also be able to handle/emit notifications.
> Test code should be very strict and fail for features/values that are not
> specified in a specific protocol version.
> Code used in the test classes should be separate to production code to be
> able to identify possible bugs in production code.
> (Following code should be considered as ”pseudo code”/idea and not as _the_
> way to go)
> Establishing a connection:
> {noformat}
> NativeConnection connection = new NativeConnection(host, port,
> minClientVersion, maxClientVersion);
> connection.setAuthentication(…);
> connection.establish(timeout);
> {noformat}
> (There could be some support to provide host+port in unit tests that start an
> "embedded" server.)
> Sending (building) frames should be possible using a single class that builds
> the frame’s byte buffer like this. Additionally it should be able to
> construct corrupt/invalid frames to check failure-resistance of the server.
> {noformat}
> NativeStream stream = connection.newStream(); // create new stream
> NativeFrame frame = new NativeFrame(version, flags, stream.getId(), opcode);
> frame.addLong(longValue);
> frame.addStringList(str1, str2, …);
> nativeConnection.send(frame);
> {noformat}
> Stream handling:
> Each frame received for a stream goes into the stream’s received-frame-queue
> and can be polled from it.
> {noformat}
> NativeStream stream = connection.getEventStream(); // or getStream(-1);
> NativeFrame frame = stream.poll(); // get next frame
> {noformat}
> Native protocol events:
> {noformat}
> NativeStream stream = connection.getEventStream(); // or getStream(-1);
> NativeFrame frame = stream.poll(); // get next frame
> // OR
> NativeFrame frame = stream.pollForOpcode(opcode); // get frame with specific
> opcode
> // OR
> NativeFrame frame = stream.pollForEvent(eventCode); // get frame with
> specific event code
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8728) Unit test support for native protocol
[
https://issues.apache.org/jira/browse/CASSANDRA-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304983#comment-14304983
]
Robert Stupp commented on CASSANDRA-8728:
-
Right - might be a bit over-engineered regarding corrupted streams and
notifications.
The actual point I started thinking about was: how can I ensure/test that a
error code new to v4 is not emitted to v2/v3 clients. First problem was how to
initiate a native protocol connection and second how to check the received
frames.
Maybe some combination of some very rudimental "client code" (just able to
establish the TCP connection and basic handshaking) plus the already existing
code could work for a start.
WDYT?
> Unit test support for native protocol
> -
>
> Key: CASSANDRA-8728
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8728
> Project: Cassandra
> Issue Type: Improvement
> Components: Tests
>Reporter: Robert Stupp
>Priority: Minor
>
> Currently we do not have any chance to directly test the native protocol „on
> the wire“. Especially when coding new protocol features, it’s more a ”blind
> flight” and basically ”hope” that the code is correct.
> Purpose of this ticket is to provide unit test code that allows testing of
> native protocol as is. It’s purpose is not to test any CQL stuff as it.
> Since the native protocol is multiplexed and as such is intended to be used
> concurrently, unit test support should allow concurrent access to a single
> connection.
> Native protocol test code should work against a single node (for unit testing
> inside C* source tree) but also be able to handle/emit notifications.
> Test code should be very strict and fail for features/values that are not
> specified in a specific protocol version.
> Code used in the test classes should be separate to production code to be
> able to identify possible bugs in production code.
> (Following code should be considered as ”pseudo code”/idea and not as _the_
> way to go)
> Establishing a connection:
> {noformat}
> NativeConnection connection = new NativeConnection(host, port,
> minClientVersion, maxClientVersion);
> connection.setAuthentication(…);
> connection.establish(timeout);
> {noformat}
> (There could be some support to provide host+port in unit tests that start an
> "embedded" server.)
> Sending (building) frames should be possible using a single class that builds
> the frame’s byte buffer like this. Additionally it should be able to
> construct corrupt/invalid frames to check failure-resistance of the server.
> {noformat}
> NativeStream stream = connection.newStream(); // create new stream
> NativeFrame frame = new NativeFrame(version, flags, stream.getId(), opcode);
> frame.addLong(longValue);
> frame.addStringList(str1, str2, …);
> nativeConnection.send(frame);
> {noformat}
> Stream handling:
> Each frame received for a stream goes into the stream’s received-frame-queue
> and can be polled from it.
> {noformat}
> NativeStream stream = connection.getEventStream(); // or getStream(-1);
> NativeFrame frame = stream.poll(); // get next frame
> {noformat}
> Native protocol events:
> {noformat}
> NativeStream stream = connection.getEventStream(); // or getStream(-1);
> NativeFrame frame = stream.poll(); // get next frame
> // OR
> NativeFrame frame = stream.pollForOpcode(opcode); // get frame with specific
> opcode
> // OR
> NativeFrame frame = stream.pollForEvent(eventCode); // get frame with
> specific event code
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8728) Unit test support for native protocol
[
https://issues.apache.org/jira/browse/CASSANDRA-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304989#comment-14304989
]
Sylvain Lebresne commented on CASSANDRA-8728:
-
bq. Maybe some combination of some very rudimental "client code" (just able to
establish the TCP connection and basic handshaking) plus the already existing
code could work for a start
Yes, and there is already some rudimental "client code" in
{{org.apache.cassandra.transport.SimpleClient}} for the purpose of testing. We
just never wrote unit tests that uses it.
> Unit test support for native protocol
> -
>
> Key: CASSANDRA-8728
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8728
> Project: Cassandra
> Issue Type: Improvement
> Components: Tests
>Reporter: Robert Stupp
>Priority: Minor
>
> Currently we do not have any chance to directly test the native protocol „on
> the wire“. Especially when coding new protocol features, it’s more a ”blind
> flight” and basically ”hope” that the code is correct.
> Purpose of this ticket is to provide unit test code that allows testing of
> native protocol as is. It’s purpose is not to test any CQL stuff as it.
> Since the native protocol is multiplexed and as such is intended to be used
> concurrently, unit test support should allow concurrent access to a single
> connection.
> Native protocol test code should work against a single node (for unit testing
> inside C* source tree) but also be able to handle/emit notifications.
> Test code should be very strict and fail for features/values that are not
> specified in a specific protocol version.
> Code used in the test classes should be separate to production code to be
> able to identify possible bugs in production code.
> (Following code should be considered as ”pseudo code”/idea and not as _the_
> way to go)
> Establishing a connection:
> {noformat}
> NativeConnection connection = new NativeConnection(host, port,
> minClientVersion, maxClientVersion);
> connection.setAuthentication(…);
> connection.establish(timeout);
> {noformat}
> (There could be some support to provide host+port in unit tests that start an
> "embedded" server.)
> Sending (building) frames should be possible using a single class that builds
> the frame’s byte buffer like this. Additionally it should be able to
> construct corrupt/invalid frames to check failure-resistance of the server.
> {noformat}
> NativeStream stream = connection.newStream(); // create new stream
> NativeFrame frame = new NativeFrame(version, flags, stream.getId(), opcode);
> frame.addLong(longValue);
> frame.addStringList(str1, str2, …);
> nativeConnection.send(frame);
> {noformat}
> Stream handling:
> Each frame received for a stream goes into the stream’s received-frame-queue
> and can be polled from it.
> {noformat}
> NativeStream stream = connection.getEventStream(); // or getStream(-1);
> NativeFrame frame = stream.poll(); // get next frame
> {noformat}
> Native protocol events:
> {noformat}
> NativeStream stream = connection.getEventStream(); // or getStream(-1);
> NativeFrame frame = stream.poll(); // get next frame
> // OR
> NativeFrame frame = stream.pollForOpcode(opcode); // get frame with specific
> opcode
> // OR
> NativeFrame frame = stream.pollForEvent(eventCode); // get frame with
> specific event code
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8728) Unit test support for native protocol
[
https://issues.apache.org/jira/browse/CASSANDRA-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304998#comment-14304998
]
Robert Stupp commented on CASSANDRA-8728:
-
SimpleClient's a good starting point. (Didn't knew it - just saw it's actually
used by cassandra-stress - so I suppose it works fine)
> Unit test support for native protocol
> -
>
> Key: CASSANDRA-8728
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8728
> Project: Cassandra
> Issue Type: Improvement
> Components: Tests
>Reporter: Robert Stupp
>Priority: Minor
>
> Currently we do not have any chance to directly test the native protocol „on
> the wire“. Especially when coding new protocol features, it’s more a ”blind
> flight” and basically ”hope” that the code is correct.
> Purpose of this ticket is to provide unit test code that allows testing of
> native protocol as is. It’s purpose is not to test any CQL stuff as it.
> Since the native protocol is multiplexed and as such is intended to be used
> concurrently, unit test support should allow concurrent access to a single
> connection.
> Native protocol test code should work against a single node (for unit testing
> inside C* source tree) but also be able to handle/emit notifications.
> Test code should be very strict and fail for features/values that are not
> specified in a specific protocol version.
> Code used in the test classes should be separate to production code to be
> able to identify possible bugs in production code.
> (Following code should be considered as ”pseudo code”/idea and not as _the_
> way to go)
> Establishing a connection:
> {noformat}
> NativeConnection connection = new NativeConnection(host, port,
> minClientVersion, maxClientVersion);
> connection.setAuthentication(…);
> connection.establish(timeout);
> {noformat}
> (There could be some support to provide host+port in unit tests that start an
> "embedded" server.)
> Sending (building) frames should be possible using a single class that builds
> the frame’s byte buffer like this. Additionally it should be able to
> construct corrupt/invalid frames to check failure-resistance of the server.
> {noformat}
> NativeStream stream = connection.newStream(); // create new stream
> NativeFrame frame = new NativeFrame(version, flags, stream.getId(), opcode);
> frame.addLong(longValue);
> frame.addStringList(str1, str2, …);
> nativeConnection.send(frame);
> {noformat}
> Stream handling:
> Each frame received for a stream goes into the stream’s received-frame-queue
> and can be polled from it.
> {noformat}
> NativeStream stream = connection.getEventStream(); // or getStream(-1);
> NativeFrame frame = stream.poll(); // get next frame
> {noformat}
> Native protocol events:
> {noformat}
> NativeStream stream = connection.getEventStream(); // or getStream(-1);
> NativeFrame frame = stream.poll(); // get next frame
> // OR
> NativeFrame frame = stream.pollForOpcode(opcode); // get frame with specific
> opcode
> // OR
> NativeFrame frame = stream.pollForEvent(eventCode); // get frame with
> specific event code
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8729) Commitlog causes read before write when overwriting
[ https://issues.apache.org/jira/browse/CASSANDRA-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305017#comment-14305017 ] Branimir Lambov commented on CASSANDRA-8729: This particual problem is caused by segment recycling. Mixing segment reuse with memory-mapped writing appears to have been a very bad idea; removing reuse solves the problem immediately. If we insist on reuse, we need to get rid of memory-mapping and necessarily use 4k padding (padding will have a benefit without reuse as well, but not that pronounced). I'm not that sold on the benefits of recycling, though. If you delete a segment file and immediately create a new one with the same size, isn't the OS supposed to reuse the space anyway? I remember that's what they did ~15yrs ago, but things have probably changed. On the other hand I _am_ seeing quite different performance writing to memmapped vs. writing to channel (using a very thin non-compressing version of the compression path of CASSANDRA-6809 with direct buffers). Memmapped appears to allow a ~20% higher throughput on Windows. I think we should get rid of the recycling, and later do the rest of the improvements you list. > Commitlog causes read before write when overwriting > --- > > Key: CASSANDRA-8729 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8729 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg > > The memory mapped commit log implementation writes directly to the page > cache. If a page is not in the cache the kernel will read it in even though > we are going to overwrite. > The way to avoid this is to write to private memory, and then pad the write > with 0s at the end so it is page (4k) aligned before writing to a file. > The commit log would benefit from being refactored into something that looks > more like a pipeline with incoming requests receiving private memory to write > in, completed buffers being submitted to a parallelized compression/checksum > step, followed by submission to another thread for writing to a file that > preserves the order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8707) Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted
[ https://issues.apache.org/jira/browse/CASSANDRA-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305068#comment-14305068 ] Benedict commented on CASSANDRA-8707: - Well, all I will say is that I wash my hands of resource cleanup problems without some variant of this patch. It has already caught several memory leaks before even being committed and used in the wild (all unrelated to the reasons I started it). bq. Still waiting for a simplification of the class hierarchy I offered one suggestion, that I think makes it worse but does simplify it. Another is to eliminate all of the XManager classes and just have them duplicate the code for lookup in a statically referenced ConcurrentMap. I did it this way first, but found the code duplication very ugly. I'm honestly not sure what the problem is here, though, as it seems to map pretty straightforwardly onto how things work. bq. I'm almost inclined to suggest that we should stop doing early opening of compaction results We would have to also stop doing summary resampling, as that was broken prior to the introduction of the linked-list chain, and is probably still subtly broken and brittle. The only distinction with this patch would be the ability to (without caveat) merge GlobalTidy and TypeTidy, but this seems like a small win, since we could do this now, as I suggested, but would still prefer not to. > Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted > > > Key: CASSANDRA-8707 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8707 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 2.1.3 > > > There are still a few bugs with resource management, especially around > SSTableReader cleanup, esp. when intermixing with compaction. This migration > should help. We can simultaneously "simplify" the logic in SSTableReader to > not track the replacement chain, only to take a new reference to each of the > underlying resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7040) Replace read/write stage with per-disk access coordination
[ https://issues.apache.org/jira/browse/CASSANDRA-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305070#comment-14305070 ] Benedict commented on CASSANDRA-7040: - bq. How dependent on CASSANDRA-5863 is this? Very bq. I'm perplexed by the comparison to CASSANDRA-5239, since this is a replica-level features and 5239 is coordinator-level. See my final comment on that ticket [for explanation|https://issues.apache.org/jira/browse/CASSANDRA-5239?focusedCommentId=14072351&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14072351] > Replace read/write stage with per-disk access coordination > -- > > Key: CASSANDRA-7040 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7040 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > As discussed in CASSANDRA-6995, current coordination of access to disk is > suboptimal: instead of ensuring disk accesses alone are coordinated, we > instead coordinate at the level of operations that may touch the disks, > ensuring only so many are proceeding at once. As such, tuning is difficult, > and we incur unnecessary delays for operations that would not touch the > disk(s). > Ideally we would instead simply use a shared coordination primitive to gate > access to the disk when we perform a rebuffer. This work would dovetail very > nicely with any work in CASSANDRA-5863, as we could prevent any blocking or > context switching for data that we know to be cached. It also, as far as I > can tell, obviates the need for CASSANDRA-5239. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8729) Commitlog causes read before write when overwriting
[ https://issues.apache.org/jira/browse/CASSANDRA-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305095#comment-14305095 ] Ariel Weisberg commented on CASSANDRA-8729: --- There are other reasons to use private memory that maybe aren't so important. For in-memory write workloads you get outliers if you have threads write to a memory mapped files. They did tend to show up in the very long tail P99.99, P99.999. With a dedicated thread draining to the filesystem you can control how much data is buffered when the filesystem is out to lunch. If you write a quick benchmark that just spits out zeroes to a file via write vs a memory mapped file do you see a difference in throughput or CPU utilization? I am skeptical that mmap is actually much faster (or even slower!). > Commitlog causes read before write when overwriting > --- > > Key: CASSANDRA-8729 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8729 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg > > The memory mapped commit log implementation writes directly to the page > cache. If a page is not in the cache the kernel will read it in even though > we are going to overwrite. > The way to avoid this is to write to private memory, and then pad the write > with 0s at the end so it is page (4k) aligned before writing to a file. > The commit log would benefit from being refactored into something that looks > more like a pipeline with incoming requests receiving private memory to write > in, completed buffers being submitted to a parallelized compression/checksum > step, followed by submission to another thread for writing to a file that > preserves the order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8559) OOM caused by large tombstone warning.
[
https://issues.apache.org/jira/browse/CASSANDRA-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305113#comment-14305113
]
J.B. Langston commented on CASSANDRA-8559:
--
I have seen another user hit this. In this case, the log message was triggered
by a bad query and an even worse data model, but it's too easy for new users to
Cassandra to stumble across this. If someone shoots themselves in the foot, we
should try not to blow their whole leg off. So I'm +1 on having a limit to the
amount of information we log.
> OOM caused by large tombstone warning.
> --
>
> Key: CASSANDRA-8559
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8559
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: 2.0.11 / 2.1
>Reporter: Dominic Letz
> Labels: tombstone
> Fix For: 2.0.13
>
> Attachments: Selection_048.png, cassandra-2.0.11-8559.txt,
> stacktrace.log
>
>
> When running with high amount of tombstones the error message generation from
> CASSANDRA-6117 can lead to out of memory situation with the default setting.
> Attached a heapdump viewed in visualvm showing how this construct created two
> 777mb strings to print the error message for a read query and then crashed
> OOM.
> {code}
> if (respectTombstoneThresholds() && columnCounter.ignored() >
> DatabaseDescriptor.getTombstoneWarnThreshold())
> {
> StringBuilder sb = new StringBuilder();
> CellNameType type = container.metadata().comparator;
> for (ColumnSlice sl : slices)
> {
> assert sl != null;
> sb.append('[');
> sb.append(type.getString(sl.start));
> sb.append('-');
> sb.append(type.getString(sl.finish));
> sb.append(']');
> }
> logger.warn("Read {} live and {} tombstoned cells in {}.{} (see
> tombstone_warn_threshold). {} columns was requested, slices={}, delInfo={}",
> columnCounter.live(), columnCounter.ignored(),
> container.metadata().ksName, container.metadata().cfName, count, sb,
> container.deletionInfo());
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8707) Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted
[ https://issues.apache.org/jira/browse/CASSANDRA-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305114#comment-14305114 ] Marcus Eriksson commented on CASSANDRA-8707: bq. simplify by not distinguishing between the global and per-descriptor-type state. bq. eliminate all of the XManager classes. bq. merge GlobalTidy and TypeTidy lets try that? > Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted > > > Key: CASSANDRA-8707 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8707 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 2.1.3 > > > There are still a few bugs with resource management, especially around > SSTableReader cleanup, esp. when intermixing with compaction. This migration > should help. We can simultaneously "simplify" the logic in SSTableReader to > not track the replacement chain, only to take a new reference to each of the > underlying resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8707) Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted
[ https://issues.apache.org/jira/browse/CASSANDRA-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305137#comment-14305137 ] Benedict commented on CASSANDRA-8707: - Like I said, I'd _really_ rather not merge GlobalTidy and TypeTidy, but can if you're really set on it. By doing so we trade some slight class hierarchy simplcity for special casing, and the latter always leads to problems further down the line. I'll offer up a version with the XManager classes removed for now, and see where that gets us. > Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted > > > Key: CASSANDRA-8707 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8707 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 2.1.3 > > > There are still a few bugs with resource management, especially around > SSTableReader cleanup, esp. when intermixing with compaction. This migration > should help. We can simultaneously "simplify" the logic in SSTableReader to > not track the replacement chain, only to take a new reference to each of the > underlying resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8707) Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted
[ https://issues.apache.org/jira/browse/CASSANDRA-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305146#comment-14305146 ] Marcus Eriksson commented on CASSANDRA-8707: Imo "special casing" is fine if the special case is the only thing you need right now, I'd prefer making it more generic when we need to bq. I'll offer up a version with the XManager classes removed for now, sounds good > Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted > > > Key: CASSANDRA-8707 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8707 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 2.1.3 > > > There are still a few bugs with resource management, especially around > SSTableReader cleanup, esp. when intermixing with compaction. This migration > should help. We can simultaneously "simplify" the logic in SSTableReader to > not track the replacement chain, only to take a new reference to each of the > underlying resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6633) Dynamic Resize of Bloom Filters
[ https://issues.apache.org/jira/browse/CASSANDRA-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305149#comment-14305149 ] Benedict commented on CASSANDRA-6633: - bq. I don't see how you can add separate spaces to that and still have a BF. What gives you the impression bloom filters have to be stored in a uniform address space? That is variant dependent. It marginally improves the false positive rate, however it is very marginal, especially as the size grows. The advantage is we can dynamically change our false positive ratio, especially under detection of severely underfulfilling our estimated ratio (in the presence of commonly looked up records hitting it). Having completely distinct address spaces for every hash function isn't requisite, though. We can have varying combinations of overlapping functions in their own isolated uniform address spaces, so that we can mix and match to get optimal space and false positive efficiency. For reference, see the paper that the current variant we use is based upon: [http://people.seas.harvard.edu/~michaelm/postscripts/esa2006a.pdf] where it addresses the theoretical false positive ratio for "partitioned" bloom filters - which is identical as the set grows to infinity - and the false positive ratios for various combinations of bit and hash function counts. The effect is generally within 1% of the desired false positive, but for very small sets and many hash functions, it can be larger, say 10% (of the false positive). For such sets we can choose not to use this method, since they will not occupy much memory anyway. > Dynamic Resize of Bloom Filters > --- > > Key: CASSANDRA-6633 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6633 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Priority: Minor > Labels: performance > Fix For: 3.0 > > > Dynamic resizing would be useful. The simplest way to achieve this is to have > separate address spaces for each hash function, so that we may > increase/decrease accuracy by simply loading/unloading another function (we > could even do interesting stuff in future like alternating the functions we > select if we find we're getting more false positives than should be expected); > Faster loading/unloading would help this, and we could achieve this by > mmapping the bloom filter representation on systems that we can mlock. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8275) Some queries with multicolumn relation do not behave properly when secondary index is used
[
https://issues.apache.org/jira/browse/CASSANDRA-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305152#comment-14305152
]
Benjamin Lerer commented on CASSANDRA-8275:
---
{quote}In 2.0, this appears to cause a regression in the dtest
CQLTests.simple_tuple_query_test(). It throws an IRE when the query should be
supported. I'm not sure if we need CASSANDRA-8613 applied at the same time to
fix this.{quote}
I had a look at the test and the failing query is effectively caused by
CASSANDRA-8613. I am working on it and it should be ready in a couple of days.
{quote}Also in 2.0, in the dtest CQLTests.composite_index_with_pk_test(), we're
not requiring ALLOW FILTERING (where I believe we should be), causing a test to
fail.{quote}
The patch was effectively introducing a regression.
> Some queries with multicolumn relation do not behave properly when secondary
> index is used
> --
>
> Key: CASSANDRA-8275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8275
> Project: Cassandra
> Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.0, 2.1.3, 2.0.13
>
> Attachments: CASSANDRA-8275-2.0.txt, CASSANDRA-8275-2.1.txt,
> CASSANDRA-8275-trunk.txt
>
>
> In the case where we perform a select using a multicolumn relation over
> multiple columns that use a secondary index the error message returned is
> wrong.
> The following unit test can be use to reproduce the problem:
> {code}
> @Test
> public void testMultipleClusteringWithIndex() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, d int, PRIMARY KEY
> (a, b, c, d))");
> createIndex("CREATE INDEX ON %s (b)");
>
> execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 0, 0,
> 0);
> execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 0, 1,
> 0);
> execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 0, 1,
> 1);
> execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 1, 0,
> 0);
> execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 1, 1,
> 0);
> execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 1, 1,
> 1);
> assertRows(execute("SELECT * FROM %s WHERE (b) = (?)", 1),
>row(0, 1, 0, 0),
>row(0, 1, 1, 0),
>row(0, 1, 1, 1)
>);
>
> assertRows(execute("SELECT * FROM %s WHERE (b, c) = (?, ?) ALLOW
> FILTERING", 1, 1),
>row(0, 1, 1, 0),
>row(0, 1, 1, 1)
>);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8707) Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted
[ https://issues.apache.org/jira/browse/CASSANDRA-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305155#comment-14305155 ] Benedict commented on CASSANDRA-8707: - Special casing is where many of our recent bugs have come from, as far as I can tell. They interplay with each other in ways we cannot easily predict, especially when focusing on one area of the codebase. We think "it'll be fine here, as it doesn't touch anything else, and I've got a handle on what it does" - then we remember validation compaction in LCS that breaks our assumption, for instance. > Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted > > > Key: CASSANDRA-8707 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8707 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 2.1.3 > > > There are still a few bugs with resource management, especially around > SSTableReader cleanup, esp. when intermixing with compaction. This migration > should help. We can simultaneously "simplify" the logic in SSTableReader to > not track the replacement chain, only to take a new reference to each of the > underlying resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8732) Make inter-node timeouts tolerate clock skew and drift
[ https://issues.apache.org/jira/browse/CASSANDRA-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305179#comment-14305179 ] Sylvain Lebresne commented on CASSANDRA-8732: - Are we talking about the timeout after which we drop messages? Cause those don't really have to be terribly precise so I'm not entirely sure it's worth adding a whole bunch of complexity for that. Unless of course their is actual evidence that it's a problem in practice. > Make inter-node timeouts tolerate clock skew and drift > -- > > Key: CASSANDRA-8732 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8732 > Project: Cassandra > Issue Type: Improvement >Reporter: Ariel Weisberg > > Right now internode timeouts rely on currentTimeMillis() (and NTP) to make > sure that tasks don't expire before they arrive. > Every receiver needs to deduce the offset between its nanoTime and the remote > nanoTime. I don't think currentTimeMillis is a good choice because it is > designed to be manipulated by operators and NTP. I would probably be > comfortable assuming that nanoTime isn't going to move in significant ways > without something that could be classified as operator error happening. > I suspect the one timing method you can rely on being accurate is nanoTime > within a node (on average) and that a node can report on its own scheduling > jitter (on average). > Finding the offset requires knowing what the network latency is in one > direction. > One way to do this would be to periodically send a ping request which > generates a series of ping responses at fixed intervals (maybe by UDP?). The > responses should corrected for scheduling jitter since the fixed intervals > may not be exactly achieved by the sender. By measuring the time deviation > between ping responses and their expected arrival time (based on the > interval) and correcting for the remotely reported scheduling jitter, you > should be able to measure latency in one direction. > A weighted moving average (only correct for drift, not readjustment) of these > measurements would eventually converge on a close answer and would not be > impacted by outlier measurements. It may also make sense to drop the largest > N samples to improve accuracy. > One you know network latency you can add that to the timestamp of each ping > and compare to the local clock and know what the offset is. > These measurements won't calculate the offset to be too small (timeouts fire > early), but could calculate the offset to be too large (timeouts fire late). > The conditions where you the offset won't be accurate are the conditions > where you also want them firing reliably. This and bootstrapping in bad > conditions is what I am most uncertain of. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305181#comment-14305181 ] Benedict commented on CASSANDRA-6936: - bq. Maybe a time will come where comparisons are our main bottleneck but we're not there atm and future storage changes will probably impact this as well. We are there already. Speak to [~jblangs...@datastax.com], for instance, who's been working with two users recently seeing CPU costs of comparison bottleneck performance. One of these customers is seeing a blistering 4MB/s of compaction throughput with their CPUs maxed out. Comparisons are pretty much the main time sink for c* when working with clustering columns, and especially collections. The big problem fields are int, bigint and timestamp. All of these are very commonly used, and trivial to make byte-order comparable. The optimisations made a little while back had a significant impact on CPU cost of merges, and they all depend on byte-order comaprability of every clustering column on the table. For such small fields the cost of the virtual invocation is a significant percentage of the time spent since the data will generally be in cache, having just been read off disk. We can avoid multiple such virtual invocations if all of the fields are byte-order comparable. It also improves instruction cache occupancy for these common methods, since they all go through the same codepath (at the time of making those optimisations, instruction cache misses were actually a significant problem, and likely worse on a live server with a more varied workload). Future storage changes largely depend on it too for delivering the best performance, as the binary trie is likely to be the most significant win. Further CASSANDRA-8731 can perhaps exploit the nature of these fields to reduce costs of merging even further. > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305181#comment-14305181 ] Benedict edited comment on CASSANDRA-6936 at 2/4/15 2:49 PM: - bq. Maybe a time will come where comparisons are our main bottleneck but we're not there atm and future storage changes will probably impact this as well. We are there already. Speak to [~jblangs...@datastax.com] and [~jshook] for instance, who've each been working with users seeing CPU costs of comparison bottleneck performance. One of these customers is seeing a blistering 4MB/s of compaction throughput with their CPUs maxed out. The other had to stop using collections entirely. Comparisons are pretty much the main time sink for c* when working with clustering columns, and especially collections. The big problem fields are int, bigint and timestamp. All of these are very commonly used, and trivial to make byte-order comparable. The optimisations made a little while back had a significant impact on CPU cost of merges, and they all depend on byte-order comaprability of every clustering column on the table. For such small fields the cost of the virtual invocation is a significant percentage of the time spent since the data will generally be in cache, having just been read off disk. We can avoid multiple such virtual invocations if all of the fields are byte-order comparable. It also improves instruction cache occupancy for these common methods, since they all go through the same codepath (at the time of making those optimisations, instruction cache misses were actually a significant problem, and likely worse on a live server with a more varied workload). Future storage changes largely depend on it too for delivering the best performance, as the binary trie is likely to be the most significant win. Further CASSANDRA-8731 can perhaps exploit the nature of these fields to reduce costs of merging even further. That all said, CASSANDRA-8731 may well help get some of the way there by itself, depending on how things pan out. was (Author: benedict): bq. Maybe a time will come where comparisons are our main bottleneck but we're not there atm and future storage changes will probably impact this as well. We are there already. Speak to [~jblangs...@datastax.com], for instance, who's been working with two users recently seeing CPU costs of comparison bottleneck performance. One of these customers is seeing a blistering 4MB/s of compaction throughput with their CPUs maxed out. Comparisons are pretty much the main time sink for c* when working with clustering columns, and especially collections. The big problem fields are int, bigint and timestamp. All of these are very commonly used, and trivial to make byte-order comparable. The optimisations made a little while back had a significant impact on CPU cost of merges, and they all depend on byte-order comaprability of every clustering column on the table. For such small fields the cost of the virtual invocation is a significant percentage of the time spent since the data will generally be in cache, having just been read off disk. We can avoid multiple such virtual invocations if all of the fields are byte-order comparable. It also improves instruction cache occupancy for these common methods, since they all go through the same codepath (at the time of making those optimisations, instruction cache misses were actually a significant problem, and likely worse on a live server with a more varied workload). Future storage changes largely depend on it too for delivering the best performance, as the binary trie is likely to be the most significant win. Further CASSANDRA-8731 can perhaps exploit the nature of these fields to reduce costs of merging even further. > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible
[jira] [Commented] (CASSANDRA-8733) List prepend reverses item order
[
https://issues.apache.org/jira/browse/CASSANDRA-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305185#comment-14305185
]
Sylvain Lebresne commented on CASSANDRA-8733:
-
+1, for both the patch and for 2.0: it's obviously a bug (no prepend method in
the history of computing science has ever done this on purpose).
> List prepend reverses item order
>
>
> Key: CASSANDRA-8733
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8733
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
> Fix For: 2.1.3, 2.0.13
>
> Attachments: 8733-2.0.txt
>
>
> If you prepend a list of items to a list collection, it will reverse the
> order of the prepend-list:
> {noformat}
> cqlsh:ks1> UPDATE foobar SET b = [2, 3] WHERE a = 0;
> cqlsh:ks1> select * from foobar;
> a | b
> ---+
> 0 | [2, 3]
> (1 rows)
> cqlsh:ks1> UPDATE foobar SET b = [0, 1] + b WHERE a = 0;
> cqlsh:ks1> select * from foobar;
> a | b
> ---+--
> 0 | [1, 0, 2, 3]
> (1 rows)
> {noformat}
> This is because we're going through the prepend-list in a normal-order
> for-loop and prepending each time (in Lists.Prepender).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8505) Invalid results are returned while secondary index are being build
[
https://issues.apache.org/jira/browse/CASSANDRA-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-8505:
Fix Version/s: (was: 3.0)
2.1.4
> Invalid results are returned while secondary index are being build
> --
>
> Key: CASSANDRA-8505
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8505
> Project: Cassandra
> Issue Type: Bug
>Reporter: Benjamin Lerer
> Fix For: 2.1.4
>
>
> If you request an index creation and then execute a query that use the index
> the results returned might be invalid until the index is fully build. This is
> caused by the fact that the table column will be marked as indexed before the
> index is ready.
> The following unit tests can be use to reproduce the problem:
> {code}
> @Test
> public void testIndexCreatedAfterInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
>
> createIndex("CREATE INDEX ON %s(b)");
>
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
>
> @Test
> public void testIndexCreatedBeforeInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> createIndex("CREATE INDEX ON %s(b)");
>
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
> {code}
> The first test will fail while the second will work.
> In my opinion the first test should reject the request as invalid (as if the
> index was not existing) until the index is fully build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8505) Invalid results are returned while secondary index are being build
[
https://issues.apache.org/jira/browse/CASSANDRA-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305197#comment-14305197
]
Sylvain Lebresne commented on CASSANDRA-8505:
-
We probably need write the patch before deciding (in theory, it would be rather
simple to have SelectStatement check if the index is built, but in practice
doing so means a query to the a system table and that's probably not acceptable
performance, so we'd need to cache the info (probably directly in
ColumnDefinition?) but we'll have to check how hairy that is exactly). That
said, even if we decide it's too hairy for 2.0, we should at least fix it in
2.1.
> Invalid results are returned while secondary index are being build
> --
>
> Key: CASSANDRA-8505
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8505
> Project: Cassandra
> Issue Type: Bug
>Reporter: Benjamin Lerer
> Fix For: 2.1.4
>
>
> If you request an index creation and then execute a query that use the index
> the results returned might be invalid until the index is fully build. This is
> caused by the fact that the table column will be marked as indexed before the
> index is ready.
> The following unit tests can be use to reproduce the problem:
> {code}
> @Test
> public void testIndexCreatedAfterInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
>
> createIndex("CREATE INDEX ON %s(b)");
>
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
>
> @Test
> public void testIndexCreatedBeforeInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> createIndex("CREATE INDEX ON %s(b)");
>
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
> {code}
> The first test will fail while the second will work.
> In my opinion the first test should reject the request as invalid (as if the
> index was not existing) until the index is fully build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8505) Invalid results are returned while secondary index are being build
[
https://issues.apache.org/jira/browse/CASSANDRA-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-8505:
Assignee: Benjamin Lerer
> Invalid results are returned while secondary index are being build
> --
>
> Key: CASSANDRA-8505
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8505
> Project: Cassandra
> Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 2.1.4
>
>
> If you request an index creation and then execute a query that use the index
> the results returned might be invalid until the index is fully build. This is
> caused by the fact that the table column will be marked as indexed before the
> index is ready.
> The following unit tests can be use to reproduce the problem:
> {code}
> @Test
> public void testIndexCreatedAfterInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
>
> createIndex("CREATE INDEX ON %s(b)");
>
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
>
> @Test
> public void testIndexCreatedBeforeInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> createIndex("CREATE INDEX ON %s(b)");
>
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
> {code}
> The first test will fail while the second will work.
> In my opinion the first test should reject the request as invalid (as if the
> index was not existing) until the index is fully build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8502) Static columns returning null for pages after first
[
https://issues.apache.org/jira/browse/CASSANDRA-8502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305202#comment-14305202
]
Sylvain Lebresne commented on CASSANDRA-8502:
-
bq. We don't allow ORDER BY unless you restrict the partition key with EQ or IN
Oh right. That does suggest the changes in {{DataRange}} are unnecessary then.
> Static columns returning null for pages after first
> ---
>
> Key: CASSANDRA-8502
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8502
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Flavien Charlon
>Assignee: Tyler Hobbs
> Fix For: 2.1.3, 2.0.13
>
> Attachments: 8502-2.0.txt, null-static-column.txt
>
>
> When paging is used for a query containing a static column, the first page
> contains the right value for the static column, but subsequent pages have
> null null for the static column instead of the expected value.
> Repro steps:
> - Create a table with a static column
> - Create a partition with 500 cells
> - Using cqlsh, query that partition
> Actual result:
> - You will see that first, the static column appears as expected, but if you
> press a key after "---MORE---", the static columns will appear as null.
> See the attached file for a repro of the output.
> I am using a single node cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8736) core dump on creating keyspaces
mike created CASSANDRA-8736:
---
Summary: core dump on creating keyspaces
Key: CASSANDRA-8736
URL: https://issues.apache.org/jira/browse/CASSANDRA-8736
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Linux
Reporter: mike
Priority: Critical
After we upgraded C* to 2.1.2, with new installation, sometimes on creating
Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
release. Thanks.
$CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f
${dbDefaultDataDir}/${VCE_DATA_SCRIPT}” that was seen yesterday I also am
seeing below on my environment when deploying.
*** glibc detected *** python: corrupted double-linked list: 0x02dc95f0
***
=== Backtrace: =
/lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
/lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
/usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
/usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
/usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
/usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
/usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
/lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
/lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
=== Memory map:
0040-00401000 r-xp 08:06 450964
/usr/bin/python
0060-00601000 rw-p 08:06 450964
/usr/bin/python
024ba000-02f29000 rw-p 00:00 0 [heap]
7fd1b000-7fd1b0172000 rw-p 00:00 0
7fd1b0172000-7fd1b400 ---p 00:00 0
7fd1b400-7fd1b4021000 rw-p 00:00 0
7fd1b4021000-7fd1b800 ---p 00:00 0
7fd1b800-7fd1b8021000 rw-p 00:00 0
7fd1b8021000-7fd1bc00 ---p 00:00 0
7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
7fd1bd7e8000-7fd1bd7e9000 ---p 00:00 0
7fd1bd7e9000-7fd1be1e9000 rw-p 00:00 0
7fd1be1e9000-7fd1be1ed000 r-xp 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be1ed000-7fd1be3ed000 ---p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ed000-7fd1be3ef000 rw-p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ef000-7fd1be3f2000 r-xp 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be3f2000-7fd1be5f1000 ---p 3000 08:06 180869
/usr/lib64/python2.6/lib-d
[jira] [Updated] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mike updated CASSANDRA-8736:
Description:
After we upgraded C* to 2.1.2, with new installation, sometimes on creating
Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
release. Thanks.
$CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f
${dbDefaultDataDir}/${VCE_DATA_SCRIPT}” that was seen yesterday I also am
seeing below on my environment when deploying.
*** glibc detected *** python: corrupted double-linked list: 0x02dc95f0
***
=== Backtrace: =
/lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
/lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
/usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
/usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
/usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
/usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
/usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
/lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
/lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
=== Memory map:
0040-00401000 r-xp 08:06 450964
/usr/bin/python
0060-00601000 rw-p 08:06 450964
/usr/bin/python
024ba000-02f29000 rw-p 00:00 0 [heap]
7fd1b000-7fd1b0172000 rw-p 00:00 0
7fd1b0172000-7fd1b400 ---p 00:00 0
7fd1b400-7fd1b4021000 rw-p 00:00 0
7fd1b4021000-7fd1b800 ---p 00:00 0
7fd1b800-7fd1b8021000 rw-p 00:00 0
7fd1b8021000-7fd1bc00 ---p 00:00 0
7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
7fd1bd7e8000-7fd1bd7e9000 ---p 00:00 0
7fd1bd7e9000-7fd1be1e9000 rw-p 00:00 0
7fd1be1e9000-7fd1be1ed000 r-xp 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be1ed000-7fd1be3ed000 ---p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ed000-7fd1be3ef000 rw-p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ef000-7fd1be3f2000 r-xp 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be3f2000-7fd1be5f1000 ---p 3000 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be5f1000-7fd1be5f2000 rw-p 2000 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be5f2000-7fd1be5f5000 r-xp 08:06 180866
[jira] [Updated] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mike updated CASSANDRA-8736:
Description:
After we upgraded C* to 2.1.2, with new installation, sometimes on creating
Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
release. Thanks.
$CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f ${dbDefaultDataDir}/${DATA_SCRIPT}”
that was seen yesterday I also am seeing below on my environment when deploying.
*** glibc detected *** python: corrupted double-linked list: 0x02dc95f0
***
=== Backtrace: =
/lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
/lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
/usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
/usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
/usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
/usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
/usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
/lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
/lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
=== Memory map:
0040-00401000 r-xp 08:06 450964
/usr/bin/python
0060-00601000 rw-p 08:06 450964
/usr/bin/python
024ba000-02f29000 rw-p 00:00 0 [heap]
7fd1b000-7fd1b0172000 rw-p 00:00 0
7fd1b0172000-7fd1b400 ---p 00:00 0
7fd1b400-7fd1b4021000 rw-p 00:00 0
7fd1b4021000-7fd1b800 ---p 00:00 0
7fd1b800-7fd1b8021000 rw-p 00:00 0
7fd1b8021000-7fd1bc00 ---p 00:00 0
7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
7fd1bd7e8000-7fd1bd7e9000 ---p 00:00 0
7fd1bd7e9000-7fd1be1e9000 rw-p 00:00 0
7fd1be1e9000-7fd1be1ed000 r-xp 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be1ed000-7fd1be3ed000 ---p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ed000-7fd1be3ef000 rw-p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ef000-7fd1be3f2000 r-xp 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be3f2000-7fd1be5f1000 ---p 3000 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be5f1000-7fd1be5f2000 rw-p 2000 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be5f2000-7fd1be5f5000 r-xp 08:06 180866
[jira] [Updated] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mike updated CASSANDRA-8736:
Description:
After we upgraded C* to 2.1.2, with new installation, sometimes on creating
Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
release. Thanks.
$CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f ${dbDefaultDataDir}/${DATA_SCRIPT}”.
*** glibc detected *** python: corrupted double-linked list: 0x02dc95f0
***
=== Backtrace: =
/lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
/lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
/usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
/usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
/usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
/usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
/usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
/lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
/lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
=== Memory map:
0040-00401000 r-xp 08:06 450964
/usr/bin/python
0060-00601000 rw-p 08:06 450964
/usr/bin/python
024ba000-02f29000 rw-p 00:00 0 [heap]
7fd1b000-7fd1b0172000 rw-p 00:00 0
7fd1b0172000-7fd1b400 ---p 00:00 0
7fd1b400-7fd1b4021000 rw-p 00:00 0
7fd1b4021000-7fd1b800 ---p 00:00 0
7fd1b800-7fd1b8021000 rw-p 00:00 0
7fd1b8021000-7fd1bc00 ---p 00:00 0
7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
7fd1bd7e8000-7fd1bd7e9000 ---p 00:00 0
7fd1bd7e9000-7fd1be1e9000 rw-p 00:00 0
7fd1be1e9000-7fd1be1ed000 r-xp 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be1ed000-7fd1be3ed000 ---p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ed000-7fd1be3ef000 rw-p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ef000-7fd1be3f2000 r-xp 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be3f2000-7fd1be5f1000 ---p 3000 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be5f1000-7fd1be5f2000 rw-p 2000 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be5f2000-7fd1be5f5000 r-xp 08:06 180866
/usr/lib64/python2.6/lib-dynload/_hashlib.so
7fd1be5f5000-7fd1be7f4000 ---
[jira] [Commented] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305224#comment-14305224 ] Aleksey Yeschenko commented on CASSANDRA-6936: -- Are we still talking about making *all* types byte-order compatible, or just a subset of them? B/c the former is almost certainly a non-option so long as we support custom types (and unfortunately it doesn't seem like dropping that is an option). > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mike updated CASSANDRA-8736:
Description:
After we upgraded C* to 2.1.2, with new installation, sometimes on creating
Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
release. Thanks.
$CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f ${dbDefaultDataDir}/${DATA_SCRIPT}”.
*** glibc detected *** python: corrupted double-linked list: 0x02dc95f0
***
=== Backtrace: =
/lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
/lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
/usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
/usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
/usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
/usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
/usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
/lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
/lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
=== Memory map:
0040-00401000 r-xp 08:06 450964
/usr/bin/python
0060-00601000 rw-p 08:06 450964
/usr/bin/python
024ba000-02f29000 rw-p 00:00 0 [heap]
7fd1b000-7fd1b0172000 rw-p 00:00 0
7fd1b0172000-7fd1b400 ---p 00:00 0
7fd1b400-7fd1b4021000 rw-p 00:00 0
7fd1b4021000-7fd1b800 ---p 00:00 0
7fd1b800-7fd1b8021000 rw-p 00:00 0
7fd1b8021000-7fd1bc00 ---p 00:00 0
7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
7fd1bd7e8000-7fd1bd7e9000 ---p 00:00 0
7fd1bd7e9000-7fd1be1e9000 rw-p 00:00 0
7fd1be1e9000-7fd1be1ed000 r-xp 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be1ed000-7fd1be3ed000 ---p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ed000-7fd1be3ef000 rw-p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ef000-7fd1be3f2000 r-xp 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be3f2000-7fd1be5f1000 ---p 3000 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be5f1000-7fd1be5f2000 rw-p 2000 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be5f2000-7fd1be5f5000 r-xp 08:06 180866
/usr/lib64/python2.6/lib-dynload/_hashlib.so
7fd1be5f5000-7fd1be7f4000 ---
[jira] [Commented] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305232#comment-14305232 ] Benedict commented on CASSANDRA-6936: - Just a subset, given the support for custom types. Ideally all non-custom types, but at least the common ones. > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mike updated CASSANDRA-8736:
Description:
After we upgraded C* to 2.1.2, with new installation, sometimes on creating
Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
release. Thanks.
$CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f ${dbDefaultDataDir}/${DATA_SCRIPT}”.
*** glibc detected *** python: corrupted double-linked list: 0x02dc95f0
***
=== Backtrace: =
/lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
/lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
/usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
/usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
/usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
/usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
/usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
/lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
/lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
=== Memory map:
0040-00401000 r-xp 08:06 450964
/usr/bin/python
0060-00601000 rw-p 08:06 450964
/usr/bin/python
024ba000-02f29000 rw-p 00:00 0 [heap]
7fd1b000-7fd1b0172000 rw-p 00:00 0
7fd1b0172000-7fd1b400 ---p 00:00 0
7fd1b400-7fd1b4021000 rw-p 00:00 0
7fd1b4021000-7fd1b800 ---p 00:00 0
7fd1b800-7fd1b8021000 rw-p 00:00 0
7fd1b8021000-7fd1bc00 ---p 00:00 0
7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
7fd1bd7e8000-7fd1bd7e9000 ---p 00:00 0
7fd1bd7e9000-7fd1be1e9000 rw-p 00:00 0
7fd1be1e9000-7fd1be1ed000 r-xp 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be1ed000-7fd1be3ed000 ---p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ed000-7fd1be3ef000 rw-p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ef000-7fd1be3f2000 r-xp 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be3f2000-7fd1be5f1000 ---p 3000 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be5f1000-7fd1be5f2000 rw-p 2000 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be5f2000-7fd1be5f5000 r-xp 08:06 180866
/usr/lib64/python2.6/lib-dynload/_hashlib.so
7fd1be5f5000-7fd1be7f4000 ---
[jira] [Updated] (CASSANDRA-7391) Partial indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-7391: -- Fix Version/s: (was: 3.0) 3.1 Moved. > Partial indexes > --- > > Key: CASSANDRA-7391 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7391 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Jonathan Ellis >Priority: Minor > Fix For: 3.1 > > > Indexing extremely common values is not useful (e.g. where 90% of rows have a > null value) because it's actually more efficient to just to a seq scan, so > indexing is just wasted effort. > Rather than requiring users to take that hit if they want anything indexed at > all, we should allow specifying a WHERE clause to CREATE INDEX specifying > which rows to include in the index. Could include UDF but useful even > without it. > http://www.postgresql.org/docs/9.3/static/indexes-partial.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7040) Replace read/write stage with per-disk access coordination
[ https://issues.apache.org/jira/browse/CASSANDRA-7040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-7040: -- Fix Version/s: (was: 3.0) 3.1 All right, let's move to 3.1 with 5863. > Replace read/write stage with per-disk access coordination > -- > > Key: CASSANDRA-7040 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7040 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.1 > > > As discussed in CASSANDRA-6995, current coordination of access to disk is > suboptimal: instead of ensuring disk accesses alone are coordinated, we > instead coordinate at the level of operations that may touch the disks, > ensuring only so many are proceeding at once. As such, tuning is difficult, > and we incur unnecessary delays for operations that would not touch the > disk(s). > Ideally we would instead simply use a shared coordination primitive to gate > access to the disk when we perform a rebuffer. This work would dovetail very > nicely with any work in CASSANDRA-5863, as we could prevent any blocking or > context switching for data that we know to be cached. It also, as far as I > can tell, obviates the need for CASSANDRA-5239. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-8736:
Priority: Minor (was: Critical)
> core dump on creating keyspaces
>
>
> Key: CASSANDRA-8736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8736
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Linux
>Reporter: mike
>Priority: Minor
>
> After we upgraded C* to 2.1.2, with new installation, sometimes on creating
> Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
> release. Thanks.
> $CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f
> ${dbDefaultDataDir}/${DATA_SCRIPT}”.
> *** glibc detected *** python: corrupted double-linked list:
> 0x02dc95f0 ***
> === Backtrace: =
> /lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
> /lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
> /usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
> /usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
> /usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
> /usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
> /usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
> /lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
> /lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
> === Memory map:
> 0040-00401000 r-xp 08:06 450964
> /usr/bin/python
> 0060-00601000 rw-p 08:06 450964
> /usr/bin/python
> 024ba000-02f29000 rw-p 00:00 0
> [heap]
> 7fd1b000-7fd1b0172000 rw-p 00:00 0
> 7fd1b0172000-7fd1b400 ---p 00:00 0
> 7fd1b400-7fd1b4021000 rw-p 00:00 0
> 7fd1b4021000-7fd1b800 ---p 00:00 0
> 7fd1b800-7fd1b8021000 rw-p 00:00 0
> 7fd1b8021000-7fd1bc00 ---p 00:00 0
> 7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
> 7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
> 7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
> 7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
> 7fd1bd7e8000-7fd1bd7e9000 ---p 00:00 0
> 7fd1bd7e9000-7fd1be1e9000 rw-p 00:00 0
> 7fd1be1e9000-7fd1be1ed000 r-xp 08:06 180906
> /usr/lib64/python2.6/lib-dynload/selectmodule.so
> 7fd1be1ed000-7fd1be3ed000 ---p 4000 08:06 180906
> /usr/lib64/python2.6/lib-dynload/selectmodule.so
> 7fd1be3ed000-7fd1be3ef000 rw-p 4000 08:06 180906
[jira] [Commented] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305277#comment-14305277
]
Brandon Williams commented on CASSANDRA-8736:
-
That really looks like a problem with your python installation/version. There
should never be a way from python code to trigger this.
> core dump on creating keyspaces
>
>
> Key: CASSANDRA-8736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8736
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Linux
>Reporter: mike
>Priority: Minor
>
> After we upgraded C* to 2.1.2, with new installation, sometimes on creating
> Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
> release. Thanks.
> $CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f
> ${dbDefaultDataDir}/${DATA_SCRIPT}”.
> *** glibc detected *** python: corrupted double-linked list:
> 0x02dc95f0 ***
> === Backtrace: =
> /lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
> /lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
> /usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
> /usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
> /usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
> /usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
> /usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
> /lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
> /lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
> === Memory map:
> 0040-00401000 r-xp 08:06 450964
> /usr/bin/python
> 0060-00601000 rw-p 08:06 450964
> /usr/bin/python
> 024ba000-02f29000 rw-p 00:00 0
> [heap]
> 7fd1b000-7fd1b0172000 rw-p 00:00 0
> 7fd1b0172000-7fd1b400 ---p 00:00 0
> 7fd1b400-7fd1b4021000 rw-p 00:00 0
> 7fd1b4021000-7fd1b800 ---p 00:00 0
> 7fd1b800-7fd1b8021000 rw-p 00:00 0
> 7fd1b8021000-7fd1bc00 ---p 00:00 0
> 7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
> 7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
> 7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
> 7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
> 7fd1bd7e8000-7fd1bd7e9000 ---p 00:00 0
> 7fd1bd7e9000-7fd1be1e9000 rw-p 00:00 0
> 7fd1be1e9000-7fd1be1ed000 r-xp 08:06 180906
> /usr/lib64/python2.6/lib-dynload/selectmodule.so
> 7fd1be1ed000-7fd1be3ed000 ---p
[jira] [Commented] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305283#comment-14305283 ] Jonathan Ellis commented on CASSANDRA-6936: --- (JB's issue: CASSANDRA-8730) It sounds tractable for 3.0 if we start soon; neither backwards compatibility with old-version sstables nor telling clients that v4 protocol includes new serialization formats should be a huge obstacle. > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8731) Optimise merges involving multiple clustering columns
[ https://issues.apache.org/jira/browse/CASSANDRA-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-8731: Description: Since the new storage format is dead in the water for the moment, we should do our best to optimise current behaviour. When merging data from multiple sstables with multiple clustering columns, currently we must incur the full costs of comparison for the entire matching prefix, and must heapify every cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for every cell we merge, where N is the number of sources we're merging. Essentially I'm proposing a trie-based merge approach as a replacement for the ManyToOne MergeIterator, wherein we treat each clustering component as a tree underwhich all Cells with a common prefix occur. We then perform a tree merge, rather than a flat merge. For byte-order fields this trie can even be a full binary-trie (although built on the fly). The advantage here is that we rapidly prune merges involving disjoint ranges, so that instead of always incurring lg(N) costs on each new record, we may often incur O(1) costs. For timeseries data, for instance, we could merge dozens of files and so long as they were non-overlapping our CPU burden would be little more than reading from a single file. On top of this, we no longer incur any of the shared prefix repetition costs, since we compare each prefix piece-wise, and only once. was: Since the new storage format is dead in the water for the moment, we should do our best to optimise current behaviour. When merging data from multiple sstables with multiple clustering columns, currently we must incur the full costs of comparison for the entire matching prefix, and must heapify every cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for every cell we merge, where N is the number of sources we're merging. Essentially I'm proposing a trie-based merge approach, wherein we treat each clustering component as a tree underwhich all Cells with a common prefix occur. We then perform a tree merge, rather than a flat merge. For byte-order fields this trie can even be a full binary-trie (although built on the fly). The advantage here is that we rapidly prune merges involving disjoint ranges, so that instead of always incurring lg(N) costs on each new record, we may often incur O(1) costs. For timeseries data, for instance, we could merge dozens of files and so long as they were non-overlapping our CPU burden would be little more than reading from a single file. On top of this, we no longer incur any of the shared prefix repetition costs, since we compare each prefix piece-wise, and only once. > Optimise merges involving multiple clustering columns > - > > Key: CASSANDRA-8731 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8731 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > Since the new storage format is dead in the water for the moment, we should > do our best to optimise current behaviour. When merging data from multiple > sstables with multiple clustering columns, currently we must incur the full > costs of comparison for the entire matching prefix, and must heapify every > cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for > every cell we merge, where N is the number of sources we're merging. > Essentially I'm proposing a trie-based merge approach as a replacement for > the ManyToOne MergeIterator, wherein we treat each clustering component as a > tree underwhich all Cells with a common prefix occur. We then perform a tree > merge, rather than a flat merge. For byte-order fields this trie can even be > a full binary-trie (although built on the fly). The advantage here is that we > rapidly prune merges involving disjoint ranges, so that instead of always > incurring lg(N) costs on each new record, we may often incur O(1) costs. For > timeseries data, for instance, we could merge dozens of files and so long as > they were non-overlapping our CPU burden would be little more than reading > from a single file. > On top of this, we no longer incur any of the shared prefix repetition costs, > since we compare each prefix piece-wise, and only once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8731) Optimise merges involving multiple clustering columns
[ https://issues.apache.org/jira/browse/CASSANDRA-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-8731: Description: Since the new storage format is dead in the water for the moment, we should do our best to optimise current behaviour. When merging data from multiple sstables with multiple clustering columns, currently we must incur the full costs of comparison for the entire matching prefix, and must heapify every cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for every cell we merge, where N is the number of sources we're merging. Essentially I'm proposing a trie-based merge approach, wherein we treat each clustering component as a tree underwhich all Cells with a common prefix occur. We then perform a tree merge, rather than a flat merge. For byte-order fields this trie can even be a full binary-trie (although built on the fly). The advantage here is that we rapidly prune merges involving disjoint ranges, so that instead of always incurring lg(N) costs on each new record, we may often incur O(1) costs. For timeseries data, for instance, we could merge dozens of files and so long as they were non-overlapping our CPU burden would be little more than reading from a single file. On top of this, we no longer incur any of the shared prefix repetition costs, since we compare each prefix piece-wise, and only once. > Optimise merges involving multiple clustering columns > - > > Key: CASSANDRA-8731 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8731 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > Since the new storage format is dead in the water for the moment, we should > do our best to optimise current behaviour. When merging data from multiple > sstables with multiple clustering columns, currently we must incur the full > costs of comparison for the entire matching prefix, and must heapify every > cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for > every cell we merge, where N is the number of sources we're merging. > Essentially I'm proposing a trie-based merge approach, wherein we treat each > clustering component as a tree underwhich all Cells with a common prefix > occur. We then perform a tree merge, rather than a flat merge. For byte-order > fields this trie can even be a full binary-trie (although built on the fly). > The advantage here is that we rapidly prune merges involving disjoint ranges, > so that instead of always incurring lg(N) costs on each new record, we may > often incur O(1) costs. For timeseries data, for instance, we could merge > dozens of files and so long as they were non-overlapping our CPU burden would > be little more than reading from a single file. > On top of this, we no longer incur any of the shared prefix repetition costs, > since we compare each prefix piece-wise, and only once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8734) Expose commit log archive status
[ https://issues.apache.org/jira/browse/CASSANDRA-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-8734: --- Reviewer: Joshua McKenzie > Expose commit log archive status > > > Key: CASSANDRA-8734 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8734 > Project: Cassandra > Issue Type: New Feature > Components: Config >Reporter: Philip S Doctor >Assignee: Chris Lohfink > Attachments: 8734-cassandra-2.1.txt > > > The operational procedure to modify commit log archiving is to edit > commitlog_archiving.properties and then perform a restart. However this has > troublesome edge cases: > 1) It is possible for people to modify commitlog_archiving.properties but > then not perform a restart > 2) It is possible for people to modify commitlog_archiving.properties only on > some nodes > 3) It is possible for people to have modified file + restart but then later > add more nodes without correct modifications. > Because of these reasons, it is operationally useful to be able to audit the > commit log archive state of a node. Simply parsing > commitlog_archiving.properties is insufficient due to #1. > I would suggest exposing either via some system table or JMX would be useful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8731) Optimise merges involving multiple clustering columns
[ https://issues.apache.org/jira/browse/CASSANDRA-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305342#comment-14305342 ] Benedict commented on CASSANDRA-8731: - Regrettably I've lost all of the discussion I had with candidates about this approach, so don't have the mathematical analysis to hand (although it gets pretty complex). [~blambov] may have the discussion I had with him; he was better at the math than me anyway. > Optimise merges involving multiple clustering columns > - > > Key: CASSANDRA-8731 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8731 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > Since the new storage format is dead in the water for the moment, we should > do our best to optimise current behaviour. When merging data from multiple > sstables with multiple clustering columns, currently we must incur the full > costs of comparison for the entire matching prefix, and must heapify every > cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for > every cell we merge, where N is the number of sources we're merging. > Essentially I'm proposing a trie-based merge approach as a replacement for > the ManyToOne MergeIterator, wherein we treat each clustering component as a > tree underwhich all Cells with a common prefix occur. We then perform a tree > merge, rather than a flat merge. For byte-order fields this trie can even be > a full binary-trie (although built on the fly). The advantage here is that we > rapidly prune merges involving disjoint ranges, so that instead of always > incurring lg(N) costs on each new record, we may often incur O(1) costs. For > timeseries data, for instance, we could merge dozens of files and so long as > they were non-overlapping our CPU burden would be little more than reading > from a single file. > On top of this, we no longer incur any of the shared prefix repetition costs, > since we compare each prefix piece-wise, and only once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8729) Commitlog causes read before write when overwriting
[ https://issues.apache.org/jira/browse/CASSANDRA-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305348#comment-14305348 ] Joshua McKenzie commented on CASSANDRA-8729: While testing for CASSANDRA-6890 (debating removing mmap path), it was pretty clear from my testing that mmap'ed I/O on Windows has a considerable advantage over buffered to a degree that linux does not. I'm of the opinion we should make efforts to memory-map our I/O on Windows wherever possible, with the known caveat that it makes deleting and renaming files more complicated (all segments have to be unmapped before either of those ops). > Commitlog causes read before write when overwriting > --- > > Key: CASSANDRA-8729 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8729 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg > > The memory mapped commit log implementation writes directly to the page > cache. If a page is not in the cache the kernel will read it in even though > we are going to overwrite. > The way to avoid this is to write to private memory, and then pad the write > with 0s at the end so it is page (4k) aligned before writing to a file. > The commit log would benefit from being refactored into something that looks > more like a pipeline with incoming requests receiving private memory to write > in, completed buffers being submitted to a parallelized compression/checksum > step, followed by submission to another thread for writing to a file that > preserves the order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305353#comment-14305353 ] Aleksey Yeschenko commented on CASSANDRA-6936: -- I'm hoping that just having CASSANDRA-8730 and CASSANDRA-8731 would be enough. Many subtle and maybe not so subtle issues with converting representations under the hood like this will come up. One would be breaking timestamp ties in reconcile. To preserve old rules and avoid corruption you'd have to convert the new representation back to the regular representation, and do a comparison on that. So cells would now have to reference the type of the value, too. bq. v4 protocol includes new serialization formats Do you mean new types, or changing serialization format of the existing types? The latter wouldn't be welcomed by driver authors. > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8731) Optimise merges involving multiple clustering columns
[ https://issues.apache.org/jira/browse/CASSANDRA-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305361#comment-14305361 ] Jonathan Ellis commented on CASSANDRA-8731: --- So you're just proposing a change to the MergeIterator, not the CF objects? > Optimise merges involving multiple clustering columns > - > > Key: CASSANDRA-8731 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8731 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > Since the new storage format is dead in the water for the moment, we should > do our best to optimise current behaviour. When merging data from multiple > sstables with multiple clustering columns, currently we must incur the full > costs of comparison for the entire matching prefix, and must heapify every > cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for > every cell we merge, where N is the number of sources we're merging. > Essentially I'm proposing a trie-based merge approach as a replacement for > the ManyToOne MergeIterator, wherein we treat each clustering component as a > tree underwhich all Cells with a common prefix occur. We then perform a tree > merge, rather than a flat merge. For byte-order fields this trie can even be > a full binary-trie (although built on the fly). The advantage here is that we > rapidly prune merges involving disjoint ranges, so that instead of always > incurring lg(N) costs on each new record, we may often incur O(1) costs. For > timeseries data, for instance, we could merge dozens of files and so long as > they were non-overlapping our CPU burden would be little more than reading > from a single file. > On top of this, we no longer incur any of the shared prefix repetition costs, > since we compare each prefix piece-wise, and only once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305367#comment-14305367 ] Aleksey Yeschenko commented on CASSANDRA-6936: -- Additionally, I wouldn't want to layer extra conversion logic on top of the already happening CASSANDRA-8099. We will have bugs there (in back and forth convertion of mutations and read commands). We are still catching bugs of this kind from CASSANDRA-3237. You don't want to make things worth by having this on top, in a single release. > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305367#comment-14305367 ] Aleksey Yeschenko edited comment on CASSANDRA-6936 at 2/4/15 4:03 PM: -- Additionally, I wouldn't want to layer extra conversion logic on top of the already happening CASSANDRA-8099. We will have bugs there (in back and forth conversion of mutations and read commands). We are still catching bugs of this kind from CASSANDRA-3237. You don't want to make things worse by having this on top, in a single release. was (Author: iamaleksey): Additionally, I wouldn't want to layer extra conversion logic on top of the already happening CASSANDRA-8099. We will have bugs there (in back and forth convertion of mutations and read commands). We are still catching bugs of this kind from CASSANDRA-3237. You don't want to make things worth by having this on top, in a single release. > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305376#comment-14305376 ] Benedict commented on CASSANDRA-6936: - Introducing new types that don't need conversion that v4 protocols can implement makes sense; but we can easily convert on the fly for earlier ones, or v4 as well. It's not challenging to do so, especially for the common fields of int, long, timestamp and UUID. Timestamp reconciliation isn't as much of a problem, since we convert them to actual native longs in Java-land; either representation is fine for those. > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305379#comment-14305379 ] Jonathan Ellis commented on CASSANDRA-6936: --- It wouldn't be welcomed by authors but it's not THAT big a deal, realistically. > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305381#comment-14305381 ] Jonathan Ellis commented on CASSANDRA-6936: --- Hmm, so basically CASSANDRA-8731's goal in life is to reduce the impact of optimizing compares (since 8731 will perform less of them). Let's see if we can do 8731 for 3.0 and see how big a problem this still is in practice at that point. > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305383#comment-14305383 ] Aleksey Yeschenko commented on CASSANDRA-6936: -- bq. Timestamp reconciliation isn't as much of a problem I was talking about a more esoteric thing - Cell#reconcile() when the timestamps are equal. Using BB.compareTo() on the value. Need to have the exact same representation as we have now, there. > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8731) Optimise merges involving multiple clustering columns
[ https://issues.apache.org/jira/browse/CASSANDRA-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305401#comment-14305401 ] Sylvain Lebresne commented on CASSANDRA-8731: - bq. For timeseries data, for instance, we could merge dozens of files and so long as they were non-overlapping our CPU burden would be little more than reading from a single file. On that specific problem, we could also use the per-sstable min/max clustering stats we have to avoid bothering merging cells form 2 sstable that we know don't overlap. It's certainly not as general as what's described above, but I suspect it's reasonably simple, while the more general description above feels a bit more involved (even though I'll admit haven't spend too much time thinking about how to implement it in practice). > Optimise merges involving multiple clustering columns > - > > Key: CASSANDRA-8731 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8731 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > Since the new storage format is dead in the water for the moment, we should > do our best to optimise current behaviour. When merging data from multiple > sstables with multiple clustering columns, currently we must incur the full > costs of comparison for the entire matching prefix, and must heapify every > cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for > every cell we merge, where N is the number of sources we're merging. > Essentially I'm proposing a trie-based merge approach as a replacement for > the ManyToOne MergeIterator, wherein we treat each clustering component as a > tree underwhich all Cells with a common prefix occur. We then perform a tree > merge, rather than a flat merge. For byte-order fields this trie can even be > a full binary-trie (although built on the fly). The advantage here is that we > rapidly prune merges involving disjoint ranges, so that instead of always > incurring lg(N) costs on each new record, we may often incur O(1) costs. For > timeseries data, for instance, we could merge dozens of files and so long as > they were non-overlapping our CPU burden would be little more than reading > from a single file. > On top of this, we no longer incur any of the shared prefix repetition costs, > since we compare each prefix piece-wise, and only once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305403#comment-14305403 ] Benedict commented on CASSANDRA-6936: - Well, the problem is that CASSANDRA-8731 only optimises compares involving multiple clustering columns. Wide rows with a single clustering column will still be very expensive - unless they're byte-order comparable, in which case 8731 could optimise those significantly as well (by creating trees within a clustering column, rather than between them) > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8731) Optimise merges involving multiple clustering columns
[ https://issues.apache.org/jira/browse/CASSANDRA-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305419#comment-14305419 ] Benedict commented on CASSANDRA-8731: - Well, the simplest implementation is very trivial: Instead of one PQ, you have a PQ for each level of clustering column. Really not very advanced at all. If you want to split intra-clustering-component to make byte-order comparable fields more efficient, that is more involved. That said, there is no harm in having both. It seems to me that it may be more involved to do what you suggest, though, especially for sstables that _do_ overlap, but don't overlap for their entirety. At the moment we just lump all our iterators together; slicing them and merging them independently actually sounds to me to be much more fiddly and tough to get right than the simplest approach outlined here. > Optimise merges involving multiple clustering columns > - > > Key: CASSANDRA-8731 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8731 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > Since the new storage format is dead in the water for the moment, we should > do our best to optimise current behaviour. When merging data from multiple > sstables with multiple clustering columns, currently we must incur the full > costs of comparison for the entire matching prefix, and must heapify every > cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for > every cell we merge, where N is the number of sources we're merging. > Essentially I'm proposing a trie-based merge approach as a replacement for > the ManyToOne MergeIterator, wherein we treat each clustering component as a > tree underwhich all Cells with a common prefix occur. We then perform a tree > merge, rather than a flat merge. For byte-order fields this trie can even be > a full binary-trie (although built on the fly). The advantage here is that we > rapidly prune merges involving disjoint ranges, so that instead of always > incurring lg(N) costs on each new record, we may often incur O(1) costs. For > timeseries data, for instance, we could merge dozens of files and so long as > they were non-overlapping our CPU burden would be little more than reading > from a single file. > On top of this, we no longer incur any of the shared prefix repetition costs, > since we compare each prefix piece-wise, and only once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-3852) use LIFO queueing policy when queue size exceeds thresholds
[
https://issues.apache.org/jira/browse/CASSANDRA-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305428#comment-14305428
]
Benedict commented on CASSANDRA-3852:
-
This won't take very long at all, we've got oodles of time before 3.0.
> use LIFO queueing policy when queue size exceeds thresholds
> ---
>
> Key: CASSANDRA-3852
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3852
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Peter Schuller
> Labels: performance
> Fix For: 3.0
>
>
> A strict FIFO policy for queueing (between stages) is detrimental to latency
> and forward progress. Whenever a node is saturated beyond incoming request
> rate, *all* requests become slow. If it is consistently saturated, you start
> effectively timing out on *all* requests.
> A much better strategy from the point of view of latency is to serve a subset
> requests quickly, and letting some time out, rather than letting all either
> time out or be slow.
> Care must be taken such that:
> * We still guarantee that requests are processed reasonably timely (we
> couldn't go strict LIFO for example as that would result in requests getting
> stuck potentially forever on a loaded node).
> * Maybe, depending on the previous point's solution, ensure that some
> requests bypass the policy and get prioritized (e.g., schema migrations, or
> anything "internal" to a node).
> A possible implementation is to go LIFO whenever there are requests in the
> queue that are older than N milliseconds (or a certain queue size, etc).
> Benefits:
> * All cases where the client is directly, or is indirectly affecting through
> other layers, a system which has limited concurrency (e.g., thread pool size
> of X to serve some incoming request rate), it is *much* better for a few
> requests to time out while most are serviced quickly, than for all requests
> to become slow, as it doesn't explode concurrency. Think any random
> non-super-advanced php app, ruby web app, java servlet based app, etc.
> Essentially, it optimizes very heavily for improved average latencies.
> * Systems with strict p95/p99/p999 requirements on latencies should greatly
> benefit from such a policy. For example, suppose you have a system at 85% of
> capacity, and it takes a write spike (or has a hiccup like GC pause, blocking
> on a commit log write, etc). Suppose the hiccup racks up 500 ms worth of
> requests. At 15% margin at steady state, that takes 500ms * 100/15 = 3.2
> seconds to recover. Instead of *all* requests for an entire 3.2 second window
> being slow, we'd serve requests quickly for 2.7 of those seconds, with the
> incoming requests during that 500 ms interval being the ones primarily
> affected. The flip side though is that once you're at the point where more
> than N percent of requests end up having to wait for others to take LIFO
> priority, the p(100-N) latencies will actually be *worse* than without this
> change (but at this point you have to consider what the root reason for those
> pXX requirements are).
> * In the case of complete saturation, it allows forward progress. Suppose
> you're taking 25% more traffic than you are able to handle. Instead of
> getting backed up and ending up essentially timing out *every single
> request*, you will succeed in processing up to 75% of them (I say "up to"
> because it depends; for example on a {{QUORUM}} request you need at least two
> of the requests from the co-ordinator to succeed so the percentage is brought
> down) and allowing clients to make forward progress and get work done, rather
> than being stuck.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8731) Optimise merges involving multiple clustering columns
[ https://issues.apache.org/jira/browse/CASSANDRA-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305410#comment-14305410 ] Benedict commented on CASSANDRA-8731: - Correct. I plan(ned) to replace the CF objects in the new storage format. 8099 does make a partial change to CF objects inviting, but may be a bit ambitious for 3.0. > Optimise merges involving multiple clustering columns > - > > Key: CASSANDRA-8731 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8731 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > Since the new storage format is dead in the water for the moment, we should > do our best to optimise current behaviour. When merging data from multiple > sstables with multiple clustering columns, currently we must incur the full > costs of comparison for the entire matching prefix, and must heapify every > cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for > every cell we merge, where N is the number of sources we're merging. > Essentially I'm proposing a trie-based merge approach as a replacement for > the ManyToOne MergeIterator, wherein we treat each clustering component as a > tree underwhich all Cells with a common prefix occur. We then perform a tree > merge, rather than a flat merge. For byte-order fields this trie can even be > a full binary-trie (although built on the fly). The advantage here is that we > rapidly prune merges involving disjoint ranges, so that instead of always > incurring lg(N) costs on each new record, we may often incur O(1) costs. For > timeseries data, for instance, we could merge dozens of files and so long as > they were non-overlapping our CPU burden would be little more than reading > from a single file. > On top of this, we no longer incur any of the shared prefix repetition costs, > since we compare each prefix piece-wise, and only once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-3852) use LIFO queueing policy when queue size exceeds thresholds
[
https://issues.apache.org/jira/browse/CASSANDRA-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305446#comment-14305446
]
Aleksey Yeschenko commented on CASSANDRA-3852:
--
Won't comment on the issue itself - helping tail latencies is always welcome.
More generally, I want to comment on "This should be tractable for 3.0 if we
start on it soon". To me, how much time we still have before 3.0 is almost
irrelevant. It's about scope, the amount of change we want to cram into a
single release, which is something we should want to bound, and not by
available time.
> use LIFO queueing policy when queue size exceeds thresholds
> ---
>
> Key: CASSANDRA-3852
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3852
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Peter Schuller
> Labels: performance
> Fix For: 3.0
>
>
> A strict FIFO policy for queueing (between stages) is detrimental to latency
> and forward progress. Whenever a node is saturated beyond incoming request
> rate, *all* requests become slow. If it is consistently saturated, you start
> effectively timing out on *all* requests.
> A much better strategy from the point of view of latency is to serve a subset
> requests quickly, and letting some time out, rather than letting all either
> time out or be slow.
> Care must be taken such that:
> * We still guarantee that requests are processed reasonably timely (we
> couldn't go strict LIFO for example as that would result in requests getting
> stuck potentially forever on a loaded node).
> * Maybe, depending on the previous point's solution, ensure that some
> requests bypass the policy and get prioritized (e.g., schema migrations, or
> anything "internal" to a node).
> A possible implementation is to go LIFO whenever there are requests in the
> queue that are older than N milliseconds (or a certain queue size, etc).
> Benefits:
> * All cases where the client is directly, or is indirectly affecting through
> other layers, a system which has limited concurrency (e.g., thread pool size
> of X to serve some incoming request rate), it is *much* better for a few
> requests to time out while most are serviced quickly, than for all requests
> to become slow, as it doesn't explode concurrency. Think any random
> non-super-advanced php app, ruby web app, java servlet based app, etc.
> Essentially, it optimizes very heavily for improved average latencies.
> * Systems with strict p95/p99/p999 requirements on latencies should greatly
> benefit from such a policy. For example, suppose you have a system at 85% of
> capacity, and it takes a write spike (or has a hiccup like GC pause, blocking
> on a commit log write, etc). Suppose the hiccup racks up 500 ms worth of
> requests. At 15% margin at steady state, that takes 500ms * 100/15 = 3.2
> seconds to recover. Instead of *all* requests for an entire 3.2 second window
> being slow, we'd serve requests quickly for 2.7 of those seconds, with the
> incoming requests during that 500 ms interval being the ones primarily
> affected. The flip side though is that once you're at the point where more
> than N percent of requests end up having to wait for others to take LIFO
> priority, the p(100-N) latencies will actually be *worse* than without this
> change (but at this point you have to consider what the root reason for those
> pXX requirements are).
> * In the case of complete saturation, it allows forward progress. Suppose
> you're taking 25% more traffic than you are able to handle. Instead of
> getting backed up and ending up essentially timing out *every single
> request*, you will succeed in processing up to 75% of them (I say "up to"
> because it depends; for example on a {{QUORUM}} request you need at least two
> of the requests from the co-ordinator to succeed so the percentage is brought
> down) and allowing clients to make forward progress and get work done, rather
> than being stuck.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8707) Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted
[ https://issues.apache.org/jira/browse/CASSANDRA-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305447#comment-14305447 ] Benedict commented on CASSANDRA-8707: - OK, uploaded a version with all of the outstanding nits addressed and the XManager hierarchy removed. It doesn't look so ugly, to be honest, so we can definitely manage with this if it makes it sufficiently clearer. I have my fingers crossed :) > Move SegmentedFile, IndexSummary and BloomFilter to utilising RefCounted > > > Key: CASSANDRA-8707 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8707 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Critical > Fix For: 2.1.3 > > > There are still a few bugs with resource management, especially around > SSTableReader cleanup, esp. when intermixing with compaction. This migration > should help. We can simultaneously "simplify" the logic in SSTableReader to > not track the replacement chain, only to take a new reference to each of the > underlying resources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-3852) use LIFO queueing policy when queue size exceeds thresholds
[
https://issues.apache.org/jira/browse/CASSANDRA-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305466#comment-14305466
]
Benedict commented on CASSANDRA-3852:
-
bq. It's about scope
Also making a general statement, but mentioning one consideration (difficulty
of changes, length of time to bed in, etc.) does not preclude the consideration
of the other aspects without explicit mention, such as how isolated the change
is, how it fits in with other parallel changes. My reading of that statement
was that these considerations were considered likely met, and thus the
consideration fell to the mentioned considerations. Of course, I could be
wrong, and I do agree with your point in general.
> use LIFO queueing policy when queue size exceeds thresholds
> ---
>
> Key: CASSANDRA-3852
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3852
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Peter Schuller
> Labels: performance
> Fix For: 3.0
>
>
> A strict FIFO policy for queueing (between stages) is detrimental to latency
> and forward progress. Whenever a node is saturated beyond incoming request
> rate, *all* requests become slow. If it is consistently saturated, you start
> effectively timing out on *all* requests.
> A much better strategy from the point of view of latency is to serve a subset
> requests quickly, and letting some time out, rather than letting all either
> time out or be slow.
> Care must be taken such that:
> * We still guarantee that requests are processed reasonably timely (we
> couldn't go strict LIFO for example as that would result in requests getting
> stuck potentially forever on a loaded node).
> * Maybe, depending on the previous point's solution, ensure that some
> requests bypass the policy and get prioritized (e.g., schema migrations, or
> anything "internal" to a node).
> A possible implementation is to go LIFO whenever there are requests in the
> queue that are older than N milliseconds (or a certain queue size, etc).
> Benefits:
> * All cases where the client is directly, or is indirectly affecting through
> other layers, a system which has limited concurrency (e.g., thread pool size
> of X to serve some incoming request rate), it is *much* better for a few
> requests to time out while most are serviced quickly, than for all requests
> to become slow, as it doesn't explode concurrency. Think any random
> non-super-advanced php app, ruby web app, java servlet based app, etc.
> Essentially, it optimizes very heavily for improved average latencies.
> * Systems with strict p95/p99/p999 requirements on latencies should greatly
> benefit from such a policy. For example, suppose you have a system at 85% of
> capacity, and it takes a write spike (or has a hiccup like GC pause, blocking
> on a commit log write, etc). Suppose the hiccup racks up 500 ms worth of
> requests. At 15% margin at steady state, that takes 500ms * 100/15 = 3.2
> seconds to recover. Instead of *all* requests for an entire 3.2 second window
> being slow, we'd serve requests quickly for 2.7 of those seconds, with the
> incoming requests during that 500 ms interval being the ones primarily
> affected. The flip side though is that once you're at the point where more
> than N percent of requests end up having to wait for others to take LIFO
> priority, the p(100-N) latencies will actually be *worse* than without this
> change (but at this point you have to consider what the root reason for those
> pXX requirements are).
> * In the case of complete saturation, it allows forward progress. Suppose
> you're taking 25% more traffic than you are able to handle. Instead of
> getting backed up and ending up essentially timing out *every single
> request*, you will succeed in processing up to 75% of them (I say "up to"
> because it depends; for example on a {{QUORUM}} request you need at least two
> of the requests from the co-ordinator to succeed so the percentage is brought
> down) and allowing clients to make forward progress and get work done, rather
> than being stuck.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8731) Optimise merges involving multiple clustering columns
[ https://issues.apache.org/jira/browse/CASSANDRA-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305480#comment-14305480 ] Sylvain Lebresne commented on CASSANDRA-8731: - bq. If you want to split intra-clustering-component to make byte-order comparable fields more efficient, that is more involved. That's really the part I was refering to. I'm totally fine with having a specialized mergeIterator for clustering prefixes that knows not to compare the same components multiple times. bq. especially for sstables that do overlap, but don't overlap for their entirety I wasn't planning to handle them. For time series withh DateTieredCompactionStrategy, most of your sstables will be non-overlapping so it would still be a win. bq. slicing them and merging them independently actually sounds to me to be much more fiddly and tough to get right than the simplest approach outlined here If by "simplest approach" you mean the one PQ per level of clustering, then I think that's really orthogonal. What I'm suggesting is really not specific to multiple clustering level, while the PQ-per-level-of-clustering only help those case. What I can agree on is that doing CASSANDRA-6936 + doing a trie-for-byte-order-comparable-fields would likely make my suggestion less useful (even though there is cases where it would still be more efficient), but my initial guess is that doing both those thing is more involved that what I'm suggesting. But not trying to shoot your idea, just mentioning a potential optimization that imo is not really very involved (in particular, note that doing what I suggest only for single partition queries would be fine imo if that makes it simpler). Anyway, I would suggest limiting the scope of this issue to "introduce a smarter merge iterator for multi-clustering" (because it's easy and I doubt it's contentious) and open other tickets for more involved byte-fiddling suggestions. > Optimise merges involving multiple clustering columns > - > > Key: CASSANDRA-8731 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8731 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > Since the new storage format is dead in the water for the moment, we should > do our best to optimise current behaviour. When merging data from multiple > sstables with multiple clustering columns, currently we must incur the full > costs of comparison for the entire matching prefix, and must heapify every > cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for > every cell we merge, where N is the number of sources we're merging. > Essentially I'm proposing a trie-based merge approach as a replacement for > the ManyToOne MergeIterator, wherein we treat each clustering component as a > tree underwhich all Cells with a common prefix occur. We then perform a tree > merge, rather than a flat merge. For byte-order fields this trie can even be > a full binary-trie (although built on the fly). The advantage here is that we > rapidly prune merges involving disjoint ranges, so that instead of always > incurring lg(N) costs on each new record, we may often incur O(1) costs. For > timeseries data, for instance, we could merge dozens of files and so long as > they were non-overlapping our CPU burden would be little more than reading > from a single file. > On top of this, we no longer incur any of the shared prefix repetition costs, > since we compare each prefix piece-wise, and only once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Fix reversal of list prepends
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 2dd6944f1 -> 2d8bddb06
Fix reversal of list prepends
Patch by Tyler Hobbs; reviewed by Sylvain Lebresne for CASSANDRA-8733
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2d8bddb0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2d8bddb0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2d8bddb0
Branch: refs/heads/cassandra-2.0
Commit: 2d8bddb064f2a6f9292878e2d40805d9db74a09d
Parents: 2dd6944
Author: Tyler Hobbs
Authored: Wed Feb 4 10:51:24 2015 -0600
Committer: Tyler Hobbs
Committed: Wed Feb 4 10:51:24 2015 -0600
--
CHANGES.txt | 1 +
NEWS.txt | 9 +
src/java/org/apache/cassandra/cql3/Lists.java | 4 ++--
3 files changed, 12 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d8bddb0/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9e9b772..375dcfe 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.13:
+ * Fix reversal of list prepends (CASSANDRA-8733)
* Prevent non-zero default_time_to_live on tables with counters
(CASSANDRA-8678)
* Fix SSTableSimpleUnsortedWriter ConcurrentModificationException
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d8bddb0/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 2bc4fe6..5a4bc3d 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,15 @@ restore snapshots created with the previous major version
using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.
+2.0.13
+==
+
+Upgrading
+-
+- Prepending a list to a list collection was erroneously resulting in
+ the prepended list being reversed upon insertion. If you were depending
+ on this buggy behavior, note that it has been corrected.
+
2.0.12
==
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d8bddb0/src/java/org/apache/cassandra/cql3/Lists.java
--
diff --git a/src/java/org/apache/cassandra/cql3/Lists.java
b/src/java/org/apache/cassandra/cql3/Lists.java
index d483dd5..f4d1d7a 100644
--- a/src/java/org/apache/cassandra/cql3/Lists.java
+++ b/src/java/org/apache/cassandra/cql3/Lists.java
@@ -385,9 +385,9 @@ public abstract class Lists
List toAdd = ((Lists.Value)value).elements;
ColumnNameBuilder column = maybeUpdatePrefix(cf.metadata(),
prefix).add(columnName.key);
-for (int i = 0; i < toAdd.size(); i++)
+for (int i = toAdd.size() - 1; i >= 0; i--)
{
-ColumnNameBuilder b = i == toAdd.size() - 1 ? column :
column.copy();
+ColumnNameBuilder b = i == 0 ? column : column.copy();
PrecisionTime pt = PrecisionTime.getNext(time);
ByteBuffer uuid =
ByteBuffer.wrap(UUIDGen.getTimeUUIDBytes(pt.millis, pt.nanos));
ByteBuffer cellName = b.add(uuid).build();
[1/2] cassandra git commit: Fix reversal of list prepends
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 8c003a2a5 -> dc74ae67a
Fix reversal of list prepends
Patch by Tyler Hobbs; reviewed by Sylvain Lebresne for CASSANDRA-8733
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2d8bddb0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2d8bddb0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2d8bddb0
Branch: refs/heads/cassandra-2.1
Commit: 2d8bddb064f2a6f9292878e2d40805d9db74a09d
Parents: 2dd6944
Author: Tyler Hobbs
Authored: Wed Feb 4 10:51:24 2015 -0600
Committer: Tyler Hobbs
Committed: Wed Feb 4 10:51:24 2015 -0600
--
CHANGES.txt | 1 +
NEWS.txt | 9 +
src/java/org/apache/cassandra/cql3/Lists.java | 4 ++--
3 files changed, 12 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d8bddb0/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9e9b772..375dcfe 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.13:
+ * Fix reversal of list prepends (CASSANDRA-8733)
* Prevent non-zero default_time_to_live on tables with counters
(CASSANDRA-8678)
* Fix SSTableSimpleUnsortedWriter ConcurrentModificationException
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d8bddb0/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 2bc4fe6..5a4bc3d 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,15 @@ restore snapshots created with the previous major version
using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.
+2.0.13
+==
+
+Upgrading
+-
+- Prepending a list to a list collection was erroneously resulting in
+ the prepended list being reversed upon insertion. If you were depending
+ on this buggy behavior, note that it has been corrected.
+
2.0.12
==
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d8bddb0/src/java/org/apache/cassandra/cql3/Lists.java
--
diff --git a/src/java/org/apache/cassandra/cql3/Lists.java
b/src/java/org/apache/cassandra/cql3/Lists.java
index d483dd5..f4d1d7a 100644
--- a/src/java/org/apache/cassandra/cql3/Lists.java
+++ b/src/java/org/apache/cassandra/cql3/Lists.java
@@ -385,9 +385,9 @@ public abstract class Lists
List toAdd = ((Lists.Value)value).elements;
ColumnNameBuilder column = maybeUpdatePrefix(cf.metadata(),
prefix).add(columnName.key);
-for (int i = 0; i < toAdd.size(); i++)
+for (int i = toAdd.size() - 1; i >= 0; i--)
{
-ColumnNameBuilder b = i == toAdd.size() - 1 ? column :
column.copy();
+ColumnNameBuilder b = i == 0 ? column : column.copy();
PrecisionTime pt = PrecisionTime.getNext(time);
ByteBuffer uuid =
ByteBuffer.wrap(UUIDGen.getTimeUUIDBytes(pt.millis, pt.nanos));
ByteBuffer cellName = b.add(uuid).build();
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt NEWS.txt src/java/org/apache/cassandra/cql3/Lists.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dc74ae67 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dc74ae67 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dc74ae67 Branch: refs/heads/cassandra-2.1 Commit: dc74ae67a6d56ada214d5296ea01efe0ece54a70 Parents: 8c003a2 2d8bddb Author: Tyler Hobbs Authored: Wed Feb 4 10:54:52 2015 -0600 Committer: Tyler Hobbs Committed: Wed Feb 4 10:54:52 2015 -0600 -- CHANGES.txt | 1 + NEWS.txt | 9 + src/java/org/apache/cassandra/cql3/Lists.java | 2 +- 3 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc74ae67/CHANGES.txt -- diff --cc CHANGES.txt index b95fd3a,375dcfe..f70db7a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,90 -1,5 +1,91 @@@ -2.0.13: +2.1.3 + * Upgrade libthrift to 0.9.2 (CASSANDRA-8685) + * Don't use the shared ref in sstableloader (CASSANDRA-8704) + * Purge internal prepared statements if related tables or + keyspaces are dropped (CASSANDRA-8693) + * (cqlsh) Handle unicode BOM at start of files (CASSANDRA-8638) + * Stop compactions before exiting offline tools (CASSANDRA-8623) + * Update tools/stress/README.txt to match current behaviour (CASSANDRA-7933) + * Fix schema from Thrift conversion with empty metadata (CASSANDRA-8695) + * Safer Resource Management (CASSANDRA-7705) + * Make sure we compact highly overlapping cold sstables with + STCS (CASSANDRA-8635) + * rpc_interface and listen_interface generate NPE on startup when specified + interface doesn't exist (CASSANDRA-8677) + * Fix ArrayIndexOutOfBoundsException in nodetool cfhistograms (CASSANDRA-8514) + * Switch from yammer metrics for nodetool cf/proxy histograms (CASSANDRA-8662) + * Make sure we don't add tmplink files to the compaction + strategy (CASSANDRA-8580) + * (cqlsh) Handle maps with blob keys (CASSANDRA-8372) + * (cqlsh) Handle DynamicCompositeType schemas correctly (CASSANDRA-8563) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Add tooling to detect hot partitions (CASSANDRA-7974) + * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) + * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) + * Don't allow starting multiple inc repairs on the same sstables (CASSANDRA-8316) + * Invalidate prepared BATCH statements when related tables + or keyspaces are dropped (CASSANDRA-8652) + * Fix missing results in secondary index queries on collections + with ALLOW FILTERING (CASSANDRA-8421) + * Expose EstimatedHistogram metrics for range slices (CASSANDRA-8627) + * (cqlsh) Escape clqshrc passwords properly (CASSANDRA-8618) + * Fix NPE when passing wrong argument in ALTER TABLE statement (CASSANDRA-8355) + * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599) + * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537) + * Fix case-sensitivity of index name on CREATE and DROP INDEX + statements (CASSANDRA-8365) + * Better detection/logging for corruption in compressed sstables (CASSANDRA-8192) + * Use the correct repairedAt value when closing writer (CASSANDRA-8570) + * (cqlsh) Handle a schema mismatch being detected on startup (CASSANDRA-8512) + * Properly calculate expected write size during compaction (CASSANDRA-8532) + * Invalidate affected prepared statements when a table's columns + are altered (CASSANDRA-7910) + * Stress - user defined writes should populate sequentally (CASSANDRA-8524) + * Fix regression in SSTableRewriter causing some rows to become unreadable + during compaction (CASSANDRA-8429) + * Run major compactions for repaired/unrepaired in parallel (CASSANDRA-8510) + * (cqlsh) Fix compression options in DESCRIBE TABLE output when compression + is disabled (CASSANDRA-8288) + * (cqlsh) Fix DESCRIBE output after keyspaces are altered (CASSANDRA-7623) + * Make sure we set lastCompactedKey correctly (CASSANDRA-8463) + * (cqlsh) Fix output of CONSISTENCY command (CASSANDRA-8507) + * (cqlsh) Fixed the handling of LIST statements (CASSANDRA-8370) + * Make sstablescrub check leveled manifest again (CASSANDRA-8432) + * Check first/last keys in sstable when giving out positions (CASSANDRA-8458) + * Disable mmap on Windows (CASSANDRA-6993) + * Add missing ConsistencyLevels to cassandra-stress (CASSANDRA-8253) + * Add auth support to cassandra-stress (CASSANDRA-7985) + * Fix ArrayIndexOutOfBoundsException when gene
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: NEWS.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/32b82879 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/32b82879 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/32b82879 Branch: refs/heads/trunk Commit: 32b82879d00c2f295d6fb5998304847dad00cfb0 Parents: 3dd1bf1 dc74ae6 Author: Tyler Hobbs Authored: Wed Feb 4 10:55:38 2015 -0600 Committer: Tyler Hobbs Committed: Wed Feb 4 10:55:38 2015 -0600 -- CHANGES.txt | 1 + NEWS.txt | 8 src/java/org/apache/cassandra/cql3/Lists.java | 2 +- 3 files changed, 10 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/32b82879/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/32b82879/NEWS.txt -- diff --cc NEWS.txt index e37e85f,e344acc..a4391b9 --- a/NEWS.txt +++ b/NEWS.txt @@@ -13,51 -13,14 +13,59 @@@ restore snapshots created with the prev 'sstableloader' tool. You can upgrade the file format of your snapshots using the provided 'sstableupgrade' tool. +3.0 +=== + +New features + + - Authentication & Authorization APIs have been updated to introduce + roles. Roles and Permissions granted to them are inherited, supporting + role based access control. The role concept supercedes that of users + and CQL constructs such as CREATE USER are deprecated but retained for + compatibility. The requirement to explicitly create Roles in Cassandra + even when auth is handled by an external system has been removed, so + authentication & authorization can be delegated to such systems in their + entirety. + - SSTable file name is changed. Now you don't have Keyspace/CF name + in file name. Also, secondary index has its own directory under parent's + directory. + + +Upgrading +- + - Pig's CqlStorage has been removed, use CqlNativeStorage instead + - IAuthenticator been updated to remove responsibility for user/role + maintenance and is now solely responsible for validating credentials, + This is primarily done via SASL, though an optional method exists for + systems which need support for the Thrift login() method. + - IRoleManager interface has been added which takes over the maintenance + functions from IAuthenticator. IAuthorizer is mainly unchanged. Auth data + in systems using the stock internal implementations PasswordAuthenticator + & CassandraAuthorizer will be automatically converted during upgrade, + with minimal operator intervention required. Custom implementations will + require modification, though these can be used in conjunction with the + stock CassandraRoleManager so providing an IRoleManager implementation + should not usually be necessary. + - Fat client support has been removed since we have push notifications to clients + - cassandra-cli has been removed. Please use cqlsh instead. + - YamlFileNetworkTopologySnitch has been removed; switch to + GossipingPropertyFileSnitch instead. + - CQL2 has been removed entirely in this release (previously deprecated + in 2.0.0). Please switch to CQL3 if you haven't already done so. + - Very large batches will now be rejected (defaults to 50kb). This + can be customized by modifying batch_size_fail_threshold_in_kb. + - The results of CQL3 queries containing an IN restriction will be ordered + in the normal order and not anymore in the order in which the column values were + specified in the IN restriction. + + 2.1.3 + = + + Upgrading + - + - Prepending a list to a list collection was erroneously resulting in + the prepended list being reversed upon insertion. If you were depending + on this buggy behavior, note that it has been corrected. 2.1.2 = http://git-wip-us.apache.org/repos/asf/cassandra/blob/32b82879/src/java/org/apache/cassandra/cql3/Lists.java --
[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt NEWS.txt src/java/org/apache/cassandra/cql3/Lists.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dc74ae67 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dc74ae67 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dc74ae67 Branch: refs/heads/trunk Commit: dc74ae67a6d56ada214d5296ea01efe0ece54a70 Parents: 8c003a2 2d8bddb Author: Tyler Hobbs Authored: Wed Feb 4 10:54:52 2015 -0600 Committer: Tyler Hobbs Committed: Wed Feb 4 10:54:52 2015 -0600 -- CHANGES.txt | 1 + NEWS.txt | 9 + src/java/org/apache/cassandra/cql3/Lists.java | 2 +- 3 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc74ae67/CHANGES.txt -- diff --cc CHANGES.txt index b95fd3a,375dcfe..f70db7a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,90 -1,5 +1,91 @@@ -2.0.13: +2.1.3 + * Upgrade libthrift to 0.9.2 (CASSANDRA-8685) + * Don't use the shared ref in sstableloader (CASSANDRA-8704) + * Purge internal prepared statements if related tables or + keyspaces are dropped (CASSANDRA-8693) + * (cqlsh) Handle unicode BOM at start of files (CASSANDRA-8638) + * Stop compactions before exiting offline tools (CASSANDRA-8623) + * Update tools/stress/README.txt to match current behaviour (CASSANDRA-7933) + * Fix schema from Thrift conversion with empty metadata (CASSANDRA-8695) + * Safer Resource Management (CASSANDRA-7705) + * Make sure we compact highly overlapping cold sstables with + STCS (CASSANDRA-8635) + * rpc_interface and listen_interface generate NPE on startup when specified + interface doesn't exist (CASSANDRA-8677) + * Fix ArrayIndexOutOfBoundsException in nodetool cfhistograms (CASSANDRA-8514) + * Switch from yammer metrics for nodetool cf/proxy histograms (CASSANDRA-8662) + * Make sure we don't add tmplink files to the compaction + strategy (CASSANDRA-8580) + * (cqlsh) Handle maps with blob keys (CASSANDRA-8372) + * (cqlsh) Handle DynamicCompositeType schemas correctly (CASSANDRA-8563) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Add tooling to detect hot partitions (CASSANDRA-7974) + * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) + * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) + * Don't allow starting multiple inc repairs on the same sstables (CASSANDRA-8316) + * Invalidate prepared BATCH statements when related tables + or keyspaces are dropped (CASSANDRA-8652) + * Fix missing results in secondary index queries on collections + with ALLOW FILTERING (CASSANDRA-8421) + * Expose EstimatedHistogram metrics for range slices (CASSANDRA-8627) + * (cqlsh) Escape clqshrc passwords properly (CASSANDRA-8618) + * Fix NPE when passing wrong argument in ALTER TABLE statement (CASSANDRA-8355) + * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599) + * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537) + * Fix case-sensitivity of index name on CREATE and DROP INDEX + statements (CASSANDRA-8365) + * Better detection/logging for corruption in compressed sstables (CASSANDRA-8192) + * Use the correct repairedAt value when closing writer (CASSANDRA-8570) + * (cqlsh) Handle a schema mismatch being detected on startup (CASSANDRA-8512) + * Properly calculate expected write size during compaction (CASSANDRA-8532) + * Invalidate affected prepared statements when a table's columns + are altered (CASSANDRA-7910) + * Stress - user defined writes should populate sequentally (CASSANDRA-8524) + * Fix regression in SSTableRewriter causing some rows to become unreadable + during compaction (CASSANDRA-8429) + * Run major compactions for repaired/unrepaired in parallel (CASSANDRA-8510) + * (cqlsh) Fix compression options in DESCRIBE TABLE output when compression + is disabled (CASSANDRA-8288) + * (cqlsh) Fix DESCRIBE output after keyspaces are altered (CASSANDRA-7623) + * Make sure we set lastCompactedKey correctly (CASSANDRA-8463) + * (cqlsh) Fix output of CONSISTENCY command (CASSANDRA-8507) + * (cqlsh) Fixed the handling of LIST statements (CASSANDRA-8370) + * Make sstablescrub check leveled manifest again (CASSANDRA-8432) + * Check first/last keys in sstable when giving out positions (CASSANDRA-8458) + * Disable mmap on Windows (CASSANDRA-6993) + * Add missing ConsistencyLevels to cassandra-stress (CASSANDRA-8253) + * Add auth support to cassandra-stress (CASSANDRA-7985) + * Fix ArrayIndexOutOfBoundsException when generating e
[1/3] cassandra git commit: Fix reversal of list prepends
Repository: cassandra
Updated Branches:
refs/heads/trunk 3dd1bf104 -> 32b82879d
Fix reversal of list prepends
Patch by Tyler Hobbs; reviewed by Sylvain Lebresne for CASSANDRA-8733
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2d8bddb0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2d8bddb0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2d8bddb0
Branch: refs/heads/trunk
Commit: 2d8bddb064f2a6f9292878e2d40805d9db74a09d
Parents: 2dd6944
Author: Tyler Hobbs
Authored: Wed Feb 4 10:51:24 2015 -0600
Committer: Tyler Hobbs
Committed: Wed Feb 4 10:51:24 2015 -0600
--
CHANGES.txt | 1 +
NEWS.txt | 9 +
src/java/org/apache/cassandra/cql3/Lists.java | 4 ++--
3 files changed, 12 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d8bddb0/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9e9b772..375dcfe 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.13:
+ * Fix reversal of list prepends (CASSANDRA-8733)
* Prevent non-zero default_time_to_live on tables with counters
(CASSANDRA-8678)
* Fix SSTableSimpleUnsortedWriter ConcurrentModificationException
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d8bddb0/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index 2bc4fe6..5a4bc3d 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -13,6 +13,15 @@ restore snapshots created with the previous major version
using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.
+2.0.13
+==
+
+Upgrading
+-
+- Prepending a list to a list collection was erroneously resulting in
+ the prepended list being reversed upon insertion. If you were depending
+ on this buggy behavior, note that it has been corrected.
+
2.0.12
==
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d8bddb0/src/java/org/apache/cassandra/cql3/Lists.java
--
diff --git a/src/java/org/apache/cassandra/cql3/Lists.java
b/src/java/org/apache/cassandra/cql3/Lists.java
index d483dd5..f4d1d7a 100644
--- a/src/java/org/apache/cassandra/cql3/Lists.java
+++ b/src/java/org/apache/cassandra/cql3/Lists.java
@@ -385,9 +385,9 @@ public abstract class Lists
List toAdd = ((Lists.Value)value).elements;
ColumnNameBuilder column = maybeUpdatePrefix(cf.metadata(),
prefix).add(columnName.key);
-for (int i = 0; i < toAdd.size(); i++)
+for (int i = toAdd.size() - 1; i >= 0; i--)
{
-ColumnNameBuilder b = i == toAdd.size() - 1 ? column :
column.copy();
+ColumnNameBuilder b = i == 0 ? column : column.copy();
PrecisionTime pt = PrecisionTime.getNext(time);
ByteBuffer uuid =
ByteBuffer.wrap(UUIDGen.getTimeUUIDBytes(pt.millis, pt.nanos));
ByteBuffer cellName = b.add(uuid).build();
[jira] [Commented] (CASSANDRA-8731) Optimise merges involving multiple clustering columns
[ https://issues.apache.org/jira/browse/CASSANDRA-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305501#comment-14305501 ] Benedict commented on CASSANDRA-8731: - bq. Anyway, I would suggest limiting the scope of this issue to "introduce a smarter merge iterator for multi-clustering" (because it's easy and I doubt it's contentious) and open other tickets for more involved byte-fiddling suggestions. Yup, that seems eminently sensible. bq. but my initial guess is that doing both those thing is more involved that what I'm suggesting Potentially, yes. Although a simple binary-trie based merge would not likely be very hard; certainly not as hard as implementing an actual binary-trie. The question is probably mostly if we get byte-order comparability for common fields. bq. then I think that's really orthogonal Close enough to agreed that I won't split hairs, and you're right, your suggestion would work for single clustering columns as well, so long as they are actually disjoint. The problem is that this is still likely to be fiddlier than it seems you're suggesting. DTCS is not likely to be non-overlapping; any two adjacent sstables will overlap a little (possibly a lot), and so you have to do something to unpick them. You could perform separate merges for the overlapping ranges, and have a rolling window, but this is also non-trivial. If we went for implementing the DTCS++ I suggested in the original ticket, of course (which would be quite achievable to deliver for 3.0) then this would guarantee non-overlapping ranges, and this could be made significantly simpler as a result. Either way, we should file a separate ticket for this and discuss it there. > Optimise merges involving multiple clustering columns > - > > Key: CASSANDRA-8731 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8731 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > Since the new storage format is dead in the water for the moment, we should > do our best to optimise current behaviour. When merging data from multiple > sstables with multiple clustering columns, currently we must incur the full > costs of comparison for the entire matching prefix, and must heapify every > cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for > every cell we merge, where N is the number of sources we're merging. > Essentially I'm proposing a trie-based merge approach as a replacement for > the ManyToOne MergeIterator, wherein we treat each clustering component as a > tree underwhich all Cells with a common prefix occur. We then perform a tree > merge, rather than a flat merge. For byte-order fields this trie can even be > a full binary-trie (although built on the fly). The advantage here is that we > rapidly prune merges involving disjoint ranges, so that instead of always > incurring lg(N) costs on each new record, we may often incur O(1) costs. For > timeseries data, for instance, we could merge dozens of files and so long as > they were non-overlapping our CPU burden would be little more than reading > from a single file. > On top of this, we no longer incur any of the shared prefix repetition costs, > since we compare each prefix piece-wise, and only once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305506#comment-14305506
]
Benjamin Lerer commented on CASSANDRA-8535:
---
This issue is preventing me to run the DTests on my windows machine.
It will be nice if somebody was willing to review the patch.
> java.lang.RuntimeException: Failed to rename XXX to YYY
> ---
>
> Key: CASSANDRA-8535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
> Project: Cassandra
> Issue Type: Bug
> Environment: Windows 2008 X64
>Reporter: Leonid Shalupov
>Assignee: Joshua McKenzie
> Fix For: 2.1.3
>
> Attachments: 8535_v1.txt
>
>
> {code}
> java.lang.RuntimeException: Failed to rename
> build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
> to
> build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
> ~[main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> ~[na:1.7.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ~[na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> Caused by: java.nio.file.FileSystemException:
> build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
> ->
> build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db:
> The process cannot access the file because it is being used by another
> process.
> at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
> ~[na:1.7.0_45]
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> ~[na:1.7.0_45]
> at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
> ~[na:1.7.0_45]
> at
> sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
> ~[na:1.7.0_45]
> at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45]
> at
> org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
> ~[main/:na]
> ... 18 common frames omitted
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8731) Optimise merges involving multiple clustering columns
[ https://issues.apache.org/jira/browse/CASSANDRA-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305522#comment-14305522 ] Sylvain Lebresne commented on CASSANDRA-8731: - bq. any two adjacent sstables will overlap a little (possibly a lot), and so you have to do something to unpick them This may not be the right ticket to discuss this after all, but again, I think you're assuming too much. We don't absolutely have to unpick them: while two adjacent sstable might overlap, you'll still have many (non-adjacent) ones that don't overlap and this can still be used to reduce comparisons. Besides, provided the merge iterator knows it deals with sstable iterators (which isn't too hard), we could alternatively use index blocks instead of global sstable min/max stats. My general point is, we could use informations we have from sstables to limit the number of comparison done, and that might not be terribly hard. Anyway, I'll create a ticket. > Optimise merges involving multiple clustering columns > - > > Key: CASSANDRA-8731 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8731 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > Since the new storage format is dead in the water for the moment, we should > do our best to optimise current behaviour. When merging data from multiple > sstables with multiple clustering columns, currently we must incur the full > costs of comparison for the entire matching prefix, and must heapify every > cell in our PriorityQueue, incurring lg(N) of these costlier comparisons for > every cell we merge, where N is the number of sources we're merging. > Essentially I'm proposing a trie-based merge approach as a replacement for > the ManyToOne MergeIterator, wherein we treat each clustering component as a > tree underwhich all Cells with a common prefix occur. We then perform a tree > merge, rather than a flat merge. For byte-order fields this trie can even be > a full binary-trie (although built on the fly). The advantage here is that we > rapidly prune merges involving disjoint ranges, so that instead of always > incurring lg(N) costs on each new record, we may often incur O(1) costs. For > timeseries data, for instance, we could merge dozens of files and so long as > they were non-overlapping our CPU burden would be little more than reading > from a single file. > On top of this, we no longer incur any of the shared prefix repetition costs, > since we compare each prefix piece-wise, and only once. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8732) Make inter-node timeouts tolerate clock skew and drift
[ https://issues.apache.org/jira/browse/CASSANDRA-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305527#comment-14305527 ] Ariel Weisberg commented on CASSANDRA-8732: --- [~benedict] asked me to log this. I don't see it as a high priority either because there is an element of operator error in not having clocks synced. It is a common error and it is a nice to have for the database to handle it as well as possible. I think issue is when one clock is far enough off timeouts can fire sooner/later then they should. Messages are dropped by receiving nodes when they are proxied intracluster according to my reading of the code so that is why it is sensitive to drift and skew. An append only workload or workload that only updates from a single writer might not notice the clock skew? Just throwing that out there. It's also possible only one node has a problematic clocks. > Make inter-node timeouts tolerate clock skew and drift > -- > > Key: CASSANDRA-8732 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8732 > Project: Cassandra > Issue Type: Improvement >Reporter: Ariel Weisberg > > Right now internode timeouts rely on currentTimeMillis() (and NTP) to make > sure that tasks don't expire before they arrive. > Every receiver needs to deduce the offset between its nanoTime and the remote > nanoTime. I don't think currentTimeMillis is a good choice because it is > designed to be manipulated by operators and NTP. I would probably be > comfortable assuming that nanoTime isn't going to move in significant ways > without something that could be classified as operator error happening. > I suspect the one timing method you can rely on being accurate is nanoTime > within a node (on average) and that a node can report on its own scheduling > jitter (on average). > Finding the offset requires knowing what the network latency is in one > direction. > One way to do this would be to periodically send a ping request which > generates a series of ping responses at fixed intervals (maybe by UDP?). The > responses should corrected for scheduling jitter since the fixed intervals > may not be exactly achieved by the sender. By measuring the time deviation > between ping responses and their expected arrival time (based on the > interval) and correcting for the remotely reported scheduling jitter, you > should be able to measure latency in one direction. > A weighted moving average (only correct for drift, not readjustment) of these > measurements would eventually converge on a close answer and would not be > impacted by outlier measurements. It may also make sense to drop the largest > N samples to improve accuracy. > One you know network latency you can add that to the timestamp of each ping > and compare to the local clock and know what the offset is. > These measurements won't calculate the offset to be too small (timeouts fire > early), but could calculate the offset to be too large (timeouts fire late). > The conditions where you the offset won't be accurate are the conditions > where you also want them firing reliably. This and bootstrapping in bad > conditions is what I am most uncertain of. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8610) Allow IF EXISTS for UPDATE statements
[
https://issues.apache.org/jira/browse/CASSANDRA-8610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-8610:
---
Assignee: Prajakta Bhosale
> Allow IF EXISTS for UPDATE statements
> -
>
> Key: CASSANDRA-8610
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8610
> Project: Cassandra
> Issue Type: Improvement
> Components: API
> Environment: Cassandra 2.1.2
>Reporter: DOAN DuyHai
>Assignee: Prajakta Bhosale
>Priority: Minor
> Attachments: 8610.patch
>
>
> While creating a hands-on exercice for Cassandra, I was facing a quite
> annoying limitation.
> Let's take this table:
> {code:sql}
> CREATE TABLE killrchat.chat_rooms(
> room_name text,
> creation_date timestamp,
> banner text,
> creator text,
> participants set,
> PRIMARY KEY(room_name));
> {code}
> Upon a new participant joining the room, to be concurrency-proof (avoiding
> mutating the participants set if the room is deleted concurrently), I would
> like to issue this query:
> {code:sql}
> UPDATE chat_rooms SET participants = participants + {'johnny'} WHERE
> room_name = 'games' IF EXISTS;
> {code}
> Unfortunately the clause IF EXISTS is not allowed for UPDATE statements.
> Similarly I tried
> {code:sql}
> UPDATE chat_rooms SET participants = participants + {'johnny'} WHERE
> room_name = 'games' IF room_name='games';
> {code}
> It doesn't work either, it is not allowed to use one column of the primary
> key as condition column for LWT (why ? mystery).
> So far, the only work-around I found is:
> {code:sql}
> UPDATE chat_rooms SET participants = participants + {'johnny'} WHERE
> room_name = 'games' IF name='games';
> {code}
> I added an extra column called *name* which is just the duplicate of the
> partition key *room_name*. It does work but is not very elegant.
> I believe there are legit use cases for UPDATE ... IF EXISTS;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-8736:
--
Description:
After we upgraded C* to 2.1.2, with new installation, sometimes on creating
Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
release. Thanks.
{noformat}
$CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f ${dbDefaultDataDir}/${DATA_SCRIPT}”.
*** glibc detected *** python: corrupted double-linked list: 0x02dc95f0
***
=== Backtrace: =
/lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
/lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
/usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
/usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
/usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
/usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
/usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
/usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
/usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
/usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
/usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
/lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
/lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
=== Memory map:
0040-00401000 r-xp 08:06 450964
/usr/bin/python
0060-00601000 rw-p 08:06 450964
/usr/bin/python
024ba000-02f29000 rw-p 00:00 0 [heap]
7fd1b000-7fd1b0172000 rw-p 00:00 0
7fd1b0172000-7fd1b400 ---p 00:00 0
7fd1b400-7fd1b4021000 rw-p 00:00 0
7fd1b4021000-7fd1b800 ---p 00:00 0
7fd1b800-7fd1b8021000 rw-p 00:00 0
7fd1b8021000-7fd1bc00 ---p 00:00 0
7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
/lib64/libgcc_s-4.4.6-20120305.so.1
7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
7fd1bd7e8000-7fd1bd7e9000 ---p 00:00 0
7fd1bd7e9000-7fd1be1e9000 rw-p 00:00 0
7fd1be1e9000-7fd1be1ed000 r-xp 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be1ed000-7fd1be3ed000 ---p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ed000-7fd1be3ef000 rw-p 4000 08:06 180906
/usr/lib64/python2.6/lib-dynload/selectmodule.so
7fd1be3ef000-7fd1be3f2000 r-xp 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be3f2000-7fd1be5f1000 ---p 3000 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be5f1000-7fd1be5f2000 rw-p 2000 08:06 180869
/usr/lib64/python2.6/lib-dynload/_json.so
7fd1be5f2000-7fd1be5f5000 r-xp 08:06 180866
/usr/lib64/python2.6/lib-dynload/_hashlib.so
[jira] [Commented] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305538#comment-14305538
]
Michael Shuler commented on CASSANDRA-8736:
---
Python 2.6? Could you provide a little more detail about this OS env, why
you're using 2.6 (we have a 2.7 dependency documented, but I'm not sure how
hard of a dep)?
> core dump on creating keyspaces
>
>
> Key: CASSANDRA-8736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8736
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Linux
>Reporter: mike
>Priority: Minor
>
> After we upgraded C* to 2.1.2, with new installation, sometimes on creating
> Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
> release. Thanks.
> {noformat}
> $CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f
> ${dbDefaultDataDir}/${DATA_SCRIPT}”.
> *** glibc detected *** python: corrupted double-linked list:
> 0x02dc95f0 ***
> === Backtrace: =
> /lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
> /lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
> /usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
> /usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
> /usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
> /usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
> /usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
> /lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
> /lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
> === Memory map:
> 0040-00401000 r-xp 08:06 450964
> /usr/bin/python
> 0060-00601000 rw-p 08:06 450964
> /usr/bin/python
> 024ba000-02f29000 rw-p 00:00 0
> [heap]
> 7fd1b000-7fd1b0172000 rw-p 00:00 0
> 7fd1b0172000-7fd1b400 ---p 00:00 0
> 7fd1b400-7fd1b4021000 rw-p 00:00 0
> 7fd1b4021000-7fd1b800 ---p 00:00 0
> 7fd1b800-7fd1b8021000 rw-p 00:00 0
> 7fd1b8021000-7fd1bc00 ---p 00:00 0
> 7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
> 7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
> 7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
> 7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
> 7fd1bd7e8000-7fd1bd7e9000 ---p 00:00 0
> 7fd1bd7e9000-7fd1be1e9000 rw-p 00:00 0
> 7fd1be1e9000-7fd1be1ed000 r-xp 08:06 180906
> /usr/lib64/python2.6/lib-dynload/selectmodule
cassandra git commit: Add support for UPDATE ... IF EXISTS
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 2d8bddb06 -> 0f5ab577b
Add support for UPDATE ... IF EXISTS
Patch by Prajakta Bhosale; reviewed by Tyler Hobbs for CASSANDRA-8610
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0f5ab577
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0f5ab577
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0f5ab577
Branch: refs/heads/cassandra-2.0
Commit: 0f5ab577bcc106aa487dd5b05ca0b991c9a96a04
Parents: 2d8bddb
Author: Prajakta Bhosale
Authored: Wed Feb 4 11:23:41 2015 -0600
Committer: Tyler Hobbs
Committed: Wed Feb 4 11:23:41 2015 -0600
--
CHANGES.txt | 1 +
pylib/cqlshlib/cql3handling.py | 2 +-
pylib/cqlshlib/helptopics.py | 3 ++-
src/java/org/apache/cassandra/cql3/Cql.g | 6 --
.../apache/cassandra/cql3/statements/UpdateStatement.java| 8 +---
5 files changed, 13 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5ab577/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 375dcfe..f4c96dc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.13:
+ * Add support for UPDATE ... IF EXISTS (CASSANDRA-8610)
* Fix reversal of list prepends (CASSANDRA-8733)
* Prevent non-zero default_time_to_live on tables with counters
(CASSANDRA-8678)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5ab577/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 0b7863c..9e9b6ad 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -15,7 +15,6 @@
# limitations under the License.
import re
-from warnings import warn
from .cqlhandling import CqlParsingRuleSet, Hint
from cql.cqltypes import (cql_types, lookup_casstype, CompositeType, UTF8Type,
ColumnToCollectionType, CounterColumnType, DateType)
@@ -715,6 +714,7 @@ syntax_rules += r'''
( "AND" [updateopt]= )* )?
"SET" ( "," )*
"WHERE"
+( "IF" "EXISTS" )?
;
::= updatecol=
( "=" update_rhs=( | )
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5ab577/pylib/cqlshlib/helptopics.py
--
diff --git a/pylib/cqlshlib/helptopics.py b/pylib/cqlshlib/helptopics.py
index 2af8db2..0b08726 100644
--- a/pylib/cqlshlib/helptopics.py
+++ b/pylib/cqlshlib/helptopics.py
@@ -563,7 +563,8 @@ class CQL3HelpTopics(CQLHelpTopics):
UPDATE [.]
[USING [TIMESTAMP ]
[AND TTL ]]
- SET name1 = value1, name2 = value2 WHERE = keyval;
+ SET name1 = value1, name2 = value2 WHERE = keyval
+ [IF EXISTS];
An UPDATE is used to write one or more columns to a record in a table.
No results are returned. The record's primary key must be completely
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5ab577/src/java/org/apache/cassandra/cql3/Cql.g
--
diff --git a/src/java/org/apache/cassandra/cql3/Cql.g
b/src/java/org/apache/cassandra/cql3/Cql.g
index a80746c..0595c25 100644
--- a/src/java/org/apache/cassandra/cql3/Cql.g
+++ b/src/java/org/apache/cassandra/cql3/Cql.g
@@ -360,18 +360,20 @@ updateStatement returns [UpdateStatement.ParsedUpdate
expr]
@init {
Attributes.Raw attrs = new Attributes.Raw();
List> operations = new
ArrayList>();
+boolean ifExists = false;
}
: K_UPDATE cf=columnFamilyName
( usingClause[attrs] )?
K_SET columnOperation[operations] (',' columnOperation[operations])*
K_WHERE wclause=whereClause
- ( K_IF conditions=updateConditions )?
+ ( K_IF ( K_EXISTS { ifExists = true; } | conditions=updateConditions ))?
{
return new UpdateStatement.ParsedUpdate(cf,
attrs,
operations,
wclause,
- conditions == null ?
Collections.>emptyList() :
conditions);
+ conditions == null ?
Collections.>emptyList() :
conditions,
+ ifExists);
[jira] [Commented] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305540#comment-14305540
]
mike commented on CASSANDRA-8736:
-
yes, that is what we have. We will try to upgrade python to see if it will
resolve the issue.
Thanks
> core dump on creating keyspaces
>
>
> Key: CASSANDRA-8736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8736
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Linux
>Reporter: mike
>Priority: Minor
>
> After we upgraded C* to 2.1.2, with new installation, sometimes on creating
> Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
> release. Thanks.
> {noformat}
> $CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f
> ${dbDefaultDataDir}/${DATA_SCRIPT}”.
> *** glibc detected *** python: corrupted double-linked list:
> 0x02dc95f0 ***
> === Backtrace: =
> /lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
> /lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
> /usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
> /usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
> /usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
> /usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
> /usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
> /lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
> /lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
> === Memory map:
> 0040-00401000 r-xp 08:06 450964
> /usr/bin/python
> 0060-00601000 rw-p 08:06 450964
> /usr/bin/python
> 024ba000-02f29000 rw-p 00:00 0
> [heap]
> 7fd1b000-7fd1b0172000 rw-p 00:00 0
> 7fd1b0172000-7fd1b400 ---p 00:00 0
> 7fd1b400-7fd1b4021000 rw-p 00:00 0
> 7fd1b4021000-7fd1b800 ---p 00:00 0
> 7fd1b800-7fd1b8021000 rw-p 00:00 0
> 7fd1b8021000-7fd1bc00 ---p 00:00 0
> 7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
> 7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
> 7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
> 7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
> 7fd1bd7e8000-7fd1bd7e9000 ---p 00:00 0
> 7fd1bd7e9000-7fd1be1e9000 rw-p 00:00 0
> 7fd1be1e9000-7fd1be1ed000 r-xp 08:06 180906
> /usr/lib64/python2.6/lib-dynload/selectmodule.so
> 7fd1be1ed000-7fd1be3ed000 ---p 4000 08:06 180906
> /usr
[jira] [Created] (CASSANDRA-8737) AdjacentDataCompactionStrategy
Benedict created CASSANDRA-8737: --- Summary: AdjacentDataCompactionStrategy Key: CASSANDRA-8737 URL: https://issues.apache.org/jira/browse/CASSANDRA-8737 Project: Cassandra Issue Type: Bug Components: Core Reporter: Benedict Assignee: Benedict Fix For: 3.0 In the original ticket for dealing with timeseries data that introduced DTCS, the first suggestion was for an approach that compacted adjacent data (by clustering columns) together until a single page (or some fixed multiple of pages) on average contained only one partition's worth of data. The idea would be to compact any sstables that overlap their clustering components, so that only one (or a fixed number) of sstables need to be queried for any clustering range. The upshot of this would be tunable compaction burden to get optimal read behaviour, more explicitly defined than the decay in DTCS. The basic idea would be to select boundary clustering prefixes based on the current data occupancy within those ranges, falling roughly along the boundaries of the existing sstables, but so that any overlapping tail falls one side or the other. We then compact all overlapping sstables, and split the results into one side or another of the boundary (or across multiple boundaries). If there are no historical updates, this gives pretty optimal behaviour; we only compact files until we get to our packing threshold (so that reads are known to be at the configured efficiency), and then stop. If updates to older records appear, they would be compacted into their boundary buckets, and left there until we had enough files in a boundary (probably following normal STCS rules) that it warranted compaction. The benefit is that such historical updates are still accounted for and bounded by comparison to DTCS, and the configuration parameters give more tunable characteristics, with explicit expectations (i.e. one seek per X bytes read in a partition; higher X may imply more compaction, lower more merges and seeks on read). It also may permit us some easy optimisations further up the stack, since we can guarantee the boundaries of overlap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8732) Make inter-node timeouts tolerate clock skew and drift
[ https://issues.apache.org/jira/browse/CASSANDRA-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305547#comment-14305547 ] Benedict commented on CASSANDRA-8732: - The simplest approach I was thinking of to bound this is to send the time remaining, as well as the expected wall clock expiry. These can both be used on the remote node to do something sensible, e.g. pick the one closest to half the timeout interval, so that we're conservative in both directions (i.e. never keeping the message too long, nor expiring them too aggressively). My biggest concern here is nodes being seen as down because clock skew temporarily got large enough to have messages get dropped much too aggressively for the response to be returned. I also agree it's not super duper pressing, I just wanted to log the ticket for discussion. But it's also pretty easy to introduce. Just send a delta along with the wall clock, and have some simple machinery on the other end to select which one to use. > Make inter-node timeouts tolerate clock skew and drift > -- > > Key: CASSANDRA-8732 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8732 > Project: Cassandra > Issue Type: Improvement >Reporter: Ariel Weisberg > > Right now internode timeouts rely on currentTimeMillis() (and NTP) to make > sure that tasks don't expire before they arrive. > Every receiver needs to deduce the offset between its nanoTime and the remote > nanoTime. I don't think currentTimeMillis is a good choice because it is > designed to be manipulated by operators and NTP. I would probably be > comfortable assuming that nanoTime isn't going to move in significant ways > without something that could be classified as operator error happening. > I suspect the one timing method you can rely on being accurate is nanoTime > within a node (on average) and that a node can report on its own scheduling > jitter (on average). > Finding the offset requires knowing what the network latency is in one > direction. > One way to do this would be to periodically send a ping request which > generates a series of ping responses at fixed intervals (maybe by UDP?). The > responses should corrected for scheduling jitter since the fixed intervals > may not be exactly achieved by the sender. By measuring the time deviation > between ping responses and their expected arrival time (based on the > interval) and correcting for the remotely reported scheduling jitter, you > should be able to measure latency in one direction. > A weighted moving average (only correct for drift, not readjustment) of these > measurements would eventually converge on a close answer and would not be > impacted by outlier measurements. It may also make sense to drop the largest > N samples to improve accuracy. > One you know network latency you can add that to the timestamp of each ping > and compare to the local clock and know what the offset is. > These measurements won't calculate the offset to be too small (timeouts fire > early), but could calculate the offset to be too large (timeouts fire late). > The conditions where you the offset won't be accurate are the conditions > where you also want them firing reliably. This and bootstrapping in bad > conditions is what I am most uncertain of. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8738) "/etc/init.d/cassandra stop" prints OK even when it doesn't work
Donald Smith created CASSANDRA-8738:
---
Summary: "/etc/init.d/cassandra stop" prints OK even when it
doesn't work
Key: CASSANDRA-8738
URL: https://issues.apache.org/jira/browse/CASSANDRA-8738
Project: Cassandra
Issue Type: Improvement
Reporter: Donald Smith
Sometimes I do {{/etc/init.d/cassandra stop}} and it prints out OK, but the
server is still running. (This happens, for example, if it's busy doing GCs.)
The current init script prints out OK after sleeping but without checking if
the process really stopped. I suggest changing it to:
{noformat}
pd0-cassandra16 ~> diff -C 1 cassandra cassandra-original
*** cassandra 2015-02-04 09:15:58.088209988 -0800
--- cassandra-original 2015-02-04 09:15:40.293767501 -0800
***
*** 69,77
sleep 5
! THE_STATUS=`$0 status`
! if [[ $THE_STATUS == *"stopped"* ]]
! then
!echo "OK"
! else
!echo "ERROR: could not stop the process: $THE_STATUS"
! fi
;;
--- 69,71
sleep 5
! echo "OK"
;;
{noformat}
Then it prints out OK only if the stop succeeded. Otherwise it prints out a
message like
{quote}
"ERROR: could not stop the process: cassandra (pid 10764) is running...
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8738) "/etc/init.d/cassandra stop" prints OK even when it doesn't work
[
https://issues.apache.org/jira/browse/CASSANDRA-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305548#comment-14305548
]
Donald Smith commented on CASSANDRA-8738:
-
Here's the change in context:
{noformat}
stop)
# Cassandra shutdown
echo -n "Shutdown Cassandra: "
su $CASSANDRA_OWNR -c "kill `cat $pid_file`"
for t in `seq 40`; do $0 status > /dev/null 2>&1 && sleep 0.5 || break;
done
# Adding a sleep here to give jmx time to wind down (CASSANDRA-4483).
Not ideal...
# Adam Holmberg suggests this, but that would break if the jmx port is
changed
# for t in `seq 40`; do netstat -tnlp | grep "0.0.0.0:7199" > /dev/null
2>&1 && sleep 0.1 || break; done
sleep 5
THE_STATUS=`$0 status`
if [[ $THE_STATUS == *"stopped"* ]]
then
echo "OK"
else
echo "ERROR: could not stop the process: $THE_STATUS"
fi
;;
{noformat}
> "/etc/init.d/cassandra stop" prints OK even when it doesn't work
>
>
> Key: CASSANDRA-8738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8738
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Donald Smith
>
> Sometimes I do {{/etc/init.d/cassandra stop}} and it prints out OK, but the
> server is still running. (This happens, for example, if it's busy doing
> GCs.) The current init script prints out OK after sleeping but without
> checking if the process really stopped. I suggest changing it to:
> {noformat}
> pd0-cassandra16 ~> diff -C 1 cassandra cassandra-original
> *** cassandra 2015-02-04 09:15:58.088209988 -0800
> --- cassandra-original 2015-02-04 09:15:40.293767501 -0800
> ***
> *** 69,77
> sleep 5
> ! THE_STATUS=`$0 status`
> ! if [[ $THE_STATUS == *"stopped"* ]]
> ! then
> !echo "OK"
> ! else
> !echo "ERROR: could not stop the process: $THE_STATUS"
> ! fi
> ;;
> --- 69,71
> sleep 5
> ! echo "OK"
> ;;
> {noformat}
> Then it prints out OK only if the stop succeeded. Otherwise it prints out a
> message like
> {quote}
> "ERROR: could not stop the process: cassandra (pid 10764) is running...
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[1/3] cassandra git commit: Add support for UPDATE ... IF EXISTS
Repository: cassandra
Updated Branches:
refs/heads/trunk 32b82879d -> 296d11b98
Add support for UPDATE ... IF EXISTS
Patch by Prajakta Bhosale; reviewed by Tyler Hobbs for CASSANDRA-8610
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0f5ab577
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0f5ab577
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0f5ab577
Branch: refs/heads/trunk
Commit: 0f5ab577bcc106aa487dd5b05ca0b991c9a96a04
Parents: 2d8bddb
Author: Prajakta Bhosale
Authored: Wed Feb 4 11:23:41 2015 -0600
Committer: Tyler Hobbs
Committed: Wed Feb 4 11:23:41 2015 -0600
--
CHANGES.txt | 1 +
pylib/cqlshlib/cql3handling.py | 2 +-
pylib/cqlshlib/helptopics.py | 3 ++-
src/java/org/apache/cassandra/cql3/Cql.g | 6 --
.../apache/cassandra/cql3/statements/UpdateStatement.java| 8 +---
5 files changed, 13 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5ab577/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 375dcfe..f4c96dc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.13:
+ * Add support for UPDATE ... IF EXISTS (CASSANDRA-8610)
* Fix reversal of list prepends (CASSANDRA-8733)
* Prevent non-zero default_time_to_live on tables with counters
(CASSANDRA-8678)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5ab577/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 0b7863c..9e9b6ad 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -15,7 +15,6 @@
# limitations under the License.
import re
-from warnings import warn
from .cqlhandling import CqlParsingRuleSet, Hint
from cql.cqltypes import (cql_types, lookup_casstype, CompositeType, UTF8Type,
ColumnToCollectionType, CounterColumnType, DateType)
@@ -715,6 +714,7 @@ syntax_rules += r'''
( "AND" [updateopt]= )* )?
"SET" ( "," )*
"WHERE"
+( "IF" "EXISTS" )?
;
::= updatecol=
( "=" update_rhs=( | )
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5ab577/pylib/cqlshlib/helptopics.py
--
diff --git a/pylib/cqlshlib/helptopics.py b/pylib/cqlshlib/helptopics.py
index 2af8db2..0b08726 100644
--- a/pylib/cqlshlib/helptopics.py
+++ b/pylib/cqlshlib/helptopics.py
@@ -563,7 +563,8 @@ class CQL3HelpTopics(CQLHelpTopics):
UPDATE [.]
[USING [TIMESTAMP ]
[AND TTL ]]
- SET name1 = value1, name2 = value2 WHERE = keyval;
+ SET name1 = value1, name2 = value2 WHERE = keyval
+ [IF EXISTS];
An UPDATE is used to write one or more columns to a record in a table.
No results are returned. The record's primary key must be completely
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5ab577/src/java/org/apache/cassandra/cql3/Cql.g
--
diff --git a/src/java/org/apache/cassandra/cql3/Cql.g
b/src/java/org/apache/cassandra/cql3/Cql.g
index a80746c..0595c25 100644
--- a/src/java/org/apache/cassandra/cql3/Cql.g
+++ b/src/java/org/apache/cassandra/cql3/Cql.g
@@ -360,18 +360,20 @@ updateStatement returns [UpdateStatement.ParsedUpdate
expr]
@init {
Attributes.Raw attrs = new Attributes.Raw();
List> operations = new
ArrayList>();
+boolean ifExists = false;
}
: K_UPDATE cf=columnFamilyName
( usingClause[attrs] )?
K_SET columnOperation[operations] (',' columnOperation[operations])*
K_WHERE wclause=whereClause
- ( K_IF conditions=updateConditions )?
+ ( K_IF ( K_EXISTS { ifExists = true; } | conditions=updateConditions ))?
{
return new UpdateStatement.ParsedUpdate(cf,
attrs,
operations,
wclause,
- conditions == null ?
Collections.>emptyList() :
conditions);
+ conditions == null ?
Collections.>emptyList() :
conditions,
+ ifExists);
}
;
h
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt pylib/cqlshlib/cql3handling.py Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bcfb652b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bcfb652b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bcfb652b Branch: refs/heads/cassandra-2.1 Commit: bcfb652b1a6dece77cce3e09ece2d0ef6fec9598 Parents: dc74ae6 0f5ab57 Author: Tyler Hobbs Authored: Wed Feb 4 11:26:55 2015 -0600 Committer: Tyler Hobbs Committed: Wed Feb 4 11:26:55 2015 -0600 -- CHANGES.txt | 1 + pylib/cqlshlib/cql3handling.py | 2 +- pylib/cqlshlib/helptopics.py | 3 ++- src/java/org/apache/cassandra/cql3/Cql.g | 6 -- .../apache/cassandra/cql3/statements/UpdateStatement.java| 8 +--- 5 files changed, 13 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/bcfb652b/CHANGES.txt -- diff --cc CHANGES.txt index f70db7a,f4c96dc..192939b --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,90 -1,5 +1,91 @@@ -2.0.13: +2.1.3 + * Upgrade libthrift to 0.9.2 (CASSANDRA-8685) + * Don't use the shared ref in sstableloader (CASSANDRA-8704) + * Purge internal prepared statements if related tables or + keyspaces are dropped (CASSANDRA-8693) + * (cqlsh) Handle unicode BOM at start of files (CASSANDRA-8638) + * Stop compactions before exiting offline tools (CASSANDRA-8623) + * Update tools/stress/README.txt to match current behaviour (CASSANDRA-7933) + * Fix schema from Thrift conversion with empty metadata (CASSANDRA-8695) + * Safer Resource Management (CASSANDRA-7705) + * Make sure we compact highly overlapping cold sstables with + STCS (CASSANDRA-8635) + * rpc_interface and listen_interface generate NPE on startup when specified + interface doesn't exist (CASSANDRA-8677) + * Fix ArrayIndexOutOfBoundsException in nodetool cfhistograms (CASSANDRA-8514) + * Switch from yammer metrics for nodetool cf/proxy histograms (CASSANDRA-8662) + * Make sure we don't add tmplink files to the compaction + strategy (CASSANDRA-8580) + * (cqlsh) Handle maps with blob keys (CASSANDRA-8372) + * (cqlsh) Handle DynamicCompositeType schemas correctly (CASSANDRA-8563) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Add tooling to detect hot partitions (CASSANDRA-7974) + * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) + * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) + * Don't allow starting multiple inc repairs on the same sstables (CASSANDRA-8316) + * Invalidate prepared BATCH statements when related tables + or keyspaces are dropped (CASSANDRA-8652) + * Fix missing results in secondary index queries on collections + with ALLOW FILTERING (CASSANDRA-8421) + * Expose EstimatedHistogram metrics for range slices (CASSANDRA-8627) + * (cqlsh) Escape clqshrc passwords properly (CASSANDRA-8618) + * Fix NPE when passing wrong argument in ALTER TABLE statement (CASSANDRA-8355) + * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599) + * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537) + * Fix case-sensitivity of index name on CREATE and DROP INDEX + statements (CASSANDRA-8365) + * Better detection/logging for corruption in compressed sstables (CASSANDRA-8192) + * Use the correct repairedAt value when closing writer (CASSANDRA-8570) + * (cqlsh) Handle a schema mismatch being detected on startup (CASSANDRA-8512) + * Properly calculate expected write size during compaction (CASSANDRA-8532) + * Invalidate affected prepared statements when a table's columns + are altered (CASSANDRA-7910) + * Stress - user defined writes should populate sequentally (CASSANDRA-8524) + * Fix regression in SSTableRewriter causing some rows to become unreadable + during compaction (CASSANDRA-8429) + * Run major compactions for repaired/unrepaired in parallel (CASSANDRA-8510) + * (cqlsh) Fix compression options in DESCRIBE TABLE output when compression + is disabled (CASSANDRA-8288) + * (cqlsh) Fix DESCRIBE output after keyspaces are altered (CASSANDRA-7623) + * Make sure we set lastCompactedKey correctly (CASSANDRA-8463) + * (cqlsh) Fix output of CONSISTENCY command (CASSANDRA-8507) + * (cqlsh) Fixed the handling of LIST statements (CASSANDRA-8370) + * Make sstablescrub check leveled manifest again (CASSANDRA-8432) + * Check first/last keys in sstable when giving out positions (CASSANDRA-8458) + * Disable mmap on Windows (CASSANDRA-6993) + * Add missing Consist
[1/2] cassandra git commit: Add support for UPDATE ... IF EXISTS
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 dc74ae67a -> bcfb652b1
Add support for UPDATE ... IF EXISTS
Patch by Prajakta Bhosale; reviewed by Tyler Hobbs for CASSANDRA-8610
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0f5ab577
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0f5ab577
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0f5ab577
Branch: refs/heads/cassandra-2.1
Commit: 0f5ab577bcc106aa487dd5b05ca0b991c9a96a04
Parents: 2d8bddb
Author: Prajakta Bhosale
Authored: Wed Feb 4 11:23:41 2015 -0600
Committer: Tyler Hobbs
Committed: Wed Feb 4 11:23:41 2015 -0600
--
CHANGES.txt | 1 +
pylib/cqlshlib/cql3handling.py | 2 +-
pylib/cqlshlib/helptopics.py | 3 ++-
src/java/org/apache/cassandra/cql3/Cql.g | 6 --
.../apache/cassandra/cql3/statements/UpdateStatement.java| 8 +---
5 files changed, 13 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5ab577/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 375dcfe..f4c96dc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.13:
+ * Add support for UPDATE ... IF EXISTS (CASSANDRA-8610)
* Fix reversal of list prepends (CASSANDRA-8733)
* Prevent non-zero default_time_to_live on tables with counters
(CASSANDRA-8678)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5ab577/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 0b7863c..9e9b6ad 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -15,7 +15,6 @@
# limitations under the License.
import re
-from warnings import warn
from .cqlhandling import CqlParsingRuleSet, Hint
from cql.cqltypes import (cql_types, lookup_casstype, CompositeType, UTF8Type,
ColumnToCollectionType, CounterColumnType, DateType)
@@ -715,6 +714,7 @@ syntax_rules += r'''
( "AND" [updateopt]= )* )?
"SET" ( "," )*
"WHERE"
+( "IF" "EXISTS" )?
;
::= updatecol=
( "=" update_rhs=( | )
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5ab577/pylib/cqlshlib/helptopics.py
--
diff --git a/pylib/cqlshlib/helptopics.py b/pylib/cqlshlib/helptopics.py
index 2af8db2..0b08726 100644
--- a/pylib/cqlshlib/helptopics.py
+++ b/pylib/cqlshlib/helptopics.py
@@ -563,7 +563,8 @@ class CQL3HelpTopics(CQLHelpTopics):
UPDATE [.]
[USING [TIMESTAMP ]
[AND TTL ]]
- SET name1 = value1, name2 = value2 WHERE = keyval;
+ SET name1 = value1, name2 = value2 WHERE = keyval
+ [IF EXISTS];
An UPDATE is used to write one or more columns to a record in a table.
No results are returned. The record's primary key must be completely
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0f5ab577/src/java/org/apache/cassandra/cql3/Cql.g
--
diff --git a/src/java/org/apache/cassandra/cql3/Cql.g
b/src/java/org/apache/cassandra/cql3/Cql.g
index a80746c..0595c25 100644
--- a/src/java/org/apache/cassandra/cql3/Cql.g
+++ b/src/java/org/apache/cassandra/cql3/Cql.g
@@ -360,18 +360,20 @@ updateStatement returns [UpdateStatement.ParsedUpdate
expr]
@init {
Attributes.Raw attrs = new Attributes.Raw();
List> operations = new
ArrayList>();
+boolean ifExists = false;
}
: K_UPDATE cf=columnFamilyName
( usingClause[attrs] )?
K_SET columnOperation[operations] (',' columnOperation[operations])*
K_WHERE wclause=whereClause
- ( K_IF conditions=updateConditions )?
+ ( K_IF ( K_EXISTS { ifExists = true; } | conditions=updateConditions ))?
{
return new UpdateStatement.ParsedUpdate(cf,
attrs,
operations,
wclause,
- conditions == null ?
Collections.>emptyList() :
conditions);
+ conditions == null ?
Collections.>emptyList() :
conditions,
+ ifExists);
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/296d11b9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/296d11b9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/296d11b9 Branch: refs/heads/trunk Commit: 296d11b985a7c15cedf8d95ca85a596db48dc470 Parents: 32b8287 bcfb652 Author: Tyler Hobbs Authored: Wed Feb 4 11:27:17 2015 -0600 Committer: Tyler Hobbs Committed: Wed Feb 4 11:27:17 2015 -0600 -- CHANGES.txt | 1 + pylib/cqlshlib/cql3handling.py | 2 +- pylib/cqlshlib/helptopics.py | 3 ++- src/java/org/apache/cassandra/cql3/Cql.g | 6 -- .../apache/cassandra/cql3/statements/UpdateStatement.java| 8 +--- 5 files changed, 13 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/296d11b9/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/296d11b9/pylib/cqlshlib/cql3handling.py -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/296d11b9/pylib/cqlshlib/helptopics.py -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/296d11b9/src/java/org/apache/cassandra/cql3/Cql.g -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/296d11b9/src/java/org/apache/cassandra/cql3/statements/UpdateStatement.java --
[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt pylib/cqlshlib/cql3handling.py Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bcfb652b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bcfb652b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bcfb652b Branch: refs/heads/trunk Commit: bcfb652b1a6dece77cce3e09ece2d0ef6fec9598 Parents: dc74ae6 0f5ab57 Author: Tyler Hobbs Authored: Wed Feb 4 11:26:55 2015 -0600 Committer: Tyler Hobbs Committed: Wed Feb 4 11:26:55 2015 -0600 -- CHANGES.txt | 1 + pylib/cqlshlib/cql3handling.py | 2 +- pylib/cqlshlib/helptopics.py | 3 ++- src/java/org/apache/cassandra/cql3/Cql.g | 6 -- .../apache/cassandra/cql3/statements/UpdateStatement.java| 8 +--- 5 files changed, 13 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/bcfb652b/CHANGES.txt -- diff --cc CHANGES.txt index f70db7a,f4c96dc..192939b --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,90 -1,5 +1,91 @@@ -2.0.13: +2.1.3 + * Upgrade libthrift to 0.9.2 (CASSANDRA-8685) + * Don't use the shared ref in sstableloader (CASSANDRA-8704) + * Purge internal prepared statements if related tables or + keyspaces are dropped (CASSANDRA-8693) + * (cqlsh) Handle unicode BOM at start of files (CASSANDRA-8638) + * Stop compactions before exiting offline tools (CASSANDRA-8623) + * Update tools/stress/README.txt to match current behaviour (CASSANDRA-7933) + * Fix schema from Thrift conversion with empty metadata (CASSANDRA-8695) + * Safer Resource Management (CASSANDRA-7705) + * Make sure we compact highly overlapping cold sstables with + STCS (CASSANDRA-8635) + * rpc_interface and listen_interface generate NPE on startup when specified + interface doesn't exist (CASSANDRA-8677) + * Fix ArrayIndexOutOfBoundsException in nodetool cfhistograms (CASSANDRA-8514) + * Switch from yammer metrics for nodetool cf/proxy histograms (CASSANDRA-8662) + * Make sure we don't add tmplink files to the compaction + strategy (CASSANDRA-8580) + * (cqlsh) Handle maps with blob keys (CASSANDRA-8372) + * (cqlsh) Handle DynamicCompositeType schemas correctly (CASSANDRA-8563) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Add tooling to detect hot partitions (CASSANDRA-7974) + * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) + * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) + * Don't allow starting multiple inc repairs on the same sstables (CASSANDRA-8316) + * Invalidate prepared BATCH statements when related tables + or keyspaces are dropped (CASSANDRA-8652) + * Fix missing results in secondary index queries on collections + with ALLOW FILTERING (CASSANDRA-8421) + * Expose EstimatedHistogram metrics for range slices (CASSANDRA-8627) + * (cqlsh) Escape clqshrc passwords properly (CASSANDRA-8618) + * Fix NPE when passing wrong argument in ALTER TABLE statement (CASSANDRA-8355) + * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599) + * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537) + * Fix case-sensitivity of index name on CREATE and DROP INDEX + statements (CASSANDRA-8365) + * Better detection/logging for corruption in compressed sstables (CASSANDRA-8192) + * Use the correct repairedAt value when closing writer (CASSANDRA-8570) + * (cqlsh) Handle a schema mismatch being detected on startup (CASSANDRA-8512) + * Properly calculate expected write size during compaction (CASSANDRA-8532) + * Invalidate affected prepared statements when a table's columns + are altered (CASSANDRA-7910) + * Stress - user defined writes should populate sequentally (CASSANDRA-8524) + * Fix regression in SSTableRewriter causing some rows to become unreadable + during compaction (CASSANDRA-8429) + * Run major compactions for repaired/unrepaired in parallel (CASSANDRA-8510) + * (cqlsh) Fix compression options in DESCRIBE TABLE output when compression + is disabled (CASSANDRA-8288) + * (cqlsh) Fix DESCRIBE output after keyspaces are altered (CASSANDRA-7623) + * Make sure we set lastCompactedKey correctly (CASSANDRA-8463) + * (cqlsh) Fix output of CONSISTENCY command (CASSANDRA-8507) + * (cqlsh) Fixed the handling of LIST statements (CASSANDRA-8370) + * Make sstablescrub check leveled manifest again (CASSANDRA-8432) + * Check first/last keys in sstable when giving out positions (CASSANDRA-8458) + * Disable mmap on Windows (CASSANDRA-6993) + * Add missing ConsistencyLeve
[jira] [Commented] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305556#comment-14305556
]
mike commented on CASSANDRA-8736:
-
here are some detail of our env:
CentOS release 6.3 (Final)
Linux version 2.6.32-431.3.1.el6.x86_64 (mockbu...@c6b10.bsys.dev.centos.org)
(gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) #1 SMP Fri Jan 3 21:39:27
UTC 2014
Python 2.6.6
java version "1.7.0_67"
Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
Thanks
> core dump on creating keyspaces
>
>
> Key: CASSANDRA-8736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8736
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Linux
>Reporter: mike
>Priority: Minor
>
> After we upgraded C* to 2.1.2, with new installation, sometimes on creating
> Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
> release. Thanks.
> {noformat}
> $CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f
> ${dbDefaultDataDir}/${DATA_SCRIPT}”.
> *** glibc detected *** python: corrupted double-linked list:
> 0x02dc95f0 ***
> === Backtrace: =
> /lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
> /lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
> /usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
> /usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
> /usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
> /usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
> /usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
> /lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
> /lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
> === Memory map:
> 0040-00401000 r-xp 08:06 450964
> /usr/bin/python
> 0060-00601000 rw-p 08:06 450964
> /usr/bin/python
> 024ba000-02f29000 rw-p 00:00 0
> [heap]
> 7fd1b000-7fd1b0172000 rw-p 00:00 0
> 7fd1b0172000-7fd1b400 ---p 00:00 0
> 7fd1b400-7fd1b4021000 rw-p 00:00 0
> 7fd1b4021000-7fd1b800 ---p 00:00 0
> 7fd1b800-7fd1b8021000 rw-p 00:00 0
> 7fd1b8021000-7fd1bc00 ---p 00:00 0
> 7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
> 7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
> 7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
> 7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
> 7fd1bd7e8000-7fd
[jira] [Commented] (CASSANDRA-8738) "/etc/init.d/cassandra stop" prints OK even when it doesn't work
[
https://issues.apache.org/jira/browse/CASSANDRA-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305557#comment-14305557
]
Michael Shuler commented on CASSANDRA-8738:
---
What version are you using? Could you verify that the current init for 3.0 is
not working for you, and if not, work this out with proper usage of
start-stop-daemon?
https://github.com/apache/cassandra/blob/trunk/debian/init
> "/etc/init.d/cassandra stop" prints OK even when it doesn't work
>
>
> Key: CASSANDRA-8738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8738
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Donald Smith
>
> Sometimes I do {{/etc/init.d/cassandra stop}} and it prints out OK, but the
> server is still running. (This happens, for example, if it's busy doing
> GCs.) The current init script prints out OK after sleeping but without
> checking if the process really stopped. I suggest changing it to:
> {noformat}
> pd0-cassandra16 ~> diff -C 1 cassandra cassandra-original
> *** cassandra 2015-02-04 09:15:58.088209988 -0800
> --- cassandra-original 2015-02-04 09:15:40.293767501 -0800
> ***
> *** 69,77
> sleep 5
> ! THE_STATUS=`$0 status`
> ! if [[ $THE_STATUS == *"stopped"* ]]
> ! then
> !echo "OK"
> ! else
> !echo "ERROR: could not stop the process: $THE_STATUS"
> ! fi
> ;;
> --- 69,71
> sleep 5
> ! echo "OK"
> ;;
> {noformat}
> Then it prints out OK only if the stop succeeded. Otherwise it prints out a
> message like
> {quote}
> ERROR: could not stop the process: cassandra (pid 10764) is running...
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8738) "/etc/init.d/cassandra stop" prints OK even when it doesn't work
[
https://issues.apache.org/jira/browse/CASSANDRA-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Donald Smith updated CASSANDRA-8738:
Description:
Sometimes I do {{/etc/init.d/cassandra stop}} and it prints out OK, but the
server is still running. (This happens, for example, if it's busy doing GCs.)
The current init script prints out OK after sleeping but without checking if
the process really stopped. I suggest changing it to:
{noformat}
pd0-cassandra16 ~> diff -C 1 cassandra cassandra-original
*** cassandra 2015-02-04 09:15:58.088209988 -0800
--- cassandra-original 2015-02-04 09:15:40.293767501 -0800
***
*** 69,77
sleep 5
! THE_STATUS=`$0 status`
! if [[ $THE_STATUS == *"stopped"* ]]
! then
!echo "OK"
! else
!echo "ERROR: could not stop the process: $THE_STATUS"
! fi
;;
--- 69,71
sleep 5
! echo "OK"
;;
{noformat}
Then it prints out OK only if the stop succeeded. Otherwise it prints out a
message like
{quote}
ERROR: could not stop the process: cassandra (pid 10764) is running...
{quote}
was:
Sometimes I do {{/etc/init.d/cassandra stop}} and it prints out OK, but the
server is still running. (This happens, for example, if it's busy doing GCs.)
The current init script prints out OK after sleeping but without checking if
the process really stopped. I suggest changing it to:
{noformat}
pd0-cassandra16 ~> diff -C 1 cassandra cassandra-original
*** cassandra 2015-02-04 09:15:58.088209988 -0800
--- cassandra-original 2015-02-04 09:15:40.293767501 -0800
***
*** 69,77
sleep 5
! THE_STATUS=`$0 status`
! if [[ $THE_STATUS == *"stopped"* ]]
! then
!echo "OK"
! else
!echo "ERROR: could not stop the process: $THE_STATUS"
! fi
;;
--- 69,71
sleep 5
! echo "OK"
;;
{noformat}
Then it prints out OK only if the stop succeeded. Otherwise it prints out a
message like
{quote}
"ERROR: could not stop the process: cassandra (pid 10764) is running...
{quote}
> "/etc/init.d/cassandra stop" prints OK even when it doesn't work
>
>
> Key: CASSANDRA-8738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8738
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Donald Smith
>
> Sometimes I do {{/etc/init.d/cassandra stop}} and it prints out OK, but the
> server is still running. (This happens, for example, if it's busy doing
> GCs.) The current init script prints out OK after sleeping but without
> checking if the process really stopped. I suggest changing it to:
> {noformat}
> pd0-cassandra16 ~> diff -C 1 cassandra cassandra-original
> *** cassandra 2015-02-04 09:15:58.088209988 -0800
> --- cassandra-original 2015-02-04 09:15:40.293767501 -0800
> ***
> *** 69,77
> sleep 5
> ! THE_STATUS=`$0 status`
> ! if [[ $THE_STATUS == *"stopped"* ]]
> ! then
> !echo "OK"
> ! else
> !echo "ERROR: could not stop the process: $THE_STATUS"
> ! fi
> ;;
> --- 69,71
> sleep 5
> ! echo "OK"
> ;;
> {noformat}
> Then it prints out OK only if the stop succeeded. Otherwise it prints out a
> message like
> {quote}
> ERROR: could not stop the process: cassandra (pid 10764) is running...
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8732) Make inter-node timeouts tolerate clock skew and drift
[ https://issues.apache.org/jira/browse/CASSANDRA-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305564#comment-14305564 ] Sylvain Lebresne commented on CASSANDRA-8732: - bq. It is a common error and it is a nice to have for the database to handle it as well as possible. Fair enough, but dopping messages sooner or later than you should doesn't feel like the biggest problem if you've made that error. Maybe adding a system that tries to detect clock skew/drift and warn the operation if it thinks it has detected some would be sensible, and once we have that maybe using it to improve message dropping timeouts will be trivial, but let say I'm not convinced that respecting timeouts is important enough in itself to justify the complexity. > Make inter-node timeouts tolerate clock skew and drift > -- > > Key: CASSANDRA-8732 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8732 > Project: Cassandra > Issue Type: Improvement >Reporter: Ariel Weisberg > > Right now internode timeouts rely on currentTimeMillis() (and NTP) to make > sure that tasks don't expire before they arrive. > Every receiver needs to deduce the offset between its nanoTime and the remote > nanoTime. I don't think currentTimeMillis is a good choice because it is > designed to be manipulated by operators and NTP. I would probably be > comfortable assuming that nanoTime isn't going to move in significant ways > without something that could be classified as operator error happening. > I suspect the one timing method you can rely on being accurate is nanoTime > within a node (on average) and that a node can report on its own scheduling > jitter (on average). > Finding the offset requires knowing what the network latency is in one > direction. > One way to do this would be to periodically send a ping request which > generates a series of ping responses at fixed intervals (maybe by UDP?). The > responses should corrected for scheduling jitter since the fixed intervals > may not be exactly achieved by the sender. By measuring the time deviation > between ping responses and their expected arrival time (based on the > interval) and correcting for the remotely reported scheduling jitter, you > should be able to measure latency in one direction. > A weighted moving average (only correct for drift, not readjustment) of these > measurements would eventually converge on a close answer and would not be > impacted by outlier measurements. It may also make sense to drop the largest > N samples to improve accuracy. > One you know network latency you can add that to the timestamp of each ping > and compare to the local clock and know what the offset is. > These measurements won't calculate the offset to be too small (timeouts fire > early), but could calculate the offset to be too large (timeouts fire late). > The conditions where you the offset won't be accurate are the conditions > where you also want them firing reliably. This and bootstrapping in bad > conditions is what I am most uncertain of. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8738) "/etc/init.d/cassandra stop" prints OK even when it doesn't work
[
https://issues.apache.org/jira/browse/CASSANDRA-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305548#comment-14305548
]
Donald Smith edited comment on CASSANDRA-8738 at 2/4/15 5:36 PM:
-
Here's the change in context:
{noformat}
stop)
# Cassandra shutdown
echo -n "Shutdown Cassandra: "
su $CASSANDRA_OWNR -c "kill `cat $pid_file`"
for t in `seq 40`; do $0 status > /dev/null 2>&1 && sleep 0.5 || break;
done
# Adding a sleep here to give jmx time to wind down (CASSANDRA-4483).
Not ideal...
# Adam Holmberg suggests this, but that would break if the jmx port is
changed
# for t in `seq 40`; do netstat -tnlp | grep "0.0.0.0:7199" > /dev/null
2>&1 && sleep 0.1 || break; done
sleep 5
THE_STATUS=`$0 status`
if [[ $THE_STATUS == *"stopped"* ]]
then
echo "OK"
else
echo "ERROR: could not stop the process: $THE_STATUS"
exit 1
fi
;;
{noformat}
was (Author: thinkerfeeler):
Here's the change in context:
{noformat}
stop)
# Cassandra shutdown
echo -n "Shutdown Cassandra: "
su $CASSANDRA_OWNR -c "kill `cat $pid_file`"
for t in `seq 40`; do $0 status > /dev/null 2>&1 && sleep 0.5 || break;
done
# Adding a sleep here to give jmx time to wind down (CASSANDRA-4483).
Not ideal...
# Adam Holmberg suggests this, but that would break if the jmx port is
changed
# for t in `seq 40`; do netstat -tnlp | grep "0.0.0.0:7199" > /dev/null
2>&1 && sleep 0.1 || break; done
sleep 5
THE_STATUS=`$0 status`
if [[ $THE_STATUS == *"stopped"* ]]
then
echo "OK"
else
echo "ERROR: could not stop the process: $THE_STATUS"
fi
;;
{noformat}
> "/etc/init.d/cassandra stop" prints OK even when it doesn't work
>
>
> Key: CASSANDRA-8738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8738
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Donald Smith
>
> Sometimes I do {{/etc/init.d/cassandra stop}} and it prints out OK, but the
> server is still running. (This happens, for example, if it's busy doing
> GCs.) The current init script prints out OK after sleeping but without
> checking if the process really stopped. I suggest changing it to:
> {noformat}
> pd0-cassandra16 ~> diff -C 1 cassandra cassandra-original
> *** cassandra 2015-02-04 09:15:58.088209988 -0800
> --- cassandra-original 2015-02-04 09:15:40.293767501 -0800
> ***
> *** 69,77
> sleep 5
> ! THE_STATUS=`$0 status`
> ! if [[ $THE_STATUS == *"stopped"* ]]
> ! then
> !echo "OK"
> ! else
> !echo "ERROR: could not stop the process: $THE_STATUS"
> !exit 1
> ! fi
> ;;
> --- 69,71
> sleep 5
> ! echo "OK"
> ;;
> {noformat}
> Then it prints out OK only if the stop succeeded. Otherwise it prints out a
> message like
> {quote}
> ERROR: could not stop the process: cassandra (pid 10764) is running...
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8738) "/etc/init.d/cassandra stop" prints OK even when it doesn't work
[
https://issues.apache.org/jira/browse/CASSANDRA-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Donald Smith updated CASSANDRA-8738:
Description:
Sometimes I do {{/etc/init.d/cassandra stop}} and it prints out OK, but the
server is still running. (This happens, for example, if it's busy doing GCs.)
The current init script prints out OK after sleeping but without checking if
the process really stopped. I suggest changing it to:
{noformat}
pd0-cassandra16 ~> diff -C 1 cassandra cassandra-original
*** cassandra 2015-02-04 09:15:58.088209988 -0800
--- cassandra-original 2015-02-04 09:15:40.293767501 -0800
***
*** 69,77
sleep 5
! THE_STATUS=`$0 status`
! if [[ $THE_STATUS == *"stopped"* ]]
! then
!echo "OK"
! else
!echo "ERROR: could not stop the process: $THE_STATUS"
!exit 1
! fi
;;
--- 69,71
sleep 5
! echo "OK"
;;
{noformat}
Then it prints out OK only if the stop succeeded. Otherwise it prints out a
message like
{quote}
ERROR: could not stop the process: cassandra (pid 10764) is running...
{quote}
was:
Sometimes I do {{/etc/init.d/cassandra stop}} and it prints out OK, but the
server is still running. (This happens, for example, if it's busy doing GCs.)
The current init script prints out OK after sleeping but without checking if
the process really stopped. I suggest changing it to:
{noformat}
pd0-cassandra16 ~> diff -C 1 cassandra cassandra-original
*** cassandra 2015-02-04 09:15:58.088209988 -0800
--- cassandra-original 2015-02-04 09:15:40.293767501 -0800
***
*** 69,77
sleep 5
! THE_STATUS=`$0 status`
! if [[ $THE_STATUS == *"stopped"* ]]
! then
!echo "OK"
! else
!echo "ERROR: could not stop the process: $THE_STATUS"
! fi
;;
--- 69,71
sleep 5
! echo "OK"
;;
{noformat}
Then it prints out OK only if the stop succeeded. Otherwise it prints out a
message like
{quote}
ERROR: could not stop the process: cassandra (pid 10764) is running...
{quote}
> "/etc/init.d/cassandra stop" prints OK even when it doesn't work
>
>
> Key: CASSANDRA-8738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8738
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Donald Smith
>
> Sometimes I do {{/etc/init.d/cassandra stop}} and it prints out OK, but the
> server is still running. (This happens, for example, if it's busy doing
> GCs.) The current init script prints out OK after sleeping but without
> checking if the process really stopped. I suggest changing it to:
> {noformat}
> pd0-cassandra16 ~> diff -C 1 cassandra cassandra-original
> *** cassandra 2015-02-04 09:15:58.088209988 -0800
> --- cassandra-original 2015-02-04 09:15:40.293767501 -0800
> ***
> *** 69,77
> sleep 5
> ! THE_STATUS=`$0 status`
> ! if [[ $THE_STATUS == *"stopped"* ]]
> ! then
> !echo "OK"
> ! else
> !echo "ERROR: could not stop the process: $THE_STATUS"
> !exit 1
> ! fi
> ;;
> --- 69,71
> sleep 5
> ! echo "OK"
> ;;
> {noformat}
> Then it prints out OK only if the stop succeeded. Otherwise it prints out a
> message like
> {quote}
> ERROR: could not stop the process: cassandra (pid 10764) is running...
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8732) Make inter-node timeouts tolerate clock skew and drift
[ https://issues.apache.org/jira/browse/CASSANDRA-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305569#comment-14305569 ] Benedict commented on CASSANDRA-8732: - An interesting question, since this is quite important to us, is actually how well even a properly managed cluster manages to keep its nodes in sync. Most users will not deploy a local GPS based ntpd; most will probably just use ntpd from an internet source. I recall at my previous place of work (admittedly several years ago we went through this) NTP on Windows had really dramatic skew corrections and could be very out of sync. Also, being out by only a few hundred milliseconds is quite achievable on Linux ntpd, depending on the prevailing conditions, and while we default to 10s timeouts, 1s timeouts are probably not uncommon. > Make inter-node timeouts tolerate clock skew and drift > -- > > Key: CASSANDRA-8732 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8732 > Project: Cassandra > Issue Type: Improvement >Reporter: Ariel Weisberg > > Right now internode timeouts rely on currentTimeMillis() (and NTP) to make > sure that tasks don't expire before they arrive. > Every receiver needs to deduce the offset between its nanoTime and the remote > nanoTime. I don't think currentTimeMillis is a good choice because it is > designed to be manipulated by operators and NTP. I would probably be > comfortable assuming that nanoTime isn't going to move in significant ways > without something that could be classified as operator error happening. > I suspect the one timing method you can rely on being accurate is nanoTime > within a node (on average) and that a node can report on its own scheduling > jitter (on average). > Finding the offset requires knowing what the network latency is in one > direction. > One way to do this would be to periodically send a ping request which > generates a series of ping responses at fixed intervals (maybe by UDP?). The > responses should corrected for scheduling jitter since the fixed intervals > may not be exactly achieved by the sender. By measuring the time deviation > between ping responses and their expected arrival time (based on the > interval) and correcting for the remotely reported scheduling jitter, you > should be able to measure latency in one direction. > A weighted moving average (only correct for drift, not readjustment) of these > measurements would eventually converge on a close answer and would not be > impacted by outlier measurements. It may also make sense to drop the largest > N samples to improve accuracy. > One you know network latency you can add that to the timestamp of each ping > and compare to the local clock and know what the offset is. > These measurements won't calculate the offset to be too small (timeouts fire > early), but could calculate the offset to be too large (timeouts fire late). > The conditions where you the offset won't be accurate are the conditions > where you also want them firing reliably. This and bootstrapping in bad > conditions is what I am most uncertain of. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305576#comment-14305576
]
Michael Shuler commented on CASSANDRA-8736:
---
It looks like centos 6.3 relies on the system python-2.6, and there does not
appear to be a native 2.7 package. You may need to search around for the best
method to install 2.7 on your system and using it for cqlsh and not disrupt the
system python. Virtualenv, building from source and installing in /usr/local
(or /opt, so it's not in $PATH, etc.)
> core dump on creating keyspaces
>
>
> Key: CASSANDRA-8736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8736
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Linux
>Reporter: mike
>Priority: Minor
>
> After we upgraded C* to 2.1.2, with new installation, sometimes on creating
> Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
> release. Thanks.
> {noformat}
> $CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f
> ${dbDefaultDataDir}/${DATA_SCRIPT}”.
> *** glibc detected *** python: corrupted double-linked list:
> 0x02dc95f0 ***
> === Backtrace: =
> /lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
> /lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
> /usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
> /usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
> /usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
> /usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
> /usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
> /lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
> /lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
> === Memory map:
> 0040-00401000 r-xp 08:06 450964
> /usr/bin/python
> 0060-00601000 rw-p 08:06 450964
> /usr/bin/python
> 024ba000-02f29000 rw-p 00:00 0
> [heap]
> 7fd1b000-7fd1b0172000 rw-p 00:00 0
> 7fd1b0172000-7fd1b400 ---p 00:00 0
> 7fd1b400-7fd1b4021000 rw-p 00:00 0
> 7fd1b4021000-7fd1b800 ---p 00:00 0
> 7fd1b800-7fd1b8021000 rw-p 00:00 0
> 7fd1b8021000-7fd1bc00 ---p 00:00 0
> 7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
> 7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
> 7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
> 7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
> 7fd1bd7e8000-7fd1bd7e9000 ---p 00
[jira] [Commented] (CASSANDRA-8738) "/etc/init.d/cassandra stop" prints OK even when it doesn't work
[
https://issues.apache.org/jira/browse/CASSANDRA-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305578#comment-14305578
]
Donald Smith commented on CASSANDRA-8738:
-
I'm using 2.0.11. But I see that 2.1.1 has the same problem. It looks like
3.0's version at https://github.com/apache/cassandra/blob/trunk/debian/init
fixes it:
{noformat}
do_stop()
{
# Return
# 0 if daemon has been stopped
# 1 if daemon was already stopped
# 2 if daemon could not be stopped
# other if a failure occurred
start-stop-daemon -K -p "$PIDFILE" -R TERM/30/KILL/5 >/dev/null
RET=$?
rm -f "$PIDFILE"
return $RET
}
{noformat}
> "/etc/init.d/cassandra stop" prints OK even when it doesn't work
>
>
> Key: CASSANDRA-8738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8738
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Donald Smith
>
> Sometimes I do {{/etc/init.d/cassandra stop}} and it prints out OK, but the
> server is still running. (This happens, for example, if it's busy doing
> GCs.) The current init script prints out OK after sleeping but without
> checking if the process really stopped. I suggest changing it to:
> {noformat}
> pd0-cassandra16 ~> diff -C 1 cassandra cassandra-original
> *** cassandra 2015-02-04 09:15:58.088209988 -0800
> --- cassandra-original 2015-02-04 09:15:40.293767501 -0800
> ***
> *** 69,77
> sleep 5
> ! THE_STATUS=`$0 status`
> ! if [[ $THE_STATUS == *"stopped"* ]]
> ! then
> !echo "OK"
> ! else
> !echo "ERROR: could not stop the process: $THE_STATUS"
> !exit 1
> ! fi
> ;;
> --- 69,71
> sleep 5
> ! echo "OK"
> ;;
> {noformat}
> Then it prints out OK only if the stop succeeded. Otherwise it prints out a
> message like
> {quote}
> ERROR: could not stop the process: cassandra (pid 10764) is running...
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8732) Make inter-node timeouts tolerate clock skew and drift
[ https://issues.apache.org/jira/browse/CASSANDRA-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305581#comment-14305581 ] Benedict commented on CASSANDRA-8732: - bq. but let say I'm not convinced that respecting timeouts is important enough in itself to justify the complexity. Is it really that complex though? What I'm suggesting is pretty simple. I won't fight for it beyond this final question, since as already stated it isn't super pressing. But it couldn't be easier: serialize the delta and the wall clock, and let the other end pick the most conservative of the two to use. > Make inter-node timeouts tolerate clock skew and drift > -- > > Key: CASSANDRA-8732 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8732 > Project: Cassandra > Issue Type: Improvement >Reporter: Ariel Weisberg > > Right now internode timeouts rely on currentTimeMillis() (and NTP) to make > sure that tasks don't expire before they arrive. > Every receiver needs to deduce the offset between its nanoTime and the remote > nanoTime. I don't think currentTimeMillis is a good choice because it is > designed to be manipulated by operators and NTP. I would probably be > comfortable assuming that nanoTime isn't going to move in significant ways > without something that could be classified as operator error happening. > I suspect the one timing method you can rely on being accurate is nanoTime > within a node (on average) and that a node can report on its own scheduling > jitter (on average). > Finding the offset requires knowing what the network latency is in one > direction. > One way to do this would be to periodically send a ping request which > generates a series of ping responses at fixed intervals (maybe by UDP?). The > responses should corrected for scheduling jitter since the fixed intervals > may not be exactly achieved by the sender. By measuring the time deviation > between ping responses and their expected arrival time (based on the > interval) and correcting for the remotely reported scheduling jitter, you > should be able to measure latency in one direction. > A weighted moving average (only correct for drift, not readjustment) of these > measurements would eventually converge on a close answer and would not be > impacted by outlier measurements. It may also make sense to drop the largest > N samples to improve accuracy. > One you know network latency you can add that to the timestamp of each ping > and compare to the local clock and know what the offset is. > These measurements won't calculate the offset to be too small (timeouts fire > early), but could calculate the offset to be too large (timeouts fire late). > The conditions where you the offset won't be accurate are the conditions > where you also want them firing reliably. This and bootstrapping in bad > conditions is what I am most uncertain of. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305576#comment-14305576
]
Michael Shuler edited comment on CASSANDRA-8736 at 2/4/15 5:40 PM:
---
It looks like centos 6.3 relies on the system python-2.6, and there does not
appear to be a native 2.7 package. You may need to search around for the best
method to install 2.7 on your system and using it for cqlsh and not disrupt the
system python. Virtualenv, building from source and installing in /usr/local
(or /opt, so it's not in $PATH, etc.)
Edit: Or, since this is a fresh install - use a modern linux version.
was (Author: mshuler):
It looks like centos 6.3 relies on the system python-2.6, and there does not
appear to be a native 2.7 package. You may need to search around for the best
method to install 2.7 on your system and using it for cqlsh and not disrupt the
system python. Virtualenv, building from source and installing in /usr/local
(or /opt, so it's not in $PATH, etc.)
> core dump on creating keyspaces
>
>
> Key: CASSANDRA-8736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8736
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Linux
>Reporter: mike
>Priority: Minor
>
> After we upgraded C* to 2.1.2, with new installation, sometimes on creating
> Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
> release. Thanks.
> {noformat}
> $CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f
> ${dbDefaultDataDir}/${DATA_SCRIPT}”.
> *** glibc detected *** python: corrupted double-linked list:
> 0x02dc95f0 ***
> === Backtrace: =
> /lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
> /lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
> /usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
> /usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
> /usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
> /usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
> /usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
> /lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
> /lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
> === Memory map:
> 0040-00401000 r-xp 08:06 450964
> /usr/bin/python
> 0060-00601000 rw-p 08:06 450964
> /usr/bin/python
> 024ba000-02f29000 rw-p 00:00 0
> [heap]
> 7fd1b000-7fd1b0172000 rw-p 00:00 0
> 7fd1b0172000-7fd1b400 ---p 00:00 0
> 7fd1b400-7fd1b4021000 rw-p 00:00 0
> 7fd1b4021000-7fd1b800 ---p 00:00 0
> 7fd1b800-7fd1b8021000 rw-p 00:00 0
> 7fd1b8021000-7fd1bc00 ---p 00:00 0
> 7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
>
[jira] [Updated] (CASSANDRA-8736) core dump on creating keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-8736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-8736:
-
Component/s: (was: Core)
Tools
> core dump on creating keyspaces
>
>
> Key: CASSANDRA-8736
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8736
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Linux
>Reporter: mike
>Priority: Minor
>
> After we upgraded C* to 2.1.2, with new installation, sometimes on creating
> Keyspaces, it failed. Is there a patch available or will it be fixed in 2.1.3
> release. Thanks.
> {noformat}
> $CASSANDRA_DIR/bin/cqlsh ${cassandraIp} -f
> ${dbDefaultDataDir}/${DATA_SCRIPT}”.
> *** glibc detected *** python: corrupted double-linked list:
> 0x02dc95f0 ***
> === Backtrace: =
> /lib64/libc.so.6(+0x76166)[0x7fd1c43b8166]
> /lib64/libc.so.6(+0x78ef4)[0x7fd1c43baef4]
> /usr/lib64/libpython2.6.so.1.0(+0x7f6f7)[0x7fd1c4ffd6f7]
> /usr/lib64/libpython2.6.so.1.0(+0xa1bb0)[0x7fd1c501fbb0]
> /usr/lib64/libpython2.6.so.1.0(+0x555bb)[0x7fd1c4fd35bb]
> /usr/lib64/libpython2.6.so.1.0(+0x6d132)[0x7fd1c4feb132]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x597)[0x7fd1c505e407]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x5304)[0x7fd1c505cbe4]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6eead)[0x7fd1c4fecead]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x3cd0)[0x7fd1c505b5b0]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalFrameEx+0x63ef)[0x7fd1c505dccf]
> /usr/lib64/libpython2.6.so.1.0(PyEval_EvalCodeEx+0x927)[0x7fd1c505e797]
> /usr/lib64/libpython2.6.so.1.0(+0x6edb0)[0x7fd1c4fecdb0]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(+0x5970f)[0x7fd1c4fd770f]
> /usr/lib64/libpython2.6.so.1.0(PyObject_Call+0x53)[0x7fd1c4fc2303]
> /usr/lib64/libpython2.6.so.1.0(PyEval_CallObjectWithKeywords+0x43)[0x7fd1c5056dd3]
> /usr/lib64/libpython2.6.so.1.0(+0x10bf2a)[0x7fd1c5089f2a]
> /lib64/libpthread.so.0(+0x79d1)[0x7fd1c4d689d1]
> /lib64/libc.so.6(clone+0x6d)[0x7fd1c442ab6d]
> === Memory map:
> 0040-00401000 r-xp 08:06 450964
> /usr/bin/python
> 0060-00601000 rw-p 08:06 450964
> /usr/bin/python
> 024ba000-02f29000 rw-p 00:00 0
> [heap]
> 7fd1b000-7fd1b0172000 rw-p 00:00 0
> 7fd1b0172000-7fd1b400 ---p 00:00 0
> 7fd1b400-7fd1b4021000 rw-p 00:00 0
> 7fd1b4021000-7fd1b800 ---p 00:00 0
> 7fd1b800-7fd1b8021000 rw-p 00:00 0
> 7fd1b8021000-7fd1bc00 ---p 00:00 0
> 7fd1bc1d-7fd1bc1e6000 r-xp 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc1e6000-7fd1bc3e5000 ---p 00016000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e5000-7fd1bc3e6000 rw-p 00015000 08:0a 917506
> /lib64/libgcc_s-4.4.6-20120305.so.1
> 7fd1bc3e6000-7fd1bc3e7000 ---p 00:00 0
> 7fd1bc3e7000-7fd1bcde7000 rw-p 00:00 0
> 7fd1bcde7000-7fd1bcde8000 ---p 00:00 0
> 7fd1bcde8000-7fd1bd7e8000 rw-p 00:00 0
> 7fd1bd7e8000-7fd1bd7e9000 ---p 00:00 0
> 7fd1bd7e9000-7fd1be1e9000 rw-p 00:00 0
> 7fd1be1e9000-7fd1be1ed000 r-xp 08:06 180906
> /usr/lib64/python2.6/lib-dynload/selectmodule.so
> 7fd1be1ed000-7fd1be3ed000 ---p 4000 08:06 180906
> /usr/lib64/python2.6/lib-dynload/selectmodule.so
> 7fd1be3ed000-7fd1be3ef000 rw-p
[jira] [Updated] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-8535:
---
Reviewer: Marcus Eriksson
oops, missed this one, I'll review it tomorrow [~JoshuaMcKenzie]
> java.lang.RuntimeException: Failed to rename XXX to YYY
> ---
>
> Key: CASSANDRA-8535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
> Project: Cassandra
> Issue Type: Bug
> Environment: Windows 2008 X64
>Reporter: Leonid Shalupov
>Assignee: Joshua McKenzie
> Fix For: 2.1.3
>
> Attachments: 8535_v1.txt
>
>
> {code}
> java.lang.RuntimeException: Failed to rename
> build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
> to
> build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
> ~[main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> ~[na:1.7.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ~[na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> Caused by: java.nio.file.FileSystemException:
> build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
> ->
> build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db:
> The process cannot access the file because it is being used by another
> process.
> at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
> ~[na:1.7.0_45]
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> ~[na:1.7.0_45]
> at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
> ~[na:1.7.0_45]
> at
> sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
> ~[na:1.7.0_45]
> at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45]
> at
> org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
> ~[main/:na]
> ... 18 common frames omitted
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8738) "/etc/init.d/cassandra stop" prints OK even when it doesn't work
[
https://issues.apache.org/jira/browse/CASSANDRA-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305590#comment-14305590
]
Brandon Williams commented on CASSANDRA-8738:
-
Not sure what you mean, we use the same init across all versions... that
function in particular is exactly the same in 2.0
> "/etc/init.d/cassandra stop" prints OK even when it doesn't work
>
>
> Key: CASSANDRA-8738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8738
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Donald Smith
>
> Sometimes I do {{/etc/init.d/cassandra stop}} and it prints out OK, but the
> server is still running. (This happens, for example, if it's busy doing
> GCs.) The current init script prints out OK after sleeping but without
> checking if the process really stopped. I suggest changing it to:
> {noformat}
> pd0-cassandra16 ~> diff -C 1 cassandra cassandra-original
> *** cassandra 2015-02-04 09:15:58.088209988 -0800
> --- cassandra-original 2015-02-04 09:15:40.293767501 -0800
> ***
> *** 69,77
> sleep 5
> ! THE_STATUS=`$0 status`
> ! if [[ $THE_STATUS == *"stopped"* ]]
> ! then
> !echo "OK"
> ! else
> !echo "ERROR: could not stop the process: $THE_STATUS"
> !exit 1
> ! fi
> ;;
> --- 69,71
> sleep 5
> ! echo "OK"
> ;;
> {noformat}
> Then it prints out OK only if the stop succeeded. Otherwise it prints out a
> message like
> {quote}
> ERROR: could not stop the process: cassandra (pid 10764) is running...
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8732) Make inter-node timeouts tolerate clock skew and drift
[ https://issues.apache.org/jira/browse/CASSANDRA-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305595#comment-14305595 ] Sylvain Lebresne commented on CASSANDRA-8732: - bq. What I'm suggesting is pretty simple. Pardon me for not having anticipated your future suggestion. I though it was obvious I was refering to Ariel's periodical ping through UDP and what not. Simply sending a tiny more info so the other end minimize it's potential error is fine complexity wise. > Make inter-node timeouts tolerate clock skew and drift > -- > > Key: CASSANDRA-8732 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8732 > Project: Cassandra > Issue Type: Improvement >Reporter: Ariel Weisberg > > Right now internode timeouts rely on currentTimeMillis() (and NTP) to make > sure that tasks don't expire before they arrive. > Every receiver needs to deduce the offset between its nanoTime and the remote > nanoTime. I don't think currentTimeMillis is a good choice because it is > designed to be manipulated by operators and NTP. I would probably be > comfortable assuming that nanoTime isn't going to move in significant ways > without something that could be classified as operator error happening. > I suspect the one timing method you can rely on being accurate is nanoTime > within a node (on average) and that a node can report on its own scheduling > jitter (on average). > Finding the offset requires knowing what the network latency is in one > direction. > One way to do this would be to periodically send a ping request which > generates a series of ping responses at fixed intervals (maybe by UDP?). The > responses should corrected for scheduling jitter since the fixed intervals > may not be exactly achieved by the sender. By measuring the time deviation > between ping responses and their expected arrival time (based on the > interval) and correcting for the remotely reported scheduling jitter, you > should be able to measure latency in one direction. > A weighted moving average (only correct for drift, not readjustment) of these > measurements would eventually converge on a close answer and would not be > impacted by outlier measurements. It may also make sense to drop the largest > N samples to improve accuracy. > One you know network latency you can add that to the timestamp of each ping > and compare to the local clock and know what the offset is. > These measurements won't calculate the offset to be too small (timeouts fire > early), but could calculate the offset to be too large (timeouts fire late). > The conditions where you the offset won't be accurate are the conditions > where you also want them firing reliably. This and bootstrapping in bad > conditions is what I am most uncertain of. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
