[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075611#comment-14075611 ] Robert Stupp commented on CASSANDRA-7438: - [~vijay2...@gmail.com] do you have a C* branch with lruc integrated? Or: what should I do to bring lruc+C* together? Is the patch up-to-date? I've pushed a new branch 'native-plugin' with the changes for native-maven-plugin - separate from the other code. Windows stuff is bit more complicated - it doesn't compile. Have to dig a bit deeper. Maybe delay Win port... Serializing Row cache alternative (Fully off heap) -- Key: CASSANDRA-7438 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438 Project: Cassandra Issue Type: Improvement Components: Core Environment: Linux Reporter: Vijay Assignee: Vijay Labels: performance Fix For: 3.0 Attachments: 0001-CASSANDRA-7438.patch Currently SerializingCache is partially off heap, keys are still stored in JVM heap as BB, * There is a higher GC costs for a reasonably big cache. * Some users have used the row cache efficiently in production for better results, but this requires careful tunning. * Overhead in Memory for the cache entries are relatively high. So the proposal for this ticket is to move the LRU cache logic completely off heap and use JNI to interact with cache. We might want to ensure that the new implementation match the existing API's (ICache), and the implementation needs to have safe memory access, low overhead in memory and less memcpy's (As much as possible). We might also want to make this cache configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075611#comment-14075611 ] Robert Stupp edited comment on CASSANDRA-7438 at 7/27/14 12:18 PM: --- [~vijay2...@gmail.com] do you have a C* branch with lruc integrated? Or: what should I do to bring lruc+C* together? Is the patch up-to-date? I've pushed a new branch 'native-plugin' with the changes for native-maven-plugin. It's separate from the other code. Works for Linux and OSX (depending on which machine the stuff's built). Windows stuff is bit more complicated - it doesn't compile. Have to dig a bit deeper. Maybe delay Win port... was (Author: snazy): [~vijay2...@gmail.com] do you have a C* branch with lruc integrated? Or: what should I do to bring lruc+C* together? Is the patch up-to-date? I've pushed a new branch 'native-plugin' with the changes for native-maven-plugin - separate from the other code. Windows stuff is bit more complicated - it doesn't compile. Have to dig a bit deeper. Maybe delay Win port... Serializing Row cache alternative (Fully off heap) -- Key: CASSANDRA-7438 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438 Project: Cassandra Issue Type: Improvement Components: Core Environment: Linux Reporter: Vijay Assignee: Vijay Labels: performance Fix For: 3.0 Attachments: 0001-CASSANDRA-7438.patch Currently SerializingCache is partially off heap, keys are still stored in JVM heap as BB, * There is a higher GC costs for a reasonably big cache. * Some users have used the row cache efficiently in production for better results, but this requires careful tunning. * Overhead in Memory for the cache entries are relatively high. So the proposal for this ticket is to move the LRU cache logic completely off heap and use JNI to interact with cache. We might want to ensure that the new implementation match the existing API's (ICache), and the implementation needs to have safe memory access, low overhead in memory and less memcpy's (As much as possible). We might also want to make this cache configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075630#comment-14075630
]
Robert Stupp commented on CASSANDRA-7438:
-
Surely not a complete list, but a start...
Java code:
* com.lruc.util.Utils.getUnsafe can be safely deleted
* com.lruc.util.Utils.extractLibraryFile
** should check return code of {{File.createNewFile}}
** Call to {{File.delete()}} for the extracted library file should be added to
{{com.lruc.util.Utils.loadNative}} since unclean shutdown (kill -9) does not
delete the so/dylib file. Possible for Unix systems - but not for Win.
* Classes com.lruc.jni.lruc, SWIGTYPE_p_item and SWIGTYPE_p_p_item are unused
(refactoring relict?)
* Generally the lruc code could be more integrated in C* code.
** Let the lruc classes implement org.apache.cassandra.io.util.DataOutputPlus
and java.io.DataInput so that they can be directly used by C*
ColumnFamilySerializer (no temporary Input/OutputStreams necessary).
** Maybe {{DataOutputPlus.write(Memory)}} can be removed in C* when lruc is
used - not sure about that.
** Implement most DataInput/Output methods in EntryInput/OutputStream to
benefit from Unsafe (e.g. Unsafe.getLong/putLong) - have seen, that you've
removed Abstract... some weeks ago ;)
** Using Unsafe for DataInput/Output of short/int/long/float/double has the
drawback that Unsafe always uses the system's byte order - not (necessarily)
the portable Java byte order. There's of course no drawback, if all
reads/writes are paired.
** {{Unsafe.copyMemory}} could be used for {{write(byte[])}}/{{read(byte[])}}.
* Naming of max_size, capacity - should use one common term which also makes
sure that it's a maximum memory size - e.g. max_size_bytes. _Capacity_ is often
used for the number of elements in a collection.
* Memory leak: {{com.lruc.api.LRUCache.hotN}} may keep references in native
code (no {{lruc_deref}} calls), if not all items are retrieved from the
iterator - remove _hotN_ or return an array/list instead?
* Generally I think all classes can be merged into a single package if only a
few a are left (see above)
C code:
* {{#define item_lock(hv) while (item_trylock(hv)) continue;}} shouldn't there
be something like a _yield_ ?
* Seems like the C code was not cleaned up after you began using
Unsafe.allocateMemory :)
* I did not follow all possible code paths (due to the previous point)
Common:
* {{prefix_delimiter}} seems to be unused
Altogether I like that :)
Serializing Row cache alternative (Fully off heap)
--
Key: CASSANDRA-7438
URL: https://issues.apache.org/jira/browse/CASSANDRA-7438
Project: Cassandra
Issue Type: Improvement
Components: Core
Environment: Linux
Reporter: Vijay
Assignee: Vijay
Labels: performance
Fix For: 3.0
Attachments: 0001-CASSANDRA-7438.patch
Currently SerializingCache is partially off heap, keys are still stored in
JVM heap as BB,
* There is a higher GC costs for a reasonably big cache.
* Some users have used the row cache efficiently in production for better
results, but this requires careful tunning.
* Overhead in Memory for the cache entries are relatively high.
So the proposal for this ticket is to move the LRU cache logic completely off
heap and use JNI to interact with cache. We might want to ensure that the new
implementation match the existing API's (ICache), and the implementation
needs to have safe memory access, low overhead in memory and less memcpy's
(As much as possible).
We might also want to make this cache configurable.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (CASSANDRA-6038) Make the inter-major schema migration barrier more targeted, and, ideally, non-existent
[ https://issues.apache.org/jira/browse/CASSANDRA-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko reassigned CASSANDRA-6038: Assignee: Aleksey Yeschenko Make the inter-major schema migration barrier more targeted, and, ideally, non-existent --- Key: CASSANDRA-6038 URL: https://issues.apache.org/jira/browse/CASSANDRA-6038 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 3.0 CASSANDRA-5845 made it so that schema changes in a major-mixed cluster are not propagated to the minorer-major nodes. This lets us perform backwards-incompatible schema changes in major releases safely - like adding the schema_triggers table, moving all the aliases to schema_columns, getting rid of the deprecated schema columns, etc. Even this limitation might be too strict, however, and should be avoided if possible (starting with at least versioning schema separately from messaging service versioning, and resorting to major-minor schema block only when truly necessary and not for every x-y pair). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075676#comment-14075676
]
Vijay commented on CASSANDRA-7438:
--
Pushed the branch to https://github.com/Vijay2win/cassandra/tree/7438
{quote}Maybe delay Win port{quote}
We should be fine, lruc is configurable with the Serialization cache.
{quote}unclean shutdown (kill -9) does not delete the so/dylib file{quote}
Yeah it works in unix, but the problem is i don't have a handle since its a
temp file after restart. So it is a best effort for cleanups.
{quote}SWIGTYPE_p_item and SWIGTYPE_p_p_item are unused {quote}
Auto generated and can be removed but will be generated every time swig is run.
{quote}Generally the lruc code could be more integrated in C* code{quote}
The problem is it produces a circular dependency, please look at
df3857e4b9637ed6a5099506e95d84de15bf2eb7 where i removed those (the DOSP added
back will still need to wrapped around by Cassandra's DOSP).
{quote}Naming of max_size, capacity{quote}
Yeah let me make it consistent, the problem was i was trying to fit everything
into Guava interface.
{quote}remove hotN or return an array/list instead{quote}
Or may be do memcpy on keys, since this doesn't need optimization (will fix).
{quote}shouldn't there be something like a yield{quote}
Actually i removed it recently adding or removing doesn't give much performance
gains, as a good citizen should add it back.
{quote}Seems like the C code was not cleaned up{quote}
This cannot be removed and needed for test cases.
Serializing Row cache alternative (Fully off heap)
--
Key: CASSANDRA-7438
URL: https://issues.apache.org/jira/browse/CASSANDRA-7438
Project: Cassandra
Issue Type: Improvement
Components: Core
Environment: Linux
Reporter: Vijay
Assignee: Vijay
Labels: performance
Fix For: 3.0
Attachments: 0001-CASSANDRA-7438.patch
Currently SerializingCache is partially off heap, keys are still stored in
JVM heap as BB,
* There is a higher GC costs for a reasonably big cache.
* Some users have used the row cache efficiently in production for better
results, but this requires careful tunning.
* Overhead in Memory for the cache entries are relatively high.
So the proposal for this ticket is to move the LRU cache logic completely off
heap and use JNI to interact with cache. We might want to ensure that the new
implementation match the existing API's (ICache), and the implementation
needs to have safe memory access, low overhead in memory and less memcpy's
(As much as possible).
We might also want to make this cache configurable.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (CASSANDRA-6977) attempting to create 10K column families fails with 100 node cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-6977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McGuire reassigned CASSANDRA-6977:
---
Assignee: Russ Hatch (was: Ryan McGuire)
attempting to create 10K column families fails with 100 node cluster
Key: CASSANDRA-6977
URL: https://issues.apache.org/jira/browse/CASSANDRA-6977
Project: Cassandra
Issue Type: Bug
Environment: 100 nodes, Ubuntu 12.04.3 LTS, AWS m1.large instances
Reporter: Daniel Meyer
Assignee: Russ Hatch
Priority: Minor
Attachments: 100_nodes_all_data.png, all_data_5_nodes.png,
keyspace_create.py, logs.tar, tpstats.txt, visualvm_tracer_data.csv
During this test we are attempting to create a total of 1K keyspaces with 10
column families each to bring the total column families to 10K. With a 5
node cluster this operation can be completed; however, it fails with 100
nodes. Please see the two charts. For the 5 node case the time required to
create each keyspace and subsequent 10 column families increases linearly
until the number of keyspaces is 1K. For a 100 node cluster there is a
sudden increase in latency between 450 keyspaces and 550 keyspaces. The test
ends when the test script times out. After the test script times out it is
impossible to reconnect to the cluster with the datastax python driver
because it cannot connect to the host:
cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers',
{'10.199.5.98': OperationTimedOut()}
It was found that running the following stress command does work from the
same machine the test script runs on.
cassandra-stress -d 10.199.5.98 -l 2 -e QUORUM -L3 -b -o INSERT
It should be noted that this test was initially done with DSE 4.0 and c*
version 2.0.5.24 and in that case it was not possible to run stress against
the cluster even locally on a node due to not finding the host.
Attached are system logs from one of the nodes, charts showing schema
creation latency for 5 and 100 node clusters and virtualvm tracer data for
cpu, memory, num_threads and gc runs, tpstat output and the test script.
The test script was on an m1.large aws instance outside of the cluster under
test.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
