[jira] [Updated] (CASSANDRA-8714) row-cache: use preloaded jemalloc w/ Unsafe
[
https://issues.apache.org/jira/browse/CASSANDRA-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-8714:
Attachment: 8714.txt
Add patch that does two things:
# modify {{bin/cassandra}} to preload libjemalloc on Linux and OSX (and set OHC
system property)
# remove {{IAllocator}} abstraction in C* code
Comments welcome
/cc [~benedict] [~aweisberg] [~vijay2...@yahoo.com]
row-cache: use preloaded jemalloc w/ Unsafe
---
Key: CASSANDRA-8714
URL: https://issues.apache.org/jira/browse/CASSANDRA-8714
Project: Cassandra
Issue Type: Sub-task
Reporter: Robert Stupp
Assignee: Robert Stupp
Fix For: 3.0
Attachments: 8714.txt
Using jemalloc via Java's {{Unsafe}} is a better alternative on Linux than
using jemalloc via JNA.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError
[
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300223#comment-14300223
]
Imri Zvik commented on CASSANDRA-8548:
--
I'm still getting this when running cleanup in 2.0.12, even after the
restart-scrub-cleanup procedure.
Nodetool Cleanup - java.lang.AssertionError
---
Key: CASSANDRA-8548
URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
Project: Cassandra
Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
Fix For: 2.0.12
Attachments: 0001-make-sure-we-unmark-compacting.patch
Needed to free up some space on a node but getting the dump below when
running nodetool cleanup.
Tried turning on debug to try to obtain additional details in the logs but
nothing gets added to the logs when running cleanup. Added:
log4j.logger.org.apache.cassandra.db=DEBUG
in log4j-server.properties
See the stack trace below:
root@cassandra-019:~# nodetool cleanup
{code}Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:267)
at
[jira] [Reopened] (CASSANDRA-8308) Windows: Commitlog access violations on unit tests
[
https://issues.apache.org/jira/browse/CASSANDRA-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp reopened CASSANDRA-8308:
-
[~JoshuaMcKenzie] cassci now also complains about TimeoutExceptions
(http://cassci.datastax.com/job/trunk_utest/1353/testReport/ and various
previous builds).
I digged a bit more into it and have a patch that solves the
TimeoutException-problem: https://gist.github.com/snazy/6f2f8203ca4ab7c19422
and a branch https://github.com/snazy/cassandra/tree/8308-post-fix
The good: the problem does only affect unit tests - not the production code.
Background:
Sometimes {{CommitLogSegmentManager.resetUnsafe()}} produces a deadlock - when
depends on 'random' - i.e. when {{COMMIT-LOG-ALLOCATOR}} thread works against
data 'corrupted' by {{resetUnsafe}}. The patch of {{resetUnsafe}} just shuts
down {{COMMIT-LOG-ALLOCATOR}} thread, does its work and restarts
{{COMMIT-LOG-ALLOCATOR}}.
Windows: Commitlog access violations on unit tests
--
Key: CASSANDRA-8308
URL: https://issues.apache.org/jira/browse/CASSANDRA-8308
Project: Cassandra
Issue Type: Bug
Reporter: Joshua McKenzie
Assignee: Joshua McKenzie
Priority: Minor
Labels: Windows
Fix For: 3.0
Attachments: 8308_v1.txt, 8308_v2.txt, 8308_v3.txt
We have four unit tests failing on trunk on Windows, all with
FileSystemException's related to the SchemaLoader:
{noformat}
[junit] Test
org.apache.cassandra.db.compaction.DateTieredCompactionStrategyTest FAILED
[junit] Test org.apache.cassandra.cql3.ThriftCompatibilityTest FAILED
[junit] Test org.apache.cassandra.io.sstable.SSTableRewriterTest FAILED
[junit] Test org.apache.cassandra.repair.LocalSyncTaskTest FAILED
{noformat}
Example error:
{noformat}
[junit] Caused by: java.nio.file.FileSystemException:
build\test\cassandra\commitlog;0\CommitLog-5-1415908745965.log: The process
cannot access the file because it is being used by another process.
[junit]
[junit] at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
[junit] at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
[junit] at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
[junit] at
sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
[junit] at
sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
[junit] at java.nio.file.Files.delete(Files.java:1079)
[junit] at
org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:125)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7970) JSON support for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300120#comment-14300120
]
Robert Stupp commented on CASSANDRA-7970:
-
I took a short look at the patch and played around a little bit with this.
Some comments:
* A test with null values in {{testFromJsonFct()}} and {{testToJsonFct()}}
would be nice (toJson handles null correctly, fromJson throws an NPE for
{{INSERT INTO tojson (k, asciival ) VALUES ( 0, fromJson(null) );}})
* Can you add a {{INSERT JSON}} variant that tackles tuples and types?
* I tried {{insert into some_table (k, asciival ) JSON \{k: 0, asciival:
foobar\};}} in cqlsh - it complains with a syntax error (obviously my bad).
Was wondering if we really need to mention the column names since they are
contained in the JSON. Couldn't a {{insert into some_table JSON ?;}} with json
{{\{k: 0, asciival: foobar\}}} also work?
* The Java Driver tries to contact a coordinator that owns the target
partition. This gets complicated with {{INSERT JSON}} since the driver would
have to parse the JSON before it actually knows the correct node. Maybe we
can discuss a follow-up that both allows {{VALUES}} and {{JSON}} - e.g.
{{INSERT INTO some_table (partitionKey, clusteringKey) VALUES (1, 2) JSON ?}},
where the {{JSON ?}} part contains more columns.
* missing license header in {{Json.java}}
* In {{FunctionCall}} you exchanged „fun“ with „fun.name()“ eliminating the
function signature in the exception message
* {{Selection.rowToJson}} : use Collections.singleton instead of
{{Arrays.asList}}
* {{Selection.rowToJson}} : would be nicer (and eliminate one String instance)
to replace sb.append(JSONValue.escape(columnName)) with something like
JSONValue.escape(columnName, sb); (make escape write to sb)
* Similar for {{AbstractType.toJSONString}} - let the implementation write to
the string builder
* {{ReversedType}} : not sure whether the impl is correct - doesn’t it need to
reverse the binary representation?
* {{FromJsonFct}} + {{ToJsonFct}} : would be nicer to have a CHM instead of
synchronized HashMap. Also these maps in these classes have {{AbstractType}} as
the key and are never evicted - means: a changed user type would remain forever
in these maps and if a UDT. This might get irrelevant when {{AbstractType}} is
removed - so not sure, whether we should fix this now - maybe open a
follow-up-ticket for this? A simple fix would be to 'statically cache' function
instances for primitive types but always create a new instance of these
functions for tuples + UDTs ; then you could pre-create the function instances
completely eliminating the need for CHM or synchronized.
Altogether: really nice work :)
JSON support for CQL
Key: CASSANDRA-7970
URL: https://issues.apache.org/jira/browse/CASSANDRA-7970
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Jonathan Ellis
Assignee: Tyler Hobbs
Fix For: 3.0
Attachments: 7970-trunk-v1.txt
JSON is popular enough that not supporting it is becoming a competitive
weakness. We can add JSON support in a way that is compatible with our
performance goals by *mapping* JSON to an existing schema: one JSON documents
maps to one CQL row.
Thus, it is NOT a goal to support schemaless documents, which is a misfeature
[1] [2] [3]. Rather, it is to allow a convenient way to easily turn a JSON
document from a service or a user into a CQL row, with all the validation
that entails.
Since we are not looking to support schemaless documents, we will not be
adding a JSON data type (CASSANDRA-6833) a la postgresql. Rather, we will
map the JSON to UDT, collections, and primitive CQL types.
Here's how this might look:
{code}
CREATE TYPE address (
street text,
city text,
zip_code int,
phones settext
);
CREATE TABLE users (
id uuid PRIMARY KEY,
name text,
addresses maptext, address
);
INSERT INTO users JSON
{‘id’: 4b856557-7153,
‘name’: ‘jbellis’,
‘address’: {“home”: {“street”: “123 Cassandra Dr”,
“city”: “Austin”,
“zip_code”: 78747,
“phones”: [2101234567]}}};
SELECT JSON id, address FROM users;
{code}
(We would also want to_json and from_json functions to allow mapping a single
column's worth of data. These would not require extra syntax.)
[1] http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/
[2] https://blog.compose.io/schema-less-is-usually-a-lie/
[3] http://dl.acm.org/citation.cfm?id=2481247
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8714) row-cache: use preloaded jemalloc w/ Unsafe
Robert Stupp created CASSANDRA-8714:
---
Summary: row-cache: use preloaded jemalloc w/ Unsafe
Key: CASSANDRA-8714
URL: https://issues.apache.org/jira/browse/CASSANDRA-8714
Project: Cassandra
Issue Type: Sub-task
Reporter: Robert Stupp
Assignee: Robert Stupp
Fix For: 3.0
Using jemalloc via Java's {{Unsafe}} is a better alternative on Linux than
using jemalloc via JNA.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8714) row-cache: use preloaded jemalloc w/ Unsafe
[
https://issues.apache.org/jira/browse/CASSANDRA-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300186#comment-14300186
]
Benedict commented on CASSANDRA-8714:
-
I agree with the principle of switching to LD_PRELOAD, but I'm not so sure
about switching the default allocator under everyone. We need to do some
comprehensive testing of the implications of that switch.
row-cache: use preloaded jemalloc w/ Unsafe
---
Key: CASSANDRA-8714
URL: https://issues.apache.org/jira/browse/CASSANDRA-8714
Project: Cassandra
Issue Type: Sub-task
Reporter: Robert Stupp
Assignee: Robert Stupp
Fix For: 3.0
Attachments: 8714.txt
Using jemalloc via Java's {{Unsafe}} is a better alternative on Linux than
using jemalloc via JNA.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8548) Nodetool Cleanup - java.lang.AssertionError
[
https://issues.apache.org/jira/browse/CASSANDRA-8548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300223#comment-14300223
]
Imri Zvik edited comment on CASSANDRA-8548 at 2/1/15 6:43 PM:
--
I'm still getting this when running cleanup in 2.0.12, even after the
restart-scrub-cleanup procedure.
*EDIT*
Actually, when I examine the stack traces, in my case they are different. I
will open a new issue regarding my instance
was (Author: imriz):
I'm still getting this when running cleanup in 2.0.12, even after the
restart-scrub-cleanup procedure.
Nodetool Cleanup - java.lang.AssertionError
---
Key: CASSANDRA-8548
URL: https://issues.apache.org/jira/browse/CASSANDRA-8548
Project: Cassandra
Issue Type: Bug
Reporter: Sebastian Estevez
Assignee: Marcus Eriksson
Fix For: 2.0.12
Attachments: 0001-make-sure-we-unmark-compacting.patch
Needed to free up some space on a node but getting the dump below when
running nodetool cleanup.
Tried turning on debug to try to obtain additional details in the logs but
nothing gets added to the logs when running cleanup. Added:
log4j.logger.org.apache.cassandra.db=DEBUG
in log4j-server.properties
See the stack trace below:
root@cassandra-019:~# nodetool cleanup
{code}Error occurred during cleanup
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at
org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:228)
at
org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:266)
at
org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1112)
at
org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2162)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at
[jira] [Commented] (CASSANDRA-8210) java.lang.AssertionError: Memory was freed exception in CompactionExecutor
[ https://issues.apache.org/jira/browse/CASSANDRA-8210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300409#comment-14300409 ] Imri Zvik commented on CASSANDRA-8210: -- I am getting a similar error when I run cleanup (using 2.0.12): [root@cassandra005 ~]# nodetool cleanup Error occurred during cleanup java.util.concurrent.ExecutionException: java.lang.AssertionError: Memory was freed at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:188) at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:234) at org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:272) at org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1115) at org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2177) at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75) at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487) at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848) at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322) at sun.rmi.transport.Transport$1.run(Transport.java:177) at sun.rmi.transport.Transport$1.run(Transport.java:174) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:173) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.AssertionError: Memory was freed at org.apache.cassandra.io.util.Memory.checkPosition(Memory.java:259) at org.apache.cassandra.io.util.Memory.getInt(Memory.java:211) at org.apache.cassandra.io.sstable.IndexSummary.getIndex(IndexSummary.java:79) at org.apache.cassandra.io.sstable.IndexSummary.getKey(IndexSummary.java:84) at org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:58) at org.apache.cassandra.io.sstable.SSTableReader.getIndexScanPosition(SSTableReader.java:602) at org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:947) at org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:910) at org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:819) at org.apache.cassandra.db.ColumnFamilyStore.getExpectedCompactedFileSize(ColumnFamilyStore.java:1088) at org.apache.cassandra.db.compaction.CompactionManager.doCleanupCompaction(CompactionManager.java:564) at
[jira] [Commented] (CASSANDRA-8518) Impose In-Flight Data Limit
[ https://issues.apache.org/jira/browse/CASSANDRA-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300786#comment-14300786 ] Benedict commented on CASSANDRA-8518: - Hi Cheng, The problem with the approach you suggest is that it could end up killing all queries on the box if there is something else eating up heap (let's say compaction is a pain point). You're right that it would be simpler, but it should be quite feasible to track an approximate count of the heap used by a query - in fact we already have many of the necessary facilities to do so, in order to impose limits on how much data can be stored in memtables. The difficulty will be plugging this into each place we generate data for a query, which remains to be seen how challenging it will be. Ideally we want to move to each class of operation on the system having an isolated allotment of memory it's permitted to eat up, so that they don't each stomp on the other. Impose In-Flight Data Limit --- Key: CASSANDRA-8518 URL: https://issues.apache.org/jira/browse/CASSANDRA-8518 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Cheng Ren Labels: performance We have been suffering from cassandra node crash due to out of memory for a long time. The heap dump from the recent crash shows there are 22 native transport request threads each of which consumes 3.3% of heap size, taking more than 70% in total. Heap dump: !https://dl-web.dropbox.com/get/attach1.png?_subject_uid=303980955w=AAAVOoncBoZ5aOPbDg2TpRkUss7B-2wlrnhUAv19b27OUA|height=400,width=600! Expanded view of one thread: !https://dl-web.dropbox.com/get/Screen%20Shot%202014-12-18%20at%204.06.29%20PM.png?_subject_uid=303980955w=AACUO4wrbxheRUxv8fwQ9P52T6gBOm5_g9zeIe8odu3V3w|height=400,width=600! The cassandra we are using now (2.0.4) utilized MemoryAwareThreadPoolExecutor as the request executor and provided a default request size estimator which constantly returns 1, meaning it limits only the number of requests being pushed to the pool. To have more fine-grained control on handling requests and better protect our node from OOM issue, we propose implementing a more precise estimator. Here is our two cents: For update/delete/insert request: Size could be estimated by adding size of all class members together. For scan query, the major part of the request is response, which can be estimated from the history data. For example if we receive a scan query on a column family for a certain token range, we keep track of its response size used as the estimated response size for later scan query on the same cf. For future requests on the same cf, response size could be calculated by token range*recorded size/ recorded token range. The request size should be estimated as (query size + estimated response size). We believe what we're proposing here can be useful for other people in the Cassandra community as well. Would you mind providing us feedbacks? Please let us know if you have any concerns or suggestions regarding this proposal. Thanks, Cheng -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/6] cassandra git commit: Fix isClientMode check in Keyspace
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 d0005a83e - 2dd6944f1
refs/heads/cassandra-2.1 9c38a103b - 2a283e10f
refs/heads/trunk 97d7d139f - 6e9aec312
Fix isClientMode check in Keyspace
patch by Jeremiah Jordan; reviewed by yukim for CASSANDRA-8687
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2dd6944f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2dd6944f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2dd6944f
Branch: refs/heads/cassandra-2.0
Commit: 2dd6944f1cf9236e2ca63f6a1fa67b020c175f8f
Parents: d0005a8
Author: Jeremiah Jordan jerem...@datastax.com
Authored: Sun Feb 1 16:07:59 2015 -0600
Committer: Yuki Morishita yu...@apache.org
Committed: Sun Feb 1 16:07:59 2015 -0600
--
CHANGES.txt| 1 +
src/java/org/apache/cassandra/db/Keyspace.java | 7 ++-
2 files changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2dd6944f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4754867..9e9b772 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -5,6 +5,7 @@
(CASSANDRA-8619)
* Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645)
* Add batch remove iterator to ABSC (CASSANDRA-8414, 8666)
+ * Fix isClientMode check in Keyspace (CASSANDRA-8687)
2.0.12:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2dd6944f/src/java/org/apache/cassandra/db/Keyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/Keyspace.java
b/src/java/org/apache/cassandra/db/Keyspace.java
index 28045f4..38b7c2b 100644
--- a/src/java/org/apache/cassandra/db/Keyspace.java
+++ b/src/java/org/apache/cassandra/db/Keyspace.java
@@ -30,10 +30,7 @@ import com.google.common.collect.Iterables;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import org.apache.cassandra.config.CFMetaData;
-import org.apache.cassandra.config.DatabaseDescriptor;
-import org.apache.cassandra.config.KSMetaData;
-import org.apache.cassandra.config.Schema;
+import org.apache.cassandra.config.*;
import org.apache.cassandra.db.commitlog.CommitLog;
import org.apache.cassandra.db.filter.QueryFilter;
import org.apache.cassandra.db.index.SecondaryIndex;
@@ -69,7 +66,7 @@ public class Keyspace
// proper directories here as well as in CassandraDaemon.
static
{
-if (!StorageService.instance.isClientMode())
+if (!(Config.isClientMode() || StorageService.instance.isClientMode()))
DatabaseDescriptor.createAllDirectories();
}
[2/6] cassandra git commit: Fix isClientMode check in Keyspace
Fix isClientMode check in Keyspace
patch by Jeremiah Jordan; reviewed by yukim for CASSANDRA-8687
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2dd6944f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2dd6944f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2dd6944f
Branch: refs/heads/cassandra-2.1
Commit: 2dd6944f1cf9236e2ca63f6a1fa67b020c175f8f
Parents: d0005a8
Author: Jeremiah Jordan jerem...@datastax.com
Authored: Sun Feb 1 16:07:59 2015 -0600
Committer: Yuki Morishita yu...@apache.org
Committed: Sun Feb 1 16:07:59 2015 -0600
--
CHANGES.txt| 1 +
src/java/org/apache/cassandra/db/Keyspace.java | 7 ++-
2 files changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2dd6944f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4754867..9e9b772 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -5,6 +5,7 @@
(CASSANDRA-8619)
* Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645)
* Add batch remove iterator to ABSC (CASSANDRA-8414, 8666)
+ * Fix isClientMode check in Keyspace (CASSANDRA-8687)
2.0.12:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2dd6944f/src/java/org/apache/cassandra/db/Keyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/Keyspace.java
b/src/java/org/apache/cassandra/db/Keyspace.java
index 28045f4..38b7c2b 100644
--- a/src/java/org/apache/cassandra/db/Keyspace.java
+++ b/src/java/org/apache/cassandra/db/Keyspace.java
@@ -30,10 +30,7 @@ import com.google.common.collect.Iterables;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import org.apache.cassandra.config.CFMetaData;
-import org.apache.cassandra.config.DatabaseDescriptor;
-import org.apache.cassandra.config.KSMetaData;
-import org.apache.cassandra.config.Schema;
+import org.apache.cassandra.config.*;
import org.apache.cassandra.db.commitlog.CommitLog;
import org.apache.cassandra.db.filter.QueryFilter;
import org.apache.cassandra.db.index.SecondaryIndex;
@@ -69,7 +66,7 @@ public class Keyspace
// proper directories here as well as in CassandraDaemon.
static
{
-if (!StorageService.instance.isClientMode())
+if (!(Config.isClientMode() || StorageService.instance.isClientMode()))
DatabaseDescriptor.createAllDirectories();
}
[4/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2a283e10 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2a283e10 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2a283e10 Branch: refs/heads/trunk Commit: 2a283e10f9183d9a0138e7622ca135cd3b001ac2 Parents: 9c38a10 2dd6944 Author: Yuki Morishita yu...@apache.org Authored: Sun Feb 1 16:10:31 2015 -0600 Committer: Yuki Morishita yu...@apache.org Committed: Sun Feb 1 16:10:31 2015 -0600 -- CHANGES.txt| 2 +- src/java/org/apache/cassandra/db/Keyspace.java | 7 ++- 2 files changed, 3 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a283e10/CHANGES.txt -- diff --cc CHANGES.txt index 129e1c9,9e9b772..40d844c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -90,7 -5,10 +90,7 @@@ Merged from 2.0 (CASSANDRA-8619) * Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645) * Add batch remove iterator to ABSC (CASSANDRA-8414, 8666) - * Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645) + * Fix isClientMode check in Keyspace (CASSANDRA-8687) - - -2.0.12: * Use more efficient slice size for querying internal secondary index tables (CASSANDRA-8550) * Fix potentially returning deleted rows with range tombstone (CASSANDRA-8558) http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a283e10/src/java/org/apache/cassandra/db/Keyspace.java -- diff --cc src/java/org/apache/cassandra/db/Keyspace.java index d27424e,38b7c2b..d92eea4 --- a/src/java/org/apache/cassandra/db/Keyspace.java +++ b/src/java/org/apache/cassandra/db/Keyspace.java @@@ -35,12 -30,8 +35,9 @@@ import com.google.common.collect.Iterab import org.slf4j.Logger; import org.slf4j.LoggerFactory; - import org.apache.cassandra.config.CFMetaData; - import org.apache.cassandra.config.DatabaseDescriptor; - import org.apache.cassandra.config.KSMetaData; - import org.apache.cassandra.config.Schema; + import org.apache.cassandra.config.*; import org.apache.cassandra.db.commitlog.CommitLog; +import org.apache.cassandra.db.commitlog.ReplayPosition; import org.apache.cassandra.db.filter.QueryFilter; import org.apache.cassandra.db.index.SecondaryIndex; import org.apache.cassandra.db.index.SecondaryIndexManager;
[6/6] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6e9aec31 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6e9aec31 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6e9aec31 Branch: refs/heads/trunk Commit: 6e9aec3125d5e549669e07f0c4b3ff22ffc9d8c2 Parents: 97d7d13 2a283e1 Author: Yuki Morishita yu...@apache.org Authored: Sun Feb 1 16:10:40 2015 -0600 Committer: Yuki Morishita yu...@apache.org Committed: Sun Feb 1 16:10:40 2015 -0600 -- --
[5/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2a283e10 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2a283e10 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2a283e10 Branch: refs/heads/cassandra-2.1 Commit: 2a283e10f9183d9a0138e7622ca135cd3b001ac2 Parents: 9c38a10 2dd6944 Author: Yuki Morishita yu...@apache.org Authored: Sun Feb 1 16:10:31 2015 -0600 Committer: Yuki Morishita yu...@apache.org Committed: Sun Feb 1 16:10:31 2015 -0600 -- CHANGES.txt| 2 +- src/java/org/apache/cassandra/db/Keyspace.java | 7 ++- 2 files changed, 3 insertions(+), 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a283e10/CHANGES.txt -- diff --cc CHANGES.txt index 129e1c9,9e9b772..40d844c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -90,7 -5,10 +90,7 @@@ Merged from 2.0 (CASSANDRA-8619) * Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645) * Add batch remove iterator to ABSC (CASSANDRA-8414, 8666) - * Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645) + * Fix isClientMode check in Keyspace (CASSANDRA-8687) - - -2.0.12: * Use more efficient slice size for querying internal secondary index tables (CASSANDRA-8550) * Fix potentially returning deleted rows with range tombstone (CASSANDRA-8558) http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a283e10/src/java/org/apache/cassandra/db/Keyspace.java -- diff --cc src/java/org/apache/cassandra/db/Keyspace.java index d27424e,38b7c2b..d92eea4 --- a/src/java/org/apache/cassandra/db/Keyspace.java +++ b/src/java/org/apache/cassandra/db/Keyspace.java @@@ -35,12 -30,8 +35,9 @@@ import com.google.common.collect.Iterab import org.slf4j.Logger; import org.slf4j.LoggerFactory; - import org.apache.cassandra.config.CFMetaData; - import org.apache.cassandra.config.DatabaseDescriptor; - import org.apache.cassandra.config.KSMetaData; - import org.apache.cassandra.config.Schema; + import org.apache.cassandra.config.*; import org.apache.cassandra.db.commitlog.CommitLog; +import org.apache.cassandra.db.commitlog.ReplayPosition; import org.apache.cassandra.db.filter.QueryFilter; import org.apache.cassandra.db.index.SecondaryIndex; import org.apache.cassandra.db.index.SecondaryIndexManager;
[3/6] cassandra git commit: Fix isClientMode check in Keyspace
Fix isClientMode check in Keyspace
patch by Jeremiah Jordan; reviewed by yukim for CASSANDRA-8687
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2dd6944f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2dd6944f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2dd6944f
Branch: refs/heads/trunk
Commit: 2dd6944f1cf9236e2ca63f6a1fa67b020c175f8f
Parents: d0005a8
Author: Jeremiah Jordan jerem...@datastax.com
Authored: Sun Feb 1 16:07:59 2015 -0600
Committer: Yuki Morishita yu...@apache.org
Committed: Sun Feb 1 16:07:59 2015 -0600
--
CHANGES.txt| 1 +
src/java/org/apache/cassandra/db/Keyspace.java | 7 ++-
2 files changed, 3 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2dd6944f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 4754867..9e9b772 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -5,6 +5,7 @@
(CASSANDRA-8619)
* Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645)
* Add batch remove iterator to ABSC (CASSANDRA-8414, 8666)
+ * Fix isClientMode check in Keyspace (CASSANDRA-8687)
2.0.12:
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2dd6944f/src/java/org/apache/cassandra/db/Keyspace.java
--
diff --git a/src/java/org/apache/cassandra/db/Keyspace.java
b/src/java/org/apache/cassandra/db/Keyspace.java
index 28045f4..38b7c2b 100644
--- a/src/java/org/apache/cassandra/db/Keyspace.java
+++ b/src/java/org/apache/cassandra/db/Keyspace.java
@@ -30,10 +30,7 @@ import com.google.common.collect.Iterables;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import org.apache.cassandra.config.CFMetaData;
-import org.apache.cassandra.config.DatabaseDescriptor;
-import org.apache.cassandra.config.KSMetaData;
-import org.apache.cassandra.config.Schema;
+import org.apache.cassandra.config.*;
import org.apache.cassandra.db.commitlog.CommitLog;
import org.apache.cassandra.db.filter.QueryFilter;
import org.apache.cassandra.db.index.SecondaryIndex;
@@ -69,7 +66,7 @@ public class Keyspace
// proper directories here as well as in CassandraDaemon.
static
{
-if (!StorageService.instance.isClientMode())
+if (!(Config.isClientMode() || StorageService.instance.isClientMode()))
DatabaseDescriptor.createAllDirectories();
}
[jira] [Commented] (CASSANDRA-8687) Keyspace should also check Config.isClientMode
[ https://issues.apache.org/jira/browse/CASSANDRA-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300736#comment-14300736 ] Yuki Morishita commented on CASSANDRA-8687: --- I think the way this should be done is to move static block to create directory to somewhere only server is needed. Though I'm fine to put this in 2.0.x since it is the safe change. Keyspace should also check Config.isClientMode -- Key: CASSANDRA-8687 URL: https://issues.apache.org/jira/browse/CASSANDRA-8687 Project: Cassandra Issue Type: Bug Components: Core, Tools Reporter: Jeremiah Jordan Assignee: Jeremiah Jordan Priority: Minor Fix For: 2.0.13 Attachments: 0001-Fix-isClientMode-check-in-Keyspace-to-also-check-Confi.txt Keyspace static init code should check Config.isClientMode as well as checking StorageService.instance.isClientMode() before trying to make directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6719) redesign loadnewsstables
[ https://issues.apache.org/jira/browse/CASSANDRA-6719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Gupte updated CASSANDRA-6719: - Attachment: 6719.patch I have been poking around this a bit and have attached a patch. It allows a path for a staging directory to be passed with `CFSMBean.loadNewSSTables` and loads new sstables from it. Consider it kinda WIP, mostly because I am not sure if its sufficient and will appreciate a review. Few questions: 1. This might be one of the dumber questions, so please bear with me, but I see `CFSMBean.loadNewSSTables` being used by `StorageService.loadNewSSTables(String, String)` as well, and eventually by NodeProbe, so I believe there will be changes required in that part of code as well. Are those changes not mentioned in the description of the ticket, because they are too obvious or is there anything else? 2. Currently if the parameter for staging directory is not passed, it will default to loading the sstables from the live data dirs. I believe this is not the expected behavior and we should force supplying a parameter, no matter what. But I want to get some opinions before I make this change. redesign loadnewsstables Key: CASSANDRA-6719 URL: https://issues.apache.org/jira/browse/CASSANDRA-6719 Project: Cassandra Issue Type: New Feature Components: Tools Reporter: Jonathan Ellis Priority: Minor Labels: lhf Fix For: 3.0 Attachments: 6719.patch CFSMBean.loadNewSSTables scans data directories for new sstables dropped there by an external agent. This is dangerous because of possible filename conflicts with existing or newly generated sstables. Instead, we should support leaving the new sstables in a separate directory (specified by a parameter, or configured as a new location in yaml) and take care of renaming as necessary automagically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6719) redesign loadnewsstables
[ https://issues.apache.org/jira/browse/CASSANDRA-6719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300771#comment-14300771 ] Chinmay Gupte commented on CASSANDRA-6719: -- Github compare just in case: https://github.com/chinmay-gupte/cassandra/compare/chinmay/redesign_load_newSSTables redesign loadnewsstables Key: CASSANDRA-6719 URL: https://issues.apache.org/jira/browse/CASSANDRA-6719 Project: Cassandra Issue Type: New Feature Components: Tools Reporter: Jonathan Ellis Priority: Minor Labels: lhf Fix For: 3.0 Attachments: 6719.patch CFSMBean.loadNewSSTables scans data directories for new sstables dropped there by an external agent. This is dangerous because of possible filename conflicts with existing or newly generated sstables. Instead, we should support leaving the new sstables in a separate directory (specified by a parameter, or configured as a new location in yaml) and take care of renaming as necessary automagically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8687) Keyspace should also check Config.isClientMode
[ https://issues.apache.org/jira/browse/CASSANDRA-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita resolved CASSANDRA-8687. --- Resolution: Fixed Keyspace should also check Config.isClientMode -- Key: CASSANDRA-8687 URL: https://issues.apache.org/jira/browse/CASSANDRA-8687 Project: Cassandra Issue Type: Bug Components: Core, Tools Reporter: Jeremiah Jordan Assignee: Jeremiah Jordan Priority: Minor Fix For: 2.1.3, 2.0.13 Attachments: 0001-Fix-isClientMode-check-in-Keyspace-to-also-check-Confi.txt Keyspace static init code should check Config.isClientMode as well as checking StorageService.instance.isClientMode() before trying to make directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (CASSANDRA-8687) Keyspace should also check Config.isClientMode
[ https://issues.apache.org/jira/browse/CASSANDRA-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita reopened CASSANDRA-8687: --- Keyspace should also check Config.isClientMode -- Key: CASSANDRA-8687 URL: https://issues.apache.org/jira/browse/CASSANDRA-8687 Project: Cassandra Issue Type: Bug Components: Core, Tools Reporter: Jeremiah Jordan Assignee: Jeremiah Jordan Priority: Minor Fix For: 2.1.3, 2.0.13 Attachments: 0001-Fix-isClientMode-check-in-Keyspace-to-also-check-Confi.txt Keyspace static init code should check Config.isClientMode as well as checking StorageService.instance.isClientMode() before trying to make directories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8715) Possible Deadlock in Cqlsh in a Kerberos-enabled environment when using COPY ... FROM ...
Eduard Tudenhoefner created CASSANDRA-8715:
--
Summary: Possible Deadlock in Cqlsh in a Kerberos-enabled
environment when using COPY ... FROM ...
Key: CASSANDRA-8715
URL: https://issues.apache.org/jira/browse/CASSANDRA-8715
Project: Cassandra
Issue Type: Bug
Environment: Cassandra 2.1.2.160, cqlsh 5.0.1, Native protocol v3
Reporter: Eduard Tudenhoefner
Priority: Critical
When running a COPY ... FROM ... command in a Kerberos environment, I see the
number of rows processed, but eventually, Cqlsh never returns. I can verify,
that all the data was copied, but the progress bar shows me the last shown info
and cqlsh hangs there and never returns.
Please note that this issue did *not* occur in the exact same environment with
*Cassandra 2.0.12.156*.
With the help of Tyler Hobbs, I investigated the problem a little bit further
and added some debug statements at specific points. For example, in the
CountdownLatch class at
https://github.com/apache/cassandra/blob/a323a1a6d5f28ced1a51ba559055283f3eb356ff/pylib/cqlshlib/async_insert.py#L35-L36
I can see that the counter always stays above zero and therefore never returns
(even when the data to be copied is already copied).
I've also seen that somehow when I type in one cqlsh command, there will be
actually two commands. Let me give you an example:
I added a debug statement just before
https://github.com/apache/cassandra/blob/d76450c7986202141f3a917b3623a4c3138c1094/bin/cqlsh#L920
{code}
cqlsh use libdata ;
2015-01-30 18:54:56,113 [DEBUG] root: STATEMENT: [('K_USE', 'use', (0, 3)),
('identifier', 'libdata', (4, 11)), ('endtoken', ';', (12, 13))]
2015-01-30 18:54:56,113 [DEBUG] root: STATEMENT: [('K_USE', 'use', (0, 3)),
('identifier', 'libdata', (4, 11)), ('endtoken', ';', (12, 13))]
{code}
and saw that all commands I enter, they end up being executed twice (same goes
for the COPY command).
If I can provide any other input for debugging purposes, please let me know.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8211) Overlapping sstables in L1+
[
https://issues.apache.org/jira/browse/CASSANDRA-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14300933#comment-14300933
]
Imri Zvik commented on CASSANDRA-8211:
--
I am still seeing this/related symptoms (CASSANDRA-8191), after upgrading to
2.0.12:
ERROR [CompactionExecutor:1116] 2015-02-01 20:47:46,448 CassandraDaemon.java
(line 199) Exception in thread Thread[CompactionExecutor:1116,1,main]
java.lang.RuntimeException: Last written key DecoratedKey(-9223372036854775808,
) = current key DecoratedKey(-9223372036854775808, ) writing into
/var/lib/cassandra/data/system/compaction_history/system-compaction_history-tmp-jb-12-Data.db
at
org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:143)
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:166)
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:167)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
I am also experiencing similar symptoms to those described in CASSANDRA-8210.
Overlapping sstables in L1+
---
Key: CASSANDRA-8211
URL: https://issues.apache.org/jira/browse/CASSANDRA-8211
Project: Cassandra
Issue Type: Bug
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson
Fix For: 2.0.12, 2.1.3
Attachments: 0001-Avoid-overlaps-in-L1-v2.patch,
0001-Avoid-overlaps-in-L1.patch
Seems we have a bug that can create overlapping sstables in L1:
{code}
WARN [main] 2014-10-28 04:09:42,295 LeveledManifest.java (line 164) At level
2, SSTableReader(path='sstable') [DecoratedKey(2838397575996053472, 00
10066059b210066059b210400100),
DecoratedKey(5516674013223138308,
001000ff2d161000ff2d160
00010400100)] overlaps
SSTableReader(path='sstable') [DecoratedKey(2839992722300822584,
0010
00229ad21000229ad210400100),
DecoratedKey(5532836928694021724,
0010034b05a610034b05a6100
000400100)]. This could be caused by a bug in
Cassandra 1.1.0 .. 1.1.3 or due to the fact that you have dropped sstables
from another node into the data directory. Sending back to L0. If
you didn't drop in sstables, and have not yet run scrub, you should do so
since you may also have rows out-of-order within an sstable
{code}
Which might manifest itself during compaction with this exception:
{code}
ERROR [CompactionExecutor:3152] 2014-10-28 00:24:06,134 CassandraDaemon.java
(line 199) Exception in thread Thread[CompactionExecutor:3152,1,main]
java.lang.RuntimeException: Last written key
DecoratedKey(5516674013223138308,
001000ff2d161000ff2d1610400100)
= current key DecoratedKey(2839992722300822584,
001000229ad21000229ad210400100)
writing into sstable
{code}
since we use LeveledScanner when compacting (the backing sstable scanner
might go beyond the start of the next sstable scanner)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
