[jira] [Created] (CASSANDRA-8532) Fix calculation of expected write size during compaction
Marcus Eriksson created CASSANDRA-8532: -- Summary: Fix calculation of expected write size during compaction Key: CASSANDRA-8532 URL: https://issues.apache.org/jira/browse/CASSANDRA-8532 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.1.3 We don't calculate expected sstable size correctly when getting the directory to compact to. Patch attached fixes that -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8532) Fix calculation of expected write size during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-8532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-8532: --- Attachment: 0001-fix-calculation-of-expected-sstable-size.patch > Fix calculation of expected write size during compaction > > > Key: CASSANDRA-8532 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8532 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.1.3 > > Attachments: 0001-fix-calculation-of-expected-sstable-size.patch > > > We don't calculate expected sstable size correctly when getting the directory > to compact to. Patch attached fixes that -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7653) Add role based access control to Cassandra
[
https://issues.apache.org/jira/browse/CASSANDRA-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14256878#comment-14256878
]
Sam Tunnicliffe commented on CASSANDRA-7653:
I've taken another run at this, this time without the consideration of
preserving backwards compatibility in IAuthenticator/IAuthorizer.
https://github.com/beobal/cassandra/compare/7653
The concept of users is replaced by roles and in the postgres style there's no
real differentiation between the two, a user being basically an alias for a
role. As such, grants are made between roles and permissions hierarchies are
enabled. Somewhat distinct from permissions, a role may have a number of
options. Another departure from the previous implementation is that when using
custom auth components there no longer exists the requirement to create
users/roles directly in the database.
h4. CQL Syntax Changes
There are no changes to existing CQL syntax, only additional statements.
{code}
CREATE ROLE [IF NOT EXISTS] [WITH PASSWORD ]
[SUPERUSER|NOSUPERUSER] [LOGIN|NOLOGIN]
ALTER ROLE [WITH PASSWORD ] [SUPERUSER|NOSUPERUSER]
[LOGIN|NOLOGIN]
DROP ROLE [IF EXISTS]
LIST ROLES [OF ] [NORECURSIVE]
{code}
The output of {{LIST ROLES}} varies depending on the privileges of the user
running the statement.
Without {{OF }} option:
* If executed by a superuser, list all roles in the system
* If executed by a non-superuser, list all roles granted to that user
With {{OF }} option:
* If executed by a superuser, list roles granted to for the specific role.
* If executed by a non-superuser, the executing user must be a member of the
specified role (either directly or through inheritence).
The {{NORECURSIVE}} option modifies {{LIST ROLES OF }} statements to
include only roles granted directly to the specified role. Memberships acquired
through inheritance are excluded.
For backwards compatibility, {{CREATE USER}}, {{ALTER USER}}, {{DROP USER}} &
{{LIST USERS}} become aliases for the equivalent role-centric statements. In
the case of {{CREATE USER}} & {{ALTER USER}} statements, the role is assumed to
have {{LOGIN}} privilege. So the two following statements are equivalent:
{code}
CREATE USER foo WITH PASSWORD 'bar' NOSUPERUSER
CREATE ROLE foo WITH PASSWORD 'bar' NOSUPERUSER LOGIN
{code}
Accordingly, the {{LOGIN}} & {{NOLOGIN}} options are not permitted with
{{CREATE}} and {{ALTER USER}}.
Granting of roles is straightforward. The default internal implementation will
disallow any grant that introduces a circular relationship.
{code}
GRANT TO
REVOKE FROM
{code}
Granting and revoking of permissions to roles is unchanged, except that
conceptually the grantee is now a role, rather than a user.
h4. Code changes
h5. IRoleManager
This is a new entity with responsibility for all aspects of role management,
creation/deletion/modification as well as checking for existence during
modification operations. A number of the functions here are simply the old user
management methods previously part of IAuthenticator.
h5. IAuthenticator
The changes here are twofold: removing responsibility for user (role)
management and making SASL the primary authentication mechanism.
h5. IAuthorizer
Minimal changes to this interface, really it's just the semantics that have
changed slightly.
h4. Schema changes for Internal Implementations
h5. Authentication
As role management is now the solely the responsibility of IRoleManager,
IAuthenticator impls generally (and PasswordAuthenticator specifically) no
longer have the means to modify role info. The benefit to that is the
simplification of IAuthenticator's responsibilities, the downside is potential
coupling between IAuthenticator and IRoleManager implementations. For example,
PasswordAuthenticator no longer maintains its own credentials table, instead it
uses the roles table managed by CassandraAuthorizer.
h5. Role Management
The primary table (roles) here is an extension of the existing users table,
previously managed by PasswordAuthenticator. In addition to superuser status of
a role, the option that determines whether the role has login privileges is
held here along with the an password (if PasswordAuthenticator is enabled).
h5. Authorization
As with IAuthorizer, there are minimal changes to the schema of the table used
by CassandraAuthorizer.
h4. Upgrading
The process for converting auth data during a rolling upgrade is fairly
straightforward. As each node is restarted, it will attempt to convert any data
in the legacy tables into the new schema. Until enough nodes to satisfy the
replication strategy for the system_auth keyspace are upgraded and so have the
new schema, this conversion will fail with the failure being reported in the
system log.
During the upgrade, PasswordAuthenticator, CassandraRoleManager &
CassandraAuthorizer will continue to use the legacy tables, so clients sh
[jira] [Updated] (CASSANDRA-7653) Add role based access control to Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-7653: --- Attachment: cql_smoke_test.py CQLSmokeTest.java > Add role based access control to Cassandra > -- > > Key: CASSANDRA-7653 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7653 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Mike Adamson >Assignee: Sam Tunnicliffe > Fix For: 3.0 > > Attachments: 7653.patch, CQLSmokeTest.java, cql_smoke_test.py > > > The current authentication model supports granting permissions to individual > users. While this is OK for small or medium organizations wanting to > implement authorization, it does not work well in large organizations because > of the overhead of having to maintain the permissions for each user. > Introducing roles into the authentication model would allow sets of > permissions to be controlled in one place as a role and then the role granted > to users. Roles should also be able to be granted to other roles to allow > hierarchical sets of permissions to be built up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7886) Coordinator should not wait for read timeouts when replicas hit Exceptions
[ https://issues.apache.org/jira/browse/CASSANDRA-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Spriegel updated CASSANDRA-7886: -- Attachment: 7886_v5_trunk.txt > Coordinator should not wait for read timeouts when replicas hit Exceptions > -- > > Key: CASSANDRA-7886 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7886 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Tested with Cassandra 2.0.8 >Reporter: Christian Spriegel >Assignee: Christian Spriegel >Priority: Minor > Labels: protocolv4 > Fix For: 3.0 > > Attachments: 7886_v1.txt, 7886_v2_trunk.txt, 7886_v3_trunk.txt, > 7886_v4_trunk.txt, 7886_v5_trunk.txt > > > *Issue* > When you have TombstoneOverwhelmingExceptions occuring in queries, this will > cause the query to be simply dropped on every data-node, but no response is > sent back to the coordinator. Instead the coordinator waits for the specified > read_request_timeout_in_ms. > On the application side this can cause memory issues, since the application > is waiting for the timeout interval for every request.Therefore, if our > application runs into TombstoneOverwhelmingExceptions, then (sooner or later) > our entire application cluster goes down :-( > *Proposed solution* > I think the data nodes should send a error message to the coordinator when > they run into a TombstoneOverwhelmingException. Then the coordinator does not > have to wait for the timeout-interval. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7886) Coordinator should not wait for read timeouts when replicas hit Exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-7886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14256948#comment-14256948
]
Christian Spriegel commented on CASSANDRA-7886:
---
Hi @thobbs!
I have a chrismas present for you, in form of a patch file ;-) I attached a v5
patch that contains the fixes.
Regarding TOE: Currently I throw TOEs as exceptions and they get logged just
like any other exception. I am not sure if this is desireable and would like to
hear your feedback. I think we have the following options:
- Leave as it is in v5, meaning TOEs get logged with stacktraces.
- Add catch blocks where neccessary and log it in user-friendly way. But it
might be in many places. Also in this case I would prefer making TOE a checked
exception. Imho TOE should not be unchecked.
- Add TOE logging to C* default exception handler. (I did not investigate yet,
but I assume there is a exceptionhandler)
- Leave it as it was before
Here a few examples how TOEs look now to the user:
TOE using a 3.0 CQLSH (still on CQL-protocol 3):
{code}
cqlsh:test> select * from test;
code=1200 [Coordinator node timed out waiting for replica nodes' responses]
message="Operation timed out - received only 0 responses."
info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
cqlsh:test>
{code}
TOE using a 2.0 CQLSH:
{code}
cqlsh:test> select * from test;
Request did not complete within rpc_timeout.
{code}
TOE with cassandra-cli:
{code}
[default@unknown] use test;
Authenticated to keyspace: test
[default@test] list test;
Using default limit of 100
Using default cell limit of 100
null
TimedOutException()
at
org.apache.cassandra.thrift.Cassandra$get_range_slices_result$get_range_slices_resultStandardScheme.read(Cassandra.java:17448)
at
org.apache.cassandra.thrift.Cassandra$get_range_slices_result$get_range_slices_resultStandardScheme.read(Cassandra.java:17397)
at
org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:17323)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:802)
at
org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:786)
at org.apache.cassandra.cli.CliClient.executeList(CliClient.java:1520)
at
org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:285)
at
org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.java:201)
at org.apache.cassandra.cli.CliMain.main(CliMain.java:331)
[default@test]
{code}
> Coordinator should not wait for read timeouts when replicas hit Exceptions
> --
>
> Key: CASSANDRA-7886
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7886
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: Tested with Cassandra 2.0.8
>Reporter: Christian Spriegel
>Assignee: Christian Spriegel
>Priority: Minor
> Labels: protocolv4
> Fix For: 3.0
>
> Attachments: 7886_v1.txt, 7886_v2_trunk.txt, 7886_v3_trunk.txt,
> 7886_v4_trunk.txt, 7886_v5_trunk.txt
>
>
> *Issue*
> When you have TombstoneOverwhelmingExceptions occuring in queries, this will
> cause the query to be simply dropped on every data-node, but no response is
> sent back to the coordinator. Instead the coordinator waits for the specified
> read_request_timeout_in_ms.
> On the application side this can cause memory issues, since the application
> is waiting for the timeout interval for every request.Therefore, if our
> application runs into TombstoneOverwhelmingExceptions, then (sooner or later)
> our entire application cluster goes down :-(
> *Proposed solution*
> I think the data nodes should send a error message to the coordinator when
> they run into a TombstoneOverwhelmingException. Then the coordinator does not
> have to wait for the timeout-interval.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6986) snapshot fails with SSTable not found if table has already been restored
[ https://issues.apache.org/jira/browse/CASSANDRA-6986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Boudreault updated CASSANDRA-6986: --- Tester: Alan Boudreault > snapshot fails with SSTable not found if table has already been restored > > > Key: CASSANDRA-6986 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6986 > Project: Cassandra > Issue Type: Bug > Components: Tools > Environment: C* 2.0.6 >Reporter: Steven Lowenthal >Assignee: Ryan McGuire >Priority: Minor > > I seem to be running into a weird restore issue. I load a database with > sstableloader, and then take a snapshot of the keyspace. I then truncate the > tables in the keyspace, replace the sstables from the snapshot, and refresh > everything. It seems to work fine. Of course the sstables get renumbered. > I then try to create a new backup of the keyspace, and this happens on one of > the tables on one of the nodes. (and the sstable has been renumbered): > Exception in thread "main" java.lang.RuntimeException: Tried to hard link to > file that does not exist > /raid0/cassandra/data/stock/trades/stock-trades-jb-18-Data.db -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8518) Cassandra Query Request Size Estimator
[ https://issues.apache.org/jira/browse/CASSANDRA-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257123#comment-14257123 ] T Jake Luciani commented on CASSANDRA-8518: --- Is this a duplicate of CASSANDRA-7402 ? > Cassandra Query Request Size Estimator > -- > > Key: CASSANDRA-8518 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8518 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Cheng Ren > > We have been suffering from cassandra node crash due to out of memory for a > long time. The heap dump from the recent crash shows there are 22 native > transport request threads each of which consumes 3.3% of heap size, taking > more than 70% in total. > Heap dump: > !https://dl-web.dropbox.com/get/attach1.png?_subject_uid=303980955&w=AAAVOoncBoZ5aOPbDg2TpRkUss7B-2wlrnhUAv19b27OUA|height=400,width=600! > Expanded view of one thread: > !https://dl-web.dropbox.com/get/Screen%20Shot%202014-12-18%20at%204.06.29%20PM.png?_subject_uid=303980955&w=AACUO4wrbxheRUxv8fwQ9P52T6gBOm5_g9zeIe8odu3V3w|height=400,width=600! > The cassandra we are using now (2.0.4) utilized MemoryAwareThreadPoolExecutor > as the request executor and provided a default request size estimator which > constantly returns 1, meaning it limits only the number of requests being > pushed to the pool. To have more fine-grained control on handling requests > and better protect our node from OOM issue, we propose implementing a more > precise estimator. > Here is our two cents: > For update/delete/insert request: Size could be estimated by adding size of > all class members together. > For scan query, the major part of the request is response, which can be > estimated from the history data. For example if we receive a scan query on a > column family for a certain token range, we keep track of its response size > used as the estimated response size for later scan query on the same cf. > For future requests on the same cf, response size could be calculated by > token range*recorded size/ recorded token range. The request size should be > estimated as (query size + estimated response size). > We believe what we're proposing here can be useful for other people in the > Cassandra community as well. Would you mind providing us feedbacks? Please > let us know if you have any concerns or suggestions regarding this proposal. > Thanks, > Cheng -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8390) The process cannot access the file because it is being used by another process
[
https://issues.apache.org/jira/browse/CASSANDRA-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Radzin updated CASSANDRA-8390:
Attachment: NoHostAvailableLogs.zip
Log files from run that reproduces NoHostAvailable problem. The {{cqlSync}}
successfully finished 11 iterations and failed while doing the iterration #12.
This is the client side exception:
{noformat}
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried
for query failed (tried: localhost/127.0.0.1:9042
(com.datastax.driver.core.TransportException: [localhost/127.0.0.1:9042]
Connection has been closed))
at
com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:65)
at
com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:259)
at
com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:175)
at
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
at
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:36)
at
com.clarisite.clingine.dataaccesslayer.cassandra.CQLTest.cqlSync(CQLTest.java:32)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
com.intellij.junit4.JUnit4TestRunnerUtil$IgnoreIgnoredTestJUnit4ClassRunner.runChild(JUnit4TestRunnerUtil.java:269)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
com.intellij.rt.execution.CommandLineWrapper.main(CommandLineWrapper.java:121)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All
host(s) tried for query failed (tried: localhost/127.0.0.1:9042
(com.datastax.driver.core.TransportException: [localhost/127.0.0.1:9042]
Connection has been closed))
at
com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:102)
at
com.datastax.driver.core.RequestHandler$1.run(RequestHandler.java:176)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
Here is the server side exception:
{noformat}
INFO 16:28:12 Compacted 4 sstables to
[c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-881,].
297,142 bytes to 295,369 (~99% of original) in 375ms = 0.751162MB/s. 16
total partitions merged to 13. Partition merge counts were {1:12, 4:1, }
ERROR 16:28:12 Unable to delete
c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-879-Data.db
(it will be removed on server restart; we'll also retry after GC)
ERROR 16:28:12 Exception in thread Thread[NonPeriodicTasks:1,5,main]
org.apache.cassandra.io.FSWriteError: java.nio.file.FileSystemException:
c:
[jira] [Comment Edited] (CASSANDRA-8390) The process cannot access the file because it is being used by another process
[
https://issues.apache.org/jira/browse/CASSANDRA-8390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257183#comment-14257183
]
Alexander Radzin edited comment on CASSANDRA-8390 at 12/23/14 4:41 PM:
---
Attached Log files (NoHostAvailable.zip) from run that reproduces
NoHostAvailable problem. The {{cqlSync}} successfully finished 11 iterations
and failed while doing the iterration #12.
This is the client side exception:
{noformat}
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried
for query failed (tried: localhost/127.0.0.1:9042
(com.datastax.driver.core.TransportException: [localhost/127.0.0.1:9042]
Connection has been closed))
at
com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:65)
at
com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:259)
at
com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:175)
at
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
at
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:36)
at
com.clarisite.clingine.dataaccesslayer.cassandra.CQLTest.cqlSync(CQLTest.java:32)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
com.intellij.junit4.JUnit4TestRunnerUtil$IgnoreIgnoredTestJUnit4ClassRunner.runChild(JUnit4TestRunnerUtil.java:269)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
com.intellij.rt.execution.CommandLineWrapper.main(CommandLineWrapper.java:121)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All
host(s) tried for query failed (tried: localhost/127.0.0.1:9042
(com.datastax.driver.core.TransportException: [localhost/127.0.0.1:9042]
Connection has been closed))
at
com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:102)
at
com.datastax.driver.core.RequestHandler$1.run(RequestHandler.java:176)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
Here is the server side exception:
{noformat}
INFO 16:28:12 Compacted 4 sstables to
[c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-881,].
297,142 bytes to 295,369 (~99% of original) in 375ms = 0.751162MB/s. 16
total partitions merged to 13. Partition merge counts were {1:12, 4:1, }
ERROR 16:28:12 Unable to delete
c:\progs\apache-cassandra-2.1.2\data\data\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-879-Data.db
(it will be removed on server restart; we'll also retry after GC)
ERROR 16:28:12 Exception in thread Thread[Non
[jira] [Created] (CASSANDRA-8533) Repair hangs on trunk
Philip Thompson created CASSANDRA-8533:
--
Summary: Repair hangs on trunk
Key: CASSANDRA-8533
URL: https://issues.apache.org/jira/browse/CASSANDRA-8533
Project: Cassandra
Issue Type: Bug
Reporter: Philip Thompson
Fix For: 3.0
The dtests {{sstableloader_compression_none_to_snappy_test}} and
{{sstableloader_compression_none_to_deflate_test}} in
{{sstable_generation_loading_test.py}} are hanging on trunk during repair. Test
output can be seen here: http://cassci.datastax.com/job/trunk_dtest/752/console
Here is the jstack output for nodetool
{code}
2014-12-23 12:11:46
Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.65-b04 mixed mode):
"Attach Listener" daemon prio=5 tid=0x7fe51301e800 nid=0x370b waiting on
condition [0x]
java.lang.Thread.State: RUNNABLE
"ClientNotifForwarder-1" daemon prio=5 tid=0x7fe514969800 nid=0x5d03
runnable [0x000113395000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
- locked <0x0007fd504398> (a java.io.BufferedInputStream)
at java.io.DataInputStream.readByte(DataInputStream.java:265)
at
sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:214)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
at
javax.management.remote.rmi.RMIConnectionImpl_Stub.fetchNotifications(Unknown
Source)
at
javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1342)
at
com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:587)
at
com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:470)
at
com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:451)
at
com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:107)
"JMX client heartbeat 2" daemon prio=5 tid=0x7fe5148f8000 nid=0x5b03
waiting on condition [0x000113292000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
com.sun.jmx.remote.internal.ClientCommunicatorAdmin$Checker.run(ClientCommunicatorAdmin.java:174)
at java.lang.Thread.run(Thread.java:745)
"GC Daemon" daemon prio=5 tid=0x7fe5148db800 nid=0x5903 in Object.wait()
[0x00011318f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0007f8009570> (a sun.misc.GC$LatencyLock)
at sun.misc.GC$Daemon.run(GC.java:117)
- locked <0x0007f8009570> (a sun.misc.GC$LatencyLock)
"RMI RenewClean-[10.150.0.64:7100]" daemon prio=5 tid=0x7fe5148d3000
nid=0x5703 in Object.wait() [0x00011308c000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x0007f8010ae0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
- locked <0x0007f8010ae0> (a java.lang.ref.ReferenceQueue$Lock)
at
sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:535)
at java.lang.Thread.run(Thread.java:745)
"RMI Scheduler(0)" daemon prio=5 tid=0x7fe51490c800 nid=0x5607 waiting on
condition [0x000112f89000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x0007f8018ad0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
"Service Thread" daemon prio=5 tid=0x7fe51480f000 nid=0x4d03 runnable
[0x0
[jira] [Updated] (CASSANDRA-8533) Nodetool Repair hangs on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-8533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-8533:
---
Summary: Nodetool Repair hangs on trunk (was: Repair hangs on trunk)
> Nodetool Repair hangs on trunk
> --
>
> Key: CASSANDRA-8533
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8533
> Project: Cassandra
> Issue Type: Bug
>Reporter: Philip Thompson
> Fix For: 3.0
>
>
> The dtests {{sstableloader_compression_none_to_snappy_test}} and
> {{sstableloader_compression_none_to_deflate_test}} in
> {{sstable_generation_loading_test.py}} are hanging on trunk during repair.
> Test output can be seen here:
> http://cassci.datastax.com/job/trunk_dtest/752/console
> Here is the jstack output for nodetool
> {code}
> 2014-12-23 12:11:46
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.65-b04 mixed mode):
> "Attach Listener" daemon prio=5 tid=0x7fe51301e800 nid=0x370b waiting on
> condition [0x]
>java.lang.Thread.State: RUNNABLE
> "ClientNotifForwarder-1" daemon prio=5 tid=0x7fe514969800 nid=0x5d03
> runnable [0x000113395000]
>java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
> - locked <0x0007fd504398> (a java.io.BufferedInputStream)
> at java.io.DataInputStream.readByte(DataInputStream.java:265)
> at
> sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:214)
> at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
> at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
> at
> javax.management.remote.rmi.RMIConnectionImpl_Stub.fetchNotifications(Unknown
> Source)
> at
> javax.management.remote.rmi.RMIConnector$RMINotifClient.fetchNotifs(RMIConnector.java:1342)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.fetchNotifs(ClientNotifForwarder.java:587)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:470)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:451)
> at
> com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:107)
> "JMX client heartbeat 2" daemon prio=5 tid=0x7fe5148f8000 nid=0x5b03
> waiting on condition [0x000113292000]
>java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> com.sun.jmx.remote.internal.ClientCommunicatorAdmin$Checker.run(ClientCommunicatorAdmin.java:174)
> at java.lang.Thread.run(Thread.java:745)
> "GC Daemon" daemon prio=5 tid=0x7fe5148db800 nid=0x5903 in Object.wait()
> [0x00011318f000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007f8009570> (a sun.misc.GC$LatencyLock)
> at sun.misc.GC$Daemon.run(GC.java:117)
> - locked <0x0007f8009570> (a sun.misc.GC$LatencyLock)
> "RMI RenewClean-[10.150.0.64:7100]" daemon prio=5 tid=0x7fe5148d3000
> nid=0x5703 in Object.wait() [0x00011308c000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0007f8010ae0> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x0007f8010ae0> (a java.lang.ref.ReferenceQueue$Lock)
> at
> sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:535)
> at java.lang.Thread.run(Thread.java:745)
> "RMI Scheduler(0)" daemon prio=5 tid=0x7fe51490c800 nid=0x5607 waiting on
> condition [0x000112f89000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0007f8018ad0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
>
[jira] [Created] (CASSANDRA-8534) The default configuration URL does not have the required file:// prefix and throws an exception if cassandra.config is not set.
Andrew Trimble created CASSANDRA-8534: - Summary: The default configuration URL does not have the required file:// prefix and throws an exception if cassandra.config is not set. Key: CASSANDRA-8534 URL: https://issues.apache.org/jira/browse/CASSANDRA-8534 Project: Cassandra Issue Type: Bug Components: Config, Core Environment: Any Reporter: Andrew Trimble Fix For: 2.1.3 Attachments: error.txt In the class org.apache.cassandra.config.YamlConfigurationLoader, the DEFAULT_CONFIGURATION is set to "cassandra.yaml". This is improperly formatted as it does not contain the prefix "file://". If this value is used, a ConfigurationException is thrown (see line 73 of the same class). A solution is to set the cassandra.config system property, but this does not solve the underlying problem. A vanilla Cassandra installation will throw this error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8525) Bloom Filter truePositive counter not updated on key cache hit
[
https://issues.apache.org/jira/browse/CASSANDRA-8525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257375#comment-14257375
]
Tyler Hobbs commented on CASSANDRA-8525:
+1
> Bloom Filter truePositive counter not updated on key cache hit
> --
>
> Key: CASSANDRA-8525
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8525
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Fix For: 2.0.12, 2.1.3
>
> Attachments: 0001-inc-truepositive-for-keycache-hits.patch
>
>
> In {{SSTableReader.getPosition()}}, we're not incrementing the bloom filter
> truePositive count when we get a key cache hit. This can lead to misleading
> FP ratios in metrics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
Leonid Shalupov created CASSANDRA-8535:
--
Summary: java.lang.RuntimeException: Failed to rename XXX to YYY
Key: CASSANDRA-8535
URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
Project: Cassandra
Issue Type: Bug
Environment: Windows 2008 X64
Reporter: Leonid Shalupov
{code}
[22:11:59] :
[junit] java.lang.RuntimeException: Failed to rename
build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-tmp-ka-14-Index.db
to
build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-14-Index.db
[22:11:59] :
[junit]at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
~[main/:na]
[22:11:59] :
[junit]at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
[22:11:59] :
[junit]at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
[22:11:59] :
[junit]at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
[22:11:59] :
[junit]at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
[22:11:59] :
[junit]at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257449#comment-14257449
]
Leonid Shalupov commented on CASSANDRA-8535:
Background: after experiencing various issues like CASSANDRA-8019 on 2.1.1 we
decided to try latest cassandra-2.1 branch.
It failed with exception I quoted in description.
This time I've set up continuous integration for Cassandra on our server:
https://teamcity.jetbrains.com/project.html?projectId=ApacheCassandra&guest=1
to reproduce it
(feel free to use it, just write me)
I run "ant test-all" there and nothing more.
And build failed with this exception:
https://teamcity.jetbrains.com/viewLog.html?tab=buildLog&buildTypeId=ApacheCassandra_TestsWindows&buildId=365436&guest=1
> java.lang.RuntimeException: Failed to rename XXX to YYY
> ---
>
> Key: CASSANDRA-8535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
> Project: Cassandra
> Issue Type: Bug
> Environment: Windows 2008 X64
>Reporter: Leonid Shalupov
>
> {code}
> [22:11:59] :
> [junit] java.lang.RuntimeException: Failed to rename
> build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-tmp-ka-14-Index.db
> to
> build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-14-Index.db
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> ~[na:1.7.0_45]
> [22:11:59] :
> [junit]at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ~[na:1.7.0_45]
> [22:11:59] :
> [junit]at
> java
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257453#comment-14257453
]
Aleksey Yeschenko commented on CASSANDRA-8535:
--
Could you at least mention the name of the failed test?
Otherwise there really isn't much we can do about it.
> java.lang.RuntimeException: Failed to rename XXX to YYY
> ---
>
> Key: CASSANDRA-8535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
> Project: Cassandra
> Issue Type: Bug
> Environment: Windows 2008 X64
>Reporter: Leonid Shalupov
>
> {code}
> [22:11:59] :
> [junit] java.lang.RuntimeException: Failed to rename
> build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-tmp-ka-14-Index.db
> to
> build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-14-Index.db
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
> ~[main/:na]
> [22:11:59] :
> [junit]at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> ~[na:1.7.0_45]
> [22:11:59] :
> [junit]at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ~[na:1.7.0_45]
> [22:11:59] :
> [junit]at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_45]
> [22:11:59] :
> [junit]at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
> [22:11:59] :
> [junit]at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> {code}
--
This messa
[jira] [Updated] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leonid Shalupov updated CASSANDRA-8535:
---
Description:
{code}
java.lang.RuntimeException: Failed to rename
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
to
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
~[main/:na]
at
org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[main/:na]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.nio.file.FileSystemException:
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
->
build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db:
The process cannot access the file because it is being used by another process.
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
~[na:1.7.0_45]
at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
~[na:1.7.0_45]
at
sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
~[na:1.7.0_45]
at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45]
at
org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184)
~[main/:na]
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
~[main/:na]
... 18 common frames omitted
{code}
was:
{code}
[22:11:59] :
[junit] java.lang.RuntimeException: Failed to rename
build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-tmp-ka-14-Index.db
to
build\test\cassandra\data;0\system\schema_columnfamilies-45f5b36024bc3f83a3631034ea4fa697\system-schema_columnfamilies-ka-14-Index.db
[22:11:59] :
[junit]at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
~[main/:na]
[22:11:59] :
[junit]at
org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
[jira] [Commented] (CASSANDRA-8374) Better support of null for UDF
[
https://issues.apache.org/jira/browse/CASSANDRA-8374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257470#comment-14257470
]
Tyler Hobbs commented on CASSANDRA-8374:
bq. If you admit that throwing an error means the function is broken then
surely that means people should unbroke their function by returning null when
they realize it is broken.
I think you may be overlooking the option of doing something other than
returning null or raising an error when a null argument is passed. There are
definitely cases where it can make sense to, say, return some default value on
null.
bq. Can you give me a few example of functions that might make sense to add to
our hardcoded functions and for which throwing an exception on null would be
reasonable, knowing that it would basically mean the function can't be used in
select clauses?
Again, I'm not saying that throwing an exception on null is the best behavior.
I think it's a good way to _alert_ users to the fact that their function is
_broken_. Fixing the function does _not_ necessarily mean changing it to
return null on null input.
An example function off the top of my head: say you're calculating a ratio of
two columns. It can make sense to return 0 instead of null when one of the two
columns is null. I admit that returning null could be okay, too, I just have a
preference for making the user explicitly aware of that behavior. It sounds
like you disagree, which is okay. I'm still -1 making {{RETURNS NULL ON NULL
INPUT}} the default, but you have reasonable arguments against that, so I'll
leave it up to you.
bq. I think the proper default behavior for aggregate function is to ignore
rows that have nulls.
Agreed.
bq. my argument is what you say: RETURNS implies that NULL is used as the
return value, which is just not true because the state isn't updated then.
Other databases generally ignore any NULL input for aggregates (e.g. Oracle
documents that explicitly) - so that could be the way to go: never call a state
function for any NULL argument (and leaving the syntax as proposed).
I think that's confusing the behavior of functions and aggregates too much.
The {{RETURNS NULL ON NULL}} option definitley makes sense for functions. The
behavior of aggregates for such functions is separate (and I agree with Sylvain
that Postgres' aggregate behavior for strict/RNON makes sense).
> Better support of null for UDF
> --
>
> Key: CASSANDRA-8374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8374
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Robert Stupp
> Fix For: 3.0
>
> Attachments: 8473-1.txt, 8473-2.txt
>
>
> Currently, every function needs to deal with it's argument potentially being
> {{null}}. There is very many case where that's just annoying, users should be
> able to define a function like:
> {noformat}
> CREATE FUNCTION addTwo(val int) RETURNS int LANGUAGE JAVA AS 'return val + 2;'
> {noformat}
> without having this crashing as soon as a column it's applied to doesn't a
> value for some rows (I'll note that this definition apparently cannot be
> compiled currently, which should be looked into).
> In fact, I think that by default methods shouldn't have to care about
> {{null}} values: if the value is {{null}}, we should not call the method at
> all and return {{null}}. There is still methods that may explicitely want to
> handle {{null}} (to return a default value for instance), so maybe we can add
> an {{ALLOW NULLS}} to the creation syntax.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8535) java.lang.RuntimeException: Failed to rename XXX to YYY
[
https://issues.apache.org/jira/browse/CASSANDRA-8535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257469#comment-14257469
]
Leonid Shalupov commented on CASSANDRA-8535:
It failed with org.apache.cassandra.cache.AutoSavingCacheTest
testSerializeAndLoadKeyCache test first, then
org.apache.cassandra.config.DefsTest dropKS and followed by many others.
Please read the log, it's filled with them:
https://teamcity.jetbrains.com/downloadBuildLog.html?buildId=365436&guest=1
> java.lang.RuntimeException: Failed to rename XXX to YYY
> ---
>
> Key: CASSANDRA-8535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8535
> Project: Cassandra
> Issue Type: Bug
> Environment: Windows 2008 X64
>Reporter: Leonid Shalupov
>
> {code}
> java.lang.RuntimeException: Failed to rename
> build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
> to
> build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:170)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:154)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:569)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:561)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:535)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.finish(SSTableWriter.java:470)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.finishAndMaybeThrow(SSTableRewriter.java:349)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:324)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SSTableRewriter.finish(SSTableRewriter.java:304)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:200)
> ~[main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> ~[na:1.7.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ~[na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> Caused by: java.nio.file.FileSystemException:
> build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-tmp-ka-5-Index.db
> ->
> build\test\cassandra\data;0\system\schema_keyspaces-b0f2235744583cdb9631c43e59ce3676\system-schema_keyspaces-ka-5-Index.db:
> The process cannot access the file because it is being used by another
> process.
> at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
> ~[na:1.7.0_45]
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> ~[na:1.7.0_45]
> at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
> ~[na:1.7.0_45]
> at
> sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
> ~[na:1.7.0_45]
> at java.nio.file.Files.move(Files.java:1345) ~[na:1.7.0_45]
> at
> org.apache.cassandra.io.util.FileUtils.atomicMoveWithFallback(FileUtils.java:184)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:166)
> ~[main/:na]
> ... 18 common frames omitted
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8518) Cassandra Query Request Size Estimator
[ https://issues.apache.org/jira/browse/CASSANDRA-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257606#comment-14257606 ] Cheng Ren commented on CASSANDRA-8518: -- Thanks for your reply. The feature you suggested is very useful, and we will definitely consider it in future upgrade. But as you mentioned in CASSANDRA-7402, it's not going to solve the underlying OOM issue caused by large response data of requests. The feature we are eager to have should help to prevent OOM in system itself. What we are proposing here is to take advantage of existing code and build a better memory estimator so that system can throttle requests. Please let us know If there is any other related issue. We also would like to hear your feedback on our proposal. Thanks > Cassandra Query Request Size Estimator > -- > > Key: CASSANDRA-8518 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8518 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Cheng Ren > > We have been suffering from cassandra node crash due to out of memory for a > long time. The heap dump from the recent crash shows there are 22 native > transport request threads each of which consumes 3.3% of heap size, taking > more than 70% in total. > Heap dump: > !https://dl-web.dropbox.com/get/attach1.png?_subject_uid=303980955&w=AAAVOoncBoZ5aOPbDg2TpRkUss7B-2wlrnhUAv19b27OUA|height=400,width=600! > Expanded view of one thread: > !https://dl-web.dropbox.com/get/Screen%20Shot%202014-12-18%20at%204.06.29%20PM.png?_subject_uid=303980955&w=AACUO4wrbxheRUxv8fwQ9P52T6gBOm5_g9zeIe8odu3V3w|height=400,width=600! > The cassandra we are using now (2.0.4) utilized MemoryAwareThreadPoolExecutor > as the request executor and provided a default request size estimator which > constantly returns 1, meaning it limits only the number of requests being > pushed to the pool. To have more fine-grained control on handling requests > and better protect our node from OOM issue, we propose implementing a more > precise estimator. > Here is our two cents: > For update/delete/insert request: Size could be estimated by adding size of > all class members together. > For scan query, the major part of the request is response, which can be > estimated from the history data. For example if we receive a scan query on a > column family for a certain token range, we keep track of its response size > used as the estimated response size for later scan query on the same cf. > For future requests on the same cf, response size could be calculated by > token range*recorded size/ recorded token range. The request size should be > estimated as (query size + estimated response size). > We believe what we're proposing here can be useful for other people in the > Cassandra community as well. Would you mind providing us feedbacks? Please > let us know if you have any concerns or suggestions regarding this proposal. > Thanks, > Cheng -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257721#comment-14257721
]
Robert Stupp commented on CASSANDRA-7438:
-
I had the opportunity to test OHC on a big machine.
First: it works - very happy about that :)
Some things I want to notice:
* high number of segments do not have any really measurable influence (default
of 2* # of cores is fine)
* throughput heavily depends on serialization (hash entry size) - Java8 gave
about 10% to 15% improvement in some tests (either on {{Unsafe.copyMemory}} or
something related like JNI barrier)
* the number of entries per bucket stays pretty low with the default load
factor of .75 - vast majority has 0 or 1 entries, some 2 or 3 and few up to 8
Issue (not solvable yet):
It works great for hash entries to approx. 64kB with good to great throughput.
Above that barrier it first works good but after some time the system spends a
huge amount of CPU time (~95%) in {{malloc()}} / {{free()}} (with jemalloc,
Unsafe.allocate is not worth discussing at all on Linux).
I tried to add some „memory buffer cache“ that caches free’d hash entries for
reuse. But it turned out that in the end it would be too complex if done right.
The current implementation is still in the code, but must be explicitly enabled
with a system property. Workloads with small entries and high number of threads
easily trigger Linux OOM protection (that kills the process). Please note that
it works with large hash entries - but throughput drops dramatically to just a
few thousand writes per second.
Some numbers (value sizes have gaussian distribution). Had to do these tests in
a hurry because I had to give back the machine. Code used during these tests is
tagged as {{0.1-SNAP-Bench}} in git. Throughput is limited by {{malloc()}} /
{{free()}} and most tests did only use 50% of available CPU capacity (on
_c3.8xlarge_ - 32 cores, Intel Xeon E5-2680v2 @2.8GHz, 64GB).
* 1k..200k value size, 32 threads, 1M keys, 90% read ratio, 32GB: 22k
writes/sec, 200k reads/sec, ~8k evictions/sec, write: 8ms (99perc), read:
3ms(99perc)
* 1k..64k value size, 500 threads, 1M keys, 90% read ratio, 32GB: 55k
writes/sec, 499k reads/sec, ~2k evictions/sec, write: .1ms (99perc), read:
.03ms(99perc)
* 1k..64k value size, 500 threads, 1M keys, 50% read ratio, 32GB: 195k
writes/sec, 195k reads/sec, ~9k evictions/sec, write: .2ms (99perc), read:
.1ms(99perc)
* 1k..64k value size, 500 threads, 1M keys, 10% read ratio, 32GB: 185k
writes/sec, 20k reads/sec, ~7k evictions/sec, write: 4ms (99perc), read:
.07ms(99perc)
* 1k..16k value size, 500 threads, 5M keys, 90% read ratio, 32GB: 110k
writes/sec, 1M reads/sec, 30k evictions/sec, write: .04ms (99perc), read:
.01ms(99perc)
* 1k..16k value size, 500 threads, 5M keys, 50% read ratio, 32GB: 420k
writes/sec, 420k reads/sec, 125k evictions/sec, write: .06ms (99perc), read:
.01ms(99perc)
* 1k..16k value size, 500 threads, 5M keys, 10% read ratio, 32GB: 435k
writes/sec, 48k reads/sec, 130k evictions/sec, write: .06ms (99perc), read:
.01ms(99perc)
* 1k..4k value size, 500 threads, 20M keys, 90% read ratio, 32GB: 140k
writes/sec, 1.25M reads/sec, 50k evictions/sec, write: .02ms (99perc), read:
.005ms(99perc)
* 1k..4k value size, 500 threads, 20M keys, 50% read ratio, 32GB: 530k
writes/sec, 530k reads/sec, 220k evictions/sec, write: .04ms (99perc), read:
.005ms(99perc)
* 1k..4k value size, 500 threads, 20M keys, 10% read ratio, 32GB: 665k
writes/sec, 74k reads/sec, 250k evcictions/sec, write: .04ms (99perc), read:
.005ms(99perc)
Command line to execute the benchmark:
{code}
java -jar ohc-benchmark/target/ohc-benchmark-0.1-SNAPSHOT.jar -rkd
'uniform(1..2000)' -wkd 'uniform(1..2000)' -vs 'gaussian(1024..4096,2)'
-r .1 -cap 320 -d 86400 -t 500 -dr 8
-r = read rate
-d = duration
-t = # of threads
-dr = # of driver threads that feed the worker threads
-rkd = read key distribution
-wkd = write key distribution
-vs = value size
-cap = capacity
{code}
Sample bucket histogram from 20M test:
{code}
[0..0]: 8118604
[1..1]: 5892298
[2..2]: 2138308
[3..3]: 518089
[4..4]: 94441
[5..5]: 13672
[6..6]: 1599
[7..7]: 189
[8..9]: 16
{code}
After trapping into that memory management issue with varying allocation sized
of some few kB to several MB, I think that it’s still worth to work on an own
off-heap memory management. Maybe some block-based approach (fixed or
variable). But that’s out of the scope of this ticket.
> Serializing Row cache alternative (Fully off heap)
> --
>
> Key: CASSANDRA-7438
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7438
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: Linux
>Reporter: Vij
[jira] [Commented] (CASSANDRA-7666) Range-segmented sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-7666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257751#comment-14257751 ] Tupshin Harper commented on CASSANDRA-7666: --- I think that it's sufficient to let this be dormant until or unless it is needed to support other features. DTCS covers most of the immediate benefit. Future possible features such as tiered storage and the ability to drop whole segments at a time, however, mean that we should not defer this one indefinitely. > Range-segmented sstables > > > Key: CASSANDRA-7666 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7666 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Jonathan Ellis >Assignee: Sam Tunnicliffe > Fix For: 3.0 > > > It would be useful to segment sstables by data range (not just token range as > envisioned by CASSANDRA-6696). > The primary use case is to allow deleting those data ranges for "free" by > dropping the sstables involved. We should also (possibly as a separate > ticket) be able to leverage this information in query planning to avoid > unnecessary sstable reads. > Relational databases typically call this "partitioning" the table, but > obviously we use that term already for something else: > http://www.postgresql.org/docs/9.1/static/ddl-partitioning.html > Tokutek's take for mongodb: > http://docs.tokutek.com/tokumx/tokumx-partitioned-collections.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
