[jira] [Commented] (CASSANDRA-1311) Support (asynchronous) triggers
[
https://issues.apache.org/jira/browse/CASSANDRA-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072731#comment-13072731
]
Martin Hentschel commented on CASSANDRA-1311:
-
Hi Steve,
Yes, this thread has been silent for quiet some time though we still believe in
our ideas. For research purposes, we evaluated our approach of asynchronous
triggers against the state-of-the-art approach of buffering work items in some
queue outside of Cassandra. Our main findings were the following:
* Our approach requires 15% less machines to perform the same tasks and reduces
overall network traffic by 30%. That is because all work is performed inside
Cassandra instead of outside of Cassandra involving more machines to host the
queue and worker threads that execute work items.
* Our approach gives you the program interface of triggers, which makes it
easier to program applications than writing code to store work items in the
queue and to execute work items with at-least-once semantics.
We now enhanced our ideas towards exactly-once semantics (which were also
favored by Jonathan Ellis in this thread) and an even easier MapReduce-like
programming model. Though, for research purposes, our prototype is not based
on Cassandra anymore. If there is enough interest in our new ideas, we think
it can be easily ported to Cassandra. An in-depth description will be
published soon at the VLDB conference ("Analytics for the Real-Time Web",
http://www.vldb.org/2011/?q=node/23).
Thanks,
Martin
> Support (asynchronous) triggers
> ---
>
> Key: CASSANDRA-1311
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1311
> Project: Cassandra
> Issue Type: New Feature
> Components: Contrib
>Reporter: Maxim Grinev
> Fix For: 1.0
>
> Attachments: HOWTO-PatchAndRunTriggerExample-update1.txt,
> HOWTO-PatchAndRunTriggerExample.txt, ImplementationDetails-update1.pdf,
> ImplementationDetails.pdf, trunk-967053.txt, trunk-984391-update1.txt,
> trunk-984391-update2.txt
>
>
> Asynchronous triggers is a basic mechanism to implement various use cases of
> asynchronous execution of application code at database side. For example to
> support indexes and materialized views, online analytics, push-based data
> propagation.
> Please find the motivation, triggers description and list of applications:
> http://maxgrinev.com/2010/07/23/extending-cassandra-with-asynchronous-triggers/
> An example of using triggers for indexing:
> http://maxgrinev.com/2010/07/23/managing-indexes-in-cassandra-using-async-triggers/
> Implementation details are attached.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2785) should export JAVA variable in the bin/cassandra and use that in the cassandra-env.sh when check for the java version
[ https://issues.apache.org/jira/browse/CASSANDRA-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072766#comment-13072766 ] Eric Evans commented on CASSANDRA-2785: --- +1 > should export JAVA variable in the bin/cassandra and use that in the > cassandra-env.sh when check for the java version > - > > Key: CASSANDRA-2785 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2785 > Project: Cassandra > Issue Type: Bug >Reporter: Jackson Chung >Assignee: paul cannon > Attachments: 0001-use-JAVA-in-cassandra-env.sh.patch.txt, > 0002-fix-usage-of-bash-n-tests.patch.txt > > > I forgot which jira we add this java -version check in the cassandra-env.sh > (for adding jamm to the javaagent), but we should probably use the variable > JAVA set in bin/cassandra (will need export) and use $JAVA instead of "java" > in the cassandra-env.sh > In a situation where JAVA_HOME may have been properly set as the Sun's java > but the PATH still have the OpenJDK's java in front, the check will fail to > add the jamm.jar, even though the cassandra jvm is properly started via the > Sun's java. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2968) AssertionError during compaction of CF with counter columns
AssertionError during compaction of CF with counter columns --- Key: CASSANDRA-2968 URL: https://issues.apache.org/jira/browse/CASSANDRA-2968 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.2 Environment: CentOS release 5.6 Reporter: Taras Puchko Having upgraded from 0.8.0 to 0.8.2 we ran nodetool compact and got Error occured during compaction java.util.concurrent.ExecutionException: java.lang.AssertionError at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) at java.util.concurrent.FutureTask.get(FutureTask.java:83) at org.apache.cassandra.db.compaction.CompactionManager.performMajor(CompactionManager.java:277) at org.apache.cassandra.db.ColumnFamilyStore.forceMajorCompaction(ColumnFamilyStore.java:1762) at org.apache.cassandra.service.StorageService.forceTableCompaction(StorageService.java:1358) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427) at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788) at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305) at sun.rmi.transport.Transport$1.run(Transport.java:159) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:155) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.AssertionError at org.apache.cassandra.db.context.CounterContext.removeOldShards(CounterContext.java:593) at org.apache.cassandra.db.CounterColumn.removeOldShards(CounterColumn.java:237) at org.apache.cassandra.db.CounterColumn.removeOldShards(CounterColumn.java:256) at org.apache.cassandra.db.compaction.PrecompactedRow.(PrecompactedRow.java:88) at org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:140) at org.apache.cassandra.db
svn commit: r1152218 - /cassandra/trunk/src/avro/internode.genavro
Author: slebresne
Date: Fri Jul 29 13:40:21 2011
New Revision: 1152218
URL: http://svn.apache.org/viewvc?rev=1152218&view=rev
Log:
Fix migration.SerializationsTest (broke by the commit of #1966)
Modified:
cassandra/trunk/src/avro/internode.genavro
Modified: cassandra/trunk/src/avro/internode.genavro
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/avro/internode.genavro?rev=1152218&r1=1152217&r2=1152218&view=diff
==
--- cassandra/trunk/src/avro/internode.genavro (original)
+++ cassandra/trunk/src/avro/internode.genavro Fri Jul 29 13:40:21 2011
@@ -57,7 +57,7 @@ protocol InterNode {
union { null, int } max_compaction_threshold = null;
union { int, null } row_cache_save_period_in_seconds = 0;
union { int, null } key_cache_save_period_in_seconds = 3600;
-union { int, null } row_cache_keys_to_save = null;
+union { null, int } row_cache_keys_to_save = null;
union { null, int } memtable_throughput_in_mb = null;
union { null, double} memtable_operations_in_millions = null;
union { null, double} merge_shards_chance = null;
[jira] [Commented] (CASSANDRA-1311) Support (asynchronous) triggers
[ https://issues.apache.org/jira/browse/CASSANDRA-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072830#comment-13072830 ] T Jake Luciani commented on CASSANDRA-1311: --- Hi Martin, I'm interested in learning more about your new approach. I think the concerns over the current approach are summed up by: bq. This sounds really, really fragile. Grafting a pseudo-master onto Cassandra replication is a bad idea. I think for version 1 of triggers we need to go with the simplest approach. I believe this is to make the triggers synchronous and run RF times. Another possibility is to make the coordinator node responsible for running/delegating the trigger before acknowledging a successful write... As we bake out new ideas we can look at improving this, but I do think a simple trigger mechanism is needed for 1.0. > Support (asynchronous) triggers > --- > > Key: CASSANDRA-1311 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1311 > Project: Cassandra > Issue Type: New Feature > Components: Contrib >Reporter: Maxim Grinev > Fix For: 1.0 > > Attachments: HOWTO-PatchAndRunTriggerExample-update1.txt, > HOWTO-PatchAndRunTriggerExample.txt, ImplementationDetails-update1.pdf, > ImplementationDetails.pdf, trunk-967053.txt, trunk-984391-update1.txt, > trunk-984391-update2.txt > > > Asynchronous triggers is a basic mechanism to implement various use cases of > asynchronous execution of application code at database side. For example to > support indexes and materialized views, online analytics, push-based data > propagation. > Please find the motivation, triggers description and list of applications: > http://maxgrinev.com/2010/07/23/extending-cassandra-with-asynchronous-triggers/ > An example of using triggers for indexing: > http://maxgrinev.com/2010/07/23/managing-indexes-in-cassandra-using-async-triggers/ > Implementation details are attached. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1152224 - /cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java
Author: slebresne Date: Fri Jul 29 14:00:25 2011 New Revision: 1152224 URL: http://svn.apache.org/viewvc?rev=1152224&view=rev Log: Fix bad fallthrough switch Modified: cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java Modified: cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java?rev=1152224&r1=1152223&r2=1152224&view=diff == --- cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/cli/CliClient.java Fri Jul 29 14:00:25 2011 @@ -1252,6 +1252,7 @@ public class CliClient break; case KEY_VALIDATION_CLASS: cfDef.setKey_validation_class(CliUtils.unescapeSQLString(mValue)); +break; case COMPACTION_STRATEGY: cfDef.setCompaction_strategy(CliUtils.unescapeSQLString(mValue)); break;
svn commit: r1152233 [3/9] - in /cassandra/trunk: conf/ src/gen-java/ src/gen-java/org/ src/gen-java/org/apache/ src/gen-java/org/apache/cassandra/ src/gen-java/org/apache/cassandra/cli/ src/gen-java/
Propchange: cassandra/trunk/src/gen-java/org/apache/cassandra/cli/CliLexer.java -- svn:eol-style = native
svn commit: r1152233 [5/9] - in /cassandra/trunk: conf/ src/gen-java/ src/gen-java/org/ src/gen-java/org/apache/ src/gen-java/org/apache/cassandra/ src/gen-java/org/apache/cassandra/cli/ src/gen-java/
Propchange: cassandra/trunk/src/gen-java/org/apache/cassandra/cli/CliParser.java
--
svn:eol-style = native
Added: cassandra/trunk/src/gen-java/org/apache/cassandra/cql/Cql.tokens
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/gen-java/org/apache/cassandra/cql/Cql.tokens?rev=1152233&view=auto
==
--- cassandra/trunk/src/gen-java/org/apache/cassandra/cql/Cql.tokens (added)
+++ cassandra/trunk/src/gen-java/org/apache/cassandra/cql/Cql.tokens Fri Jul 29
14:21:06 2011
@@ -0,0 +1,126 @@
+LETTER=75
+K_CREATE=32
+K_PRIMARY=37
+T__93=93
+T__94=94
+T__91=91
+K_USE=4
+T__92=92
+K_VALUES=23
+STRING_LITERAL=9
+T__90=90
+K_ON=41
+K_USING=11
+K_ADD=45
+K_KEY=38
+COMMENT=78
+K_TRUNCATE=47
+T__99=99
+T__98=98
+T__97=97
+T__96=96
+T__95=95
+D=61
+E=49
+F=53
+G=67
+K_COUNT=7
+T__80=80
+K_KEYSPACE=33
+K_TYPE=44
+T__81=81
+A=59
+B=70
+T__82=82
+T__83=83
+C=51
+L=50
+M=56
+N=60
+O=55
+H=58
+I=64
+J=72
+K_UPDATE=29
+K=62
+U=65
+T=52
+W=57
+V=69
+Q=68
+P=66
+S=48
+R=54
+T__85=85
+T__84=84
+T__87=87
+T__86=86
+K_TTL=25
+T__89=89
+Y=63
+X=71
+T__88=88
+Z=73
+K_INDEX=40
+K_REVERSED=17
+K_INSERT=21
+WS=77
+K_APPLY=28
+K_TIMESTAMP=24
+K_AND=19
+K_LEVEL=13
+K_BATCH=27
+UUID=46
+K_DELETE=31
+FLOAT=39
+K_SELECT=6
+K_LIMIT=15
+K_ALTER=43
+K_SET=30
+K_WHERE=14
+MULTILINE_COMMENT=79
+HEX=76
+K_INTO=22
+T__103=103
+T__104=104
+IDENT=5
+DIGIT=74
+K_FIRST=16
+K_BEGIN=26
+INTEGER=10
+RANGEOP=18
+K_CONSISTENCY=12
+K_WITH=34
+COMPIDENT=35
+T__102=102
+T__101=101
+T__100=100
+K_IN=20
+K_FROM=8
+K_COLUMNFAMILY=36
+K_DROP=42
+'counter'=94
+'>='=103
+'text'=88
+'uuid'=93
+'>'=104
+'bytea'=86
+'\*'=83
+';'=84
+'='=85
+'+'=99
+')'=81
+'date'=96
+'float'=97
+'boolean'=95
+'ascii'=87
+'double'=98
+'varint'=91
+'<='=102
+'varchar'=89
+'int'=90
+'<'=101
+'bigint'=92
+'('=80
+'-'=100
+','=82
svn commit: r1152233 [1/9] - in /cassandra/trunk: conf/ src/gen-java/ src/gen-java/org/ src/gen-java/org/apache/ src/gen-java/org/apache/cassandra/ src/gen-java/org/apache/cassandra/cli/ src/gen-java/
Author: brandonwilliams Date: Fri Jul 29 14:21:06 2011 New Revision: 1152233 URL: http://svn.apache.org/viewvc?rev=1152233&view=rev Log: Add asynchronous and half-sync/half-async thrift servers. Patch by Vijay Parthasarathy, reviewed by brandonwilliams for CASSANDRA-1405 Added: cassandra/trunk/src/gen-java/ cassandra/trunk/src/gen-java/org/ cassandra/trunk/src/gen-java/org/apache/ cassandra/trunk/src/gen-java/org/apache/cassandra/ cassandra/trunk/src/gen-java/org/apache/cassandra/cli/ cassandra/trunk/src/gen-java/org/apache/cassandra/cli/Cli.tokens cassandra/trunk/src/gen-java/org/apache/cassandra/cli/CliLexer.java (with props) cassandra/trunk/src/gen-java/org/apache/cassandra/cli/CliParser.java (with props) cassandra/trunk/src/gen-java/org/apache/cassandra/cql/ cassandra/trunk/src/gen-java/org/apache/cassandra/cql/Cql.tokens cassandra/trunk/src/gen-java/org/apache/cassandra/cql/CqlLexer.java (with props) cassandra/trunk/src/gen-java/org/apache/cassandra/cql/CqlParser.java (with props) cassandra/trunk/src/java/org/apache/cassandra/service/SocketSessionManagementService.java (with props) cassandra/trunk/src/java/org/apache/cassandra/thrift/CustomTHsHaServer.java (with props) cassandra/trunk/src/java/org/apache/cassandra/thrift/CustomTNonBlockingServer.java (with props) cassandra/trunk/src/java/org/apache/cassandra/thrift/TCustomNonblockingServerSocket.java (with props) Modified: cassandra/trunk/conf/cassandra.yaml cassandra/trunk/conf/log4j-server.properties cassandra/trunk/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java cassandra/trunk/src/java/org/apache/cassandra/config/Config.java cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraDaemon.java cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java Modified: cassandra/trunk/conf/cassandra.yaml URL: http://svn.apache.org/viewvc/cassandra/trunk/conf/cassandra.yaml?rev=1152233&r1=1152232&r2=1152233&view=diff == --- cassandra/trunk/conf/cassandra.yaml (original) +++ cassandra/trunk/conf/cassandra.yaml Fri Jul 29 14:21:06 2011 @@ -203,16 +203,35 @@ rpc_port: 9160 # enable or disable keepalive on rpc connections rpc_keepalive: true -# Cassandra uses thread-per-client for client RPC. This can -# be expensive in memory used for thread stack for a large -# enough number of clients. (Hence, connection pooling is -# very, very strongly recommended.) -# +# Cassandra provides you with a variety of options for RPC Server +# sync -> Creates one thread per connection but with a configurable number of +# threads. This can be expensive in memory used for thread stack for +# a large enough number of clients. (Hence, connection pooling is +# very, very strongly recommended.) +# +# async -> Nonblocking server implementation with one thread to serve +# rpc connections. This is not recommended for high throughput use +# cases. +# +# hsha -> half sync and half async implementation with configurable number +# of worker threads (For managing connections). IO Management is +# done by a set of threads currently equal to the number of +# processors in the system. The number of threads in the threadpool +# is configured via rpc_min_threads and rpc_max_threads. (Connection +# pooling is strongly recommended in this case too.) + +rpc_server_type: sync + # Uncomment rpc_min|max|thread to set request pool size. -# You would primarily set max as a safeguard against misbehaved -# clients; if you do hit the max, Cassandra will block until -# one disconnects before accepting more. The defaults are -# min of 16 and max unlimited. +# You would primarily set max for the sync server to safeguard against +# misbehaved clients; if you do hit the max, Cassandra will block until one +# disconnects before accepting more. The defaults are min of 16 and max +# unlimited. +# +# For the Hsha server, you would set the max so that a fair amount of resources +# are provided to the other working threads on the server. +# +# This configuration is not used for the async server. # # rpc_min_threads: 16 # rpc_max_threads: 2048 Modified: cassandra/trunk/conf/log4j-server.properties URL: http://svn.apache.org/viewvc/cassandra/trunk/conf/log4j-server.properties?rev=1152233&r1=1152232&r2=1152233&view=diff == --- cassandra/trunk/conf/log4j-server.properties (original) +++ cassandra/trunk/conf/log4j-server.properties Fri Jul 29 14:21:06 2011 @@ -39,3 +39,6 @@ log4j.appender.R.File=/var/log/cassandra #log4j.logger.org.apache.cassandra.db=DEBUG #log4j.logger.org.apache.cassandra.service.Storage
svn commit: r1152233 [7/9] - in /cassandra/trunk: conf/ src/gen-java/ src/gen-java/org/ src/gen-java/org/apache/ src/gen-java/org/apache/cassandra/ src/gen-java/org/apache/cassandra/cli/ src/gen-java/
Propchange: cassandra/trunk/src/gen-java/org/apache/cassandra/cql/CqlLexer.java -- svn:eol-style = native
svn commit: r1152233 [9/9] - in /cassandra/trunk: conf/ src/gen-java/ src/gen-java/org/ src/gen-java/org/apache/ src/gen-java/org/apache/cassandra/ src/gen-java/org/apache/cassandra/cli/ src/gen-java/
Propchange: cassandra/trunk/src/gen-java/org/apache/cassandra/cql/CqlParser.java
--
svn:eol-style = native
Modified:
cassandra/trunk/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java?rev=1152233&r1=1152232&r2=1152233&view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java
Fri Jul 29 14:21:06 2011
@@ -56,13 +56,24 @@ public class JMXEnabledThreadPoolExecuto
}
public JMXEnabledThreadPoolExecutor(int corePoolSize,
+long keepAliveTime,
+TimeUnit unit,
+BlockingQueue workQueue,
+NamedThreadFactory threadFactory,
+String jmxPath)
+{
+this(corePoolSize, corePoolSize, keepAliveTime, unit, workQueue,
threadFactory, jmxPath);
+}
+
+public JMXEnabledThreadPoolExecutor(int corePoolSize,
+int maxPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue workQueue,
NamedThreadFactory threadFactory,
String jmxPath)
{
-super(corePoolSize, keepAliveTime, unit, workQueue, threadFactory);
+super(corePoolSize, maxPoolSize, keepAliveTime, unit, workQueue,
threadFactory);
super.prestartAllCoreThreads();
MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
Modified: cassandra/trunk/src/java/org/apache/cassandra/config/Config.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/config/Config.java?rev=1152233&r1=1152232&r2=1152233&view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/config/Config.java (original)
+++ cassandra/trunk/src/java/org/apache/cassandra/config/Config.java Fri Jul 29
14:21:06 2011
@@ -67,6 +67,7 @@ public class Config
public String rpc_address;
public Integer rpc_port = 9160;
+public String rpc_server_type = "sync";
public Boolean rpc_keepalive = true;
public Integer rpc_min_threads = 16;
public Integer rpc_max_threads = Integer.MAX_VALUE;
Modified:
cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java?rev=1152233&r1=1152232&r2=1152233&view=diff
==
---
cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
(original)
+++
cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
Fri Jul 29 14:21:06 2011
@@ -48,6 +48,7 @@ import org.apache.cassandra.io.util.Mmap
import org.apache.cassandra.locator.*;
import org.apache.cassandra.scheduler.IRequestScheduler;
import org.apache.cassandra.scheduler.NoScheduler;
+import org.apache.cassandra.thrift.CassandraDaemon;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.Pair;
import org.yaml.snakeyaml.Loader;
@@ -369,6 +370,9 @@ public class DatabaseDescriptor
if (conf.compaction_throughput_mb_per_sec == null)
conf.compaction_throughput_mb_per_sec = 16;
+if
(!CassandraDaemon.rpc_server_types.contains(conf.rpc_server_type.toLowerCase()))
+throw new ConfigurationException("Unknown rpc_server_type: " +
conf.rpc_server_type);
+
/* data file and commit log directories. they get created later,
when they're needed. */
if (conf.commitlog_directory != null && conf.data_file_directories
!= null && conf.saved_caches_directory != null)
{
@@ -899,6 +903,11 @@ public class DatabaseDescriptor
{
return rpcAddress;
}
+
+public static String getRpcServerType()
+{
+return conf.rpc_server_type;
+}
public static boolean getRpcKeepAlive()
{
Added:
cassandra/trunk/src/java/org/apache/cassandra/service/SocketSessionManagementService.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/service/SocketSessionManagementService.java?rev=1152233&view=auto
==
---
cassandra/trunk/src/java/org/apache/cassandra/service/SocketSessionManagementService.java
(added)
+++
cassandra/trunk/src/java/org/apache/cassandra/service/SocketSessionManag
svn commit: r1152234 - in /cassandra/trunk/src/gen-java/org/apache/cassandra: cli/Cli.tokens cli/CliLexer.java cli/CliParser.java cql/Cql.tokens cql/CqlLexer.java cql/CqlParser.java
Author: brandonwilliams Date: Fri Jul 29 14:22:40 2011 New Revision: 1152234 URL: http://svn.apache.org/viewvc?rev=1152234&view=rev Log: Remove accidentally committed gen-java crap Removed: cassandra/trunk/src/gen-java/org/apache/cassandra/cli/Cli.tokens cassandra/trunk/src/gen-java/org/apache/cassandra/cli/CliLexer.java cassandra/trunk/src/gen-java/org/apache/cassandra/cli/CliParser.java cassandra/trunk/src/gen-java/org/apache/cassandra/cql/Cql.tokens cassandra/trunk/src/gen-java/org/apache/cassandra/cql/CqlLexer.java cassandra/trunk/src/gen-java/org/apache/cassandra/cql/CqlParser.java
svn commit: r1152238 - in /cassandra/branches/cassandra-0.8: conf/ src/java/org/apache/cassandra/concurrent/ src/java/org/apache/cassandra/config/ src/java/org/apache/cassandra/service/ src/java/org/a
Author: brandonwilliams Date: Fri Jul 29 14:29:49 2011 New Revision: 1152238 URL: http://svn.apache.org/viewvc?rev=1152238&view=rev Log: Add asynchronous and half-sync/half-async thrift servers. Patch by Vijay Parthasarathy, reviewed by brandonwilliams for CASSANDRA-1405 Added: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/SocketSessionManagementService.java (with props) cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CustomTHsHaServer.java (with props) cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CustomTNonBlockingServer.java (with props) cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/TCustomNonblockingServerSocket.java (with props) Modified: cassandra/branches/cassandra-0.8/conf/cassandra.yaml cassandra/branches/cassandra-0.8/conf/log4j-server.properties cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/Config.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java Modified: cassandra/branches/cassandra-0.8/conf/cassandra.yaml URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/conf/cassandra.yaml?rev=1152238&r1=1152237&r2=1152238&view=diff == --- cassandra/branches/cassandra-0.8/conf/cassandra.yaml (original) +++ cassandra/branches/cassandra-0.8/conf/cassandra.yaml Fri Jul 29 14:29:49 2011 @@ -196,16 +196,35 @@ rpc_port: 9160 # enable or disable keepalive on rpc connections rpc_keepalive: true -# Cassandra uses thread-per-client for client RPC. This can -# be expensive in memory used for thread stack for a large -# enough number of clients. (Hence, connection pooling is -# very, very strongly recommended.) -# +# Cassandra provides you with a variety of options for RPC Server +# sync -> Creates one thread per connection but with a configurable number of +# threads. This can be expensive in memory used for thread stack for +# a large enough number of clients. (Hence, connection pooling is +# very, very strongly recommended.) +# +# async -> Nonblocking server implementation with one thread to serve +# rpc connections. This is not recommended for high throughput use +# cases. +# +# hsha -> half sync and half async implementation with configurable number +# of worker threads (For managing connections). IO Management is +# done by a set of threads currently equal to the number of +# processors in the system. The number of threads in the threadpool +# is configured via rpc_min_threads and rpc_max_threads. (Connection +# pooling is strongly recommended in this case too.) + +rpc_server_type: sync + # Uncomment rpc_min|max|thread to set request pool size. -# You would primarily set max as a safeguard against misbehaved -# clients; if you do hit the max, Cassandra will block until -# one disconnects before accepting more. The defaults are -# min of 16 and max unlimited. +# You would primarily set max for the sync server to safeguard against +# misbehaved clients; if you do hit the max, Cassandra will block until one +# disconnects before accepting more. The defaults are min of 16 and max +# unlimited. +# +# For the Hsha server, you would set the max so that a fair amount of resources +# are provided to the other working threads on the server. +# +# This configuration is not used for the async server. # # rpc_min_threads: 16 # rpc_max_threads: 2048 Modified: cassandra/branches/cassandra-0.8/conf/log4j-server.properties URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/conf/log4j-server.properties?rev=1152238&r1=1152237&r2=1152238&view=diff == --- cassandra/branches/cassandra-0.8/conf/log4j-server.properties (original) +++ cassandra/branches/cassandra-0.8/conf/log4j-server.properties Fri Jul 29 14:29:49 2011 @@ -39,3 +39,6 @@ log4j.appender.R.File=/var/log/cassandra #log4j.logger.org.apache.cassandra.db=DEBUG #log4j.logger.org.apache.cassandra.service.StorageProxy=DEBUG +# Adding this to avoid thrift logging disconnect errors. +log4j.logger.org.apache.thrift.server.TNonblockingServer=ERROR + Modified: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java?rev=1152238&r1=1152237&r2=1152238&view=diff ===
[jira] [Updated] (CASSANDRA-47) SSTable compression
[ https://issues.apache.org/jira/browse/CASSANDRA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-47: -- Attachment: CASSANDRA-47-v5.patch I agree with Jonathan on the "let's make the compression option even more like compaction strategies". However, it may be simpler to commit this with the boolean flag and change it in another ticket, when we make the compaction algorithm "pluggable". I'm willing to take the responsibility to make sure that second ticket happens within a few days so that no-one really "sees" the intermediate state where compression is a boolean flag. Attaching a v5 that squashes v4 and v4-fixes and made the following changes/additions: * Bump thrift version * Fix a few places in CFMetaData where compression wasn't handled * Fix resetAndTruncate(), both in CSW and SW (in SW, the "reset in current buffer" optimization was broken, and for CSW, bufferOffset and chunkCount weren't reseted correctly, truncation wasn't done at all and the chunk index wasn't reseted nor truncated). Also adds a unit test for resetAndTruncate for both SW and CSW and for both the "reset in current buffer" optimization and without. * Slightly clean up CompressionMetadata: we now only keep the dataLengthOffset and write both this and the chunk count at the same time at the end. * Fix a problem with the handling of non-compressed sstable in cf where compression is activated. Previous code was expecting the sstables to be compressed as soon as the compression flag was set on the CF. Instead, the compression option is only used during writes to determine if the resulting sstable is compressed. When opening a sstable however, the presence of CompressionInfo is what is used to detect if it is compressed or not. * Add a new build target 'ant test-compression' that runs the unit tests with compression turned on for all CF. This is arguably a bit lame, and we should probably do better, but in the meantime I think it's quick and useful. * Add support for compression in stress (-I flag (supposed to mean "Inflate" since c, C and i were all taken)) The changes above are fairly simple or cosmetic (except for resetAndTruncate maybe but there the unit test now), and I think the patch is in a pretty good state so this has my +1 (great work Pavel!). I'll still leave a bit of time for someone else to check those last changes before committing. > SSTable compression > --- > > Key: CASSANDRA-47 > URL: https://issues.apache.org/jira/browse/CASSANDRA-47 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich > Labels: compression > Fix For: 1.0 > > Attachments: CASSANDRA-47-v2.patch, CASSANDRA-47-v3-rebased.patch, > CASSANDRA-47-v3.patch, CASSANDRA-47-v4-fixes.patch, CASSANDRA-47-v4.patch, > CASSANDRA-47-v5.patch, CASSANDRA-47.patch, snappy-java-1.0.3-rc4.jar > > > We should be able to do SSTable compression which would trade CPU for I/O > (almost always a good trade). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1152241 - in /cassandra/branches/cassandra-0.8: bin/cassandra conf/cassandra-env.sh debian/cassandra.postinst debian/init
Author: eevans
Date: Fri Jul 29 14:38:37 2011
New Revision: 1152241
URL: http://svn.apache.org/viewvc?rev=1152241&view=rev
Log:
honor path to java when JAVA_HOME set
Patch by Paul Cannon; reviewed by eevans for CASSANDRA-2785
Modified:
cassandra/branches/cassandra-0.8/bin/cassandra
cassandra/branches/cassandra-0.8/conf/cassandra-env.sh
cassandra/branches/cassandra-0.8/debian/cassandra.postinst
cassandra/branches/cassandra-0.8/debian/init
Modified: cassandra/branches/cassandra-0.8/bin/cassandra
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/bin/cassandra?rev=1152241&r1=1152240&r2=1152241&view=diff
==
--- cassandra/branches/cassandra-0.8/bin/cassandra (original)
+++ cassandra/branches/cassandra-0.8/bin/cassandra Fri Jul 29 14:38:37 2011
@@ -80,10 +80,10 @@ elif [ -r $CASSANDRA_INCLUDE ]; then
fi
# Use JAVA_HOME if set, otherwise look for java in PATH
-if [ -x $JAVA_HOME/bin/java ]; then
+if [ -n "$JAVA_HOME" ]; then
JAVA=$JAVA_HOME/bin/java
else
-JAVA=`which java`
+JAVA=java
fi
if [ -z $CASSANDRA_CONF -o -z $CLASSPATH ]; then
Modified: cassandra/branches/cassandra-0.8/conf/cassandra-env.sh
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/conf/cassandra-env.sh?rev=1152241&r1=1152240&r2=1152241&view=diff
==
--- cassandra/branches/cassandra-0.8/conf/cassandra-env.sh (original)
+++ cassandra/branches/cassandra-0.8/conf/cassandra-env.sh Fri Jul 29 14:38:37
2011
@@ -92,7 +92,7 @@ JMX_PORT="7199"
JVM_OPTS="$JVM_OPTS -ea"
# add the jamm javaagent
-check_openjdk=$(java -version 2>&1 | awk '{if (NR == 2) {print $1}}')
+check_openjdk=$("${JAVA:-java}" -version 2>&1 | awk '{if (NR == 2) {print
$1}}')
if [ "$check_openjdk" != "OpenJDK" ]
then
JVM_OPTS="$JVM_OPTS -javaagent:$CASSANDRA_HOME/lib/jamm-0.2.2.jar"
Modified: cassandra/branches/cassandra-0.8/debian/cassandra.postinst
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/debian/cassandra.postinst?rev=1152241&r1=1152240&r2=1152241&view=diff
==
--- cassandra/branches/cassandra-0.8/debian/cassandra.postinst (original)
+++ cassandra/branches/cassandra-0.8/debian/cassandra.postinst Fri Jul 29
14:38:37 2011
@@ -34,7 +34,7 @@ case "$1" in
cassandra
fi
-if [ -n $2 ] && dpkg --compare-versions "$2" le 0.6.4-2; then
+if [ -n "$2" ] && dpkg --compare-versions "$2" le 0.6.4-2; then
chown -R cassandra: /var/lib/cassandra
chown -R cassandra: /var/log/cassandra
fi
Modified: cassandra/branches/cassandra-0.8/debian/init
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/debian/init?rev=1152241&r1=1152240&r2=1152241&view=diff
==
--- cassandra/branches/cassandra-0.8/debian/init (original)
+++ cassandra/branches/cassandra-0.8/debian/init Fri Jul 29 14:38:37 2011
@@ -30,23 +30,15 @@ JVM_SEARCH_DIRS="/usr/lib/jvm/java-6-ope
[ -e /etc/cassandra/cassandra.yaml ] || exit 0
[ -e /etc/cassandra/cassandra-env.sh ] || exit 0
-# Read Cassandra environment file.
-. /etc/cassandra/cassandra-env.sh
-
# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME
-if [ -z "$JVM_OPTS" ]; then
-echo "Initialization failed; \$JVM_OPTS not set!" >&2
-exit 3
-fi
-
# If JAVA_HOME has not been set, try to determine it.
if [ -z "$JAVA_HOME" ]; then
# If java is in PATH, use a JAVA_HOME that corresponds to that. This is
# both consistent with how the upstream startup script works, and how
# Debian works (read: the use of alternatives to set a system JVM).
-if [ -n `which java` ]; then
+if [ -n "`which java`" ]; then
java=`which java`
# Dereference symlink(s)
while true; do
@@ -67,6 +59,15 @@ if [ -z "$JAVA_HOME" ]; then
done
fi
fi
+JAVA="$JAVA_HOME/bin/java"
+
+# Read Cassandra environment file.
+. /etc/cassandra/cassandra-env.sh
+
+if [ -z "$JVM_OPTS" ]; then
+echo "Initialization failed; \$JVM_OPTS not set!" >&2
+exit 3
+fi
# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh
[jira] [Commented] (CASSANDRA-2785) should export JAVA variable in the bin/cassandra and use that in the cassandra-env.sh when check for the java version
[ https://issues.apache.org/jira/browse/CASSANDRA-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072849#comment-13072849 ] Eric Evans commented on CASSANDRA-2785: --- committed; thanks! > should export JAVA variable in the bin/cassandra and use that in the > cassandra-env.sh when check for the java version > - > > Key: CASSANDRA-2785 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2785 > Project: Cassandra > Issue Type: Bug >Reporter: Jackson Chung >Assignee: paul cannon > Attachments: 0001-use-JAVA-in-cassandra-env.sh.patch.txt, > 0002-fix-usage-of-bash-n-tests.patch.txt > > > I forgot which jira we add this java -version check in the cassandra-env.sh > (for adding jamm to the javaagent), but we should probably use the variable > JAVA set in bin/cassandra (will need export) and use $JAVA instead of "java" > in the cassandra-env.sh > In a situation where JAVA_HOME may have been properly set as the Sun's java > but the PATH still have the OpenJDK's java in front, the check will fail to > add the jamm.jar, even though the cassandra jvm is properly started via the > Sun's java. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2785) should export JAVA variable in the bin/cassandra and use that in the cassandra-env.sh when check for the java version
[ https://issues.apache.org/jira/browse/CASSANDRA-2785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072864#comment-13072864 ] Hudson commented on CASSANDRA-2785: --- Integrated in Cassandra-0.8 #243 (See [https://builds.apache.org/job/Cassandra-0.8/243/]) honor path to java when JAVA_HOME set Patch by Paul Cannon; reviewed by eevans for CASSANDRA-2785 eevans : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152241 Files : * /cassandra/branches/cassandra-0.8/conf/cassandra-env.sh * /cassandra/branches/cassandra-0.8/debian/cassandra.postinst * /cassandra/branches/cassandra-0.8/debian/init * /cassandra/branches/cassandra-0.8/bin/cassandra > should export JAVA variable in the bin/cassandra and use that in the > cassandra-env.sh when check for the java version > - > > Key: CASSANDRA-2785 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2785 > Project: Cassandra > Issue Type: Bug >Reporter: Jackson Chung >Assignee: paul cannon > Attachments: 0001-use-JAVA-in-cassandra-env.sh.patch.txt, > 0002-fix-usage-of-bash-n-tests.patch.txt > > > I forgot which jira we add this java -version check in the cassandra-env.sh > (for adding jamm to the javaagent), but we should probably use the variable > JAVA set in bin/cassandra (will need export) and use $JAVA instead of "java" > in the cassandra-env.sh > In a situation where JAVA_HOME may have been properly set as the Sun's java > but the PATH still have the OpenJDK's java in front, the check will fail to > add the jamm.jar, even though the cassandra jvm is properly started via the > Sun's java. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1405) Switch to THsHaServer, redux
[ https://issues.apache.org/jira/browse/CASSANDRA-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072865#comment-13072865 ] Hudson commented on CASSANDRA-1405: --- Integrated in Cassandra-0.8 #243 (See [https://builds.apache.org/job/Cassandra-0.8/243/]) Add asynchronous and half-sync/half-async thrift servers. Patch by Vijay Parthasarathy, reviewed by brandonwilliams for CASSANDRA-1405 brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152238 Files : * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java * /cassandra/branches/cassandra-0.8/conf/log4j-server.properties * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/TCustomNonblockingServerSocket.java * /cassandra/branches/cassandra-0.8/conf/cassandra.yaml * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CustomTNonBlockingServer.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/SocketSessionManagementService.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraDaemon.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CassandraServer.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/CustomTHsHaServer.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/Config.java > Switch to THsHaServer, redux > > > Key: CASSANDRA-1405 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1405 > Project: Cassandra > Issue Type: Improvement > Components: API >Reporter: Jonathan Ellis >Assignee: Vijay >Priority: Minor > Fix For: 0.8.3 > > Attachments: 0001-1405-Add-Thrift-NonBlocking-types.patch, > 0001-including-validation.patch, 0001-log4j-config-change.patch, > 0002-commiting-typo-in-DTPE.patch, 1405-Thrift-Patch-SVN.patch, > libthrift-r1026391.jar, trunk-1405.patch > > > Brian's patch to CASSANDRA-876 suggested using a custom TProcessorFactory > subclass, overriding getProcessor to reset to a default state when a new > client connects. It looks like this would allow dropping > CustomTThreadPoolServer as well as allowing non-thread based servers. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1152265 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/utils/ByteBufferUtil.java src/java/org/apache/cassandra/utils/FBUtilities.java test/unit/org/apac
Author: slebresne
Date: Fri Jul 29 15:33:19 2011
New Revision: 1152265
URL: http://svn.apache.org/viewvc?rev=1152265&view=rev
Log:
Speedup bytes to hex conversions dramatically
patch by dallsopp; reviewed by slebresne for CASSANDRA-2850
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/utils/FBUtilities.java
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/utils/ByteBufferUtilTest.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1152265&r1=1152264&r2=1152265&view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Jul 29 15:33:19 2011
@@ -12,6 +12,7 @@
* use lazy initialization instead of class initialization in NodeId
(CASSANDRA-2953)
* check column family validity in nodetool repair (CASSANDRA-2933)
+ * speedup bytes to hex conversions dramatically (CASSANDRA-2850)
0.8.2
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/utils/ByteBufferUtil.java?rev=1152265&r1=1152264&r2=1152265&view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
Fri Jul 29 15:33:19 2011
@@ -445,7 +445,6 @@ public class ByteBufferUtil
return ByteBuffer.allocate(8).putDouble(0, d);
}
-
public static InputStream inputStream(ByteBuffer bytes)
{
final ByteBuffer copy = bytes.duplicate();
@@ -481,16 +480,16 @@ public class ByteBufferUtil
public static String bytesToHex(ByteBuffer bytes)
{
-StringBuilder sb = new StringBuilder();
-for (int i = bytes.position(); i < bytes.limit(); i++)
+final int offset = bytes.position();
+final int size = bytes.remaining();
+final char[] c = new char[size * 2];
+for (int i = 0; i < size; i++)
{
-int bint = bytes.get(i) & 0xff;
-if (bint <= 0xF)
-// toHexString does not 0 pad its results.
-sb.append("0");
-sb.append(Integer.toHexString(bint));
+final int bint = bytes.get(i+offset);
+c[i * 2] = FBUtilities.byteToChar[(bint & 0xf0) >> 4];
+c[1 + i * 2] = FBUtilities.byteToChar[bint & 0x0f];
}
-return sb.toString();
+return FBUtilities.wrapCharArray(c);
}
public static ByteBuffer hexToBytes(String str)
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/utils/FBUtilities.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/utils/FBUtilities.java?rev=1152265&r1=1152264&r2=1152265&view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/utils/FBUtilities.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/utils/FBUtilities.java
Fri Jul 29 15:33:19 2011
@@ -19,6 +19,7 @@
package org.apache.cassandra.utils;
import java.io.*;
+import java.lang.reflect.Constructor;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.math.BigInteger;
@@ -62,6 +63,59 @@ public class FBUtilities
private static volatile InetAddress localInetAddress_;
+private final static byte[] charToByte = new byte[256];
+// package protected for use by ByteBufferUtil. Do not modify this array !!
+static final char[] byteToChar = new char[16];
+static
+{
+for (char c = 0; c < charToByte.length; ++c)
+{
+if (c >= '0' && c <= '9')
+charToByte[c] = (byte)(c - '0');
+else if (c >= 'A' && c <= 'F')
+charToByte[c] = (byte)(c - 'A' + 10);
+else if (c >= 'a' && c <= 'f')
+charToByte[c] = (byte)(c - 'a' + 10);
+else
+charToByte[c] = (byte)-1;
+}
+
+for (int i = 0; i < 16; ++i)
+{
+byteToChar[i] = Integer.toHexString(i).charAt(0);
+}
+}
+
+/**
+ * This constructor enables us to construct a String directly by wrapping
a char array, with zero-copy.
+ * This can save time, and a lot of memory, when converting large column
values.
+ */
+private static final Constructor stringConstructor =
getProtectedConstructor(String.class, int.cla
[jira] [Commented] (CASSANDRA-1405) Switch to THsHaServer, redux
[ https://issues.apache.org/jira/browse/CASSANDRA-1405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072868#comment-13072868 ] Hudson commented on CASSANDRA-1405: --- Integrated in Cassandra #985 (See [https://builds.apache.org/job/Cassandra/985/]) Add asynchronous and half-sync/half-async thrift servers. Patch by Vijay Parthasarathy, reviewed by brandonwilliams for CASSANDRA-1405 brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152233 Files : * /cassandra/trunk/src/gen-java/org/apache/cassandra/cli/Cli.tokens * /cassandra/trunk/src/java/org/apache/cassandra/thrift/CustomTNonBlockingServer.java * /cassandra/trunk/src/gen-java/org/apache/cassandra/cql/Cql.tokens * /cassandra/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java * /cassandra/trunk/src/gen-java/org/apache/cassandra * /cassandra/trunk/conf/cassandra.yaml * /cassandra/trunk/src/gen-java/org/apache/cassandra/cql/CqlLexer.java * /cassandra/trunk/src/gen-java/org/apache/cassandra/cli/CliParser.java * /cassandra/trunk/src/gen-java/org/apache/cassandra/cli/CliLexer.java * /cassandra/trunk/src/java/org/apache/cassandra/config/Config.java * /cassandra/trunk/src/gen-java/org/apache/cassandra/cql/CqlParser.java * /cassandra/trunk/src/gen-java/org/apache * /cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraDaemon.java * /cassandra/trunk/src/java/org/apache/cassandra/thrift/CassandraServer.java * /cassandra/trunk/src/gen-java/org * /cassandra/trunk/src/java/org/apache/cassandra/thrift/CustomTHsHaServer.java * /cassandra/trunk/conf/log4j-server.properties * /cassandra/trunk/src/java/org/apache/cassandra/thrift/TCustomNonblockingServerSocket.java * /cassandra/trunk/src/gen-java/org/apache/cassandra/cql * /cassandra/trunk/src/gen-java/org/apache/cassandra/cli * /cassandra/trunk/src/java/org/apache/cassandra/service/SocketSessionManagementService.java * /cassandra/trunk/src/gen-java * /cassandra/trunk/src/java/org/apache/cassandra/concurrent/JMXEnabledThreadPoolExecutor.java > Switch to THsHaServer, redux > > > Key: CASSANDRA-1405 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1405 > Project: Cassandra > Issue Type: Improvement > Components: API >Reporter: Jonathan Ellis >Assignee: Vijay >Priority: Minor > Fix For: 0.8.3 > > Attachments: 0001-1405-Add-Thrift-NonBlocking-types.patch, > 0001-including-validation.patch, 0001-log4j-config-change.patch, > 0002-commiting-typo-in-DTPE.patch, 1405-Thrift-Patch-SVN.patch, > libthrift-r1026391.jar, trunk-1405.patch > > > Brian's patch to CASSANDRA-876 suggested using a custom TProcessorFactory > subclass, overriding getProcessor to reset to a default state when a new > client connects. It looks like this would allow dropping > CustomTThreadPoolServer as well as allowing non-thread based servers. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2850) Converting bytes to hex string is unnecessarily slow
[ https://issues.apache.org/jira/browse/CASSANDRA-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-2850: Attachment: 2850-v6_08.patch +1, committed, thanks. I've slightly updated the patch so that we use a copying String constructor in case using the package-protected one fails. Since it's not part of the public API, it avoids potential bug with some JDK that wouldn't have this constructor. > Converting bytes to hex string is unnecessarily slow > > > Key: CASSANDRA-2850 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2850 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: David Allsopp >Assignee: David Allsopp >Priority: Minor > Fix For: 0.8.3 > > Attachments: 2850-rebased.txt, 2850-v2.patch, 2850-v4.patch, > 2850-v4a.patch, 2850-v5.patch, 2850-v6_08.patch, BytesToHexBenchmark.java, > BytesToHexBenchmark2.java, BytesToHexBenchmark3.java, cassandra-2850a.diff > > > ByteBufferUtil.bytesToHex() is unnecessarily slow - it doesn't pre-size the > StringBuilder (so several re-sizes will be needed behind the scenes) and it > makes quite a few method calls per byte. > (OK, this may be a premature optimisation, but I couldn't resist, and it's a > small change) > Will attach patch shortly that speeds it up by about x3, plus benchmarking > test. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2850) Converting bytes to hex string is unnecessarily slow
[ https://issues.apache.org/jira/browse/CASSANDRA-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072867#comment-13072867 ] Sylvain Lebresne edited comment on CASSANDRA-2850 at 7/29/11 3:37 PM: -- +1, committed, thanks. I've slightly updated the patch so that we use a copying String constructor in case using the package-protected one fails. Since it's not part of the public API, it avoids potential bug with some JDK that wouldn't have this constructor (I'm attaching the committed patch for the record). was (Author: slebresne): +1, committed, thanks. I've slightly updated the patch so that we use a copying String constructor in case using the package-protected one fails. Since it's not part of the public API, it avoids potential bug with some JDK that wouldn't have this constructor. > Converting bytes to hex string is unnecessarily slow > > > Key: CASSANDRA-2850 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2850 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: David Allsopp >Assignee: David Allsopp >Priority: Minor > Fix For: 0.8.3 > > Attachments: 2850-rebased.txt, 2850-v2.patch, 2850-v4.patch, > 2850-v4a.patch, 2850-v5.patch, 2850-v6_08.patch, BytesToHexBenchmark.java, > BytesToHexBenchmark2.java, BytesToHexBenchmark3.java, cassandra-2850a.diff > > > ByteBufferUtil.bytesToHex() is unnecessarily slow - it doesn't pre-size the > StringBuilder (so several re-sizes will be needed behind the scenes) and it > makes quite a few method calls per byte. > (OK, this may be a premature optimisation, but I couldn't resist, and it's a > small change) > Will attach patch shortly that speeds it up by about x3, plus benchmarking > test. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2965) Allow cassandra to start on a Solaris machine.
[ https://issues.apache.org/jira/browse/CASSANDRA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul cannon updated CASSANDRA-2965: --- Attachment: 0002-improve-general-sh-compatibility.patch.txt 0001-Solaris-bin-sh-compatibility.patch.txt 0001: just Kjell's original patch, rebased after #2785 0002: several other places where it would be a good idea for general compatibility or safety to quote shell variables I don't have access to a Solaris machine to actually test, but this works under dash, and I'm pretty confident it's good. Kjell, would you mind giving it a few extra tests on actual Solaris? > Allow cassandra to start on a Solaris machine. > -- > > Key: CASSANDRA-2965 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2965 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Affects Versions: 0.8.2 > Environment: Solaris 10/SunOS 5.10, x86 architecture. >Reporter: Kjell Andreassen >Assignee: Kjell Andreassen >Priority: Trivial > Fix For: 0.8.3 > > Attachments: 0001-Solaris-bin-sh-compatibility.patch.txt, > 0002-improve-general-sh-compatibility.patch.txt, cassandra-0.8.2-2965.txt > > > Cassandra ($CASSANDRA_HOME/bin/cassandra) fails to start with a series of > errors, fixing one reveals the next. > These are the errors: > bin/cassandra: syntax error at line 27: `system_memory_in_mb=$' unexpected > bin/cassandra: syntax error at line 100: `check_openjdk=$' unexpected > bin/cassandra: test: argument expected -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2850) Converting bytes to hex string is unnecessarily slow
[ https://issues.apache.org/jira/browse/CASSANDRA-2850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072895#comment-13072895 ] Hudson commented on CASSANDRA-2850: --- Integrated in Cassandra-0.8 #244 (See [https://builds.apache.org/job/Cassandra-0.8/244/]) Speedup bytes to hex conversions dramatically patch by dallsopp; reviewed by slebresne for CASSANDRA-2850 slebresne : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152265 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/utils/ByteBufferUtil.java * /cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/utils/ByteBufferUtilTest.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/utils/FBUtilities.java > Converting bytes to hex string is unnecessarily slow > > > Key: CASSANDRA-2850 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2850 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: David Allsopp >Assignee: David Allsopp >Priority: Minor > Fix For: 0.8.3 > > Attachments: 2850-rebased.txt, 2850-v2.patch, 2850-v4.patch, > 2850-v4a.patch, 2850-v5.patch, 2850-v6_08.patch, BytesToHexBenchmark.java, > BytesToHexBenchmark2.java, BytesToHexBenchmark3.java, cassandra-2850a.diff > > > ByteBufferUtil.bytesToHex() is unnecessarily slow - it doesn't pre-size the > StringBuilder (so several re-sizes will be needed behind the scenes) and it > makes quite a few method calls per byte. > (OK, this may be a premature optimisation, but I couldn't resist, and it's a > small change) > Will attach patch shortly that speeds it up by about x3, plus benchmarking > test. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2898) Escape characters in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072915#comment-13072915
]
Brandon Williams commented on CASSANDRA-2898:
-
I don't see the problem:
{noformat}
cqlsh> create columnfamily foo (KEY text primary key);
cqlsh> update foo set 'fmd:' = 'foo' where key = 'test';
cqlsh> select * from foo where key = 'test';
KEY | fmd: |
test | foo |
cqlsh>
cqlsh> select 'fmd:' from foo where key = 'test';
fmd: |
foo |
cqlsh> update foo set 'fmd;' = 'foo' where key = 'test';
cqlsh> select 'fmd:'..'fmd;' from foo where key = 'test';
fmd: | fmd; |
foo | foo |
cqlsh>
{noformat}
> Escape characters in CQL
>
>
> Key: CASSANDRA-2898
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2898
> Project: Cassandra
> Issue Type: Bug
> Components: Drivers
>Reporter: Blake Visin
>Assignee: Brandon Williams
> Labels: cql
>
> When trying to get all the columns named "fmd:" in cqlsh you can not escape :
> or ;
> As per Jonathan Ellis:
> You can escape quotes but I don't think you can escape semicolons.
> Try:
> sqlsh> select 'fmd:'..'fmd;' from feeds;
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1152303 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/utils/ test/unit/org/apache/cassandra/utils/
Author: slebresne Date: Fri Jul 29 17:02:38 2011 New Revision: 1152303 URL: http://svn.apache.org/viewvc?rev=1152303&view=rev Log: merge from 0.8 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/utils/ByteBufferUtil.java cassandra/trunk/src/java/org/apache/cassandra/utils/FBUtilities.java cassandra/trunk/test/unit/org/apache/cassandra/utils/ByteBufferUtilTest.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jul 29 17:02:38 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7:1026516-1151306 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1152110 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1152110,1152265 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1152303&r1=1152302&r2=1152303&view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Fri Jul 29 17:02:38 2011 @@ -37,6 +37,7 @@ * log Java classpath on startup (CASSANDRA-2895) * keep gossipped version in sync with actual on migration coordinator (CASSANDRA-2946) + * speedup bytes to hex conversions dramatically (CASSANDRA-2850) 0.8.2 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jul 29 17:02:38 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 /cassandra/branches/cassandra-0.7/contrib:1026516-1151306 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1152110 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1152110,1152265 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jul 29 17:02:38 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1151306 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1152110 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1152110,1152265 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Jul 29 17:02:38 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1151306 /cassandra/branches/cassandra-0.7.0/interfac
[jira] [Commented] (CASSANDRA-2965) Allow cassandra to start on a Solaris machine.
[ https://issues.apache.org/jira/browse/CASSANDRA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072929#comment-13072929 ] Kjell Andreassen commented on CASSANDRA-2965: - It works beautifully. Cassandra started without warnings/errors and is talking to the other nodes on a Solaris (uname -a: SunOS 5.10 Generic_142901-11 i86pc i386 i86pc) after applying patches 1 and 2 from #2785 and patches 1 and 2 from this issue (#2965) on top of cassandra-0.8.2 (207f0a). > Allow cassandra to start on a Solaris machine. > -- > > Key: CASSANDRA-2965 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2965 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Affects Versions: 0.8.2 > Environment: Solaris 10/SunOS 5.10, x86 architecture. >Reporter: Kjell Andreassen >Assignee: Kjell Andreassen >Priority: Trivial > Fix For: 0.8.3 > > Attachments: 0001-Solaris-bin-sh-compatibility.patch.txt, > 0002-improve-general-sh-compatibility.patch.txt, cassandra-0.8.2-2965.txt > > > Cassandra ($CASSANDRA_HOME/bin/cassandra) fails to start with a series of > errors, fixing one reveals the next. > These are the errors: > bin/cassandra: syntax error at line 27: `system_memory_in_mb=$' unexpected > bin/cassandra: syntax error at line 100: `check_openjdk=$' unexpected > bin/cassandra: test: argument expected -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2965) Allow cassandra to start on a Solaris machine.
[ https://issues.apache.org/jira/browse/CASSANDRA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul cannon reassigned CASSANDRA-2965: -- Assignee: Eric Evans (was: Kjell Andreassen) +1 for 0001, 0002 then. > Allow cassandra to start on a Solaris machine. > -- > > Key: CASSANDRA-2965 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2965 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Affects Versions: 0.8.2 > Environment: Solaris 10/SunOS 5.10, x86 architecture. >Reporter: Kjell Andreassen >Assignee: Eric Evans >Priority: Trivial > Fix For: 0.8.3 > > Attachments: 0001-Solaris-bin-sh-compatibility.patch.txt, > 0002-improve-general-sh-compatibility.patch.txt, cassandra-0.8.2-2965.txt > > > Cassandra ($CASSANDRA_HOME/bin/cassandra) fails to start with a series of > errors, fixing one reveals the next. > These are the errors: > bin/cassandra: syntax error at line 27: `system_memory_in_mb=$' unexpected > bin/cassandra: syntax error at line 100: `check_openjdk=$' unexpected > bin/cassandra: test: argument expected -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2969) Queries and inserts fail on 2 node cluster when replication factor is specified.
Queries and inserts fail on 2 node cluster when replication factor is specified. Key: CASSANDRA-2969 URL: https://issues.apache.org/jira/browse/CASSANDRA-2969 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.2 Environment: 2 node Cassandra cluster with both nodes up and running. The replication factor is specified and can be 1 or 2. Client is using Hector. For production, I want to use a RF of 2 so the data is stored on both nodes and use a CL of ONE for reads and writes. Reporter: Andy Klages When a query or insert is performed, I get the following exception: me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be enough replicas present to handle consistency level. I've tried consistency levels ANY, ONE, and QUORUM, but to no avail. I'm assuming it would do this for the other levels too. Running the "get" and "set" commands on cassandra-cli always return "null" so it's encountering the same problem. Note that I don't get this problem when the keyspace doesn't have replication_factor specified. But I need the data replicated on both nodes which is why I'm specifying the RF. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2819) Split rpc timeout for read and write ops
[ https://issues.apache.org/jira/browse/CASSANDRA-2819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Melvin Wang updated CASSANDRA-2819: --- Attachment: (was: twttr-cassandra-0.8-counts-resync-rpc-rw-timeouts.diff) > Split rpc timeout for read and write ops > > > Key: CASSANDRA-2819 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2819 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Stu Hood >Assignee: Melvin Wang > Fix For: 1.0 > > > Given the vastly different latency characteristics of reads and writes, it > makes sense for them to have independent rpc timeouts internally. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2819) Split rpc timeout for read and write ops
[ https://issues.apache.org/jira/browse/CASSANDRA-2819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Melvin Wang updated CASSANDRA-2819: --- Attachment: twttr-cassandra-0.8-counts-resync-rpc-rw-timeouts.patch > Split rpc timeout for read and write ops > > > Key: CASSANDRA-2819 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2819 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Stu Hood >Assignee: Melvin Wang > Fix For: 1.0 > > Attachments: twttr-cassandra-0.8-counts-resync-rpc-rw-timeouts.patch > > > Given the vastly different latency characteristics of reads and writes, it > makes sense for them to have independent rpc timeouts internally. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2819) Split rpc timeout for read and write ops
[ https://issues.apache.org/jira/browse/CASSANDRA-2819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072953#comment-13072953 ] Melvin Wang commented on CASSANDRA-2819: Sorry for the waiting. I reverted the changes I made to Message.java and MessageDeliveryTask.java while keep the changes elsewhere. This is based on the previous discussions where we a separate ticket to make the timeout more accurate. > Split rpc timeout for read and write ops > > > Key: CASSANDRA-2819 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2819 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Stu Hood >Assignee: Melvin Wang > Fix For: 1.0 > > Attachments: twttr-cassandra-0.8-counts-resync-rpc-rw-timeouts.patch > > > Given the vastly different latency characteristics of reads and writes, it > makes sense for them to have independent rpc timeouts internally. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2825) Auto bootstrapping the 4th node in a 4 node cluster doesn't work, when no token explicitly assigned in config.
[ https://issues.apache.org/jira/browse/CASSANDRA-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-2825: Attachment: 2825-v3.txt v3 adds a check that tokenMetadata_.getEndpoint(token) is not null when checking for existing membership. This fixes the test. > Auto bootstrapping the 4th node in a 4 node cluster doesn't work, when no > token explicitly assigned in config. > -- > > Key: CASSANDRA-2825 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2825 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.8.0, 0.8.1 >Reporter: Michael Allen >Assignee: Brandon Williams > Fix For: 0.8.3 > > Attachments: 2825-v2.txt, 2825-v3.txt, 2825.txt > > > This was done in sequence. A, B, C, and D. Node A with token 0 explicitly > set in config. The rest with auto_bootstrap: true and no token explicitly > assigned. B and C work as expected. D ends up stealing C's token. > from system.log on C: > INFO [GossipStage:1] 2011-06-24 16:40:41,947 Gossiper.java (line 638) Node > /10.171.47.226 is now part of the cluster > INFO [GossipStage:1] 2011-06-24 16:40:41,947 Gossiper.java (line 606) > InetAddress /10.171.47.226 is now UP > INFO [GossipStage:1] 2011-06-24 16:42:09,432 StorageService.java (line 769) > Nodes /10.171.47.226 and /10.171.55.77 have the same token > 61078635599166706937511052402724559481. /10.171.47.226 is the new owner > WARN [GossipStage:1] 2011-06-24 16:42:09,432 TokenMetadata.java (line 120) > Token 61078635599166706937511052402724559481 changing ownership from > /10.171.55.77 to /10.171.47.226 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2858) make request dropping more accurate
[ https://issues.apache.org/jira/browse/CASSANDRA-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072959#comment-13072959 ] Melvin Wang commented on CASSANDRA-2858: Regarding to the first bullet point, why not introducing the creation time in the message header, so that whenever we need to know the remaining time we could just compare the current time with it? > make request dropping more accurate > --- > > Key: CASSANDRA-2858 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2858 > Project: Cassandra > Issue Type: Improvement >Reporter: Ryan King >Assignee: Melvin Wang >Priority: Minor > > Based on the discussion in > https://issues.apache.org/jira/browse/CASSANDRA-2819, we can make the > bookkeeping for request times more accurate. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2858) make request dropping more accurate
[ https://issues.apache.org/jira/browse/CASSANDRA-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072960#comment-13072960 ] Melvin Wang commented on CASSANDRA-2858: Regarding to the first bullet point, why not introducing the creation time in the message header, so that whenever we need to know the remaining time we could just compare the current time with it? > make request dropping more accurate > --- > > Key: CASSANDRA-2858 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2858 > Project: Cassandra > Issue Type: Improvement >Reporter: Ryan King >Assignee: Melvin Wang >Priority: Minor > > Based on the discussion in > https://issues.apache.org/jira/browse/CASSANDRA-2819, we can make the > bookkeeping for request times more accurate. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2858) make request dropping more accurate
[ https://issues.apache.org/jira/browse/CASSANDRA-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Melvin Wang updated CASSANDRA-2858: --- Comment: was deleted (was: Regarding to the first bullet point, why not introducing the creation time in the message header, so that whenever we need to know the remaining time we could just compare the current time with it?) > make request dropping more accurate > --- > > Key: CASSANDRA-2858 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2858 > Project: Cassandra > Issue Type: Improvement >Reporter: Ryan King >Assignee: Melvin Wang >Priority: Minor > > Based on the discussion in > https://issues.apache.org/jira/browse/CASSANDRA-2819, we can make the > bookkeeping for request times more accurate. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2858) make request dropping more accurate
[ https://issues.apache.org/jira/browse/CASSANDRA-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072967#comment-13072967 ] Melvin Wang commented on CASSANDRA-2858: For 3rd bullet, would it be good enough to add a 'cancel' method to IAsyncCallback so that we it got expired in ExpiringMap, cancel will be called which will unblock the get() method and possibly throwing TimeoutException? > make request dropping more accurate > --- > > Key: CASSANDRA-2858 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2858 > Project: Cassandra > Issue Type: Improvement >Reporter: Ryan King >Assignee: Melvin Wang >Priority: Minor > > Based on the discussion in > https://issues.apache.org/jira/browse/CASSANDRA-2819, we can make the > bookkeeping for request times more accurate. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2868) Native Memory Leak
[ https://issues.apache.org/jira/browse/CASSANDRA-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073006#comment-13073006 ] Jeremiah Jordan commented on CASSANDRA-2868: Depending how long the rewrite is going to take, can we get the config file option to disable gc inspector into a new 0.7.X and 0.8.X release? > Native Memory Leak > -- > > Key: CASSANDRA-2868 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2868 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Daniel Doubleday >Assignee: Brandon Williams >Priority: Minor > Fix For: 0.8.3 > > Attachments: 2868-v1.txt, 48hour_RES.png, > low-load-36-hours-initial-results.png > > > We have memory issues with long running servers. These have been confirmed by > several users in the user list. That's why I report. > The memory consumption of the cassandra java process increases steadily until > it's killed by the os because of oom (with no swap) > Our server is started with -Xmx3000M and running for around 23 days. > pmap -x shows > Total SST: 1961616 (mem mapped data and index files) > Anon RSS: 6499640 > Total RSS: 8478376 > This shows that > 3G are 'overallocated'. > We will use BRAF on one of our less important nodes to check wether it is > related to mmap and report back. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2868) Native Memory Leak
[ https://issues.apache.org/jira/browse/CASSANDRA-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-2868: Attachment: 2868-v2.txt v2 switches the GCInspector to us java.lang.managment. I don't know if it too leaks or not yet. > Native Memory Leak > -- > > Key: CASSANDRA-2868 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2868 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Daniel Doubleday >Assignee: Brandon Williams >Priority: Minor > Fix For: 0.8.3 > > Attachments: 2868-v1.txt, 2868-v2.txt, 48hour_RES.png, > low-load-36-hours-initial-results.png > > > We have memory issues with long running servers. These have been confirmed by > several users in the user list. That's why I report. > The memory consumption of the cassandra java process increases steadily until > it's killed by the os because of oom (with no swap) > Our server is started with -Xmx3000M and running for around 23 days. > pmap -x shows > Total SST: 1961616 (mem mapped data and index files) > Anon RSS: 6499640 > Total RSS: 8478376 > This shows that > 3G are 'overallocated'. > We will use BRAF on one of our less important nodes to check wether it is > related to mmap and report back. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2969) Queries and inserts fail on 2 node cluster when replication factor is specified.
[
https://issues.apache.org/jira/browse/CASSANDRA-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Klages resolved CASSANDRA-2969.
Resolution: Not A Problem
I specified the replication factor in the cassandra-cli incorrectly. The
Keyspace was set to NetworkTopologyStrategy and I didn't use the correct format
for specifying RF. I did
update keyspace xyz with strategy_options=[{replication_factor:2}]
instead of
update ... strategy_options=[{datacenter1:2}]
> Queries and inserts fail on 2 node cluster when replication factor is
> specified.
>
>
> Key: CASSANDRA-2969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2969
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.8.2
> Environment: 2 node Cassandra cluster with both nodes up and running.
> The replication factor is specified and can be 1 or 2. Client is using
> Hector. For production, I want to use a RF of 2 so the data is stored on both
> nodes and use a CL of ONE for reads and writes.
>Reporter: Andy Klages
> Labels: 2-node, replica
>
> When a query or insert is performed, I get the following exception:
> me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be
> enough replicas present to handle consistency level.
> I've tried consistency levels ANY, ONE, and QUORUM, but to no avail. I'm
> assuming it would do this for the other levels too.
> Running the "get" and "set" commands on cassandra-cli always return "null" so
> it's encountering the same problem.
> Note that I don't get this problem when the keyspace doesn't have
> replication_factor specified. But I need the data replicated on both nodes
> which is why I'm specifying the RF.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1788) reduce copies on read, write paths
[ https://issues.apache.org/jira/browse/CASSANDRA-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-1788: Attachment: 1788-v7.txt v7 is rebased to apply to current trunk. It magically has no problems now, but I didn't change anything. > reduce copies on read, write paths > -- > > Key: CASSANDRA-1788 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1788 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Fix For: 1.0 > > Attachments: 0001-setup.txt, > 0002-remove-copies-from-network-path.txt, 1788-v2.txt, 1788-v3.txt, > 1788-v4.txt, 1788-v6.txt, 1788-v7.txt, 1788.txt > > Original Estimate: 24h > Remaining Estimate: 24h > > Currently, we do _three_ unnecessary copies (that is, writing to the socket > is necessary; any other copies made are overhead) for each message: > - constructing the Message body byte[] (this is typically a call to a > ICompactSerializer[2] serialize method, but sometimes we cheat e.g. in > SchemaCheckVerbHandler's reply) > - which is copied to a buffer containing the entire Message (i.e. including > Header) when sendOneWay calls Message.serializer.serialize() > - which is copied to a newly-allocated ByteBuffer when sendOneWay calls packIt > - which is what we write to the socket > For deserialize we perform a similar orgy of copies: > - IncomingTcpConnection reads the Message length, allocates a byte[], and > reads the serialized Message into it > - ITcpC then calls Message.serializer().deserialize, which allocates a new > byte[] for the body and copies that part > - finally, the verbHandler (determined by the now-deserialized Message > header) deserializes the actual object represented by the body > Most of these are out of scope for 0.7 but I think we can at least elide the > last copy on the write path and the first on the read. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1788) reduce copies on read, write paths
[ https://issues.apache.org/jira/browse/CASSANDRA-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-1788: Attachment: (was: 1788-v7.txt) > reduce copies on read, write paths > -- > > Key: CASSANDRA-1788 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1788 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Fix For: 1.0 > > Attachments: 0001-setup.txt, > 0002-remove-copies-from-network-path.txt, 1788-v2.txt, 1788-v3.txt, > 1788-v4.txt, 1788-v6.txt, 1788.txt > > Original Estimate: 24h > Remaining Estimate: 24h > > Currently, we do _three_ unnecessary copies (that is, writing to the socket > is necessary; any other copies made are overhead) for each message: > - constructing the Message body byte[] (this is typically a call to a > ICompactSerializer[2] serialize method, but sometimes we cheat e.g. in > SchemaCheckVerbHandler's reply) > - which is copied to a buffer containing the entire Message (i.e. including > Header) when sendOneWay calls Message.serializer.serialize() > - which is copied to a newly-allocated ByteBuffer when sendOneWay calls packIt > - which is what we write to the socket > For deserialize we perform a similar orgy of copies: > - IncomingTcpConnection reads the Message length, allocates a byte[], and > reads the serialized Message into it > - ITcpC then calls Message.serializer().deserialize, which allocates a new > byte[] for the body and copies that part > - finally, the verbHandler (determined by the now-deserialized Message > header) deserializes the actual object represented by the body > Most of these are out of scope for 0.7 but I think we can at least elide the > last copy on the write path and the first on the read. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1788) reduce copies on read, write paths
[ https://issues.apache.org/jira/browse/CASSANDRA-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-1788: Comment: was deleted (was: v7 is rebased to apply to current trunk. It magically has no problems now, but I didn't change anything.) > reduce copies on read, write paths > -- > > Key: CASSANDRA-1788 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1788 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Fix For: 1.0 > > Attachments: 0001-setup.txt, > 0002-remove-copies-from-network-path.txt, 1788-v2.txt, 1788-v3.txt, > 1788-v4.txt, 1788-v6.txt, 1788.txt > > Original Estimate: 24h > Remaining Estimate: 24h > > Currently, we do _three_ unnecessary copies (that is, writing to the socket > is necessary; any other copies made are overhead) for each message: > - constructing the Message body byte[] (this is typically a call to a > ICompactSerializer[2] serialize method, but sometimes we cheat e.g. in > SchemaCheckVerbHandler's reply) > - which is copied to a buffer containing the entire Message (i.e. including > Header) when sendOneWay calls Message.serializer.serialize() > - which is copied to a newly-allocated ByteBuffer when sendOneWay calls packIt > - which is what we write to the socket > For deserialize we perform a similar orgy of copies: > - IncomingTcpConnection reads the Message length, allocates a byte[], and > reads the serialized Message into it > - ITcpC then calls Message.serializer().deserialize, which allocates a new > byte[] for the body and copies that part > - finally, the verbHandler (determined by the now-deserialized Message > header) deserializes the actual object represented by the body > Most of these are out of scope for 0.7 but I think we can at least elide the > last copy on the write path and the first on the read. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1788) reduce copies on read, write paths
[ https://issues.apache.org/jira/browse/CASSANDRA-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-1788: Attachment: 1788-v7.txt v7 rebased for trunk again. Now mysteriously has no errors. My guess is that something in CASSANDRA-1405 fixed it. > reduce copies on read, write paths > -- > > Key: CASSANDRA-1788 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1788 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Fix For: 1.0 > > Attachments: 0001-setup.txt, > 0002-remove-copies-from-network-path.txt, 1788-v2.txt, 1788-v3.txt, > 1788-v4.txt, 1788-v6.txt, 1788-v7.txt, 1788.txt > > Original Estimate: 24h > Remaining Estimate: 24h > > Currently, we do _three_ unnecessary copies (that is, writing to the socket > is necessary; any other copies made are overhead) for each message: > - constructing the Message body byte[] (this is typically a call to a > ICompactSerializer[2] serialize method, but sometimes we cheat e.g. in > SchemaCheckVerbHandler's reply) > - which is copied to a buffer containing the entire Message (i.e. including > Header) when sendOneWay calls Message.serializer.serialize() > - which is copied to a newly-allocated ByteBuffer when sendOneWay calls packIt > - which is what we write to the socket > For deserialize we perform a similar orgy of copies: > - IncomingTcpConnection reads the Message length, allocates a byte[], and > reads the serialized Message into it > - ITcpC then calls Message.serializer().deserialize, which allocates a new > byte[] for the body and copies that part > - finally, the verbHandler (determined by the now-deserialized Message > header) deserializes the actual object represented by the body > Most of these are out of scope for 0.7 but I think we can at least elide the > last copy on the write path and the first on the read. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2970) Create separate read repair settings for intra- versus inter- datacenter
Create separate read repair settings for intra- versus inter- datacenter Key: CASSANDRA-2970 URL: https://issues.apache.org/jira/browse/CASSANDRA-2970 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jeremy Hanna Priority: Minor When doing read repair, it doesn't take into account the datacenter where the replicas are. It simply does a repair based on the read repair chance. Since multi-DC configurations would benefit from a lower chance between DC, it seems reasonable to have a separate setting for read repair between DCs than for within the DC. Perhaps there could be a single property still, which would default both inter and intra for the sake of simple scenarios and backwards compatibility. Then if a more specific setting is specified (read_repair_chance_global or read_repair_chance_local), that would be used. I'm not sure if this would complicate matters too much, but it builds on CASSANDRA-982 and CASSANDRA-1530 to help with efficiency between datacenters, especially in the case of an analytics cluster versus a realtime cluster - making them tunably more independent. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2582) sstable2json: make TTL optional, possibility of incremental dumps
[ https://issues.apache.org/jira/browse/CASSANDRA-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timo Nentwig updated CASSANDRA-2582: Attachment: (was: nottl-and-lastmodified-0.7.5.patch) > sstable2json: make TTL optional, possibility of incremental dumps > - > > Key: CASSANDRA-2582 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2582 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Timo Nentwig >Priority: Trivial > > Data on live cluster is supposed to expire automatically but exported to an > archive cluster beforehand. In order to prevent expiration on the archive > cluster add an option to sstable2json to ignore TTL data. Furthermore add an > lastModified option that ignores older columns for incremental archiving. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2582) sstable2json: make TTL optional, possibility of incremental dumps
[ https://issues.apache.org/jira/browse/CASSANDRA-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timo Nentwig updated CASSANDRA-2582: Attachment: (was: nottl-and-lastmodified.patch) > sstable2json: make TTL optional, possibility of incremental dumps > - > > Key: CASSANDRA-2582 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2582 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Timo Nentwig >Priority: Trivial > > Data on live cluster is supposed to expire automatically but exported to an > archive cluster beforehand. In order to prevent expiration on the archive > cluster add an option to sstable2json to ignore TTL data. Furthermore add an > lastModified option that ignores older columns for incremental archiving. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2970) Create separate read repair settings for intra- versus inter- datacenter
[ https://issues.apache.org/jira/browse/CASSANDRA-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2970. --- Resolution: Duplicate duplicate of CASSANDRA-2506 > Create separate read repair settings for intra- versus inter- datacenter > > > Key: CASSANDRA-2970 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2970 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jeremy Hanna >Priority: Minor > > When doing read repair, it doesn't take into account the datacenter where the > replicas are. It simply does a repair based on the read repair chance. > Since multi-DC configurations would benefit from a lower chance between DC, > it seems reasonable to have a separate setting for read repair between DCs > than for within the DC. > Perhaps there could be a single property still, which would default both > inter and intra for the sake of simple scenarios and backwards compatibility. > Then if a more specific setting is specified (read_repair_chance_global or > read_repair_chance_local), that would be used. > I'm not sure if this would complicate matters too much, but it builds on > CASSANDRA-982 and CASSANDRA-1530 to help with efficiency between datacenters, > especially in the case of an analytics cluster versus a realtime cluster - > making them tunably more independent. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2969) Queries and inserts fail on 2 node cluster when replication factor is specified.
[ https://issues.apache.org/jira/browse/CASSANDRA-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073105#comment-13073105 ] Jonathan Ellis commented on CASSANDRA-2969: --- You're not the first to hit this -- we do want to make it more helpful about this. We'll do that over on CASSANDRA-2960. > Queries and inserts fail on 2 node cluster when replication factor is > specified. > > > Key: CASSANDRA-2969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2969 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.8.2 > Environment: 2 node Cassandra cluster with both nodes up and running. > The replication factor is specified and can be 1 or 2. Client is using > Hector. For production, I want to use a RF of 2 so the data is stored on both > nodes and use a CL of ONE for reads and writes. >Reporter: Andy Klages > Labels: 2-node, replica > > When a query or insert is performed, I get the following exception: > me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be > enough replicas present to handle consistency level. > I've tried consistency levels ANY, ONE, and QUORUM, but to no avail. I'm > assuming it would do this for the other levels too. > Running the "get" and "set" commands on cassandra-cli always return "null" so > it's encountering the same problem. > Note that I don't get this problem when the keyspace doesn't have > replication_factor specified. But I need the data replicated on both nodes > which is why I'm specifying the RF. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2971) Append (not add new) InetAddress info logging when starting MessagingService
Append (not add new) InetAddress info logging when starting MessagingService
Key: CASSANDRA-2971
URL: https://issues.apache.org/jira/browse/CASSANDRA-2971
Project: Cassandra
Issue Type: Improvement
Reporter: Jackson Chung
Priority: Minor
Attachments: 2971.patch
Currently we have
{code: title=MessagingService.getServerSocket(InetAddress localEp) }
logger_.info("Starting Messaging Service on port {}",
DatabaseDescriptor.getStoragePort());
{code}
We should probably just print the whole binded address. The address is an
InetSocketAddress:
{code}
InetSocketAddress address = new InetSocketAddress(localEp,
DatabaseDescriptor.getStoragePort());
try
{
ss.bind(address);
}
{code}
{code}
logger_.info("Starting Messaging Service on {}",address);
{code}
sample output with the new log:
{noformat}
INFO [main] 2011-07-29 18:54:54,018 MessagingService.java (line 226) Starting
Messaging Service on faranth/192.168.1.141:7000
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2971) Append (not add new) InetAddress info logging when starting MessagingService
[
https://issues.apache.org/jira/browse/CASSANDRA-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackson Chung updated CASSANDRA-2971:
-
Attachment: 2971.patch
> Append (not add new) InetAddress info logging when starting MessagingService
>
>
> Key: CASSANDRA-2971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2971
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jackson Chung
>Priority: Minor
> Attachments: 2971.patch
>
>
> Currently we have
> {code: title=MessagingService.getServerSocket(InetAddress localEp) }
> logger_.info("Starting Messaging Service on port {}",
> DatabaseDescriptor.getStoragePort());
> {code}
> We should probably just print the whole binded address. The address is an
> InetSocketAddress:
> {code}
> InetSocketAddress address = new InetSocketAddress(localEp,
> DatabaseDescriptor.getStoragePort());
> try
> {
> ss.bind(address);
> }
> {code}
> {code}
> logger_.info("Starting Messaging Service on {}",address);
> {code}
> sample output with the new log:
> {noformat}
> INFO [main] 2011-07-29 18:54:54,018 MessagingService.java (line 226)
> Starting Messaging Service on faranth/192.168.1.141:7000
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1152416 - in /cassandra/trunk: src/java/org/apache/cassandra/cache/FreeableMemory.java src/java/org/apache/cassandra/cache/SerializingCache.java test/unit/org/apache/cassandra/cache/Cache
Author: jbellis
Date: Sat Jul 30 02:29:32 2011
New Revision: 1152416
URL: http://svn.apache.org/viewvc?rev=1152416&view=rev
Log:
create FM objects w/ reference count of one
patch by jbellis
Modified:
cassandra/trunk/src/java/org/apache/cassandra/cache/FreeableMemory.java
cassandra/trunk/src/java/org/apache/cassandra/cache/SerializingCache.java
cassandra/trunk/test/unit/org/apache/cassandra/cache/CacheProviderTest.java
Modified:
cassandra/trunk/src/java/org/apache/cassandra/cache/FreeableMemory.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cache/FreeableMemory.java?rev=1152416&r1=1152415&r2=1152416&view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cache/FreeableMemory.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/cache/FreeableMemory.java Sat
Jul 30 02:29:32 2011
@@ -28,14 +28,17 @@ import com.sun.jna.Memory;
public class FreeableMemory extends Memory
{
-AtomicInteger references = new AtomicInteger(0);
+AtomicInteger references = new AtomicInteger(1);
public FreeableMemory(long size)
{
super(size);
}
-/** @return true if we succeed in referencing before the reference count
reaches zero */
+/**
+ * @return true if we succeed in referencing before the reference count
reaches zero.
+ * (A FreeableMemory object is created with a reference count of one.)
+ */
public boolean reference()
{
while (true)
Modified:
cassandra/trunk/src/java/org/apache/cassandra/cache/SerializingCache.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/cache/SerializingCache.java?rev=1152416&r1=1152415&r2=1152416&view=diff
==
--- cassandra/trunk/src/java/org/apache/cassandra/cache/SerializingCache.java
(original)
+++ cassandra/trunk/src/java/org/apache/cassandra/cache/SerializingCache.java
Sat Jul 30 02:29:32 2011
@@ -155,7 +155,6 @@ public class SerializingCache impl
if (mem == null)
return; // out of memory. never mind.
-mem.reference();
FreeableMemory old = map.put(key, mem);
if (old != null)
old.unreference();
Modified:
cassandra/trunk/test/unit/org/apache/cassandra/cache/CacheProviderTest.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/cache/CacheProviderTest.java?rev=1152416&r1=1152415&r2=1152416&view=diff
==
--- cassandra/trunk/test/unit/org/apache/cassandra/cache/CacheProviderTest.java
(original)
+++ cassandra/trunk/test/unit/org/apache/cassandra/cache/CacheProviderTest.java
Sat Jul 30 02:29:32 2011
@@ -48,6 +48,8 @@ public class CacheProviderTest extends S
private void simpleCase(ColumnFamily cf, ICache
cache)
{
cache.put(key1, cf);
+assert cache.get(key1) != null;
+
assertDigests(cache.get(key1), cf);
cache.put(key2, cf);
cache.put(key3, cf);
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build on builder cassandra-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1467 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra/trunk] 1152416 Blamelist: jbellis Build succeeded! sincerely, -The Buildbot
[jira] [Updated] (CASSANDRA-2643) read repair/reconciliation breaks slice based iteration at QUORUM
[ https://issues.apache.org/jira/browse/CASSANDRA-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Byron Clark updated CASSANDRA-2643: --- Attachment: CASSANDRA-2643-v2.patch [^CASSANDRA-2643-v2.patch] makes the following improvements on [^CASSANDRA-2643-poc.patch]: * Cleans up duplicated code for counting live columns. * Removes the need to store the ReadCommand in the RepairCallback. Remaining issue to be dealt with: * Storing maxLiveColumns in the resolver feels wrong, but that's where the data is generated. We need to know how many live rows the biggest return had to detect if this is actually a short read. I'm open to suggestions a better place for that count to live. > read repair/reconciliation breaks slice based iteration at QUORUM > - > > Key: CASSANDRA-2643 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2643 > Project: Cassandra > Issue Type: Bug >Affects Versions: 0.7.5 >Reporter: Peter Schuller >Assignee: Brandon Williams >Priority: Critical > Fix For: 1.0 > > Attachments: CASSANDRA-2643-poc.patch, CASSANDRA-2643-v2.patch, > reliable_short_read_0.8.sh, short_read.sh, short_read_0.8.sh, slicetest.py > > > In short, I believe iterating over columns is impossible to do reliably with > QUORUM due to the way reconciliation works. > The problem is that the SliceQueryFilter is executing locally when reading on > a node, but no attempts seem to be made to consider limits when doing > reconciliation and/or read-repair (RowRepairResolver.resolveSuperset() and > ColumnFamily.resolve()). > If a node slices and comes up with 100 columns, and another node slices and > comes up with 100 columns, some of which are unique to each side, > reconciliation results in > 100 columns in the result set. In this case the > effect is limited to "client gets more than asked for", but the columns still > accurately represent the range. This is easily triggered by my test-case. > In addition to the client receiving "too many" columns, I believe some of > them will not be satisfying the QUORUM consistency level for the same reasons > as with deletions (see discussion below). > Now, there *should* be a problem for tombstones as well, but it's more > subtle. Suppose A has: > 1 > 2 > 3 > 4 > 5 > 6 > and B has: > 1 > del 2 > del 3 > del 4 > 5 > 6 > If you now slice 1-6 with count=3 the tombstones from B will reconcile with > those from A - fine. So you end up getting 1,5,6 back. This made it a bit > difficult to trigger in a test case until I realized what was going on. At > first I was "hoping" to see a "short" iteration result, which would mean that > the process of iterating until you get a short result will cause spurious > "end of columns" and thus make it impossible to iterate correctly. > So; due to 5-6 existing (and if they didn't, you legitimately reached > end-of-columns) we do indeed get a result of size 3 which contains 1,5 and 6. > However, only node B would have contributed columns 5 and 6; so there is > actually no QUORUM consistency on the co-ordinating node with respect to > these columns. If node A and C also had 5 and 6, they would not have been > considered. > Am I wrong? > In any case; using script I'm about to attach, you can trigger the > over-delivery case very easily: > (0) disable hinted hand-off to avoid that interacting with the test > (1) start three nodes > (2) create ks 'test' with rf=3 and cf 'slicetest' > (3) ./slicetest.py hostname_of_node_C insert # let it run for a few seconds, > then ctrl-c > (4) stop node A > (5) ./slicetest.py hostname_of_node_C insert # let it run for a few seconds, > then ctrl-c > (6) start node A, wait for B and C to consider it up > (7) ./slicetest.py hostname_of_node_A slice # make A co-ordinator though it > doesn't necessarily matter > You can also pass 'delete' (random deletion of 50% of contents) or > 'deleterange' (delete all in [0.2,0.8]) to slicetest, but you don't trigger a > short read by doing that (see discussion above). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1152419 - in /cassandra/branches/cassandra-0.8: ./ src/java/org/apache/cassandra/db/compaction/ src/java/org/apache/cassandra/service/
Author: jbellis Date: Sat Jul 30 02:48:17 2011 New Revision: 1152419 URL: http://svn.apache.org/viewvc?rev=1152419&view=rev Log: Flush memtables on shutdown when durable writes are disabled patch by jbellis; reviewed by tjake for CASSANDRA-2958 Modified: cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionIterator.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1152419&r1=1152418&r2=1152419&view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Sat Jul 30 02:48:17 2011 @@ -13,6 +13,8 @@ (CASSANDRA-2953) * check column family validity in nodetool repair (CASSANDRA-2933) * speedup bytes to hex conversions dramatically (CASSANDRA-2850) + * Flush memtables on shutdown when durable writes are disabled + (CASSANDRA-2958) 0.8.2 Modified: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionIterator.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionIterator.java?rev=1152419&r1=1152418&r2=1152419&view=diff == --- cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionIterator.java (original) +++ cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionIterator.java Sat Jul 30 02:48:17 2011 @@ -27,7 +27,6 @@ import java.util.ArrayList; import java.util.Iterator; import java.util.List; -import org.apache.cassandra.service.StorageService; import org.apache.commons.collections.iterators.CollatingIterator; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -37,6 +36,7 @@ import org.apache.cassandra.io.sstable.S import org.apache.cassandra.io.sstable.SSTableReader; import org.apache.cassandra.io.sstable.SSTableScanner; import org.apache.cassandra.io.util.FileUtils; +import org.apache.cassandra.service.StorageService; import org.apache.cassandra.utils.FBUtilities; import org.apache.cassandra.utils.ReducingIterator; Modified: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java?rev=1152419&r1=1152418&r2=1152419&view=diff == --- cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java (original) +++ cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java Sat Jul 30 02:48:17 2011 @@ -25,17 +25,15 @@ import java.io.DataOutput; import java.io.IOError; import java.io.IOException; import java.security.MessageDigest; -import java.util.*; +import java.util.ArrayList; +import java.util.Iterator; +import java.util.List; import com.google.common.base.Predicates; import com.google.common.collect.Iterators; import org.apache.commons.collections.iterators.CollatingIterator; -import org.apache.cassandra.db.ColumnFamily; -import org.apache.cassandra.db.ColumnFamilyStore; -import org.apache.cassandra.db.ColumnIndexer; -import org.apache.cassandra.db.CounterColumn; -import org.apache.cassandra.db.IColumn; +import org.apache.cassandra.db.*; import org.apache.cassandra.db.marshal.AbstractType; import org.apache.cassandra.io.sstable.SSTableIdentityIterator; import org.apache.cassandra.io.util.DataOutputBuffer; Modified: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java?rev=1152419&r1=1152418&r2=1152419&view=diff == --- cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java (original) +++ cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java Sat Jul 30 02:48:17 2011 @@ -44,6 +44,7 @@ import org.apache.cassandra.concurrent.S import org.apache.cassandra.config.CFMetaData; import org.apache.cassandra.config.ConfigurationException; import org.apache.cassandra.config.DatabaseDescriptor; +import org.apache.cassandra.config.KSMetaData; import org.apache.cassandra.db.*; import org.apache.cassandra.db.commitlog.CommitLog; import org.apache.cassandra.dht.*; @@
svn commit: r1152420 - /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java
Author: jbellis
Date: Sat Jul 30 02:50:41 2011
New Revision: 1152420
URL: http://svn.apache.org/viewvc?rev=1152420&view=rev
Log:
log full MS address on startup
patch by Jackson Chung; reviewed by jbellis for CASSANDRA-2971
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java?rev=1152420&r1=1152419&r2=1152420&view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java
Sat Jul 30 02:50:41 2011
@@ -221,7 +221,7 @@ public final class MessagingService impl
else
throw e;
}
-logger_.info("Starting Messaging Service on port {}",
DatabaseDescriptor.getStoragePort());
+logger_.info("Starting Messaging Service on {}", address);
}
return ss;
}
[jira] [Resolved] (CASSANDRA-2971) Append (not add new) InetAddress info logging when starting MessagingService
[
https://issues.apache.org/jira/browse/CASSANDRA-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-2971.
---
Resolution: Fixed
Fix Version/s: 0.8.3
Reviewer: jbellis
Assignee: Jackson Chung
committed, thanks!
> Append (not add new) InetAddress info logging when starting MessagingService
>
>
> Key: CASSANDRA-2971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2971
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jackson Chung
>Assignee: Jackson Chung
>Priority: Minor
> Fix For: 0.8.3
>
> Attachments: 2971.patch
>
>
> Currently we have
> {code: title=MessagingService.getServerSocket(InetAddress localEp) }
> logger_.info("Starting Messaging Service on port {}",
> DatabaseDescriptor.getStoragePort());
> {code}
> We should probably just print the whole binded address. The address is an
> InetSocketAddress:
> {code}
> InetSocketAddress address = new InetSocketAddress(localEp,
> DatabaseDescriptor.getStoragePort());
> try
> {
> ss.bind(address);
> }
> {code}
> {code}
> logger_.info("Starting Messaging Service on {}",address);
> {code}
> sample output with the new log:
> {noformat}
> INFO [main] 2011-07-29 18:54:54,018 MessagingService.java (line 226)
> Starting Messaging Service on faranth/192.168.1.141:7000
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2819) Split rpc timeout for read and write ops
[ https://issues.apache.org/jira/browse/CASSANDRA-2819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073112#comment-13073112 ] Jonathan Ellis commented on CASSANDRA-2819: --- Thanks, Melvin! should we move MS.getMessageTimeout to an instance method of Message? This would make per-Message timeouts easier in the future as well. Let's make the default values in Config 10s as well, to avoid surprising people who upgrade but keep their old config file around. Otherwise LGTM. > Split rpc timeout for read and write ops > > > Key: CASSANDRA-2819 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2819 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Stu Hood >Assignee: Melvin Wang > Fix For: 1.0 > > Attachments: twttr-cassandra-0.8-counts-resync-rpc-rw-timeouts.patch > > > Given the vastly different latency characteristics of reads and writes, it > makes sense for them to have independent rpc timeouts internally. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1152421 - in /cassandra/branches/cassandra-0.8: CHANGES.txt bin/cassandra conf/cassandra-env.sh
Author: jbellis
Date: Sat Jul 30 03:00:09 2011
New Revision: 1152421
URL: http://svn.apache.org/viewvc?rev=1152421&view=rev
Log:
improved POSIX compatibility of start scripts
patch by Kjell Andreassen and Paul Cannon for CASSANDRA-2965
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/bin/cassandra
cassandra/branches/cassandra-0.8/conf/cassandra-env.sh
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1152421&r1=1152420&r2=1152421&view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Sat Jul 30 03:00:09 2011
@@ -15,6 +15,7 @@
* speedup bytes to hex conversions dramatically (CASSANDRA-2850)
* Flush memtables on shutdown when durable writes are disabled
(CASSANDRA-2958)
+ * improved POSIX compatibility of start scripts (CASsANDRA-2965)
0.8.2
Modified: cassandra/branches/cassandra-0.8/bin/cassandra
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/bin/cassandra?rev=1152421&r1=1152420&r2=1152421&view=diff
==
--- cassandra/branches/cassandra-0.8/bin/cassandra (original)
+++ cassandra/branches/cassandra-0.8/bin/cassandra Sat Jul 30 03:00:09 2011
@@ -61,32 +61,35 @@
# Be aware that you will be entirely responsible for populating the needed
# environment variables.
+# NB: Developers should be aware that this script should remain compatible with
+# POSIX sh and Solaris sh. This means, in particular, no $(( )) and no $( ).
+
# If an include wasn't specified in the environment, then search for one...
if [ "x$CASSANDRA_INCLUDE" = "x" ]; then
# Locations (in order) to use when searching for an include file.
for include in /usr/share/cassandra/cassandra.in.sh \
/usr/local/share/cassandra/cassandra.in.sh \
/opt/cassandra/cassandra.in.sh \
- ~/.cassandra.in.sh \
- `dirname $0`/cassandra.in.sh; do
-if [ -r $include ]; then
-. $include
+ "$HOME/.cassandra.in.sh" \
+ "`dirname $0`/cassandra.in.sh"; do
+if [ -r "$include" ]; then
+. "$include"
break
fi
done
# ...otherwise, source the specified include.
-elif [ -r $CASSANDRA_INCLUDE ]; then
-. $CASSANDRA_INCLUDE
+elif [ -r "$CASSANDRA_INCLUDE" ]; then
+. "$CASSANDRA_INCLUDE"
fi
# Use JAVA_HOME if set, otherwise look for java in PATH
if [ -n "$JAVA_HOME" ]; then
-JAVA=$JAVA_HOME/bin/java
+JAVA="$JAVA_HOME/bin/java"
else
JAVA=java
fi
-if [ -z $CASSANDRA_CONF -o -z $CLASSPATH ]; then
+if [ -z "$CASSANDRA_CONF" -o -z "$CLASSPATH" ]; then
echo "You must set the CASSANDRA_CONF and CLASSPATH vars" >&2
exit 1
fi
@@ -119,11 +122,11 @@ launch_service()
# to close stdout/stderr, but it's up to us not to background.
if [ "x$foreground" != "x" ]; then
cassandra_parms="$cassandra_parms -Dcassandra-foreground=yes"
-exec $JAVA $JVM_OPTS $cassandra_parms -cp $CLASSPATH $props $class
+exec "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" $props
"$class"
# Startup CassandraDaemon, background it, and write the pid.
else
-exec $JAVA $JVM_OPTS $cassandra_parms -cp $CLASSPATH $props $class <&-
&
-[ ! -z $pidpath ] && printf "%d" $! > $pidpath
+exec "$JAVA" $JVM_OPTS $cassandra_parms -cp "$CLASSPATH" $props
"$class" <&- &
+[ ! -z "$pidpath" ] && printf "%d" $! > "$pidpath"
fi
return $?
@@ -150,7 +153,7 @@ while true; do
exit 0
;;
-v)
-$JAVA -cp $CLASSPATH org.apache.cassandra.tools.GetVersion
+"$JAVA" -cp "$CLASSPATH" org.apache.cassandra.tools.GetVersion
exit 0
;;
-D)
Modified: cassandra/branches/cassandra-0.8/conf/cassandra-env.sh
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/conf/cassandra-env.sh?rev=1152421&r1=1152420&r2=1152421&view=diff
==
--- cassandra/branches/cassandra-0.8/conf/cassandra-env.sh (original)
+++ cassandra/branches/cassandra-0.8/conf/cassandra-env.sh Sat Jul 30 03:00:09
2011
@@ -24,10 +24,15 @@ calculate_heap_sizes()
;;
FreeBSD)
system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'`
-system_memory_in_mb=$((system_memory_in_bytes / 1024 / 1024))
+system_memory_in_mb=`expr $system_memory_in_bytes / 1024 / 1024`
system_cpu_cores=`sysctl hw.ncpu | awk '{print $2}'`
break
;;
+SunOS)
+system_memory_in_mb=`prtconf | awk '/Memory size:/ {print $3}'`
+system_cpu_cor
[jira] [Commented] (CASSANDRA-2971) Append (not add new) InetAddress info logging when starting MessagingService
[
https://issues.apache.org/jira/browse/CASSANDRA-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073119#comment-13073119
]
Hudson commented on CASSANDRA-2971:
---
Integrated in Cassandra-0.8 #245 (See
[https://builds.apache.org/job/Cassandra-0.8/245/])
log full MS address on startup
patch by Jackson Chung; reviewed by jbellis for CASSANDRA-2971
jbellis :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152420
Files :
*
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java
> Append (not add new) InetAddress info logging when starting MessagingService
>
>
> Key: CASSANDRA-2971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2971
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jackson Chung
>Assignee: Jackson Chung
>Priority: Minor
> Fix For: 0.8.3
>
> Attachments: 2971.patch
>
>
> Currently we have
> {code: title=MessagingService.getServerSocket(InetAddress localEp) }
> logger_.info("Starting Messaging Service on port {}",
> DatabaseDescriptor.getStoragePort());
> {code}
> We should probably just print the whole binded address. The address is an
> InetSocketAddress:
> {code}
> InetSocketAddress address = new InetSocketAddress(localEp,
> DatabaseDescriptor.getStoragePort());
> try
> {
> ss.bind(address);
> }
> {code}
> {code}
> logger_.info("Starting Messaging Service on {}",address);
> {code}
> sample output with the new log:
> {noformat}
> INFO [main] 2011-07-29 18:54:54,018 MessagingService.java (line 226)
> Starting Messaging Service on faranth/192.168.1.141:7000
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2965) Allow cassandra to start on a Solaris machine.
[ https://issues.apache.org/jira/browse/CASSANDRA-2965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073117#comment-13073117 ] Hudson commented on CASSANDRA-2965: --- Integrated in Cassandra-0.8 #245 (See [https://builds.apache.org/job/Cassandra-0.8/245/]) improved POSIX compatibility of start scripts patch by Kjell Andreassen and Paul Cannon for CASSANDRA-2965 jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152421 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/conf/cassandra-env.sh * /cassandra/branches/cassandra-0.8/bin/cassandra > Allow cassandra to start on a Solaris machine. > -- > > Key: CASSANDRA-2965 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2965 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Affects Versions: 0.8.2 > Environment: Solaris 10/SunOS 5.10, x86 architecture. >Reporter: Kjell Andreassen >Assignee: Kjell Andreassen >Priority: Trivial > Fix For: 0.8.3 > > Attachments: 0001-Solaris-bin-sh-compatibility.patch.txt, > 0002-improve-general-sh-compatibility.patch.txt, cassandra-0.8.2-2965.txt > > > Cassandra ($CASSANDRA_HOME/bin/cassandra) fails to start with a series of > errors, fixing one reveals the next. > These are the errors: > bin/cassandra: syntax error at line 27: `system_memory_in_mb=$' unexpected > bin/cassandra: syntax error at line 100: `check_openjdk=$' unexpected > bin/cassandra: test: argument expected -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2958) Flush memtables on shutdown when durable writes are disabled
[ https://issues.apache.org/jira/browse/CASSANDRA-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073118#comment-13073118 ] Hudson commented on CASSANDRA-2958: --- Integrated in Cassandra-0.8 #245 (See [https://builds.apache.org/job/Cassandra-0.8/245/]) Flush memtables on shutdown when durable writes are disabled patch by jbellis; reviewed by tjake for CASSANDRA-2958 jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152419 Files : * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/LazilyCompactedRow.java * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionIterator.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/service/StorageService.java > Flush memtables on shutdown when durable writes are disabled > > > Key: CASSANDRA-2958 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2958 > Project: Cassandra > Issue Type: Bug >Affects Versions: 0.8.1 >Reporter: David Phillips >Assignee: Jonathan Ellis > Fix For: 0.8.3 > > Attachments: 2958.txt > > > Memtables need to be flushed on shutdown when durable_writes is set to false, > otherwise data loss occurs as the data is not available to be replayed from > the commit log. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2894) add paging to get_count
[ https://issues.apache.org/jira/browse/CASSANDRA-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2894: -- Attachment: 2894-formatted.txt Looks good to me. Attached version w/ minor formatting changes. Two comments: - I think this stomps on the count setting in the predicate, if one is present. Granted, that's an unusual thing to do, but it's valid. - Can you add a system test (test/system/test_thrift_server.py) exercising paging? > add paging to get_count > --- > > Key: CASSANDRA-2894 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2894 > Project: Cassandra > Issue Type: Improvement > Components: API >Reporter: Jonathan Ellis >Priority: Minor > Labels: lhf > Fix For: 1.0 > > Attachments: 2894-formatted.txt, CASSANDRA-2894.patch > > > It is non-intuitive that get_count materializes the entire slice-to-count on > the coordinator node (to perform read repair and > CL.ONE consistency). Even > experienced users have been known to cause memory problems by requesting > large counts. > The user cannot page the count himself, because you need a start and stop > column to do that, and get_count only returns an integer. > So the best fix is for us to do the paging under the hood, in > CassandraServer. Add a limit to the slicepredicate they specify, and page > through it. > We could add a global setting for count_slice_size, and document that counts > of more columns than that will have higher latency (because they make > multiple calls through StorageProxy for the pages). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2972) nodetool netstats progress does not update on receiving side
nodetool netstats progress does not update on receiving side Key: CASSANDRA-2972 URL: https://issues.apache.org/jira/browse/CASSANDRA-2972 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.1 Reporter: Wojciech Meler when you add/remove node to cluster, nodetool netstats show correct results only on sending side - on receiving side you can see only 0% progress -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2973) fatal errrors after nodetool cleanup
fatal errrors after nodetool cleanup Key: CASSANDRA-2973 URL: https://issues.apache.org/jira/browse/CASSANDRA-2973 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.1 Reporter: Wojciech Meler after adding nodes to cluster & running cleanup I get scaring exceptions in log: 2011-07-30 00:00:05:506 CEST ERROR [ReadStage:2335][org.apache.cassandra.service.AbstractCassandraDaemon] Fatal exception in thread Thread[ReadStage:2335,5,main] java.io.IOError: java.io.IOException: mmap segment underflow; remaining is 4394 but 60165 requested at org.apache.cassandra.db.columniterator.IndexedSliceReader.(IndexedSliceReader.java:80) at org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:91) at org.apache.cassandra.db.columniterator.SSTableSliceIterator.(SSTableSliceIterator.java:67) at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:66) at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1292) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1189) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1146) at org.apache.cassandra.db.Table.getRow(Table.java:385) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:61) at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:69) at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: mmap segment underflow; remaining is 4394 but 60165 requested at org.apache.cassandra.io.util.MappedFileDataInput.readBytes(MappedFileDataInput.java:117) at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:389) at org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:368) at org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:194) at org.apache.cassandra.io.sstable.IndexHelper.deserializeIndex(IndexHelper.java:83) at org.apache.cassandra.db.columniterator.IndexedSliceReader.(IndexedSliceReader.java:73) ... 14 more exceptions disappeared after running scrub -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-2974) nodetool removetoken hang
nodetool removetoken hang - Key: CASSANDRA-2974 URL: https://issues.apache.org/jira/browse/CASSANDRA-2974 Project: Cassandra Issue Type: Bug Affects Versions: 0.8.1 Reporter: Wojciech Meler one node died - i tried to remove it with removetoken but it hanged: RemovalStatus: Removing token (9452287970026068429538183539771339207). Waiting for replication confirmation from [/10.0.3.78]. nodetool netstats doesn't show any streams: # nodetool -h 10.0.3.65 -p 8080 netstats Mode: Normal Not sending any streams. Not receiving any streams. Pool NameActive Pending Completed Commandsn/a 0 332578 Responses n/a 0 646405 # nodetool -h 10.0.3.66 -p 8080 netstats Mode: Normal Nothing streaming to /10.0.3.78 Not receiving any streams. Pool NameActive Pending Completed Commandsn/a 178 739797 Responses n/a 01294349 # nodetool -h 10.0.3.71 -p 8080 netstats Mode: Normal Not sending any streams. Not receiving any streams. Pool NameActive Pending Completed Commandsn/a842031299 Responses n/a 0 357749 BTW. "Nothing streaming to /10.0.3.78" is quite funny :) also nodetool ring show strange things - almost whole nodes report: 10.0.3.65 datacenter1 rack1 Up Normal 114.2 GB5.56% 0 10.0.3.77 datacenter1 rack1 Down Leaving 158.09 GB 5.56% 9452287970026068429538183539771339207 10.0.3.71 datacenter1 rack1 Up Normal 196.76 GB 5.56% 18904575940052136859076367079542678414 10.0.3.66 datacenter1 rack1 Up Normal 178.95 GB 5.56% 28356863910078205288614550619314017621 10.0.3.78 datacenter1 rack1 Up Normal 227.05 GB 5.56% 37809151880104273718152734159085356828 10.0.3.72 datacenter1 rack1 Up Normal 110.83 GB 5.56% 47261439850130342147690917698856696035 10.0.3.67 datacenter1 rack1 Up Normal 117.45 GB 5.56% 56713727820156410577229101238628035242 10.0.3.79 datacenter1 rack1 Up Normal 138.62 GB 5.56% 66166015790182479006767284778399374449 10.0.3.73 datacenter1 rack1 Up Normal 110.49 GB 5.56% 75618303760208547436305468318170713656 10.0.3.68 datacenter1 rack1 Up Normal 114.82 GB 5.56% 85070591730234615865843651857942052863 10.0.3.80 datacenter1 rack1 Up Normal 145.51 GB 5.56% 94522879700260684295381835397713392070 10.0.3.74 datacenter1 rack1 Up Normal 113.63 GB 5.56% 103975167670286752724920018937484731277 10.0.3.69 datacenter1 rack1 Up Normal 111.24 GB 5.56% 113427455640312821154458202477256070484 10.0.3.81 datacenter1 rack1 Up Normal 142.12 GB 5.56% 122879743610338889583996386017027409691 10.0.3.75 datacenter1 rack1 Up Normal 110.87 GB 5.56% 132332031580364958013534569556798748898 10.0.3.70 datacenter1 rack1 Up Normal 113.21 GB 5.56% 141784319550391026443072753096570088105 10.0.3.82 datacenter1 rack1 Up Normal 163.29 GB 5.56% 151236607520417094872610936636341427312 10.0.3.76 datacenter1 rack1 Up Normal 112.68 GB 5.56% 160688895490443163302149120176112766519 but 2 nodes which should take responsibility for removed token says: 10.0.3.65 datacenter1 rack1 Up Normal 114.2 GB5.56% 0 10.0.3.77 datacenter1 rack1 Up Leaving 158.09 GB 5.56% 9452287970026068429538183539771339207 10.0.3.71 datacenter1 rack1 Up Normal 196.76 GB 5.56% 18904575940052136859076367079542678414 10.0.3.66 datacenter1 rack1 Up Normal 178.95 GB 5.56% 28356863910078205288614550619314017621 10.0.3.78 datacenter1 rack1 Up Normal 227.05 GB 5.56% 37809151880104273718152734159085356828 10.0.3.72 datacenter1 rack1 Up Normal 110.83 GB 5.56% 47261439850130342147690917698856696035 10.0.3.67 datacenter1 rack1 Up Normal 117.45 GB 5.56% 56713727820156410577229101238628035242 10.0.3.79 datacenter1 rack1 Up Normal 138.62 GB 5.56% 66166015790182479006767284778399374449 10.0.3.73 datacenter1 rack1 Up Normal 110.49 GB 5.56% 75618303760208547436305468318170713656 10.0.3.68 datacenter1 rack1 Up Normal 114.82 GB 5.56% 85070591730234615865843651857942052863 10.0.3.80 datacenter1 rack1 Up Normal 145.51 GB 5.56% 94522879700260684295381835397713392070 10.0.3.74
[jira] [Commented] (CASSANDRA-1391) Allow Concurrent Schema Migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073124#comment-13073124 ] Jonathan Ellis commented on CASSANDRA-1391: --- Thinking about this more, the really important part is that all nodes agree on the same schema no matter what order they get the Migrations in. If we can make that guarantee, the actual conflict resolution doesn't have to be particularly good (since it will still be a rare occurrence). So what is "the simplest thing that can work" here? I think we need to be able to merge Migrations at a finer granularity. If we do not, we have problems like this: - Mutation 1 (M1) says "set default_validation_class=ascii, comment='foo'" at time T1. - M2 says "set row_cache_size=100" at time T0 < T1. If node A gets M2 first, applies it, then gets M1, it has all 3 changes made. If node B however gets M1 first, then rejects M2 because T0 < T1 (for whatever kind of clock/comparator we are talking about), nodes A and B will end up with different schemas. I think wall-clock-time plus content-based tiebreaker (like we currently do with Column values) will be just as good as more complex ordering, as long as we have the fine-grained merging. > Allow Concurrent Schema Migrations > -- > > Key: CASSANDRA-1391 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1391 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Stu Hood >Assignee: Gary Dusbabek > > CASSANDRA-1292 fixed multiple migrations started from the same node to > properly queue themselves, but it is still possible for migrations initiated > on different nodes to conflict and leave the cluster in a bad state. Since > the system_add/drop/rename methods are accessible directly from the client > API, they should be completely safe for concurrent use. > It should be possible to allow for most types of concurrent migrations by > converting the UUID schema ID into a VersionVectorClock (as provided by > CASSANDRA-580). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2972) nodetool netstats progress does not update on receiving side
[ https://issues.apache.org/jira/browse/CASSANDRA-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2972: -- Priority: Minor (was: Major) Assignee: Yuki Morishita What do you think, Yuki? > nodetool netstats progress does not update on receiving side > > > Key: CASSANDRA-2972 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2972 > Project: Cassandra > Issue Type: Bug >Affects Versions: 0.8.1 >Reporter: Wojciech Meler >Assignee: Yuki Morishita >Priority: Minor > > when you add/remove node to cluster, nodetool netstats show correct results > only on sending side - on receiving side you can see only 0% progress -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2973) fatal errrors after nodetool cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073126#comment-13073126 ] Jonathan Ellis commented on CASSANDRA-2973: --- Is it reproducible? If not, it's quite possible that it's transient hardware-caused corruption. > fatal errrors after nodetool cleanup > > > Key: CASSANDRA-2973 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2973 > Project: Cassandra > Issue Type: Bug >Affects Versions: 0.8.1 >Reporter: Wojciech Meler > > after adding nodes to cluster & running cleanup I get scaring exceptions in > log: > 2011-07-30 00:00:05:506 CEST ERROR > [ReadStage:2335][org.apache.cassandra.service.AbstractCassandraDaemon] Fatal > exception in thread Thread[ReadStage:2335,5,main] > java.io.IOError: java.io.IOException: mmap segment underflow; remaining is > 4394 but 60165 requested > at > org.apache.cassandra.db.columniterator.IndexedSliceReader.(IndexedSliceReader.java:80) > at > org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:91) > at > org.apache.cassandra.db.columniterator.SSTableSliceIterator.(SSTableSliceIterator.java:67) > at > org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:66) > at > org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80) > at > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1292) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1189) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1146) > at org.apache.cassandra.db.Table.getRow(Table.java:385) > at > org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:61) > at > org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:69) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:72) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > Caused by: java.io.IOException: mmap segment underflow; remaining is 4394 > but 60165 requested > at > org.apache.cassandra.io.util.MappedFileDataInput.readBytes(MappedFileDataInput.java:117) > at > org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:389) > at > org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:368) > at > org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:194) > at > org.apache.cassandra.io.sstable.IndexHelper.deserializeIndex(IndexHelper.java:83) > at > org.apache.cassandra.db.columniterator.IndexedSliceReader.(IndexedSliceReader.java:73) > ... 14 more > exceptions disappeared after running scrub -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1152425 - /cassandra/branches/cassandra-0.8/CHANGES.txt
Author: jbellis Date: Sat Jul 30 04:17:53 2011 New Revision: 1152425 URL: http://svn.apache.org/viewvc?rev=1152425&view=rev Log: fix issue typo Modified: cassandra/branches/cassandra-0.8/CHANGES.txt Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1152425&r1=1152424&r2=1152425&view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Sat Jul 30 04:17:53 2011 @@ -1,7 +1,7 @@ 0.8.3 * add ability to drop local reads/writes that are going to timeout (CASSANDRA-2943) - * revamp token removal process, keep gossip states for 3 days (CASSANDRA-2946) + * revamp token removal process, keep gossip states for 3 days (CASSANDRA-2496) * don't accept extra args for 0-arg nodetool commands (CASSANDRA-2740) * log unavailableexception details at debug level (CASSANDRA-2856) * expose data_dir though jmx (CASSANDRA-2770)
[jira] [Resolved] (CASSANDRA-2974) nodetool removetoken hang
[ https://issues.apache.org/jira/browse/CASSANDRA-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2974. --- Resolution: Duplicate removetoken was fairly broken prior to CASSANDRA-2496 (fixed for 0.8.3). > nodetool removetoken hang > - > > Key: CASSANDRA-2974 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2974 > Project: Cassandra > Issue Type: Bug >Affects Versions: 0.8.1 >Reporter: Wojciech Meler > > one node died - i tried to remove it with removetoken but it hanged: > RemovalStatus: Removing token (9452287970026068429538183539771339207). > Waiting for replication confirmation from [/10.0.3.78]. > nodetool netstats doesn't show any streams: > # nodetool -h 10.0.3.65 -p 8080 netstats > Mode: Normal > Not sending any streams. > Not receiving any streams. > Pool NameActive Pending Completed > Commandsn/a 0 332578 > Responses n/a 0 646405 > # nodetool -h 10.0.3.66 -p 8080 netstats > Mode: Normal > Nothing streaming to /10.0.3.78 > Not receiving any streams. > Pool NameActive Pending Completed > Commandsn/a 178 739797 > Responses n/a 01294349 > # nodetool -h 10.0.3.71 -p 8080 netstats > Mode: Normal > Not sending any streams. > Not receiving any streams. > Pool NameActive Pending Completed > Commandsn/a842031299 > Responses n/a 0 357749 > BTW. "Nothing streaming to /10.0.3.78" is quite funny :) > also nodetool ring show strange things - almost whole nodes report: > 10.0.3.65 datacenter1 rack1 Up Normal 114.2 GB5.56% > 0 > 10.0.3.77 datacenter1 rack1 Down Leaving 158.09 GB 5.56% > 9452287970026068429538183539771339207 > 10.0.3.71 datacenter1 rack1 Up Normal 196.76 GB 5.56% > 18904575940052136859076367079542678414 > 10.0.3.66 datacenter1 rack1 Up Normal 178.95 GB 5.56% > 28356863910078205288614550619314017621 > 10.0.3.78 datacenter1 rack1 Up Normal 227.05 GB 5.56% > 37809151880104273718152734159085356828 > 10.0.3.72 datacenter1 rack1 Up Normal 110.83 GB 5.56% > 47261439850130342147690917698856696035 > 10.0.3.67 datacenter1 rack1 Up Normal 117.45 GB 5.56% > 56713727820156410577229101238628035242 > 10.0.3.79 datacenter1 rack1 Up Normal 138.62 GB 5.56% > 66166015790182479006767284778399374449 > 10.0.3.73 datacenter1 rack1 Up Normal 110.49 GB 5.56% > 75618303760208547436305468318170713656 > 10.0.3.68 datacenter1 rack1 Up Normal 114.82 GB 5.56% > 85070591730234615865843651857942052863 > 10.0.3.80 datacenter1 rack1 Up Normal 145.51 GB 5.56% > 94522879700260684295381835397713392070 > 10.0.3.74 datacenter1 rack1 Up Normal 113.63 GB 5.56% > 103975167670286752724920018937484731277 > 10.0.3.69 datacenter1 rack1 Up Normal 111.24 GB 5.56% > 113427455640312821154458202477256070484 > 10.0.3.81 datacenter1 rack1 Up Normal 142.12 GB 5.56% > 122879743610338889583996386017027409691 > 10.0.3.75 datacenter1 rack1 Up Normal 110.87 GB 5.56% > 132332031580364958013534569556798748898 > 10.0.3.70 datacenter1 rack1 Up Normal 113.21 GB 5.56% > 141784319550391026443072753096570088105 > 10.0.3.82 datacenter1 rack1 Up Normal 163.29 GB 5.56% > 151236607520417094872610936636341427312 > 10.0.3.76 datacenter1 rack1 Up Normal 112.68 GB 5.56% > 160688895490443163302149120176112766519 > but 2 nodes which should take responsibility for removed token says: > 10.0.3.65 datacenter1 rack1 Up Normal 114.2 GB5.56% > 0 > 10.0.3.77 datacenter1 rack1 Up Leaving 158.09 GB 5.56% > 9452287970026068429538183539771339207 > 10.0.3.71 datacenter1 rack1 Up Normal 196.76 GB 5.56% > 18904575940052136859076367079542678414 > 10.0.3.66 datacenter1 rack1 Up Normal 178.95 GB 5.56% > 28356863910078205288614550619314017621 > 10.0.3.78 datacenter1 rack1 Up Normal 227.05 GB 5.56% > 37809151880104273718152734159085356828 > 10.0.3.72 datacenter1 rack1 Up Normal 110.83 GB 5.56% > 47261439850130342147690917698856696035 > 10.0.3.67 datacenter1 rack1 Up Normal 117.45 GB 5.56% > 56713727820156410577229101238628035242 > 10.0.3.7
[jira] [Created] (CASSANDRA-2975) Upgrade MurmurHash to version 3
Upgrade MurmurHash to version 3 --- Key: CASSANDRA-2975 URL: https://issues.apache.org/jira/browse/CASSANDRA-2975 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.8.3 Reporter: Brian Lindauer Priority: Trivial MurmurHash version 3 was finalized on June 3. It provides an enormous speedup and increased robustness over version 2, which is implemented in Cassandra. Information here: http://code.google.com/p/smhasher/ The reference implementation is here: http://code.google.com/p/smhasher/source/browse/trunk/MurmurHash3.cpp?spec=svn136&r=136 I have already done the work to port the (public domain) reference implementation to Java in the MurmurHash class and updated the BloomFilter class to use the new implementation: https://github.com/lindauer/cassandra/commit/cea6068a4a3e5d7d9509335394f9ef3350d37e93 Apart from the faster hash time, the new version only requires one call to hash() rather than 2, since it returns 128 bits of hash instead of 64. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
