[jira] [Commented] (CASSANDRA-4264) live ratio limit 64.0 is way too low

2012-05-28 Thread Radim Kolar (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284354#comment-13284354
 ] 

Radim Kolar commented on CASSANDRA-4264:


I did mvn test and MemoryMeterTest passed on same platform.

Also i noticed that for rest of column families which are not using 
supercolumns with TTL expiration live ratio reported is about 4-16x, nothing 
gets close to 64.0

 live ratio limit 64.0 is way too low
 

 Key: CASSANDRA-4264
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4264
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.8.0
 Environment: FreeBSD 8 - 64 bit
Reporter: Radim Kolar
Priority: Trivial
 Attachments: patch-liveratio.txt


 Currently live ratio is limited to 64.0. This is way too low.
 WARN [MemoryMeter:1] Memtable.java (line 181) setting live ratio to maximum 
 of 64 instead of 
 values seen in log are ofter larger than 64.0 limit. I propose to use 100.0 
 as new limit.
 ponto:(admin)log/cassandragrep to maximum of 64 system.log.1
  WARN [MemoryMeter:1] 2012-02-03 00:00:19,444 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 64.9096047648211
  WARN [MemoryMeter:1] 2012-02-08 00:00:17,379 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 68.81016452376322
  WARN [MemoryMeter:1] 2012-02-08 00:00:32,358 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 88.49747308025415
  WARN [MemoryMeter:1] 2012-02-09 00:00:08,448 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 76.2444888765154
  WARN [MemoryMeter:1] 2012-02-10 18:18:52,677 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 142.22477982642255
  WARN [MemoryMeter:1] 2012-02-20 00:00:53,753 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 88.19832386767173
  WARN [MemoryMeter:1] 2012-03-02 10:41:00,232 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 419.9607495592804
  WARN [MemoryMeter:1] 2012-03-07 14:13:15,141 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of Infinity
  WARN [MemoryMeter:1] 2012-03-08 00:01:12,766 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 94.20215772717702
  WARN [MemoryMeter:1] 2012-03-09 00:00:38,633 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 98.54003447121715
  WARN [MemoryMeter:1] 2012-03-11 00:00:13,243 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 193.14262214179965
  WARN [MemoryMeter:1] 2012-03-14 00:00:26,709 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 103.88360138951437 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4264) live ratio limit 64.0 is way too low

2012-05-28 Thread Jonathan Ellis (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-4264:
--

Attachment: 4264.txt

Ah, that does help narrow it down.

Patch attached (against 1.1, looks like the code changes do apply to 1.0 as 
well) to include supercolumn name and tombstone size in the denominator of 
liveRatio.

 live ratio limit 64.0 is way too low
 

 Key: CASSANDRA-4264
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4264
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.8.0
 Environment: FreeBSD 8 - 64 bit
Reporter: Radim Kolar
Priority: Trivial
 Attachments: 4264.txt, patch-liveratio.txt


 Currently live ratio is limited to 64.0. This is way too low.
 WARN [MemoryMeter:1] Memtable.java (line 181) setting live ratio to maximum 
 of 64 instead of 
 values seen in log are ofter larger than 64.0 limit. I propose to use 100.0 
 as new limit.
 ponto:(admin)log/cassandragrep to maximum of 64 system.log.1
  WARN [MemoryMeter:1] 2012-02-03 00:00:19,444 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 64.9096047648211
  WARN [MemoryMeter:1] 2012-02-08 00:00:17,379 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 68.81016452376322
  WARN [MemoryMeter:1] 2012-02-08 00:00:32,358 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 88.49747308025415
  WARN [MemoryMeter:1] 2012-02-09 00:00:08,448 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 76.2444888765154
  WARN [MemoryMeter:1] 2012-02-10 18:18:52,677 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 142.22477982642255
  WARN [MemoryMeter:1] 2012-02-20 00:00:53,753 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 88.19832386767173
  WARN [MemoryMeter:1] 2012-03-02 10:41:00,232 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 419.9607495592804
  WARN [MemoryMeter:1] 2012-03-07 14:13:15,141 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of Infinity
  WARN [MemoryMeter:1] 2012-03-08 00:01:12,766 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 94.20215772717702
  WARN [MemoryMeter:1] 2012-03-09 00:00:38,633 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 98.54003447121715
  WARN [MemoryMeter:1] 2012-03-11 00:00:13,243 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 193.14262214179965
  WARN [MemoryMeter:1] 2012-03-14 00:00:26,709 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 103.88360138951437 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4264) live ratio limit 64.0 is way too low

2012-05-28 Thread Jonathan Ellis (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-4264:
--

 Reviewer: hsn
Fix Version/s: 1.1.1
 Assignee: Jonathan Ellis

 live ratio limit 64.0 is way too low
 

 Key: CASSANDRA-4264
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4264
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.8.0
 Environment: FreeBSD 8 - 64 bit
Reporter: Radim Kolar
Assignee: Jonathan Ellis
Priority: Trivial
 Fix For: 1.1.1

 Attachments: 4264.txt, patch-liveratio.txt


 Currently live ratio is limited to 64.0. This is way too low.
 WARN [MemoryMeter:1] Memtable.java (line 181) setting live ratio to maximum 
 of 64 instead of 
 values seen in log are ofter larger than 64.0 limit. I propose to use 100.0 
 as new limit.
 ponto:(admin)log/cassandragrep to maximum of 64 system.log.1
  WARN [MemoryMeter:1] 2012-02-03 00:00:19,444 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 64.9096047648211
  WARN [MemoryMeter:1] 2012-02-08 00:00:17,379 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 68.81016452376322
  WARN [MemoryMeter:1] 2012-02-08 00:00:32,358 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 88.49747308025415
  WARN [MemoryMeter:1] 2012-02-09 00:00:08,448 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 76.2444888765154
  WARN [MemoryMeter:1] 2012-02-10 18:18:52,677 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 142.22477982642255
  WARN [MemoryMeter:1] 2012-02-20 00:00:53,753 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 88.19832386767173
  WARN [MemoryMeter:1] 2012-03-02 10:41:00,232 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 419.9607495592804
  WARN [MemoryMeter:1] 2012-03-07 14:13:15,141 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of Infinity
  WARN [MemoryMeter:1] 2012-03-08 00:01:12,766 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 94.20215772717702
  WARN [MemoryMeter:1] 2012-03-09 00:00:38,633 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 98.54003447121715
  WARN [MemoryMeter:1] 2012-03-11 00:00:13,243 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 193.14262214179965
  WARN [MemoryMeter:1] 2012-03-14 00:00:26,709 Memtable.java (line 181) 
 setting live ratio to maximum of 64 instead of 103.88360138951437 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4221) Error while deleting a columnfamily that is being compacted.

2012-05-28 Thread Jonathan Ellis (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-4221:
--

Attachment: 4221-v3.txt

v3 attached.  removes CompactionInfo fields that are redundant w/ the 
introduction of CFM, and removes the wait from the stop method (it doesn't help 
clean up the sstables involved any faster, so there is no point in slowing down 
the drop for it).

 Error while deleting a columnfamily that is being compacted.
 

 Key: CASSANDRA-4221
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4221
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.1.0
 Environment: ccm, dtest, cassandra-1.1. The error does not happen in 
 cassandra-1.0.
Reporter: Tyler Patterson
Assignee: Pavel Yaskevich
 Fix For: 1.1.1

 Attachments: 4221-v3.txt, CASSANDRA-4221-logging.patch, 
 CASSANDRA-4221-v2.patch, CASSANDRA-4221.patch, system.log


 The following dtest command produces an error:
 {code}export CASSANDRA_VERSION=git:cassandra-1.1; nosetests --nocapture 
 --nologcapture 
 concurrent_schema_changes_test.py:TestConcurrentSchemaChanges.load_test{code}
 Here is the error:
 {code}
 Error occured during compaction
 java.util.concurrent.ExecutionException: java.io.IOError: 
 java.io.FileNotFoundException: 
 /tmp/dtest-6ECMgy/test/node1/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-47-Data.db
  (No such file or directory)
   at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
   at java.util.concurrent.FutureTask.get(FutureTask.java:111)
   at 
 org.apache.cassandra.db.compaction.CompactionManager.performMaximal(CompactionManager.java:239)
   at 
 org.apache.cassandra.db.ColumnFamilyStore.forceMajorCompaction(ColumnFamilyStore.java:1580)
   at 
 org.apache.cassandra.service.StorageService.forceTableCompaction(StorageService.java:1770)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:616)
   at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:111)
   at 
 com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:45)
   at 
 com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:226)
   at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
   at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:251)
   at 
 com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:857)
   at 
 com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:795)
   at 
 javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1450)
   at 
 javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:90)
   at 
 javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1285)
   at 
 javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1383)
   at 
 javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:807)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:616)
   at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
   at sun.rmi.transport.Transport$1.run(Transport.java:177)
   at java.security.AccessController.doPrivileged(Native Method)
   at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
   at 
 sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
   at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
   at 
 sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:679)
 Caused by: java.io.IOError: java.io.FileNotFoundException: 
 /tmp/dtest-6ECMgy/test/node1/data/Keyspace1/Standard1/Keyspace1-Standard1-hc-47-Data.db
  (No such file or directory)
   at 
 org.apache.cassandra.io.sstable.SSTableScanner.init(SSTableScanner.java:61)
   at

[jira] [Commented] (CASSANDRA-1311) Triggers

2012-05-28 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284457#comment-13284457
 ] 

Jonathan Ellis commented on CASSANDRA-1311:
---

Pulling out the distributed commitlog idea to CASSANRA-4825

 Triggers
 

 Key: CASSANDRA-1311
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1311
 Project: Cassandra
  Issue Type: New Feature
Reporter: Maxim Grinev
 Fix For: 1.2

 Attachments: HOWTO-PatchAndRunTriggerExample-update1.txt, 
 HOWTO-PatchAndRunTriggerExample.txt, ImplementationDetails-update1.pdf, 
 ImplementationDetails.pdf, trunk-967053.txt, trunk-984391-update1.txt, 
 trunk-984391-update2.txt


 Asynchronous triggers is a basic mechanism to implement various use cases of 
 asynchronous execution of application code at database side. For example to 
 support indexes and materialized views, online analytics, push-based data 
 propagation.
 Please find the motivation, triggers description and list of applications:
 http://maxgrinev.com/2010/07/23/extending-cassandra-with-asynchronous-triggers/
 An example of using triggers for indexing:
 http://maxgrinev.com/2010/07/23/managing-indexes-in-cassandra-using-async-triggers/
 Implementation details are attached.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4285) Atomic, eventually-consistent batches

2012-05-28 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284462#comment-13284462
 ] 

Jonathan Ellis commented on CASSANDRA-4285:
---

In the trigger discussion (starting 
[here|https://issues.apache.org/jira/browse/CASSANDRA-1311?focusedCommentId=13137492page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13137492]
 and continuing 
[here|https://issues.apache.org/jira/browse/CASSANDRA-1311?focusedCommentId=13245418page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13245418])
 I proposed a distributed commitlog that the coordinator could use to retry 
partially successful batches.

To flesh that out a bit, I think there are two approaches we can take here:
# the distributed commitlog (CSCL) approach discussed in the trigger ticket.  
This is definitely more complex assuming DCL RF  1, since a write to any DCL 
replica succeeds, the commitlog write can be considered successful -- and 
timeout still means we don't know, retry. Then you have the complexity of DCL 
replay -- sort of like hint handoff, in reverse, in the sense that we need data 
from other nodes that may or may not be up at the same time as us -- and of 
course it's going to basically halve write performance.
# a local-only approach, where batches are written to a non-replicated system 
CF the way hints are now.  This would provide adequate durability when we can 
rely on Raid1/Raid10 local disks; we don't need to worry about preserving this 
data indefinitely, after all; only until it's persisted to the other replicas.  
However, this is a non-starter for cloud environments where the provider will 
just nuke VMs out from under you if there's a problem, and even for non-cloud 
environments many prefer to deploy on Raid0 instead of paying the space 
overhead for Raid10.

So I think we should
- Start with the distributed commitlog since it is more generally useful, but
- Make batch atomicity optional, so users who don't need it don't pay any 
performance penalty over what we have now

 Atomic, eventually-consistent batches
 -

 Key: CASSANDRA-4285
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4285
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis

 I discussed this in the context of triggers (CASSANDRA-1311) but it's useful 
 as a standalone feature as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4285) Atomic, eventually-consistent batches

2012-05-28 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284466#comment-13284466
 ] 

Jonathan Ellis commented on CASSANDRA-4285:
---

The DCL should be in its own keyspace, both for hygiene and so replication 
strategy can be tuned independently.  In particular I think RF=1 will be Good 
Enough for 90% of use cases; remember that our goal here is to handle the case 
when the coordinator has sent part of a batch out, but not the entire thing.  
Once the entire batch is sent out to one replica, HH and repair can take care 
of making sure that gets to the rest of the cluster.  

So it's only the relatively narrow window of sending out the rows in the batch 
that we need to worry about here -- once that is done we don't care about 
preserving the original batch anymore.  The coordinator itself is thus 
effectively a replica for the useful part of the operation, so a single other 
machine should be adequate protection.

Unfortunately we don't have a way to say store this replica anywhere but 
locally.  So maybe we do need to go to RF=2 to make sure we have another 
*non-local* copy.

Alternatively maybe we could special case this and write a hint elsewhere if we 
detect that the DCL replica would be local.

 Atomic, eventually-consistent batches
 -

 Key: CASSANDRA-4285
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4285
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis

 I discussed this in the context of triggers (CASSANDRA-1311) but it's useful 
 as a standalone feature as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4291) Oversize integer in CQL throws NumberFormatException

2012-05-28 Thread Dave Brosius (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dave Brosius updated CASSANDRA-4291:


Attachment: 4291_catch_misc_excs_processing_statements.txt

catch misc runtime exceptions from cql.getStatement and return 
InvalidRequestExceptions with 'useful messages' for these so that the 
connection from cql to the server doesn't get into a bad state.

against trunk

 Oversize integer in CQL throws NumberFormatException
 

 Key: CASSANDRA-4291
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4291
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Scott Sadler
Priority: Minor
 Attachments: 4291_catch_misc_excs_processing_statements.txt


 In CQL, the parser does not handle an oversize Integer, the client socket get 
 closed and an exception is output in the log.
 {noformat}cqlsh:TEST1 select count(*) from Items limit 10;
 TSocket read 0 bytes
 cqlsh:TEST1 select count(*) from Items limit 1;
 TSocket read 0 bytes{noformat}
 {noformat}
 ERROR 02:51:28,600 Error occurred during processing of message.
 java.lang.NumberFormatException: For input string: 100
   at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
   at java.lang.Integer.parseInt(Integer.java:461)
   at java.lang.Integer.parseInt(Integer.java:499)
   at 
 org.apache.cassandra.cql.CqlParser.selectStatement(CqlParser.java:631)
   at org.apache.cassandra.cql.CqlParser.query(CqlParser.java:221)
   at 
 org.apache.cassandra.cql.QueryProcessor.getStatement(QueryProcessor.java:951)
   at 
 org.apache.cassandra.cql.QueryProcessor.process(QueryProcessor.java:873)
   at 
 org.apache.cassandra.thrift.CassandraServer.execute_cql_query(CassandraServer.java:1234)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3542)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3530)
   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
   at 
 org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:184)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
   at java.lang.Thread.run(Thread.java:662)
 {noformat}
 The INTEGER type in Cql.g matches digits but not to any particular limit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4285) Atomic, eventually-consistent batches

2012-05-28 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284469#comment-13284469
 ] 

Jonathan Ellis commented on CASSANDRA-4285:
---

We'd also want to restrict the DCL to nodes in the same datacenter.  No sense 
in sending this data over the WAN.  Since ReplicationStrategy only depends on 
the partition key, not the coordinator location, we probably need to do 
something sneaky like having a strategy that generates N replicas in each DC, 
then special case the DCL in StorageProxy to ignore replicas outside the 
coordinator's DC.

 Atomic, eventually-consistent batches
 -

 Key: CASSANDRA-4285
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4285
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis

 I discussed this in the context of triggers (CASSANDRA-1311) but it's useful 
 as a standalone feature as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-4291) Oversize integer in CQL throws NumberFormatException

2012-05-28 Thread Jonathan Ellis (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-4291:
--

Reviewer: xedin
Assignee: Dave Brosius
  Labels: cql  (was: )

 Oversize integer in CQL throws NumberFormatException
 

 Key: CASSANDRA-4291
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4291
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Scott Sadler
Assignee: Dave Brosius
Priority: Minor
  Labels: cql
 Attachments: 4291_catch_misc_excs_processing_statements.txt


 In CQL, the parser does not handle an oversize Integer, the client socket get 
 closed and an exception is output in the log.
 {noformat}cqlsh:TEST1 select count(*) from Items limit 10;
 TSocket read 0 bytes
 cqlsh:TEST1 select count(*) from Items limit 1;
 TSocket read 0 bytes{noformat}
 {noformat}
 ERROR 02:51:28,600 Error occurred during processing of message.
 java.lang.NumberFormatException: For input string: 100
   at 
 java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
   at java.lang.Integer.parseInt(Integer.java:461)
   at java.lang.Integer.parseInt(Integer.java:499)
   at 
 org.apache.cassandra.cql.CqlParser.selectStatement(CqlParser.java:631)
   at org.apache.cassandra.cql.CqlParser.query(CqlParser.java:221)
   at 
 org.apache.cassandra.cql.QueryProcessor.getStatement(QueryProcessor.java:951)
   at 
 org.apache.cassandra.cql.QueryProcessor.process(QueryProcessor.java:873)
   at 
 org.apache.cassandra.thrift.CassandraServer.execute_cql_query(CassandraServer.java:1234)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3542)
   at 
 org.apache.cassandra.thrift.Cassandra$Processor$execute_cql_query.getResult(Cassandra.java:3530)
   at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
   at 
 org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:184)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
   at java.lang.Thread.run(Thread.java:662)
 {noformat}
 The INTEGER type in Cql.g matches digits but not to any particular limit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2118) Provide failure modes if issues with the underlying filesystem of a node

2012-05-28 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284501#comment-13284501
 ] 

Jonathan Ellis commented on CASSANDRA-2118:
---

I don't think we need more than two options.  It's common for disks to become 
readable-not-writable, but I've never heard of them being 
writable-not-readable.  Assuming that we address CASSANDRA-2116 at the right 
level of granularity (the disk) there are two sane options:

# Continue as best we can in the face of errors: If we can't write to a disk, 
log an error, mark it bad-for-writes, and continue writing to other disks.  If 
we can't read from a disk, log an error, mark it bad-for-reads-and-writes, and 
continue serving reads from other disks
# Since option one implies that we can blithely serve up stale data when the 
most recent version was on the disk that is no longer accessible, I can see the 
utility of an option to halt on error (which would allow an operator to choose 
to decommission + rebootstrap to minimize the inconsistencies observed at 
CL.ONE)

 Provide failure modes if issues with the underlying filesystem of a node
 

 Key: CASSANDRA-2118
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2118
 Project: Cassandra
  Issue Type: Sub-task
Affects Versions: 0.8 beta 1
Reporter: Chris Goffinet
Assignee: Chris Goffinet
 Attachments: 
 0001-Provide-failure-modes-if-issues-with-the-underlying-.patch, 
 0001-Provide-failure-modes-if-issues-with-the-underlying-v2.patch, 
 0001-Provide-failure-modes-if-issues-with-the-underlying-v3.patch


 CASSANDRA-2116 introduces the ability to detect FS errors. Let's provide a 
 mode in cassandra.yaml so operators can decide that in the event of failure 
 what to do:
 1) standard - means continue on all errors (default)
 2) read - means only stop  gossip/rpc server if 'reads' fail from drive, 
 writes can fail but not kill gossip/rpc server
 3) readwrite - means stop gossip/rpc server if any read or write errors.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Try to stop all compaction upon Keyspace or ColumnFamily drop patch by Pavel Yaskevich; reviewed by Jonathan Ellis for CASSANDRA-4221

2012-05-28 Thread xedin 
Updated Branches:
  refs/heads/cassandra-1.1 46722cc69 - 1d9b7f559


Try to stop all compaction upon Keyspace or ColumnFamily drop
patch by Pavel Yaskevich; reviewed by Jonathan Ellis for CASSANDRA-4221


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1d9b7f55
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1d9b7f55
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1d9b7f55

Branch: refs/heads/cassandra-1.1
Commit: 1d9b7f5597f2cd090e312e270431ab56c81de9d9
Parents: 46722cc
Author: Pavel Yaskevich xe...@apache.org
Authored: Mon May 28 20:59:14 2012 +0300
Committer: Pavel Yaskevich xe...@apache.org
Committed: Mon May 28 20:59:14 2012 +0300

--
 CHANGES.txt|1 +
 .../apache/cassandra/cache/AutoSavingCache.java|7 +--
 src/java/org/apache/cassandra/db/DefsTable.java|6 ++-
 .../db/compaction/AbstractCompactionIterable.java  |5 --
 .../cassandra/db/compaction/CompactionInfo.java|   41 +--
 .../cassandra/db/compaction/CompactionManager.java |   24 +++--
 .../cassandra/db/index/SecondaryIndexBuilder.java  |2 -
 .../org/apache/cassandra/utils/FBUtilities.java|   12 
 8 files changed, 64 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d9b7f55/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0156fe2..2709320 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -63,6 +63,7 @@
  * (cql3) Adds simple access to column timestamp and ttl (CASSANDRA-4217)
  * (cql3) Fix range queries with secondary indexes (CASSANDRA-4257)
  * Better error messages from improper input in cli (CASSANDRA-3865)
+ * Try to stop all compaction upon Keyspace or ColumnFamily drop 
(CASSANDRA-4221)
 Merged from 1.0:
  * Fix super columns bug where cache is not updated (CASSANDRA-4190)
  * fix maxTimestamp to include row tombstones (CASSANDRA-4116)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d9b7f55/src/java/org/apache/cassandra/cache/AutoSavingCache.java
--
diff --git a/src/java/org/apache/cassandra/cache/AutoSavingCache.java 
b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
index aff7aa8..659e9ec 100644
--- a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
+++ b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
@@ -192,12 +192,7 @@ public class AutoSavingCacheK extends CacheKey, V 
extends InstrumentingCacheK
 else
 type = OperationType.UNKNOWN;
 
-info = new CompactionInfo(this.hashCode(),
-  Global,
-  cacheType.toString(),
-  type,
-  0,
-  estimatedTotalBytes);
+info = new CompactionInfo(type, 0, estimatedTotalBytes);
 }
 
 public CompactionInfo getCompactionInfo()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d9b7f55/src/java/org/apache/cassandra/db/DefsTable.java
--
diff --git a/src/java/org/apache/cassandra/db/DefsTable.java 
b/src/java/org/apache/cassandra/db/DefsTable.java
index 1b37de1..be23934 100644
--- a/src/java/org/apache/cassandra/db/DefsTable.java
+++ b/src/java/org/apache/cassandra/db/DefsTable.java
@@ -35,6 +35,7 @@ import org.apache.avro.io.DecoderFactory;
 import org.apache.avro.specific.SpecificDatumReader;
 import org.apache.avro.specific.SpecificRecord;
 import org.apache.cassandra.config.*;
+import org.apache.cassandra.db.compaction.CompactionManager;
 import org.apache.cassandra.db.filter.QueryFilter;
 import org.apache.cassandra.db.filter.QueryPath;
 import org.apache.cassandra.db.marshal.AsciiType;
@@ -43,7 +44,6 @@ import org.apache.cassandra.db.migration.avro.KsDef;
 import org.apache.cassandra.net.MessagingService;
 import org.apache.cassandra.service.MigrationManager;
 import org.apache.cassandra.service.StorageService;
-import org.apache.cassandra.thrift.CfDef;
 import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.FBUtilities;
 
@@ -474,6 +474,8 @@ public class DefsTable
 KSMetaData ksm = Schema.instance.getTableDefinition(ksName);
 String snapshotName = Table.getTimestampedSnapshotName(ksName);
 
+
CompactionManager.instance.stopCompactionFor(ksm.cfMetaData().values());
+
 // remove all cfs from the table instance.
 for (CFMetaData cfm : ksm.cfMetaData().values())
 {
@@ -507,6 +509,8 @@ public class DefsTable
 Schema.instance.purge(cfm);

[1/2] git commit: merge from 1.1

2012-05-28 Thread xedin 
Updated Branches:
  refs/heads/trunk 0c8c8559a - 4ec819eaa


merge from 1.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4ec819ea
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4ec819ea
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4ec819ea

Branch: refs/heads/trunk
Commit: 4ec819eaa696b550cad8418c79d48f3fbbf89bfb
Parents: 0c8c855 1d9b7f5
Author: Pavel Yaskevich xe...@apache.org
Authored: Mon May 28 21:30:56 2012 +0300
Committer: Pavel Yaskevich xe...@apache.org
Committed: Mon May 28 21:30:56 2012 +0300

--
 CHANGES.txt|1 +
 .../apache/cassandra/cache/AutoSavingCache.java|7 +--
 src/java/org/apache/cassandra/db/DefsTable.java|5 ++
 .../db/compaction/AbstractCompactionIterable.java  |5 --
 .../cassandra/db/compaction/CompactionInfo.java|   40 +--
 .../cassandra/db/compaction/CompactionManager.java |   24 +++--
 .../cassandra/db/index/SecondaryIndexBuilder.java  |2 -
 .../org/apache/cassandra/utils/FBUtilities.java|   12 
 8 files changed, 63 insertions(+), 33 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4ec819ea/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4ec819ea/src/java/org/apache/cassandra/cache/AutoSavingCache.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4ec819ea/src/java/org/apache/cassandra/db/DefsTable.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4ec819ea/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
--
diff --cc 
src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
index 72eca8e,1eb4e9b..41441df
--- 
a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
+++ 
b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionIterable.java
@@@ -7,18 -9,18 +7,16 @@@
   * License); you may not use this file except in compliance
   * with the License.  You may obtain a copy of the License at
   *
 - *   http://www.apache.org/licenses/LICENSE-2.0
 - *
 - * Unless required by applicable law or agreed to in writing,
 - * software distributed under the License is distributed on an
 - * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 - * KIND, either express or implied.  See the License for the
 - * specific language governing permissions and limitations
 - * under the License.
 + * http://www.apache.org/licenses/LICENSE-2.0
   *
 + * Unless required by applicable law or agreed to in writing, software
 + * distributed under the License is distributed on an AS IS BASIS,
 + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 + * See the License for the specific language governing permissions and
 + * limitations under the License.
   */
 -
 +package org.apache.cassandra.db.compaction;
  
- import java.io.IOException;
- import java.util.ArrayList;
  import java.util.List;
  
  import org.slf4j.Logger;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4ec819ea/src/java/org/apache/cassandra/db/compaction/CompactionInfo.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionInfo.java
index 374d895,17d098b..02b2433
--- a/src/java/org/apache/cassandra/db/compaction/CompactionInfo.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionInfo.java
@@@ -21,7 -22,8 +21,9 @@@ import java.io.Serializable
  import java.util.HashMap;
  import java.util.Map;
  
+ import org.apache.cassandra.config.CFMetaData;
+ import org.apache.cassandra.config.Schema;
 +import org.apache.cassandra.service.StorageService;
  
  /** Implements serializable to allow structured info to be returned via JMX. 
*/
  public final class CompactionInfo implements Serializable

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4ec819ea/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4ec819ea/src/java/org/apache/cassandra/db/index/SecondaryIndexBuilder.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/4ec819ea/src/java/org/apache/cassandra/utils/FBUtilities.java
--

[2/2] git commit: Try to stop all compaction upon Keyspace or ColumnFamily drop patch by Pavel Yaskevich; reviewed by Jonathan Ellis for CASSANDRA-4221

2012-05-28 Thread xedin 
Try to stop all compaction upon Keyspace or ColumnFamily drop
patch by Pavel Yaskevich; reviewed by Jonathan Ellis for CASSANDRA-4221


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1d9b7f55
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1d9b7f55
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1d9b7f55

Branch: refs/heads/trunk
Commit: 1d9b7f5597f2cd090e312e270431ab56c81de9d9
Parents: 46722cc
Author: Pavel Yaskevich xe...@apache.org
Authored: Mon May 28 20:59:14 2012 +0300
Committer: Pavel Yaskevich xe...@apache.org
Committed: Mon May 28 20:59:14 2012 +0300

--
 CHANGES.txt|1 +
 .../apache/cassandra/cache/AutoSavingCache.java|7 +--
 src/java/org/apache/cassandra/db/DefsTable.java|6 ++-
 .../db/compaction/AbstractCompactionIterable.java  |5 --
 .../cassandra/db/compaction/CompactionInfo.java|   41 +--
 .../cassandra/db/compaction/CompactionManager.java |   24 +++--
 .../cassandra/db/index/SecondaryIndexBuilder.java  |2 -
 .../org/apache/cassandra/utils/FBUtilities.java|   12 
 8 files changed, 64 insertions(+), 34 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d9b7f55/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0156fe2..2709320 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -63,6 +63,7 @@
  * (cql3) Adds simple access to column timestamp and ttl (CASSANDRA-4217)
  * (cql3) Fix range queries with secondary indexes (CASSANDRA-4257)
  * Better error messages from improper input in cli (CASSANDRA-3865)
+ * Try to stop all compaction upon Keyspace or ColumnFamily drop 
(CASSANDRA-4221)
 Merged from 1.0:
  * Fix super columns bug where cache is not updated (CASSANDRA-4190)
  * fix maxTimestamp to include row tombstones (CASSANDRA-4116)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d9b7f55/src/java/org/apache/cassandra/cache/AutoSavingCache.java
--
diff --git a/src/java/org/apache/cassandra/cache/AutoSavingCache.java 
b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
index aff7aa8..659e9ec 100644
--- a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
+++ b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
@@ -192,12 +192,7 @@ public class AutoSavingCacheK extends CacheKey, V 
extends InstrumentingCacheK
 else
 type = OperationType.UNKNOWN;
 
-info = new CompactionInfo(this.hashCode(),
-  Global,
-  cacheType.toString(),
-  type,
-  0,
-  estimatedTotalBytes);
+info = new CompactionInfo(type, 0, estimatedTotalBytes);
 }
 
 public CompactionInfo getCompactionInfo()

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d9b7f55/src/java/org/apache/cassandra/db/DefsTable.java
--
diff --git a/src/java/org/apache/cassandra/db/DefsTable.java 
b/src/java/org/apache/cassandra/db/DefsTable.java
index 1b37de1..be23934 100644
--- a/src/java/org/apache/cassandra/db/DefsTable.java
+++ b/src/java/org/apache/cassandra/db/DefsTable.java
@@ -35,6 +35,7 @@ import org.apache.avro.io.DecoderFactory;
 import org.apache.avro.specific.SpecificDatumReader;
 import org.apache.avro.specific.SpecificRecord;
 import org.apache.cassandra.config.*;
+import org.apache.cassandra.db.compaction.CompactionManager;
 import org.apache.cassandra.db.filter.QueryFilter;
 import org.apache.cassandra.db.filter.QueryPath;
 import org.apache.cassandra.db.marshal.AsciiType;
@@ -43,7 +44,6 @@ import org.apache.cassandra.db.migration.avro.KsDef;
 import org.apache.cassandra.net.MessagingService;
 import org.apache.cassandra.service.MigrationManager;
 import org.apache.cassandra.service.StorageService;
-import org.apache.cassandra.thrift.CfDef;
 import org.apache.cassandra.utils.ByteBufferUtil;
 import org.apache.cassandra.utils.FBUtilities;
 
@@ -474,6 +474,8 @@ public class DefsTable
 KSMetaData ksm = Schema.instance.getTableDefinition(ksName);
 String snapshotName = Table.getTimestampedSnapshotName(ksName);
 
+
CompactionManager.instance.stopCompactionFor(ksm.cfMetaData().values());
+
 // remove all cfs from the table instance.
 for (CFMetaData cfm : ksm.cfMetaData().values())
 {
@@ -507,6 +509,8 @@ public class DefsTable
 Schema.instance.purge(cfm);
 Schema.instance.setTableDefinition(makeNewKeyspaceDefinition(ksm, 
cfm));
 
+

[jira] [Commented] (CASSANDRA-2116) Separate out filesystem errors from generic IOErrors

2012-05-28 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284541#comment-13284541
 ] 

Jonathan Ellis commented on CASSANDRA-2116:
---

I think you're probably right.  Ideally what I'd like is something like this:

- Code that knows what disk is involved throws an FSReadError/FSWriteError that 
*records the disk/volume in question*
- Notably this will not necessarily be the lowest-level code, which often just 
takes a DataInput or DataOutput interface.  We'll want to declare the checked 
IOException at that level to make sure we handle it higher up
- Our global uncaught exception handler can mark the disks in question bad 
(exactly where CASSANDRA-2118 looks to terminate things; could handle this over 
there to go with the existing division of code)
- As for the request that hit the exception in the first place, we just allow 
that to fail + timeout normally; little benefit to be had by complicating that 
further (which is the approach taken here and 2118; I just note it to be 
explicit)

 Separate out filesystem errors from generic IOErrors
 

 Key: CASSANDRA-2116
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2116
 Project: Cassandra
  Issue Type: Improvement
Reporter: Chris Goffinet
Priority: Minor
 Fix For: 1.2

 Attachments: 
 0001-Separate-out-filesystem-errors-from-generic-IOErrors.patch


 We throw IOErrors everywhere today in the codebase. We should separate out 
 specific errors such as (reading, writing) from filesystem into FSReadError 
 and FSWriteError. This makes it possible in the next ticket to allow certain 
 failure modes (kill the server if reads or writes fail to disk).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-4292) Per-disk I/O queues

2012-05-28 Thread Jonathan Ellis (JIRA) 
Jonathan Ellis created CASSANDRA-4292:
-

 Summary: Per-disk I/O queues
 Key: CASSANDRA-4292
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4292
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Jonathan Ellis
Priority: Minor


As noted in CASSANDRA-809, we have a certain amount of flush (and compaction) 
threads, which mix and match disk volumes indiscriminately.  It may be worth 
creating a tight thread - disk affinity, to prevent unnecessary conflict at 
that level.

OTOH as SSDs become more prevalent this becomes a non-issue.  Unclear how much 
pain this actually causes in practice in the meantime.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-809) Full disk can result in being marked down

2012-05-28 Thread Jonathan Ellis (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-809.
--

   Resolution: Duplicate
Fix Version/s: (was: 1.2)

Created https://issues.apache.org/jira/browse/CASSANDRA-4292 to pursue the 
thread-per-disk idea.  CASSANDRA-2116 and CASSANDRA-2118 address the issue of 
what to do when disks error out.

 Full disk can result in being marked down
 -

 Key: CASSANDRA-809
 URL: https://issues.apache.org/jira/browse/CASSANDRA-809
 Project: Cassandra
  Issue Type: Bug
Reporter: Ryan King
Priority: Minor

 We had a node file up the disk under one of two data directories. The result 
 was that the node stopped making progress. The problem appears to be this 
 (I'll update with more details as we find them):
 When new tasks are put onto most queues in Cassandra, if there isn't a thread 
 in the pool to handle the task immediately, the task in run in the caller's 
 thread
 (org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor:69 sets the 
 caller-runs policy).  The queue in question here is the queue that manages 
 flushes, which is enqueued to from various places in our code (and therefore 
 likely from multiple threads). Assuming that the full disk meant that no 
 threads doing flushing could make progress (it appears that way) eventually 
 any thread that calls the flush code would become stalled.
 Assuming our analysis is right (and we're still looking into it) we need to 
 make a change. Here's a proposal so far:
 SHORT TERM:
 * change the  TheadPoolExecutor policy to not be caller runs. This will let 
 other threads make progress in the event that one pool is stalled
 LONG TERM
 * It appears that there are n threads for n data directories that we flush 
 to, but they're not dedicated to a data directory. We should have a thread 
 per data directory and have that thread dedicated to that directory
 * Perhaps we could use the failure detector on disks?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-1311) Triggers

2012-05-28 Thread Jeremy Hanna (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284547#comment-13284547
 ] 

Jeremy Hanna commented on CASSANDRA-1311:
-

I think that's CASSANDRA-4285

 Triggers
 

 Key: CASSANDRA-1311
 URL: https://issues.apache.org/jira/browse/CASSANDRA-1311
 Project: Cassandra
  Issue Type: New Feature
Reporter: Maxim Grinev
 Fix For: 1.2

 Attachments: HOWTO-PatchAndRunTriggerExample-update1.txt, 
 HOWTO-PatchAndRunTriggerExample.txt, ImplementationDetails-update1.pdf, 
 ImplementationDetails.pdf, trunk-967053.txt, trunk-984391-update1.txt, 
 trunk-984391-update2.txt


 Asynchronous triggers is a basic mechanism to implement various use cases of 
 asynchronous execution of application code at database side. For example to 
 support indexes and materialized views, online analytics, push-based data 
 propagation.
 Please find the motivation, triggers description and list of applications:
 http://maxgrinev.com/2010/07/23/extending-cassandra-with-asynchronous-triggers/
 An example of using triggers for indexing:
 http://maxgrinev.com/2010/07/23/managing-indexes-in-cassandra-using-async-triggers/
 Implementation details are attached.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira