Re: Write performance compared to Postgresql

2012-04-04 Thread Jeff Williams
On three machines on the same subnet as the two cassandra nodes.

On Apr 3, 2012, at 6:40 PM, Collard, David L (Dave) wrote:

 Where is your client running?
 
 -Original Message-
 From: Jeff Williams [mailto:je...@wherethebitsroam.com] 
 Sent: Tuesday, April 03, 2012 11:09 AM
 To: user@cassandra.apache.org
 Subject: Re: Write performance compared to Postgresql
 
 Vitalii,
 
 Yep, that sounds like a good idea. Do you have any more information about how 
 you're doing that? Which client?
 
 Because even with 3 concurrent client nodes, my single postgresql server is 
 still out performing my 2 node cassandra cluster, although the gap is 
 narrowing.
 
 Jeff
 
 On Apr 3, 2012, at 4:08 PM, Vitalii Tymchyshyn wrote:
 
 Note that having tons of TCP connections is not good. We are using async 
 client to issue multiple calls over single connection at same time. You can 
 do the same.
 
 Best regards, Vitalii Tymchyshyn.
 
 03.04.12 16:18, Jeff Williams написав(ла):
 Ok, so you think the write speed is limited by the client and protocol, 
 rather than the cassandra backend? This sounds reasonable, and fits with 
 our use case, as we will have several servers writing. However, a bit 
 harder to test!
 
 Jeff
 
 On Apr 3, 2012, at 1:27 PM, Jake Luciani wrote:
 
 Hi Jeff,
 
 Writing serially over one connection will be slower. If you run many 
 threads hitting the server at once you will see throughput improve.
 
 Jake
 
 
 
 On Apr 3, 2012, at 7:08 AM, Jeff Williamsje...@wherethebitsroam.com  
 wrote:
 
 Hi,
 
 I am looking at cassandra for a logging application. We currently log to 
 a Postgresql database.
 
 I set up 2 cassandra servers for testing. I did a benchmark where I had 
 100 hashes representing logs entries, read from a json file. I then 
 looped over these to do 10,000 log inserts. I repeated the same writing 
 to a postgresql instance on one of the cassandra servers. The script is 
 attached. The cassandra writes appear to perform a lot worse. Is this 
 expected?
 
 jeff@transcoder01:~$ ruby cassandra-bm.rb
 cassandra
 3.17   0.48   3.65 ( 12.032212)
 jeff@transcoder01:~$ ruby cassandra-bm.rb
 postgres
 2.14   0.33   2.47 (  7.002601)
 
 Regards,
 Jeff
 
 cassandra-bm.rb
 
 



Re: System keyspace leak?

2012-04-04 Thread Sylvain Lebresne
On Wed, Apr 4, 2012 at 3:14 AM, David Leimbach leim...@gmail.com wrote:
 Well I just found this:

 http://wiki.apache.org/cassandra/LiveSchemaUpdates

 which explains a ton...  It looks like this particular Column Family will
 grow infinitely (it's just one row with a column per migration), so if I'm
 pounding on my Cassandra node with CREATE/DROP activity, I'm going to make
 VERY wide row.

 That tells me enough to say don't do that!  :-)

All that is true in 1.0 but know that we've fixed it in the upcoming
1.1 (http://www.datastax.com/dev/blog/the-schema-management-renaissance).

--
Sylvain



 Dave

 On Tue, Apr 3, 2012 at 6:00 PM, David Leimbach leim...@gmail.com wrote:

 I've been trying to understand the overhead of create/drop keyspace on
 Cassandra 1.0.8.  It's not free, especially when I've managed to drive up
 the LiveDiskSpaceUsed for the Migrations CF in the system keyspace up to
 over 12 MB of disk.

 I've tried doing nodetool -h localhost repair system and other nodetool
 commands to try to compact the SSTables involved with it, but it never wants
 to let go of that slowly growing space.

 The Cassandra node in question is in a ring of size 1.

 Other than clobbering my data directory, how do I get my space back?  Is
 it natural for this to grow seemingly infinitely (even though it's pretty
 small increments), or did I find a bug?

 The reason I ask is I need to run unit tests in a shared developer
 infrastructure with Cassandra, and we were having a little trouble with
 TRUNCATE on column families, but that might have been environmental (I've
 not looked deeply into it).

 Which is less expensive?  Create/Drop, or Truncate?  I don't expect
 Truncate to swell the Migration Column Family because it tracks (seemingly)
 schema changes.

 Dave





JMX APIs thru MX4J

2012-04-04 Thread Andrea Tuccia
Hello,
I'm working on a fork of Sébastien Giroux Cassandra Cluster Admin and I
wish to pull my contributions to the parent code.

Here is my repository:

https://github.com/atuccia/Cassandra-Cluster-Admin

...and Sébastien Giroux one:

https://github.com/sebgiroux/Cassandra-Cluster-Admin

I wanna add the same functionalities of OpsCenter (or the common
functions available thru the command line nodetool) as cleanup, compact,
repair, drain, decommission and so on...

I'm stuck, gone in trouble with MX4J.

http://192.168.10.91/invoke?operation=forceTableRepairobjectname=org.apache.cassandra.db%3Atype%3DStorageServicevalue0=type0=java.lang.Stringtype1=[Ljava.lang.String%3B

MBean operation: invoke method on MBean
org.apache.cassandra.db:type=StorageService
Error during MBean operation invocationMessage: count of parameter types
doesn't match count of parameter values

http://192.168.10.91/invoke?operation=forceTableRepairobjectname=org.apache.cassandra.db%3Atype%3DStorageServicevalue0=type0=java.lang.Stringtype1=[Ljava.lang.String%3Bvalue1=

MBean operation: invoke method on MBean
org.apache.cassandra.db:type=StorageService
Error during MBean operation invocationMessage: Parameter 1: cannot be
converted to type [Ljava.lang.String;

http://192.168.10.91/invoke?operation=forceTableRepairobjectname=org.apache.cassandra.db%3Atype%3DStorageService

MBean operation: invoke method on MBean
org.apache.cassandra.db:type=StorageService
Error during MBean operation invocationMessage: Operation singature has
no match in the MBean

...so, how can I call the APIs (apparently) with optional parameters?
What is that strange definition of the type [Ljava.lang.String; for
some params?

Thanks in advance!

Andrea Tuccia



Re: JMX APIs thru MX4J

2012-04-04 Thread Flavio Baronti

Hi,

I can't help you with your other questions, but the type [Ljava.lang.String; 
is an array of String objects.
More info here http://en.wikipedia.org/wiki/Java_Native_Interface#Mapping_types

Flavio


Il 4/4/2012 10:04 AM, Andrea Tuccia ha scritto:

Hello,
I'm working on a fork of Sébastien Giroux Cassandra Cluster Admin and I
wish to pull my contributions to the parent code.

Here is my repository:

https://github.com/atuccia/Cassandra-Cluster-Admin

...and Sébastien Giroux one:

https://github.com/sebgiroux/Cassandra-Cluster-Admin

I wanna add the same functionalities of OpsCenter (or the common
functions available thru the command line nodetool) as cleanup, compact,
repair, drain, decommission and so on...

I'm stuck, gone in trouble with MX4J.

http://192.168.10.91/invoke?operation=forceTableRepairobjectname=org.apache.cassandra.db%3Atype%3DStorageServicevalue0=type0=java.lang.Stringtype1=[Ljava.lang.String%3B

MBean operation: invoke method on MBean
org.apache.cassandra.db:type=StorageService
Error during MBean operation invocationMessage: count of parameter types
doesn't match count of parameter values

http://192.168.10.91/invoke?operation=forceTableRepairobjectname=org.apache.cassandra.db%3Atype%3DStorageServicevalue0=type0=java.lang.Stringtype1=[Ljava.lang.String%3Bvalue1=

MBean operation: invoke method on MBean
org.apache.cassandra.db:type=StorageService
Error during MBean operation invocationMessage: Parameter 1: cannot be
converted to type [Ljava.lang.String;

http://192.168.10.91/invoke?operation=forceTableRepairobjectname=org.apache.cassandra.db%3Atype%3DStorageService

MBean operation: invoke method on MBean
org.apache.cassandra.db:type=StorageService
Error during MBean operation invocationMessage: Operation singature has
no match in the MBean

...so, how can I call the APIs (apparently) with optional parameters?
What is that strange definition of the type [Ljava.lang.String; for
some params?

Thanks in advance!

Andrea Tuccia






Re: tombstones problem with 1.0.8

2012-04-04 Thread Ross Black
Hi Jonathan,

Thanks for your response.

We were running a compact at least once a day over the keyspace.  The
gc_grace was set to only 1 hour, so from what you said I would expect that
tombstones should be deleted after max 3 days.
When I inspected the data in the SSTables after a compact, some rows
contained millions of tombstones with many having timestamps indicating
they were older than 2 weeks.

We have recently migrated to a new schema design that avoids deleting
columns or rows.
I ran another compact once data was not being added to the new keyspace (it
only ever added new columns, never modified existing or deleted columns).
That compact deleted all of the existing tombstones, reducing our data from
~250G down to ~30G.
I assume there must have been something strange in our keyspace that
prevented tombstones from being deleted just while data was being added.

We longer delete columns so the issue is no longer critical for us, but I
am still curious as to what/why the issue was occurring just in case we
start deleting columns again ;-)

Thanks,
Ross



On 4 April 2012 09:10, Jonathan Ellis jbel...@gmail.com wrote:

 Removing expired columns actually requires two compaction passes: one
 to turn the expired column into a tombstone; one to remove the
 tombstone after gc_grace_seconds. (See
 https://issues.apache.org/jira/browse/CASSANDRA-1537.)

 Perhaps CASSANDRA-2786 was causing things to (erroneously) be cleaned
 up early enough that this helped you out in 0.8.2?

 On Wed, Mar 21, 2012 at 8:38 PM, Ross Black ross.w.bl...@gmail.com
 wrote:
  Hi,
 
  We recently moved from 0.8.2 to 1.0.8 and the behaviour seems to have
  changed so that tombstones are now not being deleted.
 
  Our application continually adds and removes columns from Cassandra.  We
  have set a short gc_grace time (3600) since our application would
  automatically delete zombies if they appear.
  Under 0.8.2, the tombstones remained at a relatively constant number.
  Under 1.0.8, the tombstones have been continually increasing so that they
  exceed the size of our real data (at this stage we have over 100G of
  tombstones).
  Even after running a full compact the new compacted SSTable contains a
  massive number of tombstones, many that are several weeks old.
 
  Have I missed some new configuration option to allow deletion of
 tombstones?
 
  I also noticed that one of the changes between 0.8.2 and 1.0.8 was
  https://issues.apache.org/jira/browse/CASSANDRA-2786 which changed code
 to
  avoid dropping tombstones when they might still be needed to shadow
 data in
  another sstable.
  Could this be having an impact since we continually add and remove
 columns
  even while a major compact is executing?
 
 
  Thanks,
  Ross
 



 --
 Jonathan Ellis
 Project Chair, Apache Cassandra
 co-founder of DataStax, the source for professional Cassandra support
 http://www.datastax.com



Re: cassandra 1.08 on java7 and win7

2012-04-04 Thread Gopala Krishna Gundeboyina
I have a cluster of four nodes all running on single machine. If I just do
create keyspace DEMO CLI gives me the same error that original post says.

If I use create the keyspace with the options you mentioned it works just
fine but the proble is it is not getting reflected on other nodes in the
cluster.

-Gopala

On Wed, Apr 4, 2012 at 10:30 AM, puneet loya puneetl...@gmail.com wrote:

 thank u Gopala :) Der is no issue with it.. Might be i was typing
 something wrong.,. Minor mistake :)


 On Tue, Apr 3, 2012 at 11:51 PM, Gopala f2001...@gmail.com wrote:

 puneet loya puneetloya at gmail.com writes:

 
 
  create keyspace DEMO
 
  with placement_strategy =
 'org.apache.cassandra.locator.NetworkTopologyStrategy'
 
  and strategy_options=[{datacenter1:1}];
 
  try it n check if it executes



 Hi Puneet,
 I have same issue. Running the command you mentioned below works for me.
 What is
 the issue here?

 Gopala





server down

2012-04-04 Thread Michael Vaknine
I am recently having problems with 1 of my 4ty cluster servers.

I would appreciate any help

 

ERROR 11:23:14,331 Fatal exception in thread Thread[MutationStage:19,5,main]

225 java.lang.InternalError: a fault occurred in a recent unsafe memory
access operation in compiled Java code

226 at
org.apache.cassandra.io.util.AbstractDataInput.readInt(AbstractDataInput.jav
a:196)

227 at
org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java
:348)

228 at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:1
20)

229 at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:8
3)

230 at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:7
3)

231 at
org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:3
7)

232 at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleS
liceReader.java:79)

233 at
org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleS
liceReader.java:40)

234 at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator
.java:140)

235 at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135
)

236 at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableS
liceIterator.java:107)

237 at
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.jav
a:145)

238 at
org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.jav
a:124)

239 at
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator
.java:98)

240 at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator
.java:140)

241 at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135
)

242 at
org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQ
ueryFilter.java:116)

243 at
org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:1
44)

244 at
org.apache.cassandra.db.CollationController.collectAllData(CollationControll
er.java:227)

245 at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationCont
roller.java:62)

246 at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilySto
re.java:1274)

247 at
org.apache.cassandra.db.ColumnFamilyStore.cacheRow(ColumnFamilyStore.java:11
41)

248 at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.
java:1170)

249 at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.
java:1122)

250 at
org.apache.cassandra.db.Table.readCurrentIndexedColumns(Table.java:504)

251 at org.apache.cassandra.db.Table.apply(Table.java:441)

252 at
org.apache.cassandra.db.commitlog.CommitLog$2.runMayThrow(CommitLog.java:338
)

253 at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)

254 at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)

255 at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)

256 at java.util.concurrent.FutureTask.run(FutureTask.java:138)

257 at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja
va:886)

258 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9
08)

259 at java.lang.Thread.run(Thread.java:619)

260  INFO 11:23:17,640 Finished reading
/var/lib/cassandra/commitlog/CommitLog-1333465745624.log

261 ERROR 11:23:17,641 Exception encountered during startup

262 java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.InternalError: a fault occurred in a recent unsafe memory access
operation in compiled Java code

263 at
org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:508)

 

Thanks

Michael



Re: data size difference between supercolumn and regular column

2012-04-04 Thread Yiming Sun
Cool, I will look into this new leveled compaction strategy and give it a
try.

BTW, Aaron, I think the last word of your message meant to say
compression, correct?

-- Y.

On Mon, Apr 2, 2012 at 9:37 PM, aaron morton aa...@thelastpickle.comwrote:

 If you have a workload with overwrites you will end up with some data
 needing compaction. Running a nightly manual compaction would remove this,
 but it will also soak up some IO so it may not be the best solution.

 I do not know if Leveled compaction would result in a smaller disk load
 for the same workload.

 I agree with other people, turn on compaction.

 Cheers

   -
 Aaron Morton
 Freelance Developer
 @aaronmorton
 http://www.thelastpickle.com

 On 3/04/2012, at 9:19 AM, Yiming Sun wrote:

 Yup Jeremiah, I learned a hard lesson on how cassandra behaves when it
 runs out of disk space :-S.I didn't try the compression, but when it
 ran out of disk space, or near running out, compaction would fail because
 it needs space to create some tmp data files.

 I shall get a tatoo that says keep it around 50% -- this is valuable tip.

 -- Y.

 On Sun, Apr 1, 2012 at 11:25 PM, Jeremiah Jordan 
 jeremiah.jor...@morningstar.com wrote:

  Is that 80% with compression?  If not, the first thing to do is turn on
 compression.  Cassandra doesn't behave well when it runs out of disk space.
  You really want to try and stay around 50%,  60-70% works, but only if it
 is spread across multiple column families, and even then you can run into
 issues when doing repairs.

  -Jeremiah



  On Apr 1, 2012, at 9:44 PM, Yiming Sun wrote:

 Thanks Aaron.  Well I guess it is possible the data files from
 sueprcolumns could've been reduced in size after compaction.

  This bring yet another question.  Say I am on a shoestring budget and
 can only put together a cluster with very limited storage space.  The first
 iteration of pushing data into cassandra would drive the disk usage up into
 the 80% range.  As time goes by, there will be updates to the data, and
 many columns will be overwritten.  If I just push the updates in, the disks
 will run out of space on all of the cluster nodes.  What would be the best
 way to handle such a situation if I cannot to buy larger disks? Do I need
 to delete the rows/columns that are going to be updated, do a compaction,
 and then insert the updates?  Or is there a better way?  Thanks

  -- Y.

 On Sat, Mar 31, 2012 at 3:28 AM, aaron morton aa...@thelastpickle.comwrote:

   does cassandra 1.0 perform some default compression?

  No.

  The on disk size depends to some degree on the work load.

  If there are a lot of overwrites or deleted you may have rows/columns
 that need to be compacted. You may have some big old SSTables that have not
 been compacted for a while.

  There is some overhead involved in the super columns: the super col
 name, length of the name and the number of columns.

  Cheers

 -
 Aaron Morton
 Freelance Developer
 @aaronmorton
 http://www.thelastpickle.com

  On 29/03/2012, at 9:47 AM, Yiming Sun wrote:

 Actually, after I read an article on cassandra 1.0 compression just now
 (
 http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-compression),
 I am more puzzled.  In our schema, we didn't specify any compression
 options -- does cassandra 1.0 perform some default compression? or is the
 data reduction purely because of the schema change?  Thanks.

  -- Y.

 On Wed, Mar 28, 2012 at 4:40 PM, Yiming Sun yiming@gmail.comwrote:

 Hi,

  We are trying to estimate the amount of storage we need for a
 production cassandra cluster.  While I was doing the calculation, I noticed
 a very dramatic difference in terms of storage space used by cassandra data
 files.

  Our previous setup consists of a single-node cassandra 0.8.x with no
 replication, and the data is stored using supercolumns, and the data files
 total about 534GB on disk.

  A few weeks ago, I put together a cluster consisting of 3 nodes
 running cassandra 1.0 with replication factor of 2, and the data is
 flattened out and stored using regular columns.  And the aggregated data
 file size is only 488GB (would be 244GB if no replication).

  This is a very dramatic reduction in terms of storage needs, and is
 certainly good news in terms of how much storage we need to provision.
  However, because of the dramatic reduction, I also would like to make sure
 it is absolutely correct before submitting it - and also get a sense of why
 there was such a difference. -- I know cassandra 1.0 does data compression,
 but does the schema change from supercolumn to regular column also help
 reduce storage usage?  Thanks.

  -- Y.










Linux Filesystem for Cassandra

2012-04-04 Thread Oleg Proudnikov
Hi,

There has been no discussion on this list on the choice of a Linux file system 
for Cassandra. Does this choice make a difference? Would you, please share, 
what filesystem you are using?

Thank you very much,
Oleg



Re: Linux Filesystem for Cassandra

2012-04-04 Thread Radim Kolar



 Would you, please share, what filesystem you are using?


zfs 28


Re: data size difference between supercolumn and regular column

2012-04-04 Thread Watanabe Maki
LeveledCompaction will use less disk space(load), but need more IO.
If your traffic is too high for your disk, you will have many pending 
compaction tasks, and large number of sstables which wait to be compacted.
Also the default sstable_size_in_mb  (5MB) will be too small for large data 
set. You should better to have test iteration with different size configuration.
Don't forget to unlimit number of file descriptors, and monitor tpstats and 
iostat.

maki

From iPhone


On 2012/04/04, at 22:19, Yiming Sun yiming@gmail.com wrote:

 Cool, I will look into this new leveled compaction strategy and give it a try.
 
 BTW, Aaron, I think the last word of your message meant to say compression, 
 correct?
 
 -- Y.
 
 On Mon, Apr 2, 2012 at 9:37 PM, aaron morton aa...@thelastpickle.com wrote:
 If you have a workload with overwrites you will end up with some data needing 
 compaction. Running a nightly manual compaction would remove this, but it 
 will also soak up some IO so it may not be the best solution. 
 
 I do not know if Leveled compaction would result in a smaller disk load for 
 the same workload. 
 
 I agree with other people, turn on compaction. 
 
 Cheers
 
 -
 Aaron Morton
 Freelance Developer
 @aaronmorton
 http://www.thelastpickle.com
 
 On 3/04/2012, at 9:19 AM, Yiming Sun wrote:
 
 Yup Jeremiah, I learned a hard lesson on how cassandra behaves when it runs 
 out of disk space :-S.I didn't try the compression, but when it ran out 
 of disk space, or near running out, compaction would fail because it needs 
 space to create some tmp data files.
 
 I shall get a tatoo that says keep it around 50% -- this is valuable tip.
 
 -- Y.
 
 On Sun, Apr 1, 2012 at 11:25 PM, Jeremiah Jordan 
 jeremiah.jor...@morningstar.com wrote:
 Is that 80% with compression?  If not, the first thing to do is turn on 
 compression.  Cassandra doesn't behave well when it runs out of disk space.  
 You really want to try and stay around 50%,  60-70% works, but only if it is 
 spread across multiple column families, and even then you can run into 
 issues when doing repairs.
 
 -Jeremiah
 
 
 
 On Apr 1, 2012, at 9:44 PM, Yiming Sun wrote:
 
 Thanks Aaron.  Well I guess it is possible the data files from sueprcolumns 
 could've been reduced in size after compaction.
 
 This bring yet another question.  Say I am on a shoestring budget and can 
 only put together a cluster with very limited storage space.  The first 
 iteration of pushing data into cassandra would drive the disk usage up into 
 the 80% range.  As time goes by, there will be updates to the data, and 
 many columns will be overwritten.  If I just push the updates in, the disks 
 will run out of space on all of the cluster nodes.  What would be the best 
 way to handle such a situation if I cannot to buy larger disks? Do  I need 
 to delete the rows/columns that are going to be updated, do a compaction, 
 and then insert the updates?  Or is there a better way?  Thanks
 
 -- Y.
 
 On Sat, Mar 31, 2012 at 3:28 AM, aaron morton aa...@thelastpickle.com 
 wrote:
 does cassandra 1.0 perform some default compression? 
 No. 
 
 The on disk size depends to some degree on the work load. 
 
 If there are a lot of overwrites or deleted you may have rows/columns that 
 need to be compacted. You may have some big old SSTables that have not been 
 compacted for a while. 
 
 There is some overhead involved in the super columns: the super col name, 
 length of the name and the number of columns.  
 
 Cheers
 
 -
 Aaron Morton
 Freelance Developer
 @aaronmorton
 http://www.thelastpickle.com
 
 On 29/03/2012, at 9:47 AM, Yiming Sun wrote:
 
 Actually, after I read an article on cassandra 1.0 compression just now ( 
 http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-0-compression), 
 I am more puzzled.  In our schema, we didn't specify any compression 
 options -- does cassandra 1.0 perform some default compression? or is the 
 data reduction purely because of the schema change?  Thanks.
 
 -- Y.
 
 On Wed, Mar 28, 2012 at 4:40 PM, Yiming Sun yiming@gmail.com wrote:
 Hi,
 
 We are trying to estimate the amount of storage we need for a production 
 cassandra cluster.  While I was doing the calculation, I noticed a very 
 dramatic difference in terms of storage space used by cassandra data files.
 
 Our previous setup consists of a single-node cassandra 0.8.x with no 
 replication, and the data is stored using supercolumns, and the data files 
 total about 534GB on disk.
 
 A few weeks ago, I put together a cluster consisting of 3 nodes running 
 cassandra 1.0 with replication factor of 2, and the data is flattened out 
 and stored using regular columns.  And the aggregated data file size is 
 only 488GB (would be 244GB if no replication).
 
 This is a very dramatic reduction in terms of storage needs, and is 
 certainly good news in terms of how much storage we need to provision.  
 However, because of the dramatic reduction, I also 

Re: Linux Filesystem for Cassandra

2012-04-04 Thread Michael Shuler
On 04/04/2012 08:40 AM, Oleg Proudnikov wrote:
 There has been no discussion on this list on the choice of a Linux
 file system for Cassandra. Does this choice make a difference? Would
 you, please share, what filesystem you are using?

Hopefully this is some good reading on the topic:

https://www.google.com/search?q=xfs+site%3Ahttp%3A%2F%2Fmail-archives.apache.org%2Fmod_mbox%2Fcassandra-user

one of the more interesting considerations:
http://mail-archives.apache.org/mod_mbox/cassandra-user/201004.mbox/%3ch2y96b607d1004131614k5382b3a5ie899989d62921...@mail.gmail.com%3E

http://wiki.apache.org/cassandra/CassandraHardware

http://wiki.apache.org/cassandra/LargeDataSetConsiderations

http://www.datastax.com/dev/blog/questions-from-the-tokyo-cassandra-conference

-- 
Kind regards,
Michael Shuler


Re: counter column family

2012-04-04 Thread aaron morton
You may have better luck with Hector specific questions on the Hector User 
Group https://groups.google.com/forum/?fromgroups#!forum/hector-users

Cheers
 
-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 4/04/2012, at 5:54 PM, Tamar Fraenkel wrote:

 Hi!
 So, if I am using Hector, I need to do:
 cassandraHostConfigurator.setRetryDownedHosts(false)?
 
 How will this affect my application generally?
 
 Thanks
 
 Tamar Fraenkel 
 Senior Software Engineer, TOK Media 
 
 tokLogo.png
 
 ta...@tok-media.com
 Tel:   +972 2 6409736 
 Mob:  +972 54 8356490 
 Fax:   +972 2 5612956 
 
 
 
 
 
 On Tue, Mar 27, 2012 at 4:25 PM, R. Verlangen ro...@us2.nl wrote:
 You should use a connection pool without retries to prevent a single 
 increment of +1 have a result of e.g. +3.
 
 
 2012/3/27 Rishabh Agrawal rishabh.agra...@impetus.co.in
 You can even define how much increment you want. But let me just warn you, as 
 far my knowledge, it has consistency issues.
 
  
 From: puneet loya [mailto:puneetl...@gmail.com] 
 Sent: Tuesday, March 27, 2012 5:59 PM
 
 
 To: user@cassandra.apache.org
 Subject: Re: counter column family
 
  
 thanxx a ton :) :)
 
  
 the counter column family works synonymous as 'auto increment' in other 
 databases rite?
 
  
 I mean we have a column of  type integer which increments with every insert.
 
  
 Am i goin the rite way??
 
  
 please reply :)
 
 On Tue, Mar 27, 2012 at 5:50 PM, R. Verlangen ro...@us2.nl wrote:
 
 create column family MyCounterColumnFamily with 
 default_validation_class=CounterColumnType and key_validation_class=UTF8Type 
 and comparator=UTF8Type; 
 
  
 There you go! Keys must be utf8, as well as the column names. Of course you 
 can change those validators.
 
  
 Cheers!
 
  
 2012/3/27 puneet loya puneetl...@gmail.com
 
 Can u give an example of create column family with counter column in it. 
 
  
  
 Please reply
 
  
  
 Regards,
 
  
 Puneet Loya
 
 
 
  
 -- 
 With kind regards,
 
  
 Robin Verlangen
 
 www.robinverlangen.nl
 
  
  
 
 
 Impetus to sponsor and exhibit at Structure Data 2012, NY; Mar 21-22. Know 
 more about our Big Data quick-start program at the event. 
 
 New Impetus webcast ‘Cloud-enabled Performance Testing vis-à-vis On-premise’ 
 available at http://bit.ly/z6zT4L. 
 
 
 NOTE: This message may contain information that is confidential, proprietary, 
 privileged or otherwise protected by law. The message is intended solely for 
 the named addressee. If received in error, please destroy and notify the 
 sender. Any use of this email is prohibited when received in error. Impetus 
 does not represent, warrant and/or guarantee, that the integrity of this 
 communication has been maintained nor that the communication is free of 
 errors, virus, interception or interference.
 
 
 
 -- 
 With kind regards,
 
 Robin Verlangen
 www.robinverlangen.nl
 
 



Re: server down

2012-04-04 Thread aaron morton
What version of cassandra are you using ? 

What java vendor / version ? 

What OS vendor / version ? 

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 4/04/2012, at 11:33 PM, Michael Vaknine wrote:

 I am recently having problems with 1 of my 4ty cluster servers.
 I would appreciate any help
  
 ERROR 11:23:14,331 Fatal exception in thread Thread[MutationStage:19,5,main]
 225 java.lang.InternalError: a fault occurred in a recent unsafe memory 
 access operation in compiled Java code
 226 at 
 org.apache.cassandra.io.util.AbstractDataInput.readInt(AbstractDataInput.java:196)
 227 at 
 org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:348)
 228 at 
 org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:120)
 229 at 
 org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:83)
 230 at 
 org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:73)
 231 at 
 org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:37)
 232 at 
 org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
 233 at 
 org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:40)
 234 at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
 235 at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
 236 at 
 org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:107)
 237 at 
 org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:145)
 238 at 
 org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:124)
 239 at 
 org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
 240 at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
 241 at 
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
 242 at 
 org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:116)
 243 at 
 org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:144)
 244 at 
 org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:227)
 245 at 
 org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:62)
 246 at 
 org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1274)
 247 at 
 org.apache.cassandra.db.ColumnFamilyStore.cacheRow(ColumnFamilyStore.java:1141)
 248 at 
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1170)
 249 at 
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1122)
 250 at 
 org.apache.cassandra.db.Table.readCurrentIndexedColumns(Table.java:504)
 251 at org.apache.cassandra.db.Table.apply(Table.java:441)
 252 at 
 org.apache.cassandra.db.commitlog.CommitLog$2.runMayThrow(CommitLog.java:338)
 253 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
 254 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
 255 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 256 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 257 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 258 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 259 at java.lang.Thread.run(Thread.java:619)
 260  INFO 11:23:17,640 Finished reading 
 /var/lib/cassandra/commitlog/CommitLog-1333465745624.log
 261 ERROR 11:23:17,641 Exception encountered during startup
 262 java.lang.RuntimeException: java.util.concurrent.ExecutionException: 
 java.lang.InternalError: a fault occurred in a recent unsafe memory access 
 operation in compiled Java code
 263 at 
 org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:508)
  
 Thanks
 Michael



Re: Attempting to avoid fatal flush due to disk space

2012-04-04 Thread aaron morton
Is cleanupDirectoriesFailover able to delete the files ? 

When you get the error is the disk actually full ?

Can you narrow this down to i cannot delete the sstables? (and what platform 
are you on). 

Cheers
 
-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 5/04/2012, at 1:57 AM, Lewis John Mcgibbney wrote:

 Hi,
 
 When writing some test code (to test our Gora-Cassandra module) for Apache 
 Gora, I'm experiencing a major problem when trying to tearDown, flush and 
 basically clean everything up. The tests consist of running our test suite 
 against an embedded Cassandra instance using the Gora API to do all sorts of 
 tasks which may highkight where there are bugs in the code.
 
 Now regardless of whether the tests are failing or not, I get the log and 
 subsequent stack trace as below which quite clearly indicates that there is 
 insufficient disk to flush. However as I don't have the luxury of making 
 changes @ runtime I'm slightly stumped as to how to solve it as all I'm 
 working with is the Cassandra server which is initiated in setUpClass.
 
 My yaml is ripped directly from the Cassandra 1.0.2 distribution so there is 
 nothing out of place in there. However maybe there are some settings which I 
 have not configured?
 
 Please indicate if I need to provide some more information to paint a clearer 
 picture of the situation.
 
 When calling tearDownClass I have
 
 if (cassandraThread != null) {
 cassandraDaemon.stop();
 cassandraDaemon.destroy();
 cassandraThread.interrupt();
 cassandraThread = null;
 }
 cleanupDirectoriesFailover(); // this recursively deletes the 
 directories Cassandra works with whilst the server is running.
 }
 
 Thank you very much in advance for any help coming this way.
 
 Lewis
 
 12/04/04 13:24:11 INFO gora.GoraTestDriver: tearing down test
 12/04/04 13:24:11 INFO migration.Migration: Applying migration 
 15541bf0-7e51-11e1--242d50cf1fff Drop keyspace: Employee
 12/04/04 13:24:11 INFO db.ColumnFamilyStore: Enqueuing flush of 
 Memtable-Migrations@9190262(6586/82325 serialized/live bytes, 1 ops)
 12/04/04 13:24:11 INFO db.Memtable: Writing 
 Memtable-Migrations@9190262(6586/82325 serialized/live bytes, 1 ops)
 12/04/04 13:24:11 INFO db.ColumnFamilyStore: Enqueuing flush of 
 Memtable-Schema@9695615(2796/34950 serialized/live bytes, 2 ops)
 12/04/04 13:24:11 INFO db.Memtable: Completed flushing 
 target/test/var/lib/cassandra/data/system/Migrations-h-2764-Data.db (6650 
 bytes)
 12/04/04 13:24:11 INFO db.Memtable: Writing 
 Memtable-Schema@9695615(2796/34950 serialized/live bytes, 2 ops)
 12/04/04 13:24:11 INFO db.Memtable: Completed flushing 
 target/test/var/lib/cassandra/data/system/Schema-h-2764-Data.db (2946 bytes)
 12/04/04 13:24:12 INFO compaction.CompactionTask: Compacted to 
 [target/test/var/lib/cassandra/data/system/Migrations-h-2765-Data.db,].  
 12,979,714 to 12,979,522 (~99% of original) bytes for 1 keys at 
 12.341213MB/s.  Time: 1,003ms.
 12/04/04 13:24:12 INFO compaction.CompactionTask: Compacting 
 [SSTableReader(path='target/test/var/lib/cassandra/data/system/Schema-h-2764-Data.db'),
  
 SSTableReader(path='target/test/var/lib/cassandra/data/system/Schema-h-2760-Data.db'),
  
 SSTableReader(path='target/test/var/lib/cassandra/data/system/Schema-h-2761-Data.db'),
  
 SSTableReader(path='target/test/var/lib/cassandra/data/system/Schema-h-2763-Data.db')]
 12/04/04 13:24:12 INFO store.DataStoreTestBase: tearing down class
 12/04/04 13:24:12 INFO service.AbstractCassandraDaemon: Cassandra shutting 
 down...
 12/04/04 13:24:12 INFO thrift.CassandraDaemon: Stop listening to thrift 
 clients
 12/04/04 13:24:12 INFO compaction.CompactionTask: Compacted to 
 [target/test/var/lib/cassandra/data/system/Schema-h-2765-Data.db,].  
 5,735,956 to 5,735,629 (~99% of original) bytes for 1,681 keys at 
 9.011404MB/s.  Time: 607ms.
 Tests run: 51, Failures: 1, Errors: 50, Skipped: 0, Time elapsed: 913.842 sec 
  FAILURE!
 12/04/04 13:25:12 INFO net.MessagingService: Shutting down MessageService...
 12/04/04 13:25:12 INFO net.MessagingService: MessagingService shutting down 
 server thread.
 12/04/04 13:25:12 INFO net.MessagingService: Waiting for in-progress requests 
 to complete
 12/04/04 13:25:12 INFO db.ColumnFamilyStore: Enqueuing flush of 
 Memtable-Versions@6081747(83/103 serialized/live bytes, 3 ops)
 12/04/04 13:25:12 INFO db.Memtable: Writing Memtable-Versions@6081747(83/103 
 serialized/live bytes, 3 ops)
 12/04/04 13:25:12 ERROR service.AbstractCassandraDaemon: Fatal exception in 
 thread Thread[FlushWriter:2,5,main]
 java.lang.RuntimeException: java.lang.RuntimeException: Insufficient disk 
 space to flush 133 bytes
 at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 

Re: Linux Filesystem for Cassandra

2012-04-04 Thread Michael Widmann
Oleg:

If you wanna use - ZFS - use smartos / openindiana and cassandra on top
dont work around with a FUSE FS.
Maybe BSD (not knowing their version of zfs / zpool)




2012/4/4 Oleg Proudnikov ol...@cloudorange.com

 Thanks, Radim!

 What OS are you using and would ZFS be a good option under Linux on EC2?

 Thank you,
 Oleg


 On 2012-04-04, at 9:42 AM, Radim Kolar wrote:

 
  Would you, please share, what filesystem you are using?
 
  zfs 28




-- 
bayoda.com - Professional Online Backup Solutions for Small and Medium
Sized Companies


Re: Attempting to avoid fatal flush due to disk space

2012-04-04 Thread Lewis John Mcgibbney
Hi Aaron,

On Wed, Apr 4, 2012 at 7:57 PM, aaron morton aa...@thelastpickle.comwrote:

 Is cleanupDirectoriesFailover able to delete the files ?


Yes, around the time that the stack traces appear (after final
teadDownClass()) you'll see my log.INFO statement and subsequent service
log indicating that CassandraDaemon.stop and CassandraDaemon.destroy have
both been called. This is backed up by my manual checks which confirm that
the test directory and files within are sucessfully deleted. So I know that
the cleanupDirectoriesFailover code is executing as planned and that things
are working as they should up until my description as above. I've pasted
the code below with some additional comments below that.

12/04/04 21:59:21 INFO store.DataStoreTestBase: tearing down class
12/04/04 21:59:21 INFO cassandra.GoraCassandraTestDriver: Shutting down
Embedded Cassandra server...
12/04/04 21:59:21 INFO service.AbstractCassandraDaemon: Cassandra shutting
down...
12/04/04 21:59:21 INFO thrift.CassandraDaemon: Stop listening to thrift
clients
Tests run: 28, Failures: 0, Errors: 28, Skipped: 0, Time elapsed: 69.043
sec  FAILURE!
12/04/04 22:00:21 INFO net.MessagingService: Shutting down MessageService...
12/04/04 22:00:21 INFO net.MessagingService: MessagingService shutting down
server thread.
12/04/04 22:00:21 INFO net.MessagingService: Waiting for in-progress
requests to complete
12/04/04 22:00:21 INFO db.ColumnFamilyStore: Enqueuing flush of
Memtable-Versions@6196605(83/103 serialized/live bytes, 3 ops)
12/04/04 22:00:21 INFO db.Memtable: Writing Memtable-Versions@6196605(83/103
serialized/live bytes, 3 ops)
12/04/04 22:00:21 ERROR service.AbstractCassandraDaemon: Fatal exception in
thread Thread[FlushWriter:2,5,main]
java.lang.RuntimeException: java.lang.RuntimeException: Insufficient disk
space to flush 133 bytes
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.RuntimeException: Insufficient disk space to flush 133
bytes
at
org.apache.cassandra.db.ColumnFamilyStore.getFlushPath(ColumnFamilyStore.java:623)
at
org.apache.cassandra.db.ColumnFamilyStore.createFlushWriter(ColumnFamilyStore.java:1850)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:250)
at org.apache.cassandra.db.Memtable.access$400(Memtable.java:47)
at org.apache.cassandra.db.Memtable$4.runMayThrow(Memtable.java:291)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more
12/04/04 22:00:21 ERROR service.AbstractCassandraDaemon: Fatal exception in
thread Thread[FlushWriter:2,5,main]
java.lang.RuntimeException: java.lang.RuntimeException: Insufficient disk
space to flush 133 bytes
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.RuntimeException: Insufficient disk space to flush 133
bytes
at
org.apache.cassandra.db.ColumnFamilyStore.getFlushPath(ColumnFamilyStore.java:623)
at
org.apache.cassandra.db.ColumnFamilyStore.createFlushWriter(ColumnFamilyStore.java:1850)
at
org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:250)
at org.apache.cassandra.db.Memtable.access$400(Memtable.java:47)
at org.apache.cassandra.db.Memtable$4.runMayThrow(Memtable.java:291)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
... 3 more



 When you get the error is the disk actually full ?


I checked the memory on disk it is certainly not full. I have well over
20GB still available!



 Can you narrow this down to i cannot delete the sstables? (and what
 platform are you on).


Well for each failed test I'm currently getting

12/04/04 21:59:21 ERROR thrift.Cassandra$Processor: Internal error
processing system_add_keyspace
java.lang.IncompatibleClassChangeError: Class
org.apache.avro.generic.GenericData$Array does not implement the requested
interface java.util.List
at org.apache.cassandra.config.KSMetaData.toAvro(KSMetaData.java:136)
at
org.apache.cassandra.db.migration.Migration.makeDefinitionMutation(Migration.java:223)
at
org.apache.cassandra.db.migration.AddKeyspace.init(AddKeyspace.java:49)
at
org.apache.cassandra.thrift.CassandraServer.system_add_keyspace(CassandraServer.java:986)
at
org.apache.cassandra.thrift.Cassandra$Processor$system_add_keyspace.process(Cassandra.java:3912)
at
org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889)
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:187)

Re: cassandra 1.08 on java7 and win7

2012-04-04 Thread Frank Hsueh
iirc, the fix for me was to add conf to the CASSANDRA_CLASSPATH var; like
so:


:okClasspath
REM Include the build\classes\main directory so it works in development
set CASSANDRA_CLASSPATH=%CASSANDRA_HOME%\conf;
%CLASSPATH%;%CASSANDRA_HOME%\build\classes\main;%CASSANDRA_HOME%\build\classes\thrift
goto runCli



On Wed, Apr 4, 2012 at 4:26 AM, Gopala Krishna Gundeboyina 
f2001...@gmail.com wrote:

 I have a cluster of four nodes all running on single machine. If I just do
 create keyspace DEMO CLI gives me the same error that original post says.

 If I use create the keyspace with the options you mentioned it works just
 fine but the proble is it is not getting reflected on other nodes in the
 cluster.

 -Gopala


 On Wed, Apr 4, 2012 at 10:30 AM, puneet loya puneetl...@gmail.com wrote:

 thank u Gopala :) Der is no issue with it.. Might be i was typing
 something wrong.,. Minor mistake :)


 On Tue, Apr 3, 2012 at 11:51 PM, Gopala f2001...@gmail.com wrote:

 puneet loya puneetloya at gmail.com writes:

 
 
  create keyspace DEMO
 
  with placement_strategy =
 'org.apache.cassandra.locator.NetworkTopologyStrategy'
 
  and strategy_options=[{datacenter1:1}];
 
  try it n check if it executes



 Hi Puneet,
 I have same issue. Running the command you mentioned below works for me.
 What is
 the issue here?

 Gopala






-- 
Frank Hsueh | frank.hs...@gmail.com


Re: Linux Filesystem for Cassandra

2012-04-04 Thread Rob Coli
On Wed, Apr 4, 2012 at 1:15 PM, Michael Widmann
michael.widm...@gmail.com wrote:
 If you wanna use - ZFS - use smartos / openindiana and cassandra on top
 dont work around with a FUSE FS.
 Maybe BSD (not knowing their version of zfs / zpool)

http://zfsonlinux.org/

(I can't vouch for it, but FYI this is non-FUSE ZFS for linux, seems
actively developed etc.)

=Rob

-- 
=Robert Coli
AIMGTALK - rc...@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb


Is there a way to update column's TTL only?

2012-04-04 Thread 金剑
Hi,

We would like to leverage Cassandra's expiration to manage our data's
lifecycle. We need to: Delete data after a period, like: 1 hour , when user
clicked the Delete button.

We need to read and insert the column in order to update the TTL, but this
is unacceptable in our system that might need to readout gigabytes of data.

Is there a way to do this?

Best Regards!

Jian Jin


Will Cassandra balance load across replicas?

2012-04-04 Thread zhiming shen
Hi,

Can any one tell me whether Cassandra can do load balancing across
replicas? How to configure it for this purpose? Thanks very much.


Best Regards,

Zhiming


Re: server down

2012-04-04 Thread Michael Vaknin
i am usuinc cassandra 1.0.3 java 6.0.17 on ubuntu 10.04
this is a stable version for 6 months now.

On Wed, Apr 4, 2012 at 9:51 PM, aaron morton aa...@thelastpickle.comwrote:

 What version of cassandra are you using ?

 What java vendor / version ?

 What OS vendor / version ?

 Cheers

 -
 Aaron Morton
 Freelance Developer
 @aaronmorton
 http://www.thelastpickle.com

 On 4/04/2012, at 11:33 PM, Michael Vaknine wrote:

 I am recently having problems with 1 of my 4ty cluster servers.
 I would appreciate any help
 ** **
 ERROR 11:23:14,331 Fatal exception in thread
 Thread[MutationStage:19,5,main]
 225 java.lang.InternalError: a fault occurred in a recent unsafe memory
 access operation in compiled Java code
 226 at
 org.apache.cassandra.io.util.AbstractDataInput.readInt(AbstractDataInput.java:196)
 
 227 at
 org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:348)
 
 228 at
 org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:120)
 
 229 at
 org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:83)
 
 230 at
 org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:73)
 
 231 at
 org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:37)
 
 232 at
 org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:79)
 
 233 at
 org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:40)
 
 234 at
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
 
 235 at
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
 
 236 at
 org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:107)
 
 237 at
 org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:145)
 
 238 at
 org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:124)
 
 239 at
 org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
 
 240 at
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
 
 241 at
 com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
 
 242 at
 org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:116)
 
 243 at
 org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:144)
 
 244 at
 org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:227)
 
 245 at
 org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:62)
 
 246 at
 org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1274)
 
 247 at
 org.apache.cassandra.db.ColumnFamilyStore.cacheRow(ColumnFamilyStore.java:1141)
 
 248 at
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1170)
 
 249 at
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1122)
 
 250 at
 org.apache.cassandra.db.Table.readCurrentIndexedColumns(Table.java:504)***
 *
 251 at org.apache.cassandra.db.Table.apply(Table.java:441)
 252 at
 org.apache.cassandra.db.commitlog.CommitLog$2.runMayThrow(CommitLog.java:338)
 
 253 at
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)***
 *
 254 at
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)***
 *
 255 at
 java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 256 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 257 at
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 
 258 at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 
 259 at java.lang.Thread.run(Thread.java:619)
 260  INFO 11:23:17,640 Finished reading
 /var/lib/cassandra/commitlog/CommitLog-1333465745624.log
 261 ERROR 11:23:17,641 Exception encountered during startup
 262 java.lang.RuntimeException: java.util.concurrent.ExecutionException:
 java.lang.InternalError: a fault occurred in a recent unsafe memory access
 operation in compiled Java code
 263 at
 org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:508)
 
 ** **
 Thanks
 Michael





Re: Will Cassandra balance load across replicas?

2012-04-04 Thread Watanabe Maki
I assume you are talking about nodes, rather than replicas.
The data distribution over ring depends on Partitioner and Replica placement 
strategy you use.
If you are using Random Partitioner and Simple Strategy, your data will be 
automatically distributed over the nodes in the ring.

maki


On 2012/04/05, at 12:31, zhiming shen zhiming.s...@gmail.com wrote:

 Hi,
 
 Can any one tell me whether Cassandra can do load balancing across replicas? 
 How to configure it for this purpose? Thanks very much.
 
 
 Best Regards,
 
 Zhiming