[jira] [Commented] (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.

2011-04-03 Thread Vijay (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015276#comment-13015276
 ] 

Vijay commented on CASSANDRA-1969:
--

hmmm... if we avoid using getByteBuffer this issue wont show up right? i think 
we dont need ByteBuffer in this ticket (we only need it for #2167)... Agree? if 
yes then i can submit changes to this patch (ByteBufferOutputStream to MemoryOS 
and BBIS to MIS).

> Use BB for row cache - To Improve GC performance.
> -
>
> Key: CASSANDRA-1969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: Linux and Mac
>Reporter: Vijay
>Assignee: Vijay
>Priority: Minor
> Attachments: 0001-Config-1969.txt, 
> 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
> 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
> 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, 
> 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, 
> 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, 
> 1969_Cache_SVN_Patch.diff, BB_Cache-1945.png, JMX-Cache-1945.png, 
> Old_Cahce-1945.png, POC-0001-Config-1945.txt, 
> POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt
>
>
> Java BB.allocateDirect() will allocate native memory out of the JVM and will 
> help reducing the GC pressure in the JVM with a large Cache.
> From some of the basic tests it shows around 50% improvement than doing a 
> normal Object cache.
> In addition this patch provide the users an option to choose 
> BB.allocateDirect or store everything in the heap.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal

2011-04-03 Thread Ed Anuff (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015274#comment-13015274
 ] 

Ed Anuff commented on CASSANDRA-2231:
-

One more thing that's come up in testing, we're finding that only the last 
end-of-component byte is allowed to be non-zero.  In lines 231-233 of your 
path, you do this check in AbstractCompositeType.validate():

+byte b = bb.get();
+if (b != 0 && bb.remaining() != 0)
+throw new MarshalException("Invalid bytes remaining after an 
end-of-component at component" + i);
+

Is this check necessary or would it be ok if the end-of-component byte could be 
non-zero in any component?  We're usually doing the less-than-equals or 
greater-than-equals comparisons on the first component value, not the last, 
which is usually just a discriminator value to prevent duplicate column names.

> Add CompositeType comparer to the comparers provided in 
> org.apache.cassandra.db.marshal
> ---
>
> Key: CASSANDRA-2231
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Contrib
>Affects Versions: 0.7.3
>Reporter: Ed Anuff
>Assignee: Sylvain Lebresne
>Priority: Minor
> Fix For: 0.7.5
>
> Attachments: CompositeType-and-DynamicCompositeType.patch, 
> edanuff-CassandraCompositeType-1e253c4.zip
>
>
> CompositeType is a custom comparer that makes it possible to create 
> comparable composite values out of the basic types that Cassandra currently 
> supports, such as Long, UUID, etc.  This is very useful in both the creation 
> of custom inverted indexes using columns in a skinny row, where each column 
> name is a composite value, and also when using Cassandra's built-in secondary 
> index support, where it can be used to encode the values in the columns that 
> Cassandra indexes.  One scenario for the usage of these is documented here: 
> http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html.  Source for 
> contribution is attached and has been previously maintained on github here: 
> https://github.com/edanuff/CassandraCompositeType

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal

2011-04-03 Thread Todd Nine (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015273#comment-13015273
 ] 

Todd Nine commented on CASSANDRA-2231:
--

Hi Sylvain,
  I seem to have encountered a bug in the comparator.  I'm using the composite 
to perform Cassandra based intersections of fields during queries.  For 
instance, say a user defines an index as this.

status + unitId.


The write would always contain values of +0++0 when using the 
-1 0 and 1 fields.  If the user enters a query such as this


status > 100 && status < 300 && unitId = 10

I would need to construct a column scan of the following to get  correct result 
sets.

start => 100+1+10+0

end => 300+0+10+1


However on in the validate function I'm receiving the error "Invalid bytes 
remaining after an end-of-component at component".  This seems incorrect to me. 
 We're ultimately attempting to apply any equality operand and transform it to 
a range scan for the given field in the composite.  This means that -1, or 1 
could appear after any component in the composite, not just the last one.  Can 
you please add this functionality/remove this verification check?

Thanks,
Todd

> Add CompositeType comparer to the comparers provided in 
> org.apache.cassandra.db.marshal
> ---
>
> Key: CASSANDRA-2231
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Contrib
>Affects Versions: 0.7.3
>Reporter: Ed Anuff
>Assignee: Sylvain Lebresne
>Priority: Minor
> Fix For: 0.7.5
>
> Attachments: CompositeType-and-DynamicCompositeType.patch, 
> edanuff-CassandraCompositeType-1e253c4.zip
>
>
> CompositeType is a custom comparer that makes it possible to create 
> comparable composite values out of the basic types that Cassandra currently 
> supports, such as Long, UUID, etc.  This is very useful in both the creation 
> of custom inverted indexes using columns in a skinny row, where each column 
> name is a composite value, and also when using Cassandra's built-in secondary 
> index support, where it can be used to encode the values in the columns that 
> Cassandra indexes.  One scenario for the usage of these is documented here: 
> http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html.  Source for 
> contribution is attached and has been previously maintained on github here: 
> https://github.com/edanuff/CassandraCompositeType

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.

2011-04-03 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015265#comment-13015265
 ] 

Jonathan Ellis commented on CASSANDRA-1969:
---

Doesn't that still leaves us with the segfaulting problem from map entries for 
freed buffers (the other part of JNA-179)?

> Use BB for row cache - To Improve GC performance.
> -
>
> Key: CASSANDRA-1969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: Linux and Mac
>Reporter: Vijay
>Assignee: Vijay
>Priority: Minor
> Attachments: 0001-Config-1969.txt, 
> 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
> 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
> 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, 
> 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, 
> 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, 
> 1969_Cache_SVN_Patch.diff, BB_Cache-1945.png, JMX-Cache-1945.png, 
> Old_Cahce-1945.png, POC-0001-Config-1945.txt, 
> POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt
>
>
> Java BB.allocateDirect() will allocate native memory out of the JVM and will 
> help reducing the GC pressure in the JVM with a large Cache.
> From some of the basic tests it shows around 50% improvement than doing a 
> normal Object cache.
> In addition this patch provide the users an option to choose 
> BB.allocateDirect or store everything in the heap.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Issue Comment Edited] (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.

2011-04-03 Thread Vijay (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015264#comment-13015264
 ] 

Vijay edited comment on CASSANDRA-1969 at 4/4/11 3:34 AM:
--

used reflection to change the map into a synchronized WeakHashMap in 
SerializingCache

{code}
static 
{
hackJNA();
}

/**
 * this is a hack to make the WeakHashMap thread-safe. 
 * Bug ID: JNA-179 {@link http://java.net/jira/browse/JNA-179}
 */
private static void hackJNA()
{
try
{
Field h = Memory.class.getDeclaredField("buffers");
h.setAccessible(true);
Map map = Collections.synchronizedMap(new WeakHashMap());
h.set(null, map);
}
catch (Exception e)
{
throw new RuntimeException(e);
}
}
{code}

  was (Author: vijay2...@yahoo.com):
used reflection to change the map into a synchronized WeakHashMap in 
SerializingCache

static 
{
hackJNA();
}

/**
 * this is a hack to make the WeakHashMap thread-safe. 
 * Bug ID: JNA-179 {@link http://java.net/jira/browse/JNA-179}
 */
private static void hackJNA()
{
try
{
Field h = Memory.class.getDeclaredField("buffers");
h.setAccessible(true);
Map map = Collections.synchronizedMap(new WeakHashMap());
h.set(null, map);
}
catch (Exception e)
{
throw new RuntimeException(e);
}
}
  
> Use BB for row cache - To Improve GC performance.
> -
>
> Key: CASSANDRA-1969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: Linux and Mac
>Reporter: Vijay
>Assignee: Vijay
>Priority: Minor
> Attachments: 0001-Config-1969.txt, 
> 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
> 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
> 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, 
> 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, 
> 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, 
> 1969_Cache_SVN_Patch.diff, BB_Cache-1945.png, JMX-Cache-1945.png, 
> Old_Cahce-1945.png, POC-0001-Config-1945.txt, 
> POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt
>
>
> Java BB.allocateDirect() will allocate native memory out of the JVM and will 
> help reducing the GC pressure in the JVM with a large Cache.
> From some of the basic tests it shows around 50% improvement than doing a 
> normal Object cache.
> In addition this patch provide the users an option to choose 
> BB.allocateDirect or store everything in the heap.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.

2011-04-03 Thread Vijay (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015264#comment-13015264
 ] 

Vijay commented on CASSANDRA-1969:
--

used reflection to change the map into a synchronized WeakHashMap in 
SerializingCache

static 
{
hackJNA();
}

/**
 * this is a hack to make the WeakHashMap thread-safe. 
 * Bug ID: JNA-179 {@link http://java.net/jira/browse/JNA-179}
 */
private static void hackJNA()
{
try
{
Field h = Memory.class.getDeclaredField("buffers");
h.setAccessible(true);
Map map = Collections.synchronizedMap(new WeakHashMap());
h.set(null, map);
}
catch (Exception e)
{
throw new RuntimeException(e);
}
}

> Use BB for row cache - To Improve GC performance.
> -
>
> Key: CASSANDRA-1969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: Linux and Mac
>Reporter: Vijay
>Assignee: Vijay
>Priority: Minor
> Attachments: 0001-Config-1969.txt, 
> 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
> 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
> 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, 
> 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, 
> 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, 
> 1969_Cache_SVN_Patch.diff, BB_Cache-1945.png, JMX-Cache-1945.png, 
> Old_Cahce-1945.png, POC-0001-Config-1945.txt, 
> POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt
>
>
> Java BB.allocateDirect() will allocate native memory out of the JVM and will 
> help reducing the GC pressure in the JVM with a large Cache.
> From some of the basic tests it shows around 50% improvement than doing a 
> normal Object cache.
> In addition this patch provide the users an option to choose 
> BB.allocateDirect or store everything in the heap.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.

2011-04-03 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015263#comment-13015263
 ] 

Jonathan Ellis commented on CASSANDRA-1969:
---

what is the hack/workaround?

> Use BB for row cache - To Improve GC performance.
> -
>
> Key: CASSANDRA-1969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: Linux and Mac
>Reporter: Vijay
>Assignee: Vijay
>Priority: Minor
> Attachments: 0001-Config-1969.txt, 
> 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
> 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
> 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, 
> 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, 
> 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, 
> 1969_Cache_SVN_Patch.diff, BB_Cache-1945.png, JMX-Cache-1945.png, 
> Old_Cahce-1945.png, POC-0001-Config-1945.txt, 
> POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt
>
>
> Java BB.allocateDirect() will allocate native memory out of the JVM and will 
> help reducing the GC pressure in the JVM with a large Cache.
> From some of the basic tests it shows around 50% improvement than doing a 
> normal Object cache.
> In addition this patch provide the users an option to choose 
> BB.allocateDirect or store everything in the heap.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


[Cassandra Wiki] Update of "FrontPage" by DrewKutcharian

2011-04-03 Thread Apache Wiki
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for 
change notification.

The "FrontPage" page has been changed by DrewKutcharian.
http://wiki.apache.org/cassandra/FrontPage?action=diff&rev1=58&rev2=59

--

   * [[UseCases|Simple Use Cases and Solutions]] -- please help complete
   * [[FAQ]]
   * [[Counters]]
+  * [[SecondaryIndexes]]
  
  == Advanced Setup and Tuning ==
   * [[StorageConfiguration|Storage Configuration]]


[Cassandra Wiki] Update of "SecondaryIndexes" by DrewKutcharian

2011-04-03 Thread Apache Wiki
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for 
change notification.

The "SecondaryIndexes" page has been changed by DrewKutcharian.
http://wiki.apache.org/cassandra/SecondaryIndexes

--

New page:
'''Articles/Blogs'''
 * A good article by Jonathan Ellis: 
http://www.datastax.com/dev/blog/whats-new-cassandra-07-secondary-indexes

'''FAQ for Secondary Indexes'''

 * Q: Are there any limitations beside the hash properties (no between 
queries)? Like size or memory, etc?
   A: No.

 * Q: Are they distributed? If so, how does that work? How are they stored on 
the nodes?
   A: Each node only indexes data that it holds locally.

 * Q: When you write a new row, when/how does the index get updated? What I 
would like to know is the atomicity of the operation, is the "index write" part 
of the "row write"?
   A: The row and index updates are one atomic operation.

 * Q: Is there a difference between creating a secondary index vs creating an 
"index" CF manually such as "users_by_country"?
   A: Yes. First, when creating your own index, a node may index data held by 
another node. Second, updates to the index and data are not atomic.


[jira] [Updated] (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.

2011-04-03 Thread Vijay (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vijay updated CASSANDRA-1969:
-

Attachment: 1969_Cache_SVN_Patch.diff

Rebased to trunk + a hack/workaround for JNA-179, i have tested this patch and 
i think with this change we should commit this patch... the performance looks 
very good with this update.

> Use BB for row cache - To Improve GC performance.
> -
>
> Key: CASSANDRA-1969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: Linux and Mac
>Reporter: Vijay
>Assignee: Vijay
>Priority: Minor
> Attachments: 0001-Config-1969.txt, 
> 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, 
> 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, 
> 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, 
> 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, 
> 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, 
> 1969_Cache_SVN_Patch.diff, BB_Cache-1945.png, JMX-Cache-1945.png, 
> Old_Cahce-1945.png, POC-0001-Config-1945.txt, 
> POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt
>
>
> Java BB.allocateDirect() will allocate native memory out of the JVM and will 
> help reducing the GC pressure in the JVM with a large Cache.
> From some of the basic tests it shows around 50% improvement than doing a 
> normal Object cache.
> In addition this patch provide the users an option to choose 
> BB.allocateDirect or store everything in the heap.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-2401) getColumnFamily() return null, which is not checked in ColumnFamilyStore.java scan() method, causing Timeout Exception in query

2011-04-03 Thread Roland Gude (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015193#comment-13015193
 ] 

Roland Gude commented on CASSANDRA-2401:


can you provide some unit tests that reproduce your error? i'd like to look 
into it, but i am not sure whether i understand the issue correctly.

> getColumnFamily() return null, which is not checked in ColumnFamilyStore.java 
> scan() method, causing Timeout Exception in query
> ---
>
> Key: CASSANDRA-2401
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2401
> Project: Cassandra
>  Issue Type: Bug
>Affects Versions: 0.7.4
> Environment: Hector 0.7.0-28, Cassandra 0.7.4, Windows 7, Eclipse
>Reporter: Tey Kar Shiang
>
> ColumnFamilyStore.java, line near 1680, "ColumnFamily data = 
> getColumnFamily(new QueryFilter(dk, path, firstFilter))", the data is 
> returned null, causing NULL exception in "satisfies(data, clause, primary)" 
> which is not captured. The callback got timeout and return a Timeout 
> exception to Hector.
> The data is empty, as I traced, I have the the columns Count as 0 in 
> removeDeletedCF(), which return the null there. (I am new and trying to 
> understand the logics around still). Instead of crash to NULL, could we 
> bypass the data?
> About my test:
> A stress-test program to add, modify and delete data to keyspace. I have 30 
> threads simulate concurrent users to perform the actions above, and do a 
> query to all rows periodically. I have Column Family with rows (as File) and 
> columns as index (e.g. userID, fileType).
> No issue on the first day of test, and stopped for 3 days. I restart the test 
> on 4th day, 1 of the users failed to query the files (timeout exception 
> received). Most of the users are still okay with the query.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-2191) Multithread across compaction buckets

2011-04-03 Thread Aaron Morton (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015150#comment-13015150
 ] 

Aaron Morton commented on CASSANDRA-2191:
-

4) Sounds reasonable if throttling is on.
 
6) I'm not familiar with the bloom filter optimization you mentioned. However 
it seems that more than anything else the major flag in doCompaction() 
indicates if the compaction is running on all sstables, regardless of how the 
process was triggered. i.e. the first ever minor compaction would also be 
marked as major by this logic. PrecompactedRow and LazilyCompactedRow will 
purge rows if the major flag is set or the key is only present in the sstables 
under compaction. I'm not sure why the extra check is there for minor 
compactions, but it looks like losing the fact the a major/manual compaction 
was started could change the purge behaviour. 

I'm also trying to understand if the isKeyInRemainingSSTables() in the 
AbstractCompactedRow sub classes could be affected by multithreading. e.g. CF 
with two buckets, high min compaction threshold so longer compaction, two 
concurrent minor compactions one in each bucket, row A in both buckets, if 
either thread processes row A before the other finishes it would stop that 
thread purging the row, is there a race condition that stops both threads 
purging the row?

9) We do not use the value in the compactions map, could we set it to the 
current system time when beginCompaction() is called and use that to the sort 
the list ? was not a biggie 



> Multithread across compaction buckets
> -
>
> Key: CASSANDRA-2191
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2191
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: Stu Hood
>Assignee: Stu Hood
>Priority: Critical
>  Labels: compaction
> Fix For: 0.8
>
> Attachments: 0001-Add-a-compacting-set-to-DataTracker.txt, 
> 0002-Use-the-compacting-set-of-sstables-to-schedule-multith.txt, 
> 0003-Expose-multiple-compactions-via-JMX-and-deprecate-sing.txt, 
> 0004-Try-harder-to-close-scanners-in-compaction-close.txt
>
>
> This ticket overlaps with CASSANDRA-1876 to a degree, but the approaches and 
> reasoning are different enough to open a separate issue.
> The problem with compactions currently is that they compact the set of 
> sstables that existed the moment the compaction started. This means that for 
> longer running compactions (even when running as fast as possible on the 
> hardware), a very large number of new sstables might be created in the 
> meantime. We have observed this proliferation of sstables killing performance 
> during major/high-bucketed compactions.
> One approach would be to pause compactions in upper buckets (containing 
> larger files) when compactions in lower buckets become possible. While this 
> would likely solve the problem with read performance, it does not actually 
> help us perform compaction any faster, which is a reasonable requirement for 
> other situations.
> Instead, we need to be able to perform any compactions that are currently 
> required in parallel, independent of what bucket they might be in.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-2414) IntegerType isn't noted in cli

2011-04-03 Thread Pavel Yaskevich (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015143#comment-13015143
 ] 

Pavel Yaskevich commented on CASSANDRA-2414:


+1

> IntegerType isn't noted in cli
> --
>
> Key: CASSANDRA-2414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2414
> Project: Cassandra
>  Issue Type: Improvement
>Affects Versions: 0.7.0
>Reporter: Joaquin Casares
>Assignee: Joaquin Casares
>Priority: Trivial
> Fix For: 0.7.5
>
> Attachments: 2414.diff
>
>
> The cli's help doesn't list all comparator types.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Commented] (CASSANDRA-786) RPM Packages

2011-04-03 Thread Oded Arbel (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015139#comment-13015139
 ] 

Oded Arbel commented on CASSANDRA-786:
--

With current release (Cassandra 0.7.4) the spec file does not allow one to 
build successfully on CentOS 5.5 due to missing build dependencies and other 
issues. Should I open a new ticket for that?

> RPM Packages
> 
>
> Key: CASSANDRA-786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-786
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Contrib
>Reporter: Daniel Lundin
>Assignee: Nick Bailey
>Priority: Minor
>  Labels: distribution, packaging, rpm
> Fix For: 0.7 beta 2
>
> Attachments: 768-update-spec-for-trunk.diff, 786-adjust-jars.patch, 
> apache-cassandra.spec, cassandra.spec, cassandra.spec
>
>
> RPM packages (and debs of course) would be nice,especially now that cassandra 
> is maturing and gaining more interest.
> Lowering the threshold for getting cassandra running and getting started is 
> also important.
> I think the RabbitMQ project has an admirable "Download and install" 
> experience, not to mention the rather cute "2 min guarantee". Definitely a 
> good inspiration.
> I've been studying Cloudera's Hadoop packages, which are very nice, and 
> really appreciate the separate packages for configuration.
> This allows easy deployment of node configuration to a cluster.
> I'll have a spec file for building RHEL5 / CentOS packages ready for review 
> and attached here in a bit.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira