[jira] [Comment Edited] (CASSANDRA-5903) Integer overflow in OffHeapBitSet when bloomfilter 2GB
[
https://issues.apache.org/jira/browse/CASSANDRA-5903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13746118#comment-13746118
]
Taylan Develioglu edited comment on CASSANDRA-5903 at 8/21/13 3:08 PM:
---
Sadly that wasn't sufficient, there's another overflow in
OffHeapBitSet.deserialize:
ERROR [SSTableBatchOpen:6] 2013-08-21 15:29:51,799 CassandraDaemon.java (line
192) Exception in thread Thread[SSTableBatchOpen:6,5,main]
java.lang.IllegalArgumentException
at org.apache.cassandra.io.util.Memory.allocate(Memory.java:58)
at
org.apache.cassandra.utils.obs.OffHeapBitSet.deserialize(OffHeapBitSet.java:123)
at
org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilterSerializer.java:46)
at
org.apache.cassandra.utils.Murmur2BloomFilter$Murmur2BloomFilterSerializer.deserialize(Murmur2BloomFilter.java:40)
at
org.apache.cassandra.utils.FilterFactory.deserialize(FilterFactory.java:71)
at
org.apache.cassandra.io.sstable.SSTableReader.loadBloomFilter(SSTableReader.java:365)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:195)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:153)
at
org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:258)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
{code}
public static OffHeapBitSet deserialize(DataInput dis) throws IOException
{
int byteCount = dis.readInt() * 8;
Memory memory = RefCountedMemory.allocate(byteCount);
for (int i = 0; i byteCount;)
{
long v = dis.readLong();
memory.setByte(i++, (byte) (v 0));
memory.setByte(i++, (byte) (v 8));
memory.setByte(i++, (byte) (v 16));
memory.setByte(i++, (byte) (v 24));
memory.setByte(i++, (byte) (v 32));
memory.setByte(i++, (byte) (v 40));
memory.setByte(i++, (byte) (v 48));
memory.setByte(i++, (byte) (v 56));
}
return new OffHeapBitSet(memory);
}
{code}
was (Author: tdevelioglu):
Sadly that wasn't sufficient, there's another overflow in
OffHeapBitSet.deserialize:
{code}
public static OffHeapBitSet deserialize(DataInput dis) throws IOException
{
int byteCount = dis.readInt() * 8;
Memory memory = RefCountedMemory.allocate(byteCount);
for (int i = 0; i byteCount;)
{
long v = dis.readLong();
memory.setByte(i++, (byte) (v 0));
memory.setByte(i++, (byte) (v 8));
memory.setByte(i++, (byte) (v 16));
memory.setByte(i++, (byte) (v 24));
memory.setByte(i++, (byte) (v 32));
memory.setByte(i++, (byte) (v 40));
memory.setByte(i++, (byte) (v 48));
memory.setByte(i++, (byte) (v 56));
}
return new OffHeapBitSet(memory);
}
{code}
Integer overflow in OffHeapBitSet when bloomfilter 2GB
Key: CASSANDRA-5903
URL: https://issues.apache.org/jira/browse/CASSANDRA-5903
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Taylan Develioglu
Assignee: Vijay
Fix For: 1.2.9
Attachments: 0001-CASSANDRA-5903.patch
In org.apache.cassandra.utils.obs.OffHeapBitSet.
byteCount overflows and causes an IllegalArgument exception in
Memory.allocate when bloomfilter is 2GB.
Suggest changing byteCount to long.
{code:title=OffHeapBitSet.java}
public OffHeapBitSet(long numBits)
{
// OpenBitSet.bits2words calculation is there for backward
compatibility.
int byteCount = OpenBitSet.bits2words(numBits) * 8;
bytes = RefCountedMemory.allocate(byteCount);
// flush/clear the existing memory.
clear();
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5903) Integer overflow in OffHeapBitSet when bloomfilter 2GB
[
https://issues.apache.org/jira/browse/CASSANDRA-5903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13746118#comment-13746118
]
Taylan Develioglu edited comment on CASSANDRA-5903 at 8/21/13 3:09 PM:
---
Sadly that wasn't sufficient, there's another overflow in
OffHeapBitSet.deserialize:
{code}
public static OffHeapBitSet deserialize(DataInput dis) throws IOException
{
int byteCount = dis.readInt() * 8;
Memory memory = RefCountedMemory.allocate(byteCount);
for (int i = 0; i byteCount;)
{
long v = dis.readLong();
memory.setByte(i++, (byte) (v 0));
memory.setByte(i++, (byte) (v 8));
memory.setByte(i++, (byte) (v 16));
memory.setByte(i++, (byte) (v 24));
memory.setByte(i++, (byte) (v 32));
memory.setByte(i++, (byte) (v 40));
memory.setByte(i++, (byte) (v 48));
memory.setByte(i++, (byte) (v 56));
}
return new OffHeapBitSet(memory);
}
{code}
was (Author: tdevelioglu):
Sadly that wasn't sufficient, there's another overflow in
OffHeapBitSet.deserialize:
ERROR [SSTableBatchOpen:6] 2013-08-21 15:29:51,799 CassandraDaemon.java (line
192) Exception in thread Thread[SSTableBatchOpen:6,5,main]
java.lang.IllegalArgumentException
at org.apache.cassandra.io.util.Memory.allocate(Memory.java:58)
at
org.apache.cassandra.utils.obs.OffHeapBitSet.deserialize(OffHeapBitSet.java:123)
at
org.apache.cassandra.utils.BloomFilterSerializer.deserialize(BloomFilterSerializer.java:46)
at
org.apache.cassandra.utils.Murmur2BloomFilter$Murmur2BloomFilterSerializer.deserialize(Murmur2BloomFilter.java:40)
at
org.apache.cassandra.utils.FilterFactory.deserialize(FilterFactory.java:71)
at
org.apache.cassandra.io.sstable.SSTableReader.loadBloomFilter(SSTableReader.java:365)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:195)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:153)
at
org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:258)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
{code}
public static OffHeapBitSet deserialize(DataInput dis) throws IOException
{
int byteCount = dis.readInt() * 8;
Memory memory = RefCountedMemory.allocate(byteCount);
for (int i = 0; i byteCount;)
{
long v = dis.readLong();
memory.setByte(i++, (byte) (v 0));
memory.setByte(i++, (byte) (v 8));
memory.setByte(i++, (byte) (v 16));
memory.setByte(i++, (byte) (v 24));
memory.setByte(i++, (byte) (v 32));
memory.setByte(i++, (byte) (v 40));
memory.setByte(i++, (byte) (v 48));
memory.setByte(i++, (byte) (v 56));
}
return new OffHeapBitSet(memory);
}
{code}
Integer overflow in OffHeapBitSet when bloomfilter 2GB
Key: CASSANDRA-5903
URL: https://issues.apache.org/jira/browse/CASSANDRA-5903
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Taylan Develioglu
Assignee: Vijay
Fix For: 1.2.9
Attachments: 0001-CASSANDRA-5903.patch
In org.apache.cassandra.utils.obs.OffHeapBitSet.
byteCount overflows and causes an IllegalArgument exception in
Memory.allocate when bloomfilter is 2GB.
Suggest changing byteCount to long.
{code:title=OffHeapBitSet.java}
public OffHeapBitSet(long numBits)
{
// OpenBitSet.bits2words calculation is there for backward
compatibility.
int byteCount = OpenBitSet.bits2words(numBits) * 8;
bytes = RefCountedMemory.allocate(byteCount);
// flush/clear the existing memory.
clear();
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5903) Integer overflow in OffHeapBitSet when bloomfilter 2GB
[
https://issues.apache.org/jira/browse/CASSANDRA-5903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13746836#comment-13746836
]
Vijay edited comment on CASSANDRA-5903 at 8/21/13 8:59 PM:
---
Not sure if we still need this patch, attaching it just in case :) Ignored the
test since we need 4 GB to test it function.
was (Author: vijay2...@yahoo.com):
Not sure if we still need this patch, attaching it just in case :)
Integer overflow in OffHeapBitSet when bloomfilter 2GB
Key: CASSANDRA-5903
URL: https://issues.apache.org/jira/browse/CASSANDRA-5903
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Taylan Develioglu
Assignee: Vijay
Labels: patch
Fix For: 1.2.9
Attachments: 0001-CASSANDRA-5903-check.patch,
0001-CASSANDRA-5903.patch, 0002-CASSANDRA-5903.patch
In org.apache.cassandra.utils.obs.OffHeapBitSet.
byteCount overflows and causes an IllegalArgument exception in
Memory.allocate when bloomfilter is 2GB.
Suggest changing byteCount to long.
{code:title=OffHeapBitSet.java}
public OffHeapBitSet(long numBits)
{
// OpenBitSet.bits2words calculation is there for backward
compatibility.
int byteCount = OpenBitSet.bits2words(numBits) * 8;
bytes = RefCountedMemory.allocate(byteCount);
// flush/clear the existing memory.
clear();
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5903) Integer overflow in OffHeapBitSet when bloomfilter 2GB
[
https://issues.apache.org/jira/browse/CASSANDRA-5903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13745244#comment-13745244
]
Vijay edited comment on CASSANDRA-5903 at 8/20/13 6:31 PM:
---
I can change the byte count to long,
As a side note, i am not sure if we are addressing the right issue. From the
stack trace the byteCount should be 228805104 which is 228 MB
(OpenBitSet.bits2words(1830440832L) * 8L) / ((1830440832L/64) * 8) which should
fit in a integer.
was (Author: vijay2...@yahoo.com):
I can change the byte count to long,
As a side note, i am not sure if we are addressing the right issue. From the
stack trace the byteCount should be 228805104 which is 228 MB
(OpenBitSet.bits2words(1830440832L) * 8L) which should fit in a integer.
Integer overflow in OffHeapBitSet when bloomfilter 2GB
Key: CASSANDRA-5903
URL: https://issues.apache.org/jira/browse/CASSANDRA-5903
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Taylan Develioglu
Assignee: Vijay
Fix For: 1.2.9
In org.apache.cassandra.utils.obs.OffHeapBitSet.
byteCount overflows and causes an IllegalArgument exception in
Memory.allocate when bloomfilter is 2GB.
Suggest changing byteCount to long.
{code:title=OffHeapBitSet.java}
public OffHeapBitSet(long numBits)
{
// OpenBitSet.bits2words calculation is there for backward
compatibility.
int byteCount = OpenBitSet.bits2words(numBits) * 8;
bytes = RefCountedMemory.allocate(byteCount);
// flush/clear the existing memory.
clear();
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
