[jira] [Updated] (CASSANDRA-12744) Randomness of stress distributions is not good
[ https://issues.apache.org/jira/browse/CASSANDRA-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Slater updated CASSANDRA-12744: --- Reviewer: T Jake Luciani Status: Patch Available (was: Open) After some more digging, I've come to the conclusion that the issue is that the JDKRandomGenerator creates close random numbers when seeded with close values. So, when running with a small range of potential seeds (from the population) you end up with different random doubles which all round to the same long value. The attached patch multiplies the generated seed so that max seed values are of the order of 10^22. I've tested this against a couple of the failed dtests and pass OK. In addition, I get the following results from a range of YAML files: Example 1: table: test5 table_definition: | CREATE TABLE test5 ( pk int, val text, PRIMARY KEY (pk) ) columnspec: - name: pk size: fixed(64) population: uniform(1..500) user profile=... ops(insert=1) n=1000 cl=ALL no-warmup -rate threads=5 -node 127.0.0.1 without multiplier - 47 rows with multiplier - 490 rows table: test4 table_definition: | CREATE TABLE test4 ( pk int, pk2 text, val text, PRIMARY KEY ((pk,pk2)) ) columnspec: - name: pk size: fixed(2) population: uniform(1..5) - name: pk2 size: fixed(2) population: uniform(1..5) user profile=... ops(insert=1) n=1000 cl=ALL no-warmup -rate threads=5 -node 127.0.0.1 without multipler - 1 row with multiplier - 25 rows table: test4 table_definition: | CREATE TABLE test4 ( pk int, pk2 text, val text, PRIMARY KEY ((pk,pk2)) ) columnspec: - name: pk size: fixed(2) population: uniform(1..500M) - name: pk2 size: fixed(2) population: uniform(1..5) user profile=... ops(insert=1) n=1000 cl=ALL no-warmup -rate threads=5 -node 127.0.0.1 without multipler - 1000 row with multiplier - 1000 rows === table: test7 table_definition: | CREATE TABLE test7 ( pk int, pk2 text, ck1 text, val text, PRIMARY KEY ((pk,pk2), ck1) ) columnspec: - name: pk size: fixed(2) population: uniform(1..100) - name: pk2 size: fixed(4) population: uniform(1..1) - name: pk2 size: fixed(4) population: uniform(1..1000) user profile=... ops(insert=1) n=10 cl=ALL no-warmup -rate threads=5 -node 127.0.0.1 without multipler - 10342 row with multiplier - 63387 rows = table_definition: | CREATE TABLE test7 ( pk int, pk2 text, ck1 text, val text, PRIMARY KEY ((pk,pk2), ck1) ) columnspec: - name: pk size: fixed(4) population: seq(1..100) - name: pk2 size: fixed(10) population: seq(1..1) - name: pk2 size: fixed(10) cluster: uniform(1..1000) population: seq(1..1000) user profile=... ops(insert=1) n=10 cl=ALL no-warmup -rate threads=5 -node 127.0.0.1 without multiplier - 25000 row with multiplier - 43304 rows > Randomness of stress distributions is not good > -- > > Key: CASSANDRA-12744 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12744 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: T Jake Luciani >Assignee: Ben Slater >Priority: Minor > Labels: stress > Fix For: 4.0 > > > The randomness of our distributions is pretty bad. We are using the > JDKRandomGenerator() but in testing of uniform(1..3) we see for 100 > iterations it's only outputting 3. If you bump it to 10k it hits all 3 > values. > I made a change to just use the default commons math random generator and now > see all 3 values for n=10 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-12744) Randomness of stress distributions is not good
[ https://issues.apache.org/jira/browse/CASSANDRA-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Slater updated CASSANDRA-12744: --- Attachment: CASSANDRA_12744_SeedManager_changes-trunk.patch > Randomness of stress distributions is not good > -- > > Key: CASSANDRA-12744 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12744 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: T Jake Luciani >Assignee: Ben Slater >Priority: Minor > Labels: stress > Fix For: 4.0 > > Attachments: CASSANDRA_12744_SeedManager_changes-trunk.patch > > > The randomness of our distributions is pretty bad. We are using the > JDKRandomGenerator() but in testing of uniform(1..3) we see for 100 > iterations it's only outputting 3. If you bump it to 10k it hits all 3 > values. > I made a change to just use the default commons math random generator and now > see all 3 values for n=10 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-12744) Randomness of stress distributions is not good
[ https://issues.apache.org/jira/browse/CASSANDRA-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16028803#comment-16028803 ] Ben Slater edited comment on CASSANDRA-12744 at 5/30/17 7:18 AM: - After some more digging, I've come to the conclusion that the issue is that the JDKRandomGenerator creates close random numbers when seeded with close values. So, when running with a small range of potential seeds (from the population) you end up with different random doubles which all round to the same long value. The attached patch multiplies the generated seed so that max seed values are of the order of 10^22. I've tested this against a couple of the failed dtests and pass OK. In addition, I get the following results from a range of YAML files (without multiplier result is unmodified trunk, with multiplier is with this patch applied): Example 1: table: test5 table_definition: | CREATE TABLE test5 ( pk int, val text, PRIMARY KEY (pk) ) columnspec: - name: pk size: fixed(64) population: uniform(1..500) user profile=... ops(insert=1) n=1000 cl=ALL no-warmup -rate threads=5 -node 127.0.0.1 without multiplier - 47 rows with multiplier - 490 rows table: test4 table_definition: | CREATE TABLE test4 ( pk int, pk2 text, val text, PRIMARY KEY ((pk,pk2)) ) columnspec: - name: pk size: fixed(2) population: uniform(1..5) - name: pk2 size: fixed(2) population: uniform(1..5) user profile=... ops(insert=1) n=1000 cl=ALL no-warmup -rate threads=5 -node 127.0.0.1 without multipler - 1 row with multiplier - 25 rows table: test4 table_definition: | CREATE TABLE test4 ( pk int, pk2 text, val text, PRIMARY KEY ((pk,pk2)) ) columnspec: - name: pk size: fixed(2) population: uniform(1..500M) - name: pk2 size: fixed(2) population: uniform(1..5) user profile=... ops(insert=1) n=1000 cl=ALL no-warmup -rate threads=5 -node 127.0.0.1 without multipler - 1000 row with multiplier - 1000 rows === table: test7 table_definition: | CREATE TABLE test7 ( pk int, pk2 text, ck1 text, val text, PRIMARY KEY ((pk,pk2), ck1) ) columnspec: - name: pk size: fixed(2) population: uniform(1..100) - name: pk2 size: fixed(4) population: uniform(1..1) - name: pk2 size: fixed(4) population: uniform(1..1000) user profile=... ops(insert=1) n=10 cl=ALL no-warmup -rate threads=5 -node 127.0.0.1 without multipler - 10342 row with multiplier - 63387 rows = table_definition: | CREATE TABLE test7 ( pk int, pk2 text, ck1 text, val text, PRIMARY KEY ((pk,pk2), ck1) ) columnspec: - name: pk size: fixed(4) population: seq(1..100) - name: pk2 size: fixed(10) population: seq(1..1) - name: pk2 size: fixed(10) cluster: uniform(1..1000) population: seq(1..1000) user profile=... ops(insert=1) n=10 cl=ALL no-warmup -rate threads=5 -node 127.0.0.1 without multiplier - 25000 row with multiplier - 43304 rows was (Author: slater_ben): After some more digging, I've come to the conclusion that the issue is that the JDKRandomGenerator creates close random numbers when seeded with close values. So, when running with a small range of potential seeds (from the population) you end up with different random doubles which all round to the same long value. The attached patch multiplies the generated seed so that max seed values are of the order of 10^22. I've tested this against a couple of the failed dtests and pass OK. In addition, I get the following results from a range of YAML files: Example 1: table: test5 table_definition: | CREATE TABLE test5 ( pk int, val text, PRIMARY KEY (pk) ) columnspec: - name: pk size: fixed(64) population: uniform(1..500) user profile=... ops(insert=1) n=1000 cl=ALL no-warmup -rate threads=5 -node 127.0.0.1 without multiplier - 47 rows with multiplier - 490 rows table: test4 table_definition: | CREATE TABLE test4 ( pk int, pk2 text, val text, PRIMARY KEY ((pk,pk2)) ) columnspec: - name: pk size: fixed(2) population: uniform(1..5) - name: pk2 size: fixed(2) population: uniform(1..5) user profile=... ops(insert=1) n=1000 cl=ALL no-warmup -rate threads=5 -node 127.0.0.1 without multipler - 1 row with multiplier - 25 rows table: test4 table_definition: | CREATE TABLE test4 ( pk int, pk2 text, val text, PRIMARY KEY
[jira] [Commented] (CASSANDRA-13068) Fully expired sstable not dropped when running out of disk space
[
https://issues.apache.org/jira/browse/CASSANDRA-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16028817#comment-16028817
]
Lerh Chuan Low commented on CASSANDRA-13068:
Hi [~krummas],
Here's the new branch:
https://github.com/apache/cassandra/compare/trunk...juiceblender:cassandra-13068
I've added some 3 different tests using the example, but I've also added an
extra check
{code}
final Set fullyExpiredSSTables =
controller.getFullyExpiredSSTables();
final Set actuallyCompact =
Sets.difference(transaction.originals(), fullyExpiredSSTables);
// note that we need to do a rough estimate early if we can fit the
compaction on disk - this is pessimistic, but
// since we might remove sstables from the compaction in
checkAvailableDiskSpace it needs to be done here
// If there are no fully expired SSTables, check available disk
space. Otherwise, we want to try and always compact
// fully expired SSTables
if (fullyExpiredSSTables.isEmpty())
checkAvailableDiskSpace(actuallyCompact);
{code}
This is to make the test I wrote
{{testExpiredSSTablesStillGetDroppedWithNoDiskSpace()}} work. I am not sure if
this is the right way to do it or if it was intentionally left that way -
basically if you have expired SSTables and SSTables that are too big, we will
throw the run time exception instead of try and compact the expired SSTables
away. I don't know if there's ordering of compactions at this point though, so
I don't know if this is a safe operation to do.
Feel free to let me know about any feedback. There seem to be other ways to go
about this, such as {{checkAvailableDiskSpace}} should just estimate fully
expired SSTables to have compacted file size 0 instead...?
> Fully expired sstable not dropped when running out of disk space
>
>
> Key: CASSANDRA-13068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13068
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
> Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> If a fully expired sstable is larger than the remaining disk space we won't
> run the compaction that can drop the sstable (ie, in our disk space check
> should not include the fully expired sstables)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-11871) Allow to aggregate by time intervals
[
https://issues.apache.org/jira/browse/CASSANDRA-11871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer reassigned CASSANDRA-11871:
--
Assignee: (was: Benjamin Lerer)
> Allow to aggregate by time intervals
>
>
> Key: CASSANDRA-11871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11871
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Benjamin Lerer
> Fix For: 4.x
>
>
> For time series data it can be usefull to aggregate by time intervals.
> The idea would be to add support for one or several functions in the {{GROUP
> BY}} clause.
> Regarding the implementation, even if in general I also prefer to follow the
> SQL syntax, I do not believe it will be a good fit for Cassandra.
> If we have a table like:
> {code}
> CREATE TABLE trades
> {
> symbol text,
> date date,
> time time,
> priceMantissa int,
> priceExponent tinyint,
> volume int,
> PRIMARY KEY ((symbol, date), time)
> };
> {code}
> The trades will be inserted with an increasing time and sorted in the same
> order. As we can have to process a large amount of data, we want to try to
> limit ourself to the cases where we can build the groups on the flight (which
> is not a requirement in the SQL world).
> If we want to get the number of trades per minutes with the SQL syntax we
> will have to write:
> {{SELECT hour(time), minute(time), count() FROM Trades WHERE symbol = 'AAPL'
> AND date = '2016-01-11' GROUP BY hour(time), minute(time);}}
> which is fine. The problem is that if the user invert by mistake the
> functions like that:
> {{SELECT hour(time), minute(time), count() FROM Trades WHERE symbol = 'AAPL'
> AND date = '2016-01-11' GROUP BY minute(time), hour(time);}}
> the query will return weird results.
> The only way to prevent that would be to check the function order and make
> sure that we do not allow to skip functions (e.g. {{GROUP BY hour(time),
> second(time)}}).
> In my opinion a function like {{floor(, )}} will be
> much better as it does not allow for this type of mistakes and is much more
> flexible (you can create 5 minutes buckets if you want to).
> {{SELECT floor(time, m), count() FROM Trades WHERE symbol = 'AAPL' AND date =
> '2016-01-11' GROUP BY floor(time, m);}}
> An important aspect to keep in mind with a function like {{floor}} is the
> starting point. For a query like: {{SELECT floor(time, m), count() FROM
> Trades WHERE symbol = 'AAPL' AND date = '2016-01-11' AND time >= '01:30:00'
> AND time =< '07:30:00' GROUP BY floor(time, 2h);}}, I think that ideally the
> result should return 3 groups: {{01:30:00}}, {{03:30:00}} and {{05:30:00}}.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13072) Cassandra failed to run on Linux-aarch64
[
https://issues.apache.org/jira/browse/CASSANDRA-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-13072:
---
Status: Patch Available (was: Open)
> Cassandra failed to run on Linux-aarch64
>
>
> Key: CASSANDRA-13072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13072
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Hardware: ARM aarch64
> OS: Ubuntu 16.04.1 LTS
>Reporter: Jun He
>Assignee: Benjamin Lerer
> Labels: incompatible
> Attachments: compat_report.html
>
>
> Steps to reproduce:
> 1. Download cassandra latest source
> 2. Build it with "ant"
> 3. Run with "./bin/cassandra". Daemon is crashed with following error message:
> {quote}
> INFO 05:30:21 Initializing system.schema_functions
> INFO 05:30:21 Initializing system.schema_aggregates
> ERROR 05:30:22 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
> at
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:97)
> ~[main/:na]
> at org.apache.cassandra.io.util.Memory.(Memory.java:74)
> ~[main/:na]
> at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:519)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:497)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:480)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.createFlushWriter(Memtable.java:439)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:371)
> ~[main/:na]
> at org.apache.cassandra.db.Memtable.flush(Memtable.java:332)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> {quote}
> Analyze:
> This issue is caused by bundled jna-4.0.0.jar which doesn't come with aarch64
> native support. Replace lib/jna-4.0.0.jar with jna-4.2.0.jar from
> http://central.maven.org/maven2/net/java/dev/jna/jna/4.2.0/ can fix this
> problem.
> Attached is the binary compatibility report of jna.jar between 4.0 and 4.2.
> The result is good (97.4%). So is there possibility to upgrade jna to 4.2.0
> in upstream? Should there be any kind of tests to execute, please kindly
> point me. Thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13072) Cassandra failed to run on Linux-aarch64
[
https://issues.apache.org/jira/browse/CASSANDRA-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029336#comment-16029336
]
Benjamin Lerer commented on CASSANDRA-13072:
I pushed a patch for
[2.2|https://github.com/apache/cassandra/compare/trunk...blerer:13072-2.2],
[3.0|https://github.com/apache/cassandra/compare/trunk...blerer:13072-3.0] and
[3.11|https://github.com/apache/cassandra/compare/trunk...blerer:13072-3.11].
The 3 patches are the same. They just upgrade the JNA version to {{4.4.0}}.
I ran it on our internal CI and the results looks good.
[~ifesdjeen] could you review?
> Cassandra failed to run on Linux-aarch64
>
>
> Key: CASSANDRA-13072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13072
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Hardware: ARM aarch64
> OS: Ubuntu 16.04.1 LTS
>Reporter: Jun He
>Assignee: Benjamin Lerer
> Labels: incompatible
> Attachments: compat_report.html
>
>
> Steps to reproduce:
> 1. Download cassandra latest source
> 2. Build it with "ant"
> 3. Run with "./bin/cassandra". Daemon is crashed with following error message:
> {quote}
> INFO 05:30:21 Initializing system.schema_functions
> INFO 05:30:21 Initializing system.schema_aggregates
> ERROR 05:30:22 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
> at
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:97)
> ~[main/:na]
> at org.apache.cassandra.io.util.Memory.(Memory.java:74)
> ~[main/:na]
> at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:519)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:497)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:480)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.createFlushWriter(Memtable.java:439)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:371)
> ~[main/:na]
> at org.apache.cassandra.db.Memtable.flush(Memtable.java:332)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> {quote}
> Analyze:
> This issue is caused by bundled jna-4.0.0.jar which doesn't come with aarch64
> native support. Replace lib/jna-4.0.0.jar with jna-4.2.0.jar from
> http://central.maven.org/maven2/net/java/dev/jna/jna/4.2.0/ can fix this
> problem.
> Attached is the binary compatibility report of jna.jar between 4.0 and 4.2.
> The result is good (97.4%). So is there possibility to upgrade jna to 4.2.0
> in upstream? Should there be any kind of tests to execute, please kindly
> point me. Thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13072) Cassandra failed to run on Linux-aarch64
[
https://issues.apache.org/jira/browse/CASSANDRA-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029389#comment-16029389
]
Jason Brown commented on CASSANDRA-13072:
-
fwiw, we upgraded to jna 4.3 in CASSANDRA-13300.
More importantly, I thought we didn't upgrade the dependencies until major
revs, which in this case would be 3.11 and 4.0 (trunk). [~iamaleksey], what do
you think about upgrading on 2.2 & 3.0?
> Cassandra failed to run on Linux-aarch64
>
>
> Key: CASSANDRA-13072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13072
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Hardware: ARM aarch64
> OS: Ubuntu 16.04.1 LTS
>Reporter: Jun He
>Assignee: Benjamin Lerer
> Labels: incompatible
> Attachments: compat_report.html
>
>
> Steps to reproduce:
> 1. Download cassandra latest source
> 2. Build it with "ant"
> 3. Run with "./bin/cassandra". Daemon is crashed with following error message:
> {quote}
> INFO 05:30:21 Initializing system.schema_functions
> INFO 05:30:21 Initializing system.schema_aggregates
> ERROR 05:30:22 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
> at
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:97)
> ~[main/:na]
> at org.apache.cassandra.io.util.Memory.(Memory.java:74)
> ~[main/:na]
> at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:519)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:497)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:480)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.createFlushWriter(Memtable.java:439)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:371)
> ~[main/:na]
> at org.apache.cassandra.db.Memtable.flush(Memtable.java:332)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> {quote}
> Analyze:
> This issue is caused by bundled jna-4.0.0.jar which doesn't come with aarch64
> native support. Replace lib/jna-4.0.0.jar with jna-4.2.0.jar from
> http://central.maven.org/maven2/net/java/dev/jna/jna/4.2.0/ can fix this
> problem.
> Attached is the binary compatibility report of jna.jar between 4.0 and 4.2.
> The result is good (97.4%). So is there possibility to upgrade jna to 4.2.0
> in upstream? Should there be any kind of tests to execute, please kindly
> point me. Thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13546) Getting error while adding a node in existing cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-13546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-13546:
Description:
Getting error after adding a node in existing cluster, error are coming after 1
hour after started service on seed node and due to this load got increased
vastly on seed node not on new node.
Below errors from system.log on seed node:
{code}
ERROR [NonPeriodicTasks:1] 2017-05-18 10:03:01,819 CassandraDaemon.java:153 -
Exception in thread Thread[NonPeriodicTasks:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.free(Memory.java:300)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.utils.obs.OffHeapBitSet.close(OffHeapBitSet.java:143)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at org.apache.cassandra.utils.BloomFilter.close(BloomFilter.java:116)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableReader$6.run(SSTableReader.java:645)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_71]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_71]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown
Source) ~[na:1.7.0_71]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source) ~[na:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_71]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_71]
INFO [CompactionExecutor:162] 2017-05-18 10:03:01,820
ColumnFamilyStore.java:840 - Enqueuing flush of compactions_in_progress: 164
(0%) on-heap, 0 (0%) off-heap
INFO [MemtableFlushWriter:152] 2017-05-18 10:03:01,821 Memtable.java:325 -
Writing Memtable-compactions_in_progress@731506998(0 serialized bytes, 1 ops,
0%/0% of on/off-heap limit)
INFO [MemtableFlushWriter:152] 2017-05-18 10:03:01,825 Memtable.java:364 -
Completed flushing
/var/lib/cassandra/data/system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/system-compactions_in_progress-ka-52434-Data.db
(42 bytes) for commitlog position ReplayPosition(segmentId=1495106933696,
position=9156004)
ERROR [CompactionExecutor:162] 2017-05-18 10:03:01,829 CassandraDaemon.java:153
-
Exception in thread Thread[CompactionExecutor:162,1,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.util.Memory.free(Memory.java:300)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.utils.obs.OffHeapBitSet.close(OffHeapBitSet.java:143)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at org.apache.cassandra.utils.BloomFilter.close(BloomFilter.java:116)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableWriter.abort(SSTableWriter.java:345)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.io.sstable.SSTableRewriter.abort(SSTableRewriter.java:198)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:204)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:75)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:232)
~[apache-cassandra-2.1.2.jar:2.1.2-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
~[na:1.7.0_71]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
[na:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
[na:1.7.0_71]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_71]
INFO [main] 2017-05-18 11:06:56,032 CassandraDaemon.java:89 - Hostname:
csdb2-dl.intermesh.net
INFO [main] 2017-05-18 11:06:56,061 YamlConfigurationLoader.java:92 - Loading
settings from file:/etc/cassandra/default.conf/cassandra.yaml
{code}
was:
Getting error after adding a node in existing cluster, error are coming after 1
hour after started service on seed node and due to this load g
[jira] [Commented] (CASSANDRA-13555) Thread leak during repair
[
https://issues.apache.org/jira/browse/CASSANDRA-13555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029421#comment-16029421
]
T Jake Luciani commented on CASSANDRA-13555:
The issue here is we aren't finishing off the validations when the repair
session terminates:
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/repair/RepairSession.java#L302
So just need to walk that array and set results to null. Same with the
syncTasks.
[~szhou] would you be able to make a patch (and ideally a dtest)?
> Thread leak during repair
> -
>
> Key: CASSANDRA-13555
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13555
> Project: Cassandra
> Issue Type: Bug
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>
> The symptom is similar to what happened in [CASSANDRA-13204 |
> https://issues.apache.org/jira/browse/CASSANDRA-13204] that the thread
> waiting forever doing nothing. This one happened during "nodetool repair -pr
> -seq -j 1" in production but I can easily simulate the problem with just
> "nodetool repair" in dev environment (CCM). I'm trying to explain what
> happened with 3.0.13 code base.
> 1. One node is down while doing repair. This is the error I saw in production:
> {code}
> ERROR [GossipTasks:1] 2017-05-19 15:00:10,545 RepairSession.java:334 -
> [repair #bc9a3cd1-3ca3-11e7-a44a-e30923ac9336] session completed with the
> following error
> java.io.IOException: Endpoint /10.185.43.15 died
> at
> org.apache.cassandra.repair.RepairSession.convict(RepairSession.java:333)
> ~[apache-cassandra-3.0.11.jar:3.0.11]
> at
> org.apache.cassandra.gms.FailureDetector.interpret(FailureDetector.java:306)
> [apache-cassandra-3.0.11.jar:3.0.11]
> at org.apache.cassandra.gms.Gossiper.doStatusCheck(Gossiper.java:766)
> [apache-cassandra-3.0.11.jar:3.0.11]
> at org.apache.cassandra.gms.Gossiper.access$800(Gossiper.java:66)
> [apache-cassandra-3.0.11.jar:3.0.11]
> at org.apache.cassandra.gms.Gossiper$GossipTask.run(Gossiper.java:181)
> [apache-cassandra-3.0.11.jar:3.0.11]
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> [apache-cassandra-3.0.11.jar:3.0.11]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_121]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_121]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_121]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [apache-cassandra-3.0.11.jar:3.0.11]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
> {code}
> 2. At this moment the repair coordinator hasn't received the response
> (MerkleTrees) for the node that was marked down. This means, RepairJob#run
> will never return because it waits for validations to finish:
> {code}
> // Wait for validation to complete
> Futures.getUnchecked(validations);
> {code}
> Be noted that all RepairJob's (as Runnable) run on a shared executor created
> in RepairRunnable#runMayThrow, while all snapshot, validation and sync'ing
> happen on a per-RepairSession "taskExecutor". The RepairJob#run will only
> return when it receives MerkleTrees (or null) from all endpoints for a given
> column family and token range.
> As evidence of the thread leak, below is from the thread dump. I can also get
> the same stack trace when simulating the same issue in dev environment.
> {code}
> "Repair#129:56" #406373 daemon prio=5 os_prio=0 tid=0x7fc495028400
> nid=0x1a77d waiting on condition [0x7fc02153]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0002d7c00198> (a
> com.google.common.util.concurrent.AbstractFuture$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchroniz
[jira] [Commented] (CASSANDRA-13555) Thread leak during repair
[
https://issues.apache.org/jira/browse/CASSANDRA-13555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029431#comment-16029431
]
T Jake Luciani commented on CASSANDRA-13555:
Just realised you already found that :)
We only block on validations because we want to throttle the validations (since
they require compactors). If we don't it can overwhelm the replicas with
pending work for each subrange
> Thread leak during repair
> -
>
> Key: CASSANDRA-13555
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13555
> Project: Cassandra
> Issue Type: Bug
>Reporter: Simon Zhou
>Assignee: Simon Zhou
>
> The symptom is similar to what happened in [CASSANDRA-13204 |
> https://issues.apache.org/jira/browse/CASSANDRA-13204] that the thread
> waiting forever doing nothing. This one happened during "nodetool repair -pr
> -seq -j 1" in production but I can easily simulate the problem with just
> "nodetool repair" in dev environment (CCM). I'm trying to explain what
> happened with 3.0.13 code base.
> 1. One node is down while doing repair. This is the error I saw in production:
> {code}
> ERROR [GossipTasks:1] 2017-05-19 15:00:10,545 RepairSession.java:334 -
> [repair #bc9a3cd1-3ca3-11e7-a44a-e30923ac9336] session completed with the
> following error
> java.io.IOException: Endpoint /10.185.43.15 died
> at
> org.apache.cassandra.repair.RepairSession.convict(RepairSession.java:333)
> ~[apache-cassandra-3.0.11.jar:3.0.11]
> at
> org.apache.cassandra.gms.FailureDetector.interpret(FailureDetector.java:306)
> [apache-cassandra-3.0.11.jar:3.0.11]
> at org.apache.cassandra.gms.Gossiper.doStatusCheck(Gossiper.java:766)
> [apache-cassandra-3.0.11.jar:3.0.11]
> at org.apache.cassandra.gms.Gossiper.access$800(Gossiper.java:66)
> [apache-cassandra-3.0.11.jar:3.0.11]
> at org.apache.cassandra.gms.Gossiper$GossipTask.run(Gossiper.java:181)
> [apache-cassandra-3.0.11.jar:3.0.11]
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118)
> [apache-cassandra-3.0.11.jar:3.0.11]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_121]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_121]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_121]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79)
> [apache-cassandra-3.0.11.jar:3.0.11]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
> {code}
> 2. At this moment the repair coordinator hasn't received the response
> (MerkleTrees) for the node that was marked down. This means, RepairJob#run
> will never return because it waits for validations to finish:
> {code}
> // Wait for validation to complete
> Futures.getUnchecked(validations);
> {code}
> Be noted that all RepairJob's (as Runnable) run on a shared executor created
> in RepairRunnable#runMayThrow, while all snapshot, validation and sync'ing
> happen on a per-RepairSession "taskExecutor". The RepairJob#run will only
> return when it receives MerkleTrees (or null) from all endpoints for a given
> column family and token range.
> As evidence of the thread leak, below is from the thread dump. I can also get
> the same stack trace when simulating the same issue in dev environment.
> {code}
> "Repair#129:56" #406373 daemon prio=5 os_prio=0 tid=0x7fc495028400
> nid=0x1a77d waiting on condition [0x7fc02153]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0002d7c00198> (a
> com.google.common.util.concurrent.AbstractFuture$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> at
> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:285)
> at
> com.g
[jira] [Commented] (CASSANDRA-13072) Cassandra failed to run on Linux-aarch64
[
https://issues.apache.org/jira/browse/CASSANDRA-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029437#comment-16029437
]
Benjamin Lerer commented on CASSANDRA-13072:
bq. More importantly, I thought we didn't upgrade the dependencies until major
revs, which in this case would be 3.11 and 4.0 (trunk).
In case of problem like this one were some people cannot even start C*, I think
it really make sense to allow it.
> Cassandra failed to run on Linux-aarch64
>
>
> Key: CASSANDRA-13072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13072
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Hardware: ARM aarch64
> OS: Ubuntu 16.04.1 LTS
>Reporter: Jun He
>Assignee: Benjamin Lerer
> Labels: incompatible
> Attachments: compat_report.html
>
>
> Steps to reproduce:
> 1. Download cassandra latest source
> 2. Build it with "ant"
> 3. Run with "./bin/cassandra". Daemon is crashed with following error message:
> {quote}
> INFO 05:30:21 Initializing system.schema_functions
> INFO 05:30:21 Initializing system.schema_aggregates
> ERROR 05:30:22 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
> at
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:97)
> ~[main/:na]
> at org.apache.cassandra.io.util.Memory.(Memory.java:74)
> ~[main/:na]
> at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:519)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:497)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:480)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.createFlushWriter(Memtable.java:439)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:371)
> ~[main/:na]
> at org.apache.cassandra.db.Memtable.flush(Memtable.java:332)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> {quote}
> Analyze:
> This issue is caused by bundled jna-4.0.0.jar which doesn't come with aarch64
> native support. Replace lib/jna-4.0.0.jar with jna-4.2.0.jar from
> http://central.maven.org/maven2/net/java/dev/jna/jna/4.2.0/ can fix this
> problem.
> Attached is the binary compatibility report of jna.jar between 4.0 and 4.2.
> The result is good (97.4%). So is there possibility to upgrade jna to 4.2.0
> in upstream? Should there be any kind of tests to execute, please kindly
> point me. Thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-8272) 2ndary indexes can return stale data
[
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029457#comment-16029457
]
Andrés de la Peña commented on CASSANDRA-8272:
--
bq. For the row-filter aware counter however, I don't think we can aford to
have it not be a stopping transformation: we very much rely on that stopping to
not OOM nodes (and generally read the whole database on a read), whether it be
for user limits or paging. I'm not sure I understand why stopping it a concern
in this case however?
If I understand it right, the idea is to have a {{DataLimits}} (associated to a
{{RowFilter}}) that doesn't filter nor count rows that don't satisfy the
filter. Any possible deleted index entry from a replica could be required to
discard the possible stale results of another replica, so they shouldn't be
filtered by {{DataLimits}}. So, if the query limit requires {{n}} rows, we
should return not more than {{n}} rows satisfying the filter, and *all* the
rows not satisfying the index but being pointed by a deleted index entry. Is
this correct? If so, we can't stop reading when we have {{n}} rows satisfying
the filter, we should keep reading the all the remaining rows pointed by
deleted index entries, independently of the limit and with the subsequent
impact on performance.
> 2ndary indexes can return stale data
>
>
> Key: CASSANDRA-8272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Andrés de la Peña
> Fix For: 3.0.x
>
>
> When replica return 2ndary index results, it's possible for a single replica
> to return a stale result and that result will be sent back to the user,
> potentially failing the CL contract.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v text);
> CREATE INDEX ON test(v);
> INSERT INTO test(k, v) VALUES (0, 'foo');
> {noformat}
> with every replica up to date. Now, suppose that the following queries are
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v = 'bar' WHERE k = 0;
> SELECT * FROM test WHERE v = 'foo';
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before
> having applied the insert, then the now stale result will be returned (since
> C will return it and A or B will return nothing).
> A potential solution would be that when we read a tombstone in the index (and
> provided we make the index inherit the gcGrace of it's parent CF), instead of
> skipping that tombstone, we'd insert in the result a corresponding range
> tombstone.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13072) Cassandra failed to run on Linux-aarch64
[
https://issues.apache.org/jira/browse/CASSANDRA-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029437#comment-16029437
]
Benjamin Lerer edited comment on CASSANDRA-13072 at 5/30/17 2:12 PM:
-
bq. More importantly, I thought we didn't upgrade the dependencies until major
revs, which in this case would be 3.11 and 4.0 (trunk).
In case of problem like this one where some people cannot even start C*, I
think it really makes sense to allow it.
was (Author: blerer):
bq. More importantly, I thought we didn't upgrade the dependencies until major
revs, which in this case would be 3.11 and 4.0 (trunk).
In case of problem like this one were some people cannot even start C*, I think
it really make sense to allow it.
> Cassandra failed to run on Linux-aarch64
>
>
> Key: CASSANDRA-13072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13072
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Hardware: ARM aarch64
> OS: Ubuntu 16.04.1 LTS
>Reporter: Jun He
>Assignee: Benjamin Lerer
> Labels: incompatible
> Attachments: compat_report.html
>
>
> Steps to reproduce:
> 1. Download cassandra latest source
> 2. Build it with "ant"
> 3. Run with "./bin/cassandra". Daemon is crashed with following error message:
> {quote}
> INFO 05:30:21 Initializing system.schema_functions
> INFO 05:30:21 Initializing system.schema_aggregates
> ERROR 05:30:22 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
> at
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:97)
> ~[main/:na]
> at org.apache.cassandra.io.util.Memory.(Memory.java:74)
> ~[main/:na]
> at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:519)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:497)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:480)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.createFlushWriter(Memtable.java:439)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:371)
> ~[main/:na]
> at org.apache.cassandra.db.Memtable.flush(Memtable.java:332)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> {quote}
> Analyze:
> This issue is caused by bundled jna-4.0.0.jar which doesn't come with aarch64
> native support. Replace lib/jna-4.0.0.jar with jna-4.2.0.jar from
> http://central.maven.org/maven2/net/java/dev/jna/jna/4.2.0/ can fix this
> problem.
> Attached is the binary compatibility report of jna.jar between 4.0 and 4.2.
> The result is good (97.4%). So is there possibility to upgrade jna to 4.2.0
> in upstream? Should there be any kind of tests to execute, please kindly
> point me. Thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12744) Randomness of stress distributions is not good
[ https://issues.apache.org/jira/browse/CASSANDRA-12744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029477#comment-16029477 ] T Jake Luciani commented on CASSANDRA-12744: good find! I re-started the tests with your patch > Randomness of stress distributions is not good > -- > > Key: CASSANDRA-12744 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12744 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: T Jake Luciani >Assignee: Ben Slater >Priority: Minor > Labels: stress > Fix For: 4.0 > > Attachments: CASSANDRA_12744_SeedManager_changes-trunk.patch > > > The randomness of our distributions is pretty bad. We are using the > JDKRandomGenerator() but in testing of uniform(1..3) we see for 100 > iterations it's only outputting 3. If you bump it to 10k it hits all 3 > values. > I made a change to just use the default commons math random generator and now > see all 3 values for n=10 -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index
[
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029787#comment-16029787
]
Robert Stupp commented on CASSANDRA-7396:
-
As discussed offline with Benjamin, let's keep it as it is.
Note for other readers: Adding _any_ restriction after a slice restriction is
effectively pointless. For example, {{m\[1..\]\[..2\]}} can and should be
written as {{m\[1..2\]}}, {{m\[1..\]\[2\]}} as {{m\[1..\]}}.
Therefore: +1, ship it!
> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
> Labels: cql, docs-impacting
> Fix For: 4.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]." (Selecting a UDT subfield
> is already supported.)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-13562) nodes in cluster gets into split-brain mode
Jaydeepkumar Chovatia created CASSANDRA-13562: - Summary: nodes in cluster gets into split-brain mode Key: CASSANDRA-13562 URL: https://issues.apache.org/jira/browse/CASSANDRA-13562 Project: Cassandra Issue Type: Bug Components: Core Reporter: Jaydeepkumar Chovatia Fix For: 3.0.x We have seen nodes in Cassandra (3.0.11) ring gets into split-brain somehow. We don't know exact reproducible steps but here is our observation: Let's assume we have 5 node cluster n1,n2,n3,n4,n5. In this bug when do nodetool status on each node then each one has different view of DN node e.g. n1 sees n3 as DN and other nodes are UN n3 sees n4 as DN and other nodes are UN n4 sees n5 as DN and other nodes are UN and so on... One thing we have observed is once n/w link is broken and restored then sometimes nodes go into this split-brain mode but we still don't have exact reproducible steps. Please let us know if I am missing anything specific here. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10130) Node failure during 2i update after streaming can have incomplete 2i when restarted
[
https://issues.apache.org/jira/browse/CASSANDRA-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029960#comment-16029960
]
Caleb Rackliffe commented on CASSANDRA-10130:
-
bq. We could either add a new field to SSTableAddedNotification to indicate
wether the added sstables come from flush or external sstables or create two
new notifications which would inherit from SSTableAddedNotification to maintain
backward compatibility, perhaps: SSTableFlushedNotification and
ExternalSSTableLoadedNotification.
The {{SSTableFlushedNotification}} approach sounds cleaner, mostly because it
could carry an unambiguously non-null reference to the flushed {{Memtable}}.
> Node failure during 2i update after streaming can have incomplete 2i when
> restarted
> ---
>
> Key: CASSANDRA-10130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10130
> Project: Cassandra
> Issue Type: Bug
> Components: Coordination
>Reporter: Yuki Morishita
>Assignee: Andrés de la Peña
>Priority: Minor
>
> Since MV/2i update happens after SSTables are received, node failure during
> MV/2i update can leave received SSTables live when restarted while MV/2i are
> partially up to date.
> We can add some kind of tracking mechanism to automatically rebuild at the
> startup, or at least warn user when the node restarts.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13072) Cassandra failed to run on Linux-aarch64
[
https://issues.apache.org/jira/browse/CASSANDRA-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029989#comment-16029989
]
Aleksey Yeschenko commented on CASSANDRA-13072:
---
Also, +1.
> Cassandra failed to run on Linux-aarch64
>
>
> Key: CASSANDRA-13072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13072
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Hardware: ARM aarch64
> OS: Ubuntu 16.04.1 LTS
>Reporter: Jun He
>Assignee: Benjamin Lerer
> Labels: incompatible
> Attachments: compat_report.html
>
>
> Steps to reproduce:
> 1. Download cassandra latest source
> 2. Build it with "ant"
> 3. Run with "./bin/cassandra". Daemon is crashed with following error message:
> {quote}
> INFO 05:30:21 Initializing system.schema_functions
> INFO 05:30:21 Initializing system.schema_aggregates
> ERROR 05:30:22 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
> at
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:97)
> ~[main/:na]
> at org.apache.cassandra.io.util.Memory.(Memory.java:74)
> ~[main/:na]
> at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:519)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:497)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:480)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.createFlushWriter(Memtable.java:439)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:371)
> ~[main/:na]
> at org.apache.cassandra.db.Memtable.flush(Memtable.java:332)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> {quote}
> Analyze:
> This issue is caused by bundled jna-4.0.0.jar which doesn't come with aarch64
> native support. Replace lib/jna-4.0.0.jar with jna-4.2.0.jar from
> http://central.maven.org/maven2/net/java/dev/jna/jna/4.2.0/ can fix this
> problem.
> Attached is the binary compatibility report of jna.jar between 4.0 and 4.2.
> The result is good (97.4%). So is there possibility to upgrade jna to 4.2.0
> in upstream? Should there be any kind of tests to execute, please kindly
> point me. Thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13072) Cassandra failed to run on Linux-aarch64
[
https://issues.apache.org/jira/browse/CASSANDRA-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029988#comment-16029988
]
Aleksey Yeschenko commented on CASSANDRA-13072:
---
bq. What do you think about upgrading on 2.2 & 3.0?
It's more of a guideline than a hard rule; I agree with Benjamin here, it gotta
be done in this case.
> Cassandra failed to run on Linux-aarch64
>
>
> Key: CASSANDRA-13072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13072
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Hardware: ARM aarch64
> OS: Ubuntu 16.04.1 LTS
>Reporter: Jun He
>Assignee: Benjamin Lerer
> Labels: incompatible
> Attachments: compat_report.html
>
>
> Steps to reproduce:
> 1. Download cassandra latest source
> 2. Build it with "ant"
> 3. Run with "./bin/cassandra". Daemon is crashed with following error message:
> {quote}
> INFO 05:30:21 Initializing system.schema_functions
> INFO 05:30:21 Initializing system.schema_aggregates
> ERROR 05:30:22 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
> at
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:97)
> ~[main/:na]
> at org.apache.cassandra.io.util.Memory.(Memory.java:74)
> ~[main/:na]
> at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:519)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:497)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:480)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.createFlushWriter(Memtable.java:439)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:371)
> ~[main/:na]
> at org.apache.cassandra.db.Memtable.flush(Memtable.java:332)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> {quote}
> Analyze:
> This issue is caused by bundled jna-4.0.0.jar which doesn't come with aarch64
> native support. Replace lib/jna-4.0.0.jar with jna-4.2.0.jar from
> http://central.maven.org/maven2/net/java/dev/jna/jna/4.2.0/ can fix this
> problem.
> Attached is the binary compatibility report of jna.jar between 4.0 and 4.2.
> The result is good (97.4%). So is there possibility to upgrade jna to 4.2.0
> in upstream? Should there be any kind of tests to execute, please kindly
> point me. Thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13072) Cassandra failed to run on Linux-aarch64
[
https://issues.apache.org/jira/browse/CASSANDRA-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-13072:
--
Reviewer: Aleksey Yeschenko
> Cassandra failed to run on Linux-aarch64
>
>
> Key: CASSANDRA-13072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13072
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Hardware: ARM aarch64
> OS: Ubuntu 16.04.1 LTS
>Reporter: Jun He
>Assignee: Benjamin Lerer
> Labels: incompatible
> Attachments: compat_report.html
>
>
> Steps to reproduce:
> 1. Download cassandra latest source
> 2. Build it with "ant"
> 3. Run with "./bin/cassandra". Daemon is crashed with following error message:
> {quote}
> INFO 05:30:21 Initializing system.schema_functions
> INFO 05:30:21 Initializing system.schema_aggregates
> ERROR 05:30:22 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
> at
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:97)
> ~[main/:na]
> at org.apache.cassandra.io.util.Memory.(Memory.java:74)
> ~[main/:na]
> at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:519)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:497)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:480)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.createFlushWriter(Memtable.java:439)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:371)
> ~[main/:na]
> at org.apache.cassandra.db.Memtable.flush(Memtable.java:332)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> {quote}
> Analyze:
> This issue is caused by bundled jna-4.0.0.jar which doesn't come with aarch64
> native support. Replace lib/jna-4.0.0.jar with jna-4.2.0.jar from
> http://central.maven.org/maven2/net/java/dev/jna/jna/4.2.0/ can fix this
> problem.
> Attached is the binary compatibility report of jna.jar between 4.0 and 4.2.
> The result is good (97.4%). So is there possibility to upgrade jna to 4.2.0
> in upstream? Should there be any kind of tests to execute, please kindly
> point me. Thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13072) Cassandra failed to run on Linux-aarch64
[
https://issues.apache.org/jira/browse/CASSANDRA-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-13072:
--
Status: Ready to Commit (was: Patch Available)
> Cassandra failed to run on Linux-aarch64
>
>
> Key: CASSANDRA-13072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13072
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Hardware: ARM aarch64
> OS: Ubuntu 16.04.1 LTS
>Reporter: Jun He
>Assignee: Benjamin Lerer
> Labels: incompatible
> Attachments: compat_report.html
>
>
> Steps to reproduce:
> 1. Download cassandra latest source
> 2. Build it with "ant"
> 3. Run with "./bin/cassandra". Daemon is crashed with following error message:
> {quote}
> INFO 05:30:21 Initializing system.schema_functions
> INFO 05:30:21 Initializing system.schema_aggregates
> ERROR 05:30:22 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
> at
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:97)
> ~[main/:na]
> at org.apache.cassandra.io.util.Memory.(Memory.java:74)
> ~[main/:na]
> at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:519)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:497)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:480)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.createFlushWriter(Memtable.java:439)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:371)
> ~[main/:na]
> at org.apache.cassandra.db.Memtable.flush(Memtable.java:332)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> {quote}
> Analyze:
> This issue is caused by bundled jna-4.0.0.jar which doesn't come with aarch64
> native support. Replace lib/jna-4.0.0.jar with jna-4.2.0.jar from
> http://central.maven.org/maven2/net/java/dev/jna/jna/4.2.0/ can fix this
> problem.
> Attached is the binary compatibility report of jna.jar between 4.0 and 4.2.
> The result is good (97.4%). So is there possibility to upgrade jna to 4.2.0
> in upstream? Should there be any kind of tests to execute, please kindly
> point me. Thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13072) Cassandra failed to run on Linux-aarch64
[
https://issues.apache.org/jira/browse/CASSANDRA-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030008#comment-16030008
]
Jeff Jirsa commented on CASSANDRA-13072:
FWIW, I don't disagree we need to do this, but for ppc64le, we spent a lot of
time saying we shouldn't make changes for other platforms that didnt have
public tests; we should try to get public tests for platforms, especially if
we're going to bump JNA in a branch that is almost 2 years old (2.2).
> Cassandra failed to run on Linux-aarch64
>
>
> Key: CASSANDRA-13072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13072
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Hardware: ARM aarch64
> OS: Ubuntu 16.04.1 LTS
>Reporter: Jun He
>Assignee: Benjamin Lerer
> Labels: incompatible
> Attachments: compat_report.html
>
>
> Steps to reproduce:
> 1. Download cassandra latest source
> 2. Build it with "ant"
> 3. Run with "./bin/cassandra". Daemon is crashed with following error message:
> {quote}
> INFO 05:30:21 Initializing system.schema_functions
> INFO 05:30:21 Initializing system.schema_aggregates
> ERROR 05:30:22 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
> at
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:97)
> ~[main/:na]
> at org.apache.cassandra.io.util.Memory.(Memory.java:74)
> ~[main/:na]
> at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
> ~[main/:na]
> at
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
> ~[main/:na]
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:519)
> ~[main/:na]
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:497)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:480)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.createFlushWriter(Memtable.java:439)
> ~[main/:na]
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:371)
> ~[main/:na]
> at org.apache.cassandra.db.Memtable.flush(Memtable.java:332)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_111]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ~[na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> {quote}
> Analyze:
> This issue is caused by bundled jna-4.0.0.jar which doesn't come with aarch64
> native support. Replace lib/jna-4.0.0.jar with jna-4.2.0.jar from
> http://central.maven.org/maven2/net/java/dev/jna/jna/4.2.0/ can fix this
> problem.
> Attached is the binary compatibility report of jna.jar between 4.0 and 4.2.
> The result is good (97.4%). So is there possibility to upgrade jna to 4.2.0
> in upstream? Should there be any kind of tests to execute, please kindly
> point me. Thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13004) Corruption while adding a column to a table
[
https://issues.apache.org/jira/browse/CASSANDRA-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030026#comment-16030026
]
Jeff Jirsa commented on CASSANDRA-13004:
So I think two teams are trying to diagnose this in parallel, and I want to
make sure we're communicating more broadly.
Both of us (myself, and the folks at datastax) believe this is in read repair.
Both of us believe that the fundamental problem is the subset encoding of
columns ( from CASSANDRA-9894 ), where our concept of column filters ("*" = all
columns) and read repair interact in unpleasant ways.
I'm running some tests locally with
[this|https://github.com/jeffjirsa/cassandra/commit/5206ed2b9c18c03d6edf2682ff8b162e1d2b7486]
patch, which so far seems to solve it, but let's not pretend this is a
blessing that it's the only or right fix yet (as you can see, there are no
tests, so please don't run this patch in prod, it's offered as-is without
warranty).
> Corruption while adding a column to a table
> ---
>
> Key: CASSANDRA-13004
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13004
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata
>Reporter: Stanislav Vishnevskiy
>Priority: Blocker
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> We had the following schema in production.
> {code:none}
> CREATE TYPE IF NOT EXISTS discord_channels.channel_recipient (
> nick text
> );
> CREATE TYPE IF NOT EXISTS discord_channels.channel_permission_overwrite (
> id bigint,
> type int,
> allow_ int,
> deny int
> );
> CREATE TABLE IF NOT EXISTS discord_channels.channels (
> id bigint,
> guild_id bigint,
> type tinyint,
> name text,
> topic text,
> position int,
> owner_id bigint,
> icon_hash text,
> recipients map>,
> permission_overwrites map>,
> bitrate int,
> user_limit int,
> last_pin_timestamp timestamp,
> last_message_id bigint,
> PRIMARY KEY (id)
> );
> {code}
> And then we executed the following alter.
> {code:none}
> ALTER TABLE discord_channels.channels ADD application_id bigint;
> {code}
> And one row (that we can tell) got corrupted at the same time and could no
> longer be read from the Python driver.
> {code:none}
> [E 161206 01:56:58 geventreactor:141] Error decoding response from Cassandra.
> ver(4); flags(); stream(27); op(8); offset(9); len(887); buffer:
> '\x84\x00\x00\x1b\x08\x00\x00\x03w\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x0f\x00\x10discord_channels\x00\x08channels\x00\x02id\x00\x02\x00\x0eapplication_id\x00\x02\x00\x07bitrate\x00\t\x00\x08guild_id\x00\x02\x00\ticon_hash\x00\r\x00\x0flast_message_id\x00\x02\x00\x12last_pin_timestamp\x00\x0b\x00\x04name\x00\r\x00\x08owner_id\x00\x02\x00\x15permission_overwrites\x00!\x00\x02\x000\x00\x10discord_channels\x00\x1cchannel_permission_overwrite\x00\x04\x00\x02id\x00\x02\x00\x04type\x00\t\x00\x06allow_\x00\t\x00\x04deny\x00\t\x00\x08position\x00\t\x00\nrecipients\x00!\x00\x02\x000\x00\x10discord_channels\x00\x11channel_recipient\x00\x01\x00\x04nick\x00\r\x00\x05topic\x00\r\x00\x04type\x00\x14\x00\nuser_limit\x00\t\x00\x00\x00\x01\x00\x00\x00\x08\x03\x8a\x19\x8e\xf8\x82\x00\x01\xff\xff\xff\xff\x00\x00\x00\x04\x00\x00\xfa\x00\x00\x00\x00\x08\x00\x00\xfa\x00\x00\xf8G\xc5\x00\x00\x00\x00\x00\x00\x00\x08\x03\x8b\xc0\xb5nB\x00\x02\x00\x00\x00\x08G\xc5\xffI\x98\xc4\xb4(\x00\x00\x00\x03\x8b\xc0\xa8\xff\xff\xff\xff\x00\x00\x01<\x00\x00\x00\x06\x00\x00\x00\x08\x03\x81L\xea\xfc\x82\x00\n\x00\x00\x00$\x00\x00\x00\x08\x03\x81L\xea\xfc\x82\x00\n\x00\x00\x00\x04\x00\x00\x00\x01\x00\x00\x00\x04\x00\x00\x08\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x08\x03\x8a\x1e\xe6\x8b\x80\x00\n\x00\x00\x00$\x00\x00\x00\x08\x03\x8a\x1e\xe6\x8b\x80\x00\n\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x040\x07\xf8Q\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x08\x03\x8a\x1f\x1b{\x82\x00\x00\x00\x00\x00$\x00\x00\x00\x08\x03\x8a\x1f\x1b{\x82\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x04\x00\x07\xf8Q\x00\x00\x00\x04\x10\x00\x00\x00\x00\x00\x00\x08\x03\x8a\x1fH6\x82\x00\x01\x00\x00\x00$\x00\x00\x00\x08\x03\x8a\x1fH6\x82\x00\x01\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x04\x00\x05\xe8A\x00\x00\x00\x04\x10\x02\x00\x00\x00\x00\x00\x08\x03\x8a+=\xca\xc0\x00\n\x00\x00\x00$\x00\x00\x00\x08\x03\x8a+=\xca\xc0\x00\n\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x08\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x08\x03\x8a\x8f\x979\x80\x00\n\x00\x00\x00$\x00\x00\x00\x08\x03\x8a\x8f\x979\x80\x00\n\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x04\x00
>
> \x08\x01\x00\x00\x00\x04\xc4\xb4(\x00\xff\xff\xff\xff\x00\x00\x00O[f\x80Q\x07general\x05\xf8G\xc5\xffI\x98\xc4\xb4(\x00\xf8O[f\x80Q\x00\x00\x00\x02\x04\xf8O[f\x80Q\x00\xf8G\xc5\x
[jira] [Commented] (CASSANDRA-7960) cassandra-stress should support LWT
[ https://issues.apache.org/jira/browse/CASSANDRA-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030175#comment-16030175 ] Jay Zhuang commented on CASSANDRA-7960: --- Hi [~jjirsa] [~kohlisankalp] [~bdeggleston] this is the patch I used to benchmark CASSANDRA-13508, would you please check and commit the change? > cassandra-stress should support LWT > --- > > Key: CASSANDRA-7960 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7960 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Benedict >Assignee: Jay Zhuang >Priority: Minor > Labels: stress > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-7960) cassandra-stress should support LWT
[ https://issues.apache.org/jira/browse/CASSANDRA-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030175#comment-16030175 ] Jay Zhuang edited comment on CASSANDRA-7960 at 5/30/17 9:35 PM: Hi [~jjirsa], [~kohlisankalp], [~bdeggleston] this is the patch I used to benchmark CASSANDRA-13508, would you please check and commit the change? was (Author: jay.zhuang): Hi [~jjirsa] [~kohlisankalp] [~bdeggleston] this is the patch I used to benchmark CASSANDRA-13508, would you please check and commit the change? > cassandra-stress should support LWT > --- > > Key: CASSANDRA-7960 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7960 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Benedict >Assignee: Jay Zhuang >Priority: Minor > Labels: stress > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13508) Make system.paxos table compaction strategy configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jay Zhuang updated CASSANDRA-13508:
---
Status: Open (was: Patch Available)
> Make system.paxos table compaction strategy configurable
>
>
> Key: CASSANDRA-13508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13508
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
> Labels: core, paxos
> Fix For: 4.0, 4.x
>
> Attachments: test11.png, test2.png
>
>
> The default compaction strategy for {{system.paxos}} table is LCS for
> performance reason: CASSANDRA-7753. But for CAS heavily used cluster, the
> system is busy with {{system.paxos}} compaction.
> As the data in {{paxos}} table are TTL'ed, TWCS might be a better fit. In our
> test, it significantly reduced the number of compaction without impacting the
> latency too much:
> !test11.png!
> The time window for TWCS is set to 2 minutes for the test.
> Here is the p99 latency impact:
> !test2.png!
> the yellow one is LCS, the purple one is TWCS. Average p99 has about 10%
> increase.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-9608) Support Java 9
[
https://issues.apache.org/jira/browse/CASSANDRA-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030429#comment-16030429
]
Paul Sandoz commented on CASSANDRA-9608:
The JMX RMI connector was moved into it's own module, java.management.rmi. See:
https://bugs.openjdk.java.net/browse/JDK-8173607
As a result the class com.sun.jmx.remote.internal.RMIExporter was moved to the
package com.sun.jmx.remote.internal.rmi.
The Cassandra class org.apache.cassandra.utils.JMXServerUtils depends on
RMIExporter.
In total JMXServerUtils depends on the following internal classes:
com.sun.jmx.remote.internal.RMIExporter
com.sun.jmx.remote.security.JMXPluggableAuthenticator
sun.rmi.registry.RegistryImpl
sun.rmi.server.UnicastServerRef2
> Support Java 9
> --
>
> Key: CASSANDRA-9608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9608
> Project: Cassandra
> Issue Type: Task
>Reporter: Robert Stupp
>Priority: Minor
>
> This ticket is intended to group all issues found to support Java 9 in the
> future.
> From what I've found out so far:
> * Maven dependency {{com.sun:tools:jar:0}} via cobertura cannot be resolved.
> It can be easily solved using this patch:
> {code}
> - artifactId="cobertura"/>
> + artifactId="cobertura">
> +
> +
> {code}
> * Another issue is that {{sun.misc.Unsafe}} no longer contains the methods
> {{monitorEnter}} + {{monitorExit}}. These methods are used by
> {{o.a.c.utils.concurrent.Locks}} which is only used by
> {{o.a.c.db.AtomicBTreeColumns}}.
> I don't mind to start working on this yet since Java 9 is in a too early
> development phase.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13508) Make system.paxos table compaction strategy configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-13508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030443#comment-16030443
]
Jay Zhuang commented on CASSANDRA-13508:
[~bdeggleston] You're right, the configuration is lost after restart. To make
it configurable in {{cassandra.yaml}}, what do you think about the options like
this:
{noformat}
system_paxos_compaction_strategy:
- class_name: LeveledCompactionStrategy
parameters:
- sstable_size_in_mb: "160MB"
tombstone_threshold: "0.2"
{noformat}
> Make system.paxos table compaction strategy configurable
>
>
> Key: CASSANDRA-13508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13508
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
> Labels: core, paxos
> Fix For: 4.0, 4.x
>
> Attachments: test11.png, test2.png
>
>
> The default compaction strategy for {{system.paxos}} table is LCS for
> performance reason: CASSANDRA-7753. But for CAS heavily used cluster, the
> system is busy with {{system.paxos}} compaction.
> As the data in {{paxos}} table are TTL'ed, TWCS might be a better fit. In our
> test, it significantly reduced the number of compaction without impacting the
> latency too much:
> !test11.png!
> The time window for TWCS is set to 2 minutes for the test.
> Here is the p99 latency impact:
> !test2.png!
> the yellow one is LCS, the purple one is TWCS. Average p99 has about 10%
> increase.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13548) system.paxos improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-13548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030457#comment-16030457
]
Jay Zhuang commented on CASSANDRA-13548:
How about making Paxos TTL configurable:
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/SystemKeyspace.java#L1067
? Why it has to store at least 3 hours Paxos data?
> system.paxos improvements
> -
>
> Key: CASSANDRA-13548
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13548
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Jeff Jirsa
>Priority: Minor
> Fix For: 4.x
>
>
> There are a few practical changes we can make to {{system.paxos}} that will
> improve (especially read) performance for LWT:
> - We should decrease the compression chunk size for situations where we have
> to go to disk
> - We can change the primary key structure so that the row key and CFID are
> both part of the partition key, which will decrease LCS compaction activity
> in use cases where a row key is common across tables, and one table is
> updated more frequently than the other.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-13559) Schema version id mismatch while upgrading to 3.0.13
[
https://issues.apache.org/jira/browse/CASSANDRA-13559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefania reassigned CASSANDRA-13559:
Assignee: Stefania
> Schema version id mismatch while upgrading to 3.0.13
>
>
> Key: CASSANDRA-13559
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13559
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jay Zhuang
>Assignee: Stefania
>Priority: Blocker
>
> As the order of SchemaKeyspace is changed ([6991556 |
> https://github.com/apache/cassandra/commit/6991556e431a51575744248a4c484270c4f918c9],
> CASSANDRA-12213), the result of function
> [{{calculateSchemaDigest}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L311]
> is also changed for the same schema. Which causes schema mismatch while
> upgrading 3.0.x -> 3.0.13.
> It could cause cassandra fail to start because Unknown CF exception. And
> streaming will fail:
> {noformat}
> ERROR [main] 2017-05-26 18:58:57,572 CassandraDaemon.java:709 - Exception
> encountered during startup
> java.lang.IllegalArgumentException: Unknown CF
> 83c8eae0-3a65-11e7-9a27-e17fd11571e3
> {noformat}
> {noformat}
> WARN [MessagingService-Incoming-/IP] 2017-05-26 19:27:11,523
> IncomingTcpConnection.java:101 - UnknownColumnFamilyException reading from
> socket; closing
> org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find table for
> cfId 922b7940-3a65-11e7-adf3-a3ff55d9bcf1. If a table was just created, this
> is likely due to the schema not being fully propagated. Please wait for
> schema agreement on table creation.
> {noformat}
> Restart the new node will cause:
> {noformat}
> Exception (java.lang.NoSuchFieldError) encountered during startup: ALL
> java.lang.NoSuchFieldError: ALL
> at
> org.apache.cassandra.service.ClientState.(ClientState.java:67)
> at
> org.apache.cassandra.cql3.QueryProcessor$InternalStateInstance.(QueryProcessor.java:155)
> at
> org.apache.cassandra.cql3.QueryProcessor$InternalStateInstance.(QueryProcessor.java:149)
> at
> org.apache.cassandra.cql3.QueryProcessor.internalQueryState(QueryProcessor.java:163)
> at
> org.apache.cassandra.cql3.QueryProcessor.prepareInternal(QueryProcessor.java:286)
> at
> org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:294)
> at
> org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:900)
> at
> org.apache.cassandra.service.StartupChecks$9.execute(StartupChecks.java:354)
> at
> org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:110)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:179)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:697)
> {noformat}
> I would suggest to have the older list back for digest calculation and
> release 3.0.14.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13559) Schema version id mismatch while upgrading to 3.0.13
[
https://issues.apache.org/jira/browse/CASSANDRA-13559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030574#comment-16030574
]
Stefania commented on CASSANDRA-13559:
--
I agree this is a problem, thanks for reporting it. Since we need both the new
and old orders, I prefer to only change the order for flushing and restore the
old order for everything else. I am going to prepare a patch shortly.
> Schema version id mismatch while upgrading to 3.0.13
>
>
> Key: CASSANDRA-13559
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13559
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jay Zhuang
>Assignee: Stefania
>Priority: Blocker
>
> As the order of SchemaKeyspace is changed ([6991556 |
> https://github.com/apache/cassandra/commit/6991556e431a51575744248a4c484270c4f918c9],
> CASSANDRA-12213), the result of function
> [{{calculateSchemaDigest}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L311]
> is also changed for the same schema. Which causes schema mismatch while
> upgrading 3.0.x -> 3.0.13.
> It could cause cassandra fail to start because Unknown CF exception. And
> streaming will fail:
> {noformat}
> ERROR [main] 2017-05-26 18:58:57,572 CassandraDaemon.java:709 - Exception
> encountered during startup
> java.lang.IllegalArgumentException: Unknown CF
> 83c8eae0-3a65-11e7-9a27-e17fd11571e3
> {noformat}
> {noformat}
> WARN [MessagingService-Incoming-/IP] 2017-05-26 19:27:11,523
> IncomingTcpConnection.java:101 - UnknownColumnFamilyException reading from
> socket; closing
> org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find table for
> cfId 922b7940-3a65-11e7-adf3-a3ff55d9bcf1. If a table was just created, this
> is likely due to the schema not being fully propagated. Please wait for
> schema agreement on table creation.
> {noformat}
> Restart the new node will cause:
> {noformat}
> Exception (java.lang.NoSuchFieldError) encountered during startup: ALL
> java.lang.NoSuchFieldError: ALL
> at
> org.apache.cassandra.service.ClientState.(ClientState.java:67)
> at
> org.apache.cassandra.cql3.QueryProcessor$InternalStateInstance.(QueryProcessor.java:155)
> at
> org.apache.cassandra.cql3.QueryProcessor$InternalStateInstance.(QueryProcessor.java:149)
> at
> org.apache.cassandra.cql3.QueryProcessor.internalQueryState(QueryProcessor.java:163)
> at
> org.apache.cassandra.cql3.QueryProcessor.prepareInternal(QueryProcessor.java:286)
> at
> org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:294)
> at
> org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:900)
> at
> org.apache.cassandra.service.StartupChecks$9.execute(StartupChecks.java:354)
> at
> org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:110)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:179)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:697)
> {noformat}
> I would suggest to have the older list back for digest calculation and
> release 3.0.14.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13559) Schema version id mismatch while upgrading to 3.0.13
[
https://issues.apache.org/jira/browse/CASSANDRA-13559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030598#comment-16030598
]
Jeff Jirsa commented on CASSANDRA-13559:
Any thoughts on an upgrade path from 3.0.13 - 3.0.14 Stefania?
> Schema version id mismatch while upgrading to 3.0.13
>
>
> Key: CASSANDRA-13559
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13559
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Jay Zhuang
>Assignee: Stefania
>Priority: Blocker
>
> As the order of SchemaKeyspace is changed ([6991556 |
> https://github.com/apache/cassandra/commit/6991556e431a51575744248a4c484270c4f918c9],
> CASSANDRA-12213), the result of function
> [{{calculateSchemaDigest}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L311]
> is also changed for the same schema. Which causes schema mismatch while
> upgrading 3.0.x -> 3.0.13.
> It could cause cassandra fail to start because Unknown CF exception. And
> streaming will fail:
> {noformat}
> ERROR [main] 2017-05-26 18:58:57,572 CassandraDaemon.java:709 - Exception
> encountered during startup
> java.lang.IllegalArgumentException: Unknown CF
> 83c8eae0-3a65-11e7-9a27-e17fd11571e3
> {noformat}
> {noformat}
> WARN [MessagingService-Incoming-/IP] 2017-05-26 19:27:11,523
> IncomingTcpConnection.java:101 - UnknownColumnFamilyException reading from
> socket; closing
> org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find table for
> cfId 922b7940-3a65-11e7-adf3-a3ff55d9bcf1. If a table was just created, this
> is likely due to the schema not being fully propagated. Please wait for
> schema agreement on table creation.
> {noformat}
> Restart the new node will cause:
> {noformat}
> Exception (java.lang.NoSuchFieldError) encountered during startup: ALL
> java.lang.NoSuchFieldError: ALL
> at
> org.apache.cassandra.service.ClientState.(ClientState.java:67)
> at
> org.apache.cassandra.cql3.QueryProcessor$InternalStateInstance.(QueryProcessor.java:155)
> at
> org.apache.cassandra.cql3.QueryProcessor$InternalStateInstance.(QueryProcessor.java:149)
> at
> org.apache.cassandra.cql3.QueryProcessor.internalQueryState(QueryProcessor.java:163)
> at
> org.apache.cassandra.cql3.QueryProcessor.prepareInternal(QueryProcessor.java:286)
> at
> org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:294)
> at
> org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:900)
> at
> org.apache.cassandra.service.StartupChecks$9.execute(StartupChecks.java:354)
> at
> org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:110)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:179)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:569)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:697)
> {noformat}
> I would suggest to have the older list back for digest calculation and
> release 3.0.14.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
