[jira] [Commented] (CASSANDRA-12478) cassandra stress still uses CFMetaData.compile()
[
https://issues.apache.org/jira/browse/CASSANDRA-12478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15440972#comment-15440972
]
Robert Stupp commented on CASSANDRA-12478:
--
Hint: before you rebase, check CASSANDRA-12550 - the new methods in {{Config}}
are moved to {{DatabaseDescriptor}} and the existing {{set/isClientMode()}} are
deprecated. The cleanest way would be to use client mode - i.e. call
{{DatabaseDescriptor.clientInitialization()}} (Config.setClientMode(true) is
not sufficient).
Want to note, that using {{QueryProcessor}} class is not the right way IMO
since QP itself has some implicit dependencies (in the static initializer and
via class dependencies). However, if the reason is just to get the keyspace and
table name, you can directly use
{{CQLFragmentParser.parseAnyUnhandled(CqlParser::query, queryStr);}} as in
{{QueryProcessor.parseStatement}} - that's much cleaner and should have no
"evil" dependencies (i.e. threads, file system, etc).
To clarify the different init modes in DD: client just gets an empty {{Config}}
object, tools gets most of the stuff and is intended for stuff like sstable
tools, daemon is, well, for the daemon.
If you want to ensure, that stress does not start threads or initializes stuff
like commit-log-handling, you could add a new utest for stress using the
{{ToolsTester}} class and do something like in the following code snippet (not
tested). This helps to ensure that no unwanted stuff ist initialized/started.
{code}
@RunWith(OrderedJUnit4ClassRunner.class)
public class StressToolTest extends ToolsTester
{
@Test
public void testSSTableExpiredBlockers_NoArgs()
{
runTool(1, "org.apache.cassandra.stress.Stress", );
assertNoUnexpectedThreadsStarted(null, null); // may need to add some
thread name regex, if stress starts threads depending on cl args
assertSchemaNotLoaded();
assertCLSMNotLoaded();
assertSystemKSNotLoaded();
assertKeyspaceNotLoaded();
assertServerNotLoaded();
}
{code}
> cassandra stress still uses CFMetaData.compile()
>
>
> Key: CASSANDRA-12478
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12478
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Denis Ranger
> Labels: stress
> Fix For: 3.0.x
>
> Attachments:
> 0001-Replaced-using-CFMetaData.compile-in-cassandra-stres.patch
>
>
> Using CFMetaData.compile() on a client tool causes permission problems. To
> reproduce:
> * Start cassandra under user _cassandra_
> * Run {{chmod -R go-rwx /var/lib/cassandra}} to deny access to other users.
> * Use a non-root user to run {{cassandra-stress}}
> This produces an access denied message on {{/var/lib/cassandra/commitlog}}.
> The attached fix uses client-mode functionality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: Replace Config.setClientMode with DatabaseDescriptor.clientInit()
Repository: cassandra
Updated Branches:
refs/heads/trunk a8743c6e3 -> 29239c765
Replace Config.setClientMode with DatabaseDescriptor.clientInit()
patch by Robert Stupp; reviewed by Jeremiah Jordan for CASSANDRA-12550
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/29239c76
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/29239c76
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/29239c76
Branch: refs/heads/trunk
Commit: 29239c765c22a7c1cb3a64476ebf0dfa9e2f9da0
Parents: a8743c6
Author: Robert Stupp
Authored: Sat Aug 27 09:59:11 2016 +0200
Committer: Robert Stupp
Committed: Sat Aug 27 09:59:11 2016 +0200
--
CHANGES.txt | 2 +-
NEWS.txt| 3 +-
.../org/apache/cassandra/config/Config.java | 27 ++
.../cassandra/config/DatabaseDescriptor.java| 88 ++--
.../org/apache/cassandra/config/Schema.java | 2 +-
src/java/org/apache/cassandra/db/Keyspace.java | 2 +-
.../cassandra/io/sstable/CQLSSTableWriter.java | 2 +-
.../io/sstable/format/SSTableReader.java| 4 +-
.../locator/DynamicEndpointSnitch.java | 7 +-
.../apache/cassandra/service/ClientState.java | 3 +-
.../io/sstable/CQLSSTableWriterLongTest.java| 6 --
.../cassandra/streaming/LongStreamingTest.java | 6 --
.../test/microbench/MutationBench.java | 2 +-
.../io/sstable/CQLSSTableWriterClientTest.java | 18 ++--
.../io/sstable/CQLSSTableWriterTest.java| 6 --
15 files changed, 105 insertions(+), 73 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/29239c76/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b3975da..3dd46de 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -9,7 +9,7 @@
* Extend read/write failure messages with a map of replica addresses
to error codes in the v5 native protocol (CASSANDRA-12311)
* Fix rebuild of SASI indexes with existing index files (CASSANDRA-12374)
- * Let DatabaseDescriptor not implicitly startup services (CASSANDRA-9054)
+ * Let DatabaseDescriptor not implicitly startup services (CASSANDRA-9054,
12550)
* Fix clustering indexes in presence of static columns in SASI
(CASSANDRA-12378)
* Fix queries on columns with reversed type on SASI indexes (CASSANDRA-12223)
* Added slow query log (CASSANDRA-12403)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/29239c76/NEWS.txt
--
diff --git a/NEWS.txt b/NEWS.txt
index a22fdd8..bb21c3c 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -87,7 +87,8 @@ Upgrading
a standalone tool or client application, use the
DatabaseDescriptor.toolInitialization() or
DatabaseDescriptor.clientInitialization() methods. Tool initialization
sets up partitioner,
snitch, encryption context. Client initialization just applies the
configuration but does not
- setup anything.
+ setup anything. Instead of using Config.setClientMode() or
Config.isClientMode(), which are
+ deprecated now, use one of the appropiate new methods in
DatabaseDescriptor.
3.8
===
http://git-wip-us.apache.org/repos/asf/cassandra/blob/29239c76/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java
b/src/java/org/apache/cassandra/config/Config.java
index 7248031..87f664d 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -257,7 +257,6 @@ public class Config
public volatile int counter_cache_keys_to_save = Integer.MAX_VALUE;
private static boolean isClientMode = false;
-private static boolean isToolsMode = false;
public Integer file_cache_size_in_mb;
@@ -360,6 +359,10 @@ public class Config
outboundBindAny = value;
}
+/**
+ * @deprecated migrate to {@link DatabaseDescriptor#isClientInitialized()}
+ */
+@Deprecated
public static boolean isClientMode()
{
return isClientMode;
@@ -368,31 +371,15 @@ public class Config
/**
* Client mode means that the process is a pure client, that uses C* code
base but does
* not read or write local C* database files.
+ *
+ * @deprecated migrate to {@link
DatabaseDescriptor#clientInitialization(boolean)}
*/
+@Deprecated
public static void setClientMode(boolean clientMode)
{
isClientMode = clientMode;
}
-public static boolean isToolsMode()
-{
-return isToolsMode;
-}
-
-/**
- * Tools mode means that the process is a standalone (offline) C
[jira] [Updated] (CASSANDRA-12550) Replace Config.setClientMode with DatabaseDescriptor.clientInit()
[
https://issues.apache.org/jira/browse/CASSANDRA-12550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-12550:
-
Resolution: Fixed
Fix Version/s: (was: 3.x)
3.10
Status: Resolved (was: Patch Available)
Thanks!
Committed as
[29239c765c22a7c1cb3a64476ebf0dfa9e2f9da0|https://github.com/apache/cassandra/commit/29239c765c22a7c1cb3a64476ebf0dfa9e2f9da0]
to [trunk|https://github.com/apache/cassandra/tree/trunk]
> Replace Config.setClientMode with DatabaseDescriptor.clientInit()
> -
>
> Key: CASSANDRA-12550
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12550
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Fix For: 3.10
>
>
> After CASSANDRA-9054, uses of {{CQLSSTableWriter}} may become screwed, since
> it just calls {{Config.setClientMode(true)}} and implicit initialization of
> DD has been removed with 9754.
> This ticket is to
> * deprecate Config.setClientMode() + Config.isClientMode()
> * replace usages of Config.setClientMode() with the methods in DD
> Further, the patch cleans up the check-methods introduced by 9754.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11889) LogRecord: file system race condition may cause verify() to fail
[
https://issues.apache.org/jira/browse/CASSANDRA-11889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15441817#comment-15441817
]
Stefano Ortolani commented on CASSANDRA-11889:
--
Based on what I understood 3.0.9 does't get frozen like 3.9, so I am still
hoping for this to make it to 3.0.9 if possible :)
> LogRecord: file system race condition may cause verify() to fail
>
>
> Key: CASSANDRA-11889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11889
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 3.0.x, 3.x
>
>
> The following exception was reported in CASSANDRA-11470. It occurred whilst
> listing files with compaction in progress:
> {code}
> WARN [CompactionExecutor:2006] 2016-05-23 18:23:31,694
> BigTableWriter.java:171 - Writing large partition
> test_keyspace/test_columnfamily:eda6b9c36f8df6fe596492c3438d7a38e9b109a6
> (123663388 bytes)
> INFO [IndexSummaryManager:1] 2016-05-23 18:24:23,731
> IndexSummaryRedistribution.java:74 - Redistributing index summaries
> WARN [CompactionExecutor:2006] 2016-05-23 18:24:56,669
> BigTableWriter.java:171 - Writing large partition
> test_keyspace/test_columnfamily:05b6b424194dd19ab7cfbcd53c4979768cd859e9
> (256286063 bytes)
> WARN [CompactionExecutor:2006] 2016-05-23 18:26:23,575
> BigTableWriter.java:171 - Writing large partition
> test_keyspace/test_columnfamily:04e9fac15552b9ae77c27a6cb8d0fd11fdcc24d7
> (212445557 bytes)
> INFO [CompactionExecutor:2005] 2016-05-23 18:29:26,839
> LeveledManifest.java:437 - Adding high-level (L3)
> BigTableReader(path='/data/cassandra/data/test_keyspace/test_columnfamily_2-d29dd71045a811e59aff6776bf484396/ma-61041-big-Data.db')
> to candidates
> WARN [CompactionExecutor:2006] 2016-05-23 18:30:34,154
> BigTableWriter.java:171 - Writing large partition
> test_keyspace/test_columnfamily:edbe6f178503be90911dbf29a55b97a4b095a9ec
> (183852539 bytes)
> INFO [CompactionExecutor:2006] 2016-05-23 18:31:21,080
> LeveledManifest.java:437 - Adding high-level (L3)
> BigTableReader(path='/data/cassandra/data/test_keyspace/test_columnfamily_2-d29dd71045a811e59aff6776bf484396/ma-61042-big-Data.db')
> to candidates
> ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-23 18:31:21,207
> LogFile.java:173 - Unexpected files detected for sstable [ma-91034-big],
> record
> [REMOVE:[/data/cassandra/data/test_keyspace/test_columnfamily-3996ce80b7ac11e48a9b6776bf484396/ma-91034-big,1463992176000,8][457420186]]:
> last update time [00:00:00] should have been [08:29:36]
> ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-23 18:31:21,208
> ScheduledReporter.java:119 - RuntimeException thrown from
> GraphiteReporter#report. Exception was suppressed.
> java.lang.RuntimeException: Failed to list files in
> /data/cassandra/data/test_keyspace/test_columnfamily-3996ce80b7ac11e48a9b6776bf484396
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:57)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:547)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> org.apache.cassandra.db.Directories$SSTableLister.filter(Directories.java:691)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> org.apache.cassandra.db.Directories$SSTableLister.listFiles(Directories.java:662)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> org.apache.cassandra.db.Directories$TrueFilesSizeVisitor.(Directories.java:981)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> org.apache.cassandra.db.Directories.getTrueAllocatedSizeIn(Directories.java:893)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> org.apache.cassandra.db.Directories.trueSnapshotsSize(Directories.java:883)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> org.apache.cassandra.db.ColumnFamilyStore.trueSnapshotsSize(ColumnFamilyStore.java:2332)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> org.apache.cassandra.metrics.TableMetrics$32.getValue(TableMetrics.java:637)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> org.apache.cassandra.metrics.TableMetrics$32.getValue(TableMetrics.java:634)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:692)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:686)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:281)
> ~[metrics-graphite-3.1.0.jar:3.1.0]
> at
> com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteRepor
[jira] [Commented] (CASSANDRA-12499) Row cache does not cache partitions on tables without clustering keys
[
https://issues.apache.org/jira/browse/CASSANDRA-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15441819#comment-15441819
]
Jeff Jirsa commented on CASSANDRA-12499:
Branch here:
https://github.com/jeffjirsa/cassandra/commit/e7cb6c3409b889174d32c4eec9f60da380f39d3d
Believe patch applies cleanly to 3.0 -> trunk (or it did a few days ago when
written)
testall shows no failures:
http://cassci.datastax.com/job/jeffjirsa-cassandra-12499-testall/lastBuild/testReport/
dtest shows one failure that appears unrelated:
http://cassci.datastax.com/job/jeffjirsa-cassandra-12499-dtest/
> Row cache does not cache partitions on tables without clustering keys
> -
>
> Key: CASSANDRA-12499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12499
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Labels: Performance
>
> {code}
> MLSEA-JJIRSA01:~ jjirsa$ ccm start
> MLSEA-JJIRSA01:~ jjirsa$ echo "DESCRIBE TABLE test.test; " | ccm node1 cqlsh
> CREATE TABLE test.test (
> id int PRIMARY KEY,
> v text
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': '100'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> MLSEA-JJIRSA01:~ jjirsa$ ccm node1 nodetool info | grep Row
> Row Cache : entries 0, size 0 bytes, capacity 100 MiB, 0 hits, 0
> requests, NaN recent hit rate, 0 save period in seconds
> MLSEA-JJIRSA01:~ jjirsa$ echo "INSERT INTO test.test(id,v) VALUES(1, 'a'); "
> | ccm node1 cqlsh
> MLSEA-JJIRSA01:~ jjirsa$ echo "SELECT * FROM test.test WHERE id=1; " | ccm
> node1 cqlsh
> id | v
> +---
> 1 | a
> (1 rows)
> MLSEA-JJIRSA01:~ jjirsa$ ccm node1 nodetool info | grep Row
> Row Cache : entries 0, size 0 bytes, capacity 100 MiB, 0 hits, 0
> requests, NaN recent hit rate, 0 save period in seconds
> MLSEA-JJIRSA01:~ jjirsa$ echo "SELECT * FROM test.test WHERE id=1; " | ccm
> node1 cqlsh
> id | v
> +---
> 1 | a
> (1 rows)
> MLSEA-JJIRSA01:~ jjirsa$ ccm node1 nodetool info | grep Row
> Row Cache : entries 0, size 0 bytes, capacity 100 MiB, 0 hits, 0
> requests, NaN recent hit rate, 0 save period in seconds
> MLSEA-JJIRSA01:~ jjirsa$
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12499) Row cache does not cache partitions on tables without clustering keys
[
https://issues.apache.org/jira/browse/CASSANDRA-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-12499:
---
Status: Patch Available (was: Open)
> Row cache does not cache partitions on tables without clustering keys
> -
>
> Key: CASSANDRA-12499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12499
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
> Labels: Performance
>
> {code}
> MLSEA-JJIRSA01:~ jjirsa$ ccm start
> MLSEA-JJIRSA01:~ jjirsa$ echo "DESCRIBE TABLE test.test; " | ccm node1 cqlsh
> CREATE TABLE test.test (
> id int PRIMARY KEY,
> v text
> ) WITH bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': '100'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> MLSEA-JJIRSA01:~ jjirsa$ ccm node1 nodetool info | grep Row
> Row Cache : entries 0, size 0 bytes, capacity 100 MiB, 0 hits, 0
> requests, NaN recent hit rate, 0 save period in seconds
> MLSEA-JJIRSA01:~ jjirsa$ echo "INSERT INTO test.test(id,v) VALUES(1, 'a'); "
> | ccm node1 cqlsh
> MLSEA-JJIRSA01:~ jjirsa$ echo "SELECT * FROM test.test WHERE id=1; " | ccm
> node1 cqlsh
> id | v
> +---
> 1 | a
> (1 rows)
> MLSEA-JJIRSA01:~ jjirsa$ ccm node1 nodetool info | grep Row
> Row Cache : entries 0, size 0 bytes, capacity 100 MiB, 0 hits, 0
> requests, NaN recent hit rate, 0 save period in seconds
> MLSEA-JJIRSA01:~ jjirsa$ echo "SELECT * FROM test.test WHERE id=1; " | ccm
> node1 cqlsh
> id | v
> +---
> 1 | a
> (1 rows)
> MLSEA-JJIRSA01:~ jjirsa$ ccm node1 nodetool info | grep Row
> Row Cache : entries 0, size 0 bytes, capacity 100 MiB, 0 hits, 0
> requests, NaN recent hit rate, 0 save period in seconds
> MLSEA-JJIRSA01:~ jjirsa$
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: CASSANDRA-12544 Portability Flaw: Locale Dependent Comparison
Repository: cassandra
Updated Branches:
refs/heads/trunk 29239c765 -> af4ebfffc
CASSANDRA-12544 Portability Flaw: Locale Dependent Comparison
use Locale.ENGLISH as table meta data is always english
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/af4ebfff
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/af4ebfff
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/af4ebfff
Branch: refs/heads/trunk
Commit: af4ebfffcfad203563c8eeadf68b2c3b0ecf2f1c
Parents: 29239c7
Author: Dave Brosius
Authored: Sat Aug 27 14:32:10 2016 -0400
Committer: Dave Brosius
Committed: Sat Aug 27 14:32:10 2016 -0400
--
src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/af4ebfff/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
--
diff --git a/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
index 05233bd..9502860 100644
--- a/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
+++ b/src/java/org/apache/cassandra/schema/LegacySchemaMigrator.java
@@ -282,7 +282,7 @@ public final class LegacySchemaMigrator
AbstractType rawComparator =
TypeParser.parse(tableRow.getString("comparator"));
AbstractType subComparator = tableRow.has("subcomparator") ?
TypeParser.parse(tableRow.getString("subcomparator")) : null;
-boolean isSuper =
"super".equals(tableRow.getString("type").toLowerCase());
+boolean isSuper =
"super".equals(tableRow.getString("type").toLowerCase(Locale.ENGLISH));
boolean isCompound = rawComparator instanceof CompositeType || isSuper;
/*
[jira] [Assigned] (CASSANDRA-12544) Portability Flaw: Locale Dependent Comparison
[
https://issues.apache.org/jira/browse/CASSANDRA-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius reassigned CASSANDRA-12544:
Assignee: Dave Brosius
> Portability Flaw: Locale Dependent Comparison
> -
>
> Key: CASSANDRA-12544
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12544
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Eduardo Aguinaga
>Assignee: Dave Brosius
>
> Overview:
> In May through June of 2016 a static analysis was performed on version 3.0.5
> of the Cassandra source code. The analysis included an automated analysis
> using HP Fortify v4.21 SCA and a manual analysis utilizing SciTools
> Understand v4. The results of that analysis includes the issue below.
> Issue:
> In the file LegacySchemaMigrator.java on line 286 there is a portability
> problem with the call to toLowerCase() because it has different locales which
> may lead to unexpected output. This may also circumvent custom validation
> routines.
> {code:java}
> LegacySchemaMigrator.java, lines 286-289:
> 286 boolean isSuper =
> "super".equals(tableRow.getString("type").toLowerCase());
> 287 boolean isDense = tableRow.has("is_dense")
> 288 ? tableRow.getBoolean("is_dense")
> 289 : calculateIsDense(rawComparator, columnRows);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-12544) Portability Flaw: Locale Dependent Comparison
[
https://issues.apache.org/jira/browse/CASSANDRA-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius resolved CASSANDRA-12544.
--
Resolution: Fixed
Fix Version/s: 3.10
commit af4ebfffcfad203563c8eeadf68b2c3b0ecf2f1c
table meta data is always english, so Locale.ENGLISH is fine
> Portability Flaw: Locale Dependent Comparison
> -
>
> Key: CASSANDRA-12544
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12544
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Eduardo Aguinaga
>Assignee: Dave Brosius
> Fix For: 3.10
>
>
> Overview:
> In May through June of 2016 a static analysis was performed on version 3.0.5
> of the Cassandra source code. The analysis included an automated analysis
> using HP Fortify v4.21 SCA and a manual analysis utilizing SciTools
> Understand v4. The results of that analysis includes the issue below.
> Issue:
> In the file LegacySchemaMigrator.java on line 286 there is a portability
> problem with the call to toLowerCase() because it has different locales which
> may lead to unexpected output. This may also circumvent custom validation
> routines.
> {code:java}
> LegacySchemaMigrator.java, lines 286-289:
> 286 boolean isSuper =
> "super".equals(tableRow.getString("type").toLowerCase());
> 287 boolean isDense = tableRow.has("is_dense")
> 288 ? tableRow.getBoolean("is_dense")
> 289 : calculateIsDense(rawComparator, columnRows);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12547) Privacy Violation
[
https://issues.apache.org/jira/browse/CASSANDRA-12547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius updated CASSANDRA-12547:
-
Priority: Trivial (was: Major)
> Privacy Violation
> -
>
> Key: CASSANDRA-12547
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12547
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Eduardo Aguinaga
>Priority: Trivial
>
> Overview:
> In May through June of 2016 a static analysis was performed on version 3.0.5
> of the Cassandra source code. The analysis included an automated analysis
> using HP Fortify v4.21 SCA and a manual analysis utilizing SciTools
> Understand v4. The results of that analysis includes the issue below.
> Issue:
> In the file GetCompactionThreshold.java on line 46 the method execute()
> mishandles sensitive information. Sensitive information should be handled
> carefully to avoid divulging it to unauthorized parties.
> {code:java}
> GetCompactionThreshold.java, lines 44-47:
> 44 ColumnFamilyStoreMBean cfsProxy = probe.getCfsProxy(ks, cf);
> 45 System.out.println("Current compaction thresholds for " + ks + "/" + cf +
> ": \n" +
> 46 " min = " + cfsProxy.getMinimumCompactionThreshold() + ", " +
> 47 " max = " + cfsProxy.getMaximumCompactionThreshold());
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12546) Privacy Violation
[
https://issues.apache.org/jira/browse/CASSANDRA-12546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius updated CASSANDRA-12546:
-
Priority: Trivial (was: Major)
> Privacy Violation
> -
>
> Key: CASSANDRA-12546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12546
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Eduardo Aguinaga
>Priority: Trivial
>
> Overview:
> In May through June of 2016 a static analysis was performed on version 3.0.5
> of the Cassandra source code. The analysis included an automated analysis
> using HP Fortify v4.21 SCA and a manual analysis utilizing SciTools
> Understand v4. The results of that analysis includes the issue below.
> Issue:
> In the file DigestMismatchException.java on line 30 the method
> DigestMismatchException() mishandles sensitive information. Sensitive
> information should be handled carefully to avoid divulging it to unauthorized
> parties.
> {code:java}
> DigestMismatchException.java, lines 28-34:
> 28 public DigestMismatchException(DecoratedKey key, ByteBuffer digest1,
> ByteBuffer digest2)
> 29 {
> 30 super(String.format("Mismatch for key %s (%s vs %s)",
> 31 key.toString(),
> 32 ByteBufferUtil.bytesToHex(digest1),
> 33 ByteBufferUtil.bytesToHex(digest2)));
> 34 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12549) Unreleased Resource: Streams
[
https://issues.apache.org/jira/browse/CASSANDRA-12549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius updated CASSANDRA-12549:
-
Priority: Trivial (was: Major)
> Unreleased Resource: Streams
>
>
> Key: CASSANDRA-12549
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12549
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Eduardo Aguinaga
>Priority: Trivial
>
> Overview:
> In May through June of 2016 a static analysis was performed on version 3.0.5
> of the Cassandra source code. The analysis included an automated analysis
> using HP Fortify v4.21 SCA and a manual analysis utilizing SciTools
> Understand v4. The results of that analysis includes the issue below.
> Issue:
> The function csQueryMetadata() in CloudstackSnitch.java sometimes fails to
> release a system resource allocated by getInputStream() on line 141. Stream
> not closed on exception.
> The two lines of code within the finally block can throw exceptions. Code
> that can throw an exception should not be in a finally block or the code
> within the finally block should be isolated within its own try/catch.
> {code:java}
> CloudstackSnitch.java, lines 131-149:
> 131 try
> 132 {
> 133 conn.setRequestMethod("GET");
> 134 if (conn.getResponseCode() != 200)
> 135 {
> 136 throw new ConfigurationException("CloudstackSnitch was unable to
> query metadata.");
> 137 }
> 138
> 139 int cl = conn.getContentLength();
> 140 byte[] b = new byte[cl];
> 141 is = new DataInputStream(new
> BufferedInputStream(conn.getInputStream()));
> 142 is.readFully(b);
> 143 return new String(b, StandardCharsets.UTF_8);
> 144 }
> 145 finally
> 146 {
> 147 FileUtils.close(is);
> 148 conn.disconnect();
> 149 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12543) Portability Flaw: Locale Dependent Comparison
[
https://issues.apache.org/jira/browse/CASSANDRA-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius updated CASSANDRA-12543:
-
Priority: Trivial (was: Major)
> Portability Flaw: Locale Dependent Comparison
> -
>
> Key: CASSANDRA-12543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12543
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Eduardo Aguinaga
>Priority: Trivial
>
> Overview:
> In May through June of 2016 a static analysis was performed on version 3.0.5
> of the Cassandra source code. The analysis included an automated analysis
> using HP Fortify v4.21 SCA and a manual analysis utilizing SciTools
> Understand v4. The results of that analysis includes the issue below.
> Issue:
> In the file ThriftConversion.java on lines 197 and 679 there is a portability
> problem with the call to toLowerCase() because it has different locales which
> may lead to unexpected output. This may also circumvent custom validation
> routines.
> ThriftConversion.java, lines 197-202:
> {code:java}
> 197 switch (thriftColumnType.toLowerCase())
> 198 {
> 199 case "standard": return false;
> 200 case "super": return true;
> 201 default: throw new
> org.apache.cassandra.exceptions.InvalidRequestException("Invalid column type
> " + thriftColumnType);
> 202 }
> {code}
> ThriftConversion.java, lines 679-691:
> {code:java}
> 679 switch (caching.toUpperCase())
> 680 {
> 681 case "ALL":
> 682 return CachingParams.CACHE_EVERYTHING;
> 683 case "ROWS_ONLY":
> 684 return new CachingParams(false, Integer.MAX_VALUE);
> 685 case "KEYS_ONLY":
> 686 return CachingParams.CACHE_KEYS;
> 687 case "NONE":
> 688 return CachingParams.CACHE_NOTHING;
> 689 default:
> 690 throw new ConfigurationException(String.format("Invalid value %s
> for caching parameter", caching));
> 691 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12541) Portability Flaw: Locale Dependent Comparison
[
https://issues.apache.org/jira/browse/CASSANDRA-12541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius updated CASSANDRA-12541:
-
Priority: Trivial (was: Major)
> Portability Flaw: Locale Dependent Comparison
> -
>
> Key: CASSANDRA-12541
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12541
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Eduardo Aguinaga
>Priority: Trivial
>
> Overview:
> In May through June of 2016 a static analysis was performed on version 3.0.5
> of the Cassandra source code. The analysis included an automated analysis
> using HP Fortify v4.21 SCA and a manual analysis utilizing SciTools
> Understand v4. The results of that analysis includes the issue below.
> Issue:
> In the file CqlRecordWriter.java on line 128 there is a portability problem
> with the call to toLowerCase() because it has different locales which may
> lead to unexpected output. This may also circumvent custom validation
> routines.
> {code:java}
> CqlRecordWriter.java, lines 127-130:
> 127 String cqlQuery = CqlConfigHelper.getOutputCql(conf).trim();
> 128 if (cqlQuery.toLowerCase().startsWith("insert"))
> 129 throw new UnsupportedOperationException("INSERT with CqlRecordWriter
> is not supported, please use UPDATE/DELETE statement");
> 130 cql = appendKeyWhereClauses(cqlQuery);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12544) Portability Flaw: Locale Dependent Comparison
[
https://issues.apache.org/jira/browse/CASSANDRA-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius updated CASSANDRA-12544:
-
Priority: Trivial (was: Major)
> Portability Flaw: Locale Dependent Comparison
> -
>
> Key: CASSANDRA-12544
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12544
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Eduardo Aguinaga
>Assignee: Dave Brosius
>Priority: Trivial
> Fix For: 3.10
>
>
> Overview:
> In May through June of 2016 a static analysis was performed on version 3.0.5
> of the Cassandra source code. The analysis included an automated analysis
> using HP Fortify v4.21 SCA and a manual analysis utilizing SciTools
> Understand v4. The results of that analysis includes the issue below.
> Issue:
> In the file LegacySchemaMigrator.java on line 286 there is a portability
> problem with the call to toLowerCase() because it has different locales which
> may lead to unexpected output. This may also circumvent custom validation
> routines.
> {code:java}
> LegacySchemaMigrator.java, lines 286-289:
> 286 boolean isSuper =
> "super".equals(tableRow.getString("type").toLowerCase());
> 287 boolean isDense = tableRow.has("is_dense")
> 288 ? tableRow.getBoolean("is_dense")
> 289 : calculateIsDense(rawComparator, columnRows);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12545) Portability Flaw: Locale Dependent Comparison
[
https://issues.apache.org/jira/browse/CASSANDRA-12545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius updated CASSANDRA-12545:
-
Priority: Trivial (was: Major)
> Portability Flaw: Locale Dependent Comparison
> -
>
> Key: CASSANDRA-12545
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12545
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Eduardo Aguinaga
>Priority: Trivial
>
> Overview:
> In May through June of 2016 a static analysis was performed on version 3.0.5
> of the Cassandra source code. The analysis included an automated analysis
> using HP Fortify v4.21 SCA and a manual analysis utilizing SciTools
> Understand v4. The results of that analysis includes the issue below.
> Issue:
> In the file CoalescingStrategies.java on line 502 there is a portability
> problem with the call to toLowerCase() because it has different locales which
> may lead to unexpected output. This may also circumvent custom validation
> routines.
> {code:java}
> CoalescingStrategies.java, lines 502-519:
> 502 String strategyCleaned = strategy.trim().toUpperCase();
> 503 switch(strategyCleaned)
> 504 {
> 505 case "MOVINGAVERAGE":
> 506 classname = MovingAverageCoalescingStrategy.class.getName();
> 507 break;
> 508 case "FIXED":
> 509 classname = FixedCoalescingStrategy.class.getName();
> 510 break;
> 511 case "TIMEHORIZON":
> 512 classname =
> TimeHorizonMovingAverageCoalescingStrategy.class.getName();
> 513 break;
> 514 case "DISABLED":
> 515 classname = DisabledCoalescingStrategy.class.getName();
> 516 break;
> 517 default:
> 518 classname = strategy;
> 519 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12542) Portability Flaw: Locale Dependent Comparison
[
https://issues.apache.org/jira/browse/CASSANDRA-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius updated CASSANDRA-12542:
-
Priority: Trivial (was: Major)
> Portability Flaw: Locale Dependent Comparison
> -
>
> Key: CASSANDRA-12542
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12542
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Eduardo Aguinaga
>Priority: Trivial
>
> Overview:
> In May through June of 2016 a static analysis was performed on version 3.0.5
> of the Cassandra source code. The analysis included an automated analysis
> using HP Fortify v4.21 SCA and a manual analysis utilizing SciTools
> Understand v4. The results of that analysis includes the issue below.
> Issue:
> In the file SpeculativeRetryParam.java on lines 91 and 103 there are a
> portability problems with the call to toLowerCase() because it has different
> locales which may lead to unexpected output. This may also circumvent custom
> validation routines.
> SpeculativeRetryParam.java, lines 91-101:
> {code:java}
> 091 if (value.toLowerCase().endsWith("ms"))
> 092 {
> 093 try
> 094 {
> 095 return custom(Double.parseDouble(value.substring(0,
> value.length() - "ms".length(;
> 096 }
> 097 catch (IllegalArgumentException e)
> 098 {
> 099 throw new ConfigurationException(format("Invalid value %s for
> option '%s'", value, TableParams.Option.SPECULATIVE_RETRY));
> 100 }
> 101 }
> {code}
> SpeculativeRetryParam.java, lines 103-121:
> {code:java}
> 103 if (value.toUpperCase().endsWith(Kind.PERCENTILE.toString()))
> 104 {
> 105 double threshold;
> 106 try
> 107 {
> 108 threshold = Double.parseDouble(value.substring(0, value.length()
> - Kind.PERCENTILE.toString().length()));
> 109 }
> 110 catch (IllegalArgumentException e)
> 111 {
> 112 throw new ConfigurationException(format("Invalid value %s for
> option '%s'", value, TableParams.Option.SPECULATIVE_RETRY));
> 113 }
> 114
> 115 if (threshold >= 0.0 && threshold <= 100.0)
> 116 return percentile(threshold);
> 117
> 118 throw new ConfigurationException(format("Invalid value %s for
> PERCENTILE option '%s': must be between 0.0 and 100.0",
> 119 value,
> 120
> TableParams.Option.SPECULATIVE_RETRY));
> 121 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12548) Improper Neutralization of Special Elements used in a Command ('Command Injection'), Improper Neutralization of Special Elements used in an OS Command ('OS Command I
[
https://issues.apache.org/jira/browse/CASSANDRA-12548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius updated CASSANDRA-12548:
-
Priority: Minor (was: Major)
> Improper Neutralization of Special Elements used in a Command ('Command
> Injection'), Improper Neutralization of Special Elements used in an OS
> Command ('OS Command Injection')
> ---
>
> Key: CASSANDRA-12548
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12548
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Eduardo Aguinaga
>Priority: Minor
>
> Overview:
> In May through June of 2016 a static analysis was performed on version 3.0.5
> of the Cassandra source code. The analysis included an automated analysis
> using HP Fortify v4.21 SCA and a manual analysis utilizing SciTools
> Understand v4. The results of that analysis includes the issue below.
> Issue:
> The software constructs all or part of a command using externally-influenced
> input from an upstream component, but it does not neutralize or incorrectly
> neutralizes special elements that could modify the intended command when it
> is sent to a downstream component.
> The source file CommitLogArchiver.java retrieves commands stored as system
> properties and executes these commands after replacing tokens with relevant
> data. The commands retrieved are not verified to ensure that the commands do
> not contain malicious content. An adversary could perform part of its attack
> on Cassandra by replacing the "archive_command" or "restore_command" property
> with commands to achieve their goal.
> Line numbers 141, 163, 251 and 265
> {code:java}
> CommitLogArchiver.java, lines 91-92:
> 91 String archiveCommand = commitlog_commands.getProperty("archive_command");
> 92 String restoreCommand = commitlog_commands.getProperty("restore_command");
> CommitLogArchiver.java, lines 129-144:
> 129 public void maybeArchive(final CommitLogSegment segment)
> 130 {
> 131 if (Strings.isNullOrEmpty(archiveCommand))
> 132 return;
> 133
> 134 archivePending.put(segment.getName(), executor.submit(new
> WrappedRunnable()
> 135 {
> 136 protected void runMayThrow() throws IOException
> 137 {
> 138 segment.waitForFinalSync();
> 139 String command = archiveCommand.replace("%name",
> segment.getName());
> 140 command = command.replace("%path", segment.getPath());
> 141 exec(command);
> 142 }
> 143 }));
> 144 }
> CommitLogArchiver.java, lines 261-266:
> 261 private void exec(String command) throws IOException
> 262 {
> 263 ProcessBuilder pb = new ProcessBuilder(command.split(" "));
> 264 pb.redirectErrorStream(true);
> 265 FBUtilities.exec(pb);
> 266 }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
