Merge branch 'cassandra-2.2' into cassandra-3.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4b47d711 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4b47d711 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4b47d711 Branch: refs/heads/cassandra-3.0 Commit: 4b47d711ec5dd8e9b31679c2810f57503077985a Parents: 8210075 83fcc92 Author: Robert Stupp <sn...@snazy.de> Authored: Wed Oct 28 20:13:49 2015 +0100 Committer: Robert Stupp <sn...@snazy.de> Committed: Wed Oct 28 20:13:49 2015 +0100 ---------------------------------------------------------------------- CHANGES.txt | 1 + NEWS.txt | 4 ++++ conf/cassandra-env.sh | 23 +++++++++----------- conf/cassandra.yaml | 13 ----------- .../org/apache/cassandra/config/Config.java | 3 --- 5 files changed, 15 insertions(+), 29 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b47d711/CHANGES.txt ---------------------------------------------------------------------- diff --cc CHANGES.txt index c945bd2,cc4a8bd..3f2f493 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,21 -1,10 +1,22 @@@ -2.2.4 - * Deprecate memory_allocator in cassandra.yaml (CASSANDRA-10581) +3.0 ++ * Remove memory_allocator paramter from cassandra.yaml (CASSANDRA-10581) + * Execute the metadata reload task of all registered indexes on CFS::reload (CASSANDRA-10604) + * Fix thrift cas operations with defined columns (CASSANDRA-10576) + * Fix PartitionUpdate.operationCount()for updates with static column operations (CASSANDRA-10606) + * Fix thrift get() queries with defined columns (CASSANDRA-10586) + * Fix marking of indexes as built and removed (CASSANDRA-10601) + * Skip initialization of non-registered 2i instances, remove Index::getIndexName (CASSANDRA-10595) + * Fix batches on multiple tables (CASSANDRA-10554) + * Ensure compaction options are validated when updating KeyspaceMetadata (CASSANDRA-10569) + * Flatten Iterator Transformation Hierarchy (CASSANDRA-9975) + * Remove token generator (CASSANDRA-5261) + * RolesCache should not be created for any authenticator that does not requireAuthentication (CASSANDRA-10562) + * Fix LogTransaction checking only a single directory for files (CASSANDRA-10421) + * Fix handling of range tombstones when reading old format sstables (CASSANDRA-10360) + * Aggregate with Initial Condition fails with C* 3.0 (CASSANDRA-10367) +Merged from 2.2: * Expose phi values from failure detector via JMX and tweak debug and trace logging (CASSANDRA-9526) - * Fix RangeNamesQueryPager (CASSANDRA-10509) - * Deprecate Pig support (CASSANDRA-10542) - * Reduce contention getting instances of CompositeType (CASSANDRA-10433) Merged from 2.1: * Support encrypted and plain traffic on the same port (CASSANDRA-10559) * Do STCS in DTCS windows (CASSANDRA-10276) http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b47d711/NEWS.txt ---------------------------------------------------------------------- diff --cc NEWS.txt index dd20970,b67814a..aa19fcb --- a/NEWS.txt +++ b/NEWS.txt @@@ -13,110 -13,6 +13,111 @@@ restore snapshots created with the prev 'sstableloader' tool. You can upgrade the file format of your snapshots using the provided 'sstableupgrade' tool. +3.0 +=== + +New features +------------ + - EACH_QUORUM is now a supported consistency level for read requests. + - Support for IN restrictions on any partition key component or clustering key + as well as support for EQ and IN multicolumn restrictions has been added to + UPDATE and DELETE statement. + - Support for single-column and multi-colum slice restrictions (>, >=, <= and <) + has been added to DELETE statements + - nodetool rebuild_index accepts the index argument without + the redundant table name + - Materialized Views, which allow for server-side denormalization, is now + available. Materialized views provide an alternative to secondary indexes + for non-primary key queries, and perform much better for indexing high + cardinality columns. + See http://www.datastax.com/dev/blog/new-in-cassandra-3-0-materialized-views + - Hinted handoff has been completely rewritten. Hints are now stored in flat + files, with less overhead for storage and more efficient dispatch. + See CASSANDRA-6230 for full details. + - Option to not purge unrepaired tombstones. To avoid users having data resurrected + if repair has not been run within gc_grace_seconds, an option has been added to + only allow tombstones from repaired sstables to be purged. To enable, set the + compaction option 'only_purge_repaired_tombstones':true but keep in mind that if + you do not run repair for a long time, you will keep all tombstones around which + can cause other problems. + - Enabled warning on GC taking longer than 1000ms. See + cassandra.yaml:gc_warn_threshold_in_ms + +Upgrading +--------- + - token-generator tool has been removed. + - Upgrade to 3.0 is supported from Cassandra 2.1 versions greater or equal to 2.1.9, + or Cassandra 2.2 versions greater or equal to 2.2.2. Upgrade from Cassandra 2.0 and + older versions is not supported. + - The 'memtable_allocation_type: offheap_objects' option has been removed. It should + be re-introduced in a future release and you can follow CASSANDRA-9472 to know more. ++ - Configuration parameter memory_allocator in cassandra.yaml has been removed. + - The native protocol versions 1 and 2 are not supported anymore. + - Max mutation size is now configurable via max_mutation_size_in_kb setting in + cassandra.yaml; the default is half the size commitlog_segment_size_in_mb * 1024. + - 3.0 requires Java 8u40 or later. + - Garbage collection options were moved from cassandra-env to jvm.options file. + - New transaction log files have been introduced to replace the compactions_in_progress + system table, temporary file markers (tmp and tmplink) and sstable ancerstors. + Therefore, compaction metadata no longer contains ancestors. Transaction log files + list sstable descriptors involved in compactions and other operations such as flushing + and streaming. Use the sstableutil tool to list any sstable files currently involved + in operations not yet completed, which previously would have been marked as temporary. + A transaction log file contains one sstable per line, with the prefix "add:" or "remove:". + They also contain a special line "commit", only inserted at the end when the transaction + is committed. On startup we use these files to cleanup any partial transactions that were + in progress when the process exited. If the commit line is found, we keep new sstables + (those with the "add" prefix) and delete the old sstables (those with the "remove" prefix), + vice-versa if the commit line is missing. Should you lose or delete these log files, + both old and new sstable files will be kept as live files, which will result in duplicated + sstables. These files are protected by incremental checksums so you should not manually + edit them. When restoring a full backup or moving sstable files, you should clean-up + any left over transactions and their temporary files first. You can use this command: + ===> sstableutil -c ks table + See CASSANDRA-7066 for full details. + - New write stages have been added for batchlog and materialized view mutations + you can set their size in cassandra.yaml + - User defined functions are now executed in a sandbox. + To use UDFs and UDAs, you have to enable them in cassandra.yaml. + - New SSTable version 'la' with improved bloom-filter false-positive handling + compared to previous version 'ka' used in 2.2 and 2.1. Running sstableupgrade + is not necessary but recommended. + - Before upgrading to 3.0, make sure that your cluster is in complete agreement + (schema versions outputted by `nodetool describecluster` are all the same). + - Schema metadata is now stored in the new `system_schema` keyspace, and + legacy `system.schema_*` tables are now gone; see CASSANDRA-6717 for details. + - Pig's support has been removed. + - Hadoop BulkOutputFormat and BulkRecordWriter have been removed; use + CqlBulkOutputFormat and CqlBulkRecordWriter instead. + - Hadoop ColumnFamilyInputFormat and ColumnFamilyOutputFormat have been removed; + use CqlInputFormat and CqlOutputFormat instead. + - Hadoop ColumnFamilyRecordReader and ColumnFamilyRecordWriter have been removed; + use CqlRecordReader and CqlRecordWriter instead. + - hinted_handoff_enabled in cassandra.yaml no longer supports a list of data centers. + To specify a list of excluded data centers when hinted_handoff_enabled is set to true, + use hinted_handoff_disabled_datacenters, see CASSANDRA-9035 for details. + - The `sstable_compression` and `chunk_length_kb` compression options have been deprecated. + The new options are `class` and `chunk_length_in_kb`. Disabling compression should now + be done by setting the new option `enabled` to `false`. + - The compression option `crc_check_chance` became a top-level table option, but is currently + enforced only against tables with enabled compression. + - Only map syntax is now allowed for caching options. ALL/NONE/KEYS_ONLY/ROWS_ONLY syntax + has been deprecated since 2.1.0 and is being removed in 3.0.0. + - The 'index_interval' option for 'CREATE TABLE' statements, which has been deprecated + since 2.1 and replaced with the 'min_index_interval' and 'max_index_interval' options, + has now been removed. + - Batchlog entries are now stored in a new table - system.batches. + The old one has been deprecated. + - JMX methods set/getCompactionStrategyClass have been removed, use + set/getCompactionParameters or set/getCompactionParametersJson instead. + - SizeTieredCompactionStrategy parameter cold_reads_to_omit has been removed. + - The secondary index API has been comprehensively reworked. This will be a breaking + change for any custom index implementations, which should now look to implement + the new org.apache.cassandra.index.Index interface. New syntax has been added to create + and query row-based indexes, which are not explicitly linked to a single column in the + base table. + + 2.2.4 ===== http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b47d711/conf/cassandra-env.sh ---------------------------------------------------------------------- diff --cc conf/cassandra-env.sh index 1d72070,bb20964..2f0ae34 --- a/conf/cassandra-env.sh +++ b/conf/cassandra-env.sh @@@ -291,11 -250,53 +293,6 @@@ if [ "x$CASSANDRA_HEAPDUMP_DIR" != "x" JVM_OPTS="$JVM_OPTS -XX:HeapDumpPath=$CASSANDRA_HEAPDUMP_DIR/cassandra-`date +%s`-pid$$.hprof" fi - # Configure the following for JEMallocAllocator and if jemalloc is not available in the system - # library path (Example: /usr/local/lib/). Usually "make install" will do the right thing. - # export LD_LIBRARY_PATH=<JEMALLOC_HOME>/lib/ - # JVM_OPTS="$JVM_OPTS -Djava.library.path=<JEMALLOC_HOME>/lib/" - -startswith() { [ "${1#$2}" != "$1" ]; } - -# Per-thread stack size. -JVM_OPTS="$JVM_OPTS -Xss256k" - -# Larger interned string table, for gossip's benefit (CASSANDRA-6410) -JVM_OPTS="$JVM_OPTS -XX:StringTableSize=1000003" - -# GC tuning options -JVM_OPTS="$JVM_OPTS -XX:+UseParNewGC" -JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC" -JVM_OPTS="$JVM_OPTS -XX:+CMSParallelRemarkEnabled" -JVM_OPTS="$JVM_OPTS -XX:SurvivorRatio=8" -JVM_OPTS="$JVM_OPTS -XX:MaxTenuringThreshold=1" -JVM_OPTS="$JVM_OPTS -XX:CMSInitiatingOccupancyFraction=75" -JVM_OPTS="$JVM_OPTS -XX:+UseCMSInitiatingOccupancyOnly" -JVM_OPTS="$JVM_OPTS -XX:+UseTLAB" -JVM_OPTS="$JVM_OPTS -XX:+PerfDisableSharedMem" -JVM_OPTS="$JVM_OPTS -XX:CompileCommandFile=$CASSANDRA_CONF/hotspot_compiler" -JVM_OPTS="$JVM_OPTS -XX:CMSWaitDuration=10000" - -# note: bash evals '1.7.x' as > '1.7' so this is really a >= 1.7 jvm check -if { [ "$JVM_VERSION" \> "1.7" ] && [ "$JVM_VERSION" \< "1.8.0" ] && [ "$JVM_PATCH_VERSION" -ge "60" ]; } || [ "$JVM_VERSION" \> "1.8" ] ; then - JVM_OPTS="$JVM_OPTS -XX:+CMSParallelInitialMarkEnabled -XX:+CMSEdenChunksRecordAlways -XX:CMSWaitDuration=10000" -fi - -if [ "$JVM_ARCH" = "64-Bit" ] ; then - JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark" -fi - -# GC logging options -- uncomment to enable -# JVM_OPTS="$JVM_OPTS -XX:+PrintGCDetails" -# JVM_OPTS="$JVM_OPTS -XX:+PrintGCDateStamps" -# JVM_OPTS="$JVM_OPTS -XX:+PrintHeapAtGC" -# JVM_OPTS="$JVM_OPTS -XX:+PrintTenuringDistribution" -# JVM_OPTS="$JVM_OPTS -XX:+PrintGCApplicationStoppedTime" -# JVM_OPTS="$JVM_OPTS -XX:+PrintPromotionFailure" -# JVM_OPTS="$JVM_OPTS -XX:PrintFLSStatistics=1" -# JVM_OPTS="$JVM_OPTS -Xloggc:/var/log/cassandra/gc-`date +%s`.log" -# If you are using JDK 6u34 7u2 or later you can enable GC log rotation -# don't stick the date in the log name if rotation is on. -# JVM_OPTS="$JVM_OPTS -Xloggc:/var/log/cassandra/gc.log" -# JVM_OPTS="$JVM_OPTS -XX:+UseGCLogFileRotation" -# JVM_OPTS="$JVM_OPTS -XX:NumberOfGCLogFiles=10" -# JVM_OPTS="$JVM_OPTS -XX:GCLogFileSize=10M" -- # uncomment to have Cassandra JVM listen for remote debuggers/profilers on port 1414 # JVM_OPTS="$JVM_OPTS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=1414" http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b47d711/conf/cassandra.yaml ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b47d711/src/java/org/apache/cassandra/config/Config.java ---------------------------------------------------------------------- diff --cc src/java/org/apache/cassandra/config/Config.java index bcd736c,a8be5f4..9e0698f --- a/src/java/org/apache/cassandra/config/Config.java +++ b/src/java/org/apache/cassandra/config/Config.java @@@ -220,20 -218,12 +220,17 @@@ public class Confi public volatile int counter_cache_save_period = 7200; public volatile int counter_cache_keys_to_save = Integer.MAX_VALUE; -- @Deprecated -- public String memory_allocator; -- private static boolean isClientMode = false; - public Integer file_cache_size_in_mb; + public Integer file_cache_size_in_mb = 512; + + public boolean buffer_pool_use_heap_if_exhausted = true; + + public DiskOptimizationStrategy disk_optimization_strategy = DiskOptimizationStrategy.ssd; + + public double disk_optimization_estimate_percentile = 0.95; + + public double disk_optimization_page_cross_chance = 0.1; public boolean inter_dc_tcp_nodelay = true;
