Merge branch 'cassandra-3.0' into cassandra-3.X
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eca51a03 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eca51a03 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eca51a03 Branch: refs/heads/cassandra-3.X Commit: eca51a0348141f45ea9bb31b76571ed81284c32c Parents: 5f3b359 6e716c6 Author: Robert Stupp <sn...@snazy.de> Authored: Thu Jan 5 22:26:48 2017 +0100 Committer: Robert Stupp <sn...@snazy.de> Committed: Thu Jan 5 22:26:48 2017 +0100 ---------------------------------------------------------------------- CHANGES.txt | 2 + NEWS.txt | 3 ++ doc/cql3/CQL.textile | 2 +- lib/jsr223/clojure/README.txt | 8 --- lib/jsr223/groovy/README.txt | 35 ------------- lib/jsr223/jaskell/README.txt | 5 -- lib/jsr223/jruby/README.txt | 54 -------------------- lib/jsr223/jython/README.txt | 33 ------------ lib/jsr223/scala/README.txt | 37 -------------- .../cql3/functions/ScriptBasedUDFunction.java | 35 +++++-------- .../cql3/validation/entities/UFScriptTest.java | 3 +- .../validation/entities/UFSecurityTest.java | 12 ++++- 12 files changed, 31 insertions(+), 198 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/eca51a03/CHANGES.txt ---------------------------------------------------------------------- diff --cc CHANGES.txt index cacb7cd,77a310d..66c2ee6 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,136 -1,5 +1,138 @@@ -3.0.11 - * Fixed flacky SSTableRewriterTest: check file counts before calling validateCFS (CASSANDRA-12348) +3.12 + * Use new token allocation for non bootstrap case as well (CASSANDRA-13080) + * Avoid byte-array copy when key cache is disabled (CASSANDRA-13084) + * More fixes to the TokenAllocator (CASSANDRA-12990) + * Require forceful decommission if number of nodes is less than replication factor (CASSANDRA-12510) + * Allow IN restrictions on column families with collections (CASSANDRA-12654) + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034) + * nodetool stopdaemon errors out (CASSANDRA-13030) + * Log message size in trace message in OutboundTcpConnection (CASSANDRA-13028) + * Add timeUnit Days for cassandra-stress (CASSANDRA-13029) + * Add mutation size and batch metrics (CASSANDRA-12649) + * Add method to get size of endpoints to TokenMetadata (CASSANDRA-12999) + * Fix primary index calculation for SASI (CASSANDRA-12910) + * Expose time spent waiting in thread pool queue (CASSANDRA-8398) + * Conditionally update index built status to avoid unnecessary flushes (CASSANDRA-12969) + * NoReplicationTokenAllocator should work with zero replication factor (CASSANDRA-12983) + * cqlsh auto completion: refactor definition of compaction strategy options (CASSANDRA-12946) + * Add support for arithmetic operators (CASSANDRA-11935) + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954) ++Merged from 3.0: ++ * Remove support for non-JavaScript UDFs (CASSANDRA-12883) + + +3.10 + * Fixed query monitoring for range queries (CASSANDRA-13050) + * Remove outboundBindAny configuration property (CASSANDRA-12673) + * Use correct bounds for all-data range when filtering (CASSANDRA-12666) + * Remove timing window in test case (CASSANDRA-12875) + * Resolve unit testing without JCE security libraries installed (CASSANDRA-12945) + * Fix inconsistencies in cassandra-stress load balancing policy (CASSANDRA-12919) + * Fix validation of non-frozen UDT cells (CASSANDRA-12916) + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903) + * Fix Murmur3PartitionerTest (CASSANDRA-12858) + * Move cqlsh syntax rules into separate module and allow easier customization (CASSANDRA-12897) + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283) + * Fix cassandra-stress truncate option (CASSANDRA-12695) + * Fix crossNode value when receiving messages (CASSANDRA-12791) + * Don't load MX4J beans twice (CASSANDRA-12869) + * Extend native protocol request flags, add versions to SUPPORTED, and introduce ProtocolVersion enum (CASSANDRA-12838) + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836) + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845) + * Properly format IPv6 addresses when logging JMX service URL (CASSANDRA-12454) + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777) + * Use non-token restrictions for bounds when token restrictions are overridden (CASSANDRA-12419) + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803) + * Use different build directories for Eclipse and Ant (CASSANDRA-12466) + * Avoid potential AttributeError in cqlsh due to no table metadata (CASSANDRA-12815) + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster (CASSANDRA-12812) + * Upgrade commons-codec to 1.9 (CASSANDRA-12790) + * Make the fanout size for LeveledCompactionStrategy to be configurable (CASSANDRA-11550) + * Add duration data type (CASSANDRA-11873) + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784) + * Improve sum aggregate functions (CASSANDRA-12417) + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in CASSANDRA-10876 (CASSANDRA-12761) + * cqlsh fails to format collections when using aliases (CASSANDRA-11534) + * Check for hash conflicts in prepared statements (CASSANDRA-12733) + * Exit query parsing upon first error (CASSANDRA-12598) + * Fix cassandra-stress to use single seed in UUID generation (CASSANDRA-12729) + * CQLSSTableWriter does not allow Update statement (CASSANDRA-12450) + * Config class uses boxed types but DD exposes primitive types (CASSANDRA-12199) + * Add pre- and post-shutdown hooks to Storage Service (CASSANDRA-12461) + * Add hint delivery metrics (CASSANDRA-12693) + * Remove IndexInfo cache from FileIndexInfoRetriever (CASSANDRA-12731) + * ColumnIndex does not reuse buffer (CASSANDRA-12502) + * cdc column addition still breaks schema migration tasks (CASSANDRA-12697) + * Upgrade metrics-reporter dependencies (CASSANDRA-12089) + * Tune compaction thread count via nodetool (CASSANDRA-12248) + * Add +=/-= shortcut syntax for update queries (CASSANDRA-12232) + * Include repair session IDs in repair start message (CASSANDRA-12532) + * Add a blocking task to Index, run before joining the ring (CASSANDRA-12039) + * Fix NPE when using CQLSSTableWriter (CASSANDRA-12667) + * Support optional backpressure strategies at the coordinator (CASSANDRA-9318) + * Make randompartitioner work with new vnode allocation (CASSANDRA-12647) + * Fix cassandra-stress graphing (CASSANDRA-12237) + * Allow filtering on partition key columns for queries without secondary indexes (CASSANDRA-11031) + * Fix Cassandra Stress reporting thread model and precision (CASSANDRA-12585) + * Add JMH benchmarks.jar (CASSANDRA-12586) + * Cleanup uses of AlterTableStatementColumn (CASSANDRA-12567) + * Add keep-alive to streaming (CASSANDRA-11841) + * Tracing payload is passed through newSession(..) (CASSANDRA-11706) + * avoid deleting non existing sstable files and improve related log messages (CASSANDRA-12261) + * json/yaml output format for nodetool compactionhistory (CASSANDRA-12486) + * Retry all internode messages once after a connection is + closed and reopened (CASSANDRA-12192) + * Add support to rebuild from targeted replica (CASSANDRA-9875) + * Add sequence distribution type to cassandra stress (CASSANDRA-12490) + * "SELECT * FROM foo LIMIT ;" does not error out (CASSANDRA-12154) + * Define executeLocally() at the ReadQuery Level (CASSANDRA-12474) + * Extend read/write failure messages with a map of replica addresses + to error codes in the v5 native protocol (CASSANDRA-12311) + * Fix rebuild of SASI indexes with existing index files (CASSANDRA-12374) + * Let DatabaseDescriptor not implicitly startup services (CASSANDRA-9054, 12550) + * Fix clustering indexes in presence of static columns in SASI (CASSANDRA-12378) + * Fix queries on columns with reversed type on SASI indexes (CASSANDRA-12223) + * Added slow query log (CASSANDRA-12403) + * Count full coordinated request against timeout (CASSANDRA-12256) + * Allow TTL with null value on insert and update (CASSANDRA-12216) + * Make decommission operation resumable (CASSANDRA-12008) + * Add support to one-way targeted repair (CASSANDRA-9876) + * Remove clientutil jar (CASSANDRA-11635) + * Fix compaction throughput throttle (CASSANDRA-12366, CASSANDRA-12717) + * Delay releasing Memtable memory on flush until PostFlush has finished running (CASSANDRA-12358) + * Cassandra stress should dump all setting on startup (CASSANDRA-11914) + * Make it possible to compact a given token range (CASSANDRA-10643) + * Allow updating DynamicEndpointSnitch properties via JMX (CASSANDRA-12179) + * Collect metrics on queries by consistency level (CASSANDRA-7384) + * Add support for GROUP BY to SELECT statement (CASSANDRA-10707) + * Deprecate memtable_cleanup_threshold and update default for memtable_flush_writers (CASSANDRA-12228) + * Upgrade to OHC 0.4.4 (CASSANDRA-12133) + * Add version command to cassandra-stress (CASSANDRA-12258) + * Create compaction-stress tool (CASSANDRA-11844) + * Garbage-collecting compaction operation and schema option (CASSANDRA-7019) + * Add beta protocol flag for v5 native protocol (CASSANDRA-12142) + * Support filtering on non-PRIMARY KEY columns in the CREATE + MATERIALIZED VIEW statement's WHERE clause (CASSANDRA-10368) + * Unify STDOUT and SYSTEMLOG logback format (CASSANDRA-12004) + * COPY FROM should raise error for non-existing input files (CASSANDRA-12174) + * Faster write path (CASSANDRA-12269) + * Option to leave omitted columns in INSERT JSON unset (CASSANDRA-11424) + * Support json/yaml output in nodetool tpstats (CASSANDRA-12035) + * Expose metrics for successful/failed authentication attempts (CASSANDRA-10635) + * Prepend snapshot name with "truncated" or "dropped" when a snapshot + is taken before truncating or dropping a table (CASSANDRA-12178) + * Optimize RestrictionSet (CASSANDRA-12153) + * cqlsh does not automatically downgrade CQL version (CASSANDRA-12150) + * Omit (de)serialization of state variable in UDAs (CASSANDRA-9613) + * Create a system table to expose prepared statements (CASSANDRA-8831) + * Reuse DataOutputBuffer from ColumnIndex (CASSANDRA-11970) + * Remove DatabaseDescriptor dependency from SegmentedFile (CASSANDRA-11580) + * Add supplied username to authentication error messages (CASSANDRA-12076) + * Remove pre-startup check for open JMX port (CASSANDRA-12074) + * Remove compaction Severity from DynamicEndpointSnitch (CASSANDRA-11738) + * Restore resumable hints delivery (CASSANDRA-11960) + * Properly report LWT contention (CASSANDRA-12626) +Merged from 3.0: * Fix deserialization of 2.x DeletedCells (CASSANDRA-12620) * Add parent repair session id to anticompaction log message (CASSANDRA-12186) * Improve contention handling on failure to acquire MV lock for streaming and hints (CASSANDRA-12905) http://git-wip-us.apache.org/repos/asf/cassandra/blob/eca51a03/NEWS.txt ---------------------------------------------------------------------- diff --cc NEWS.txt index e6c8c10,b4e6551..9f376ff --- a/NEWS.txt +++ b/NEWS.txt @@@ -21,125 -21,23 +21,128 @@@ Upgradin - Specifying the default_time_to_live option when creating or altering a materialized view was erroneously accepted (and ignored). It is now properly rejected. + - Only Java and JavaScript are now supported UDF languages. + The sandbox in 3.0 already prevented the use of script languages except Java + and JavaScript. -3.0.10 -===== +3.10 +==== -Upgrading ---------- - - memtable_allocation_type: offheap_buffers is no longer allowed to be specified in the 3.0 series. - This was an oversight that can cause segfaults. Offheap was re-introduced in 3.4 see CASSANDRA-11039 - and CASSANDRA-9472 for details. +New features +------------ + - New `DurationType` (cql duration). See CASSANDRA-11873 + - Runtime modification of concurrent_compactors is now available via nodetool + - Support for the assignment operators +=/-= has been added for update queries. + - An Index implementation may now provide a task which runs prior to joining + the ring. See CASSANDRA-12039 + - Filtering on partition key columns is now also supported for queries without + secondary indexes. + - A slow query log has been added: slow queries will be logged at DEBUG level. + For more details refer to CASSANDRA-12403 and slow_query_log_timeout_in_ms + in cassandra.yaml. + - Support for GROUP BY queries has been added. + - A new compaction-stress tool has been added to test the throughput of compaction + for any cassandra-stress user schema. see compaction-stress help for how to use. + - Compaction can now take into account overlapping tables that don't take part + in the compaction to look for deleted or overwritten data in the compacted tables. + Then such data is found, it can be safely discarded, which in turn should enable + the removal of tombstones over that data. + + The behavior can be engaged in two ways: + - as a "nodetool garbagecollect -g CELL/ROW" operation, which applies + single-table compaction on all sstables to discard deleted data in one step. + - as a "provide_overlapping_tombstones:CELL/ROW/NONE" compaction strategy flag, + which uses overlapping tables as a source of deletions/overwrites during all + compactions. + The argument specifies the granularity at which deleted data is to be found: + - If ROW is specified, only whole deleted rows (or sets of rows) will be + discarded. + - If CELL is specified, any columns whose value is overwritten or deleted + will also be discarded. + - NONE (default) specifies the old behavior, overlapping tables are not used to + decide when to discard data. + Which option to use depends on your workload, both ROW and CELL increase the + disk load on compaction (especially with the size-tiered compaction strategy), + with CELL being more resource-intensive. Both should lead to better read + performance if deleting rows (resp. overwriting or deleting cells) is common. + - Prepared statements are now persisted in the table prepared_statements in + the system keyspace. Upon startup, this table is used to preload all + previously prepared statements - i.e. in many cases clients do not need to + re-prepare statements against restarted nodes. + - cqlsh can now connect to older Cassandra versions by downgrading the native + protocol version. Please note that this is currently not part of our release + testing and, as a consequence, it is not guaranteed to work in all cases. + See CASSANDRA-12150 for more details. + - Snapshots that are automatically taken before a table is dropped or truncated + will have a "dropped" or "truncated" prefix on their snapshot tag name. + - Metrics are exposed for successful and failed authentication attempts. + These can be located using the object names org.apache.cassandra.metrics:type=Client,name=AuthSuccess + and org.apache.cassandra.metrics:type=Client,name=AuthFailure respectively. + - Add support to "unset" JSON fields in prepared statements by specifying DEFAULT UNSET. + See CASSANDRA-11424 for details + - Allow TTL with null value on insert and update. It will be treated as equivalent to inserting a 0. + - Removed outboundBindAny configuration property. See CASSANDRA-12673 for details. + +Upgrading +--------- + - Request timeouts in cassandra.yaml (read_request_timeout_in_ms, etc) now apply to the + "full" request time on the coordinator. Previously, they only covered the time from + when the coordinator sent a message to a replica until the time that the replica + responded. Additionally, the previous behavior was to reset the timeout when performing + a read repair, making a second read to fix a short read, and when subranges were read + as part of a range scan or secondary index query. In 3.10 and higher, the timeout + is no longer reset for these "subqueries". The entire request must complete within + the specified timeout. As a consequence, your timeouts may need to be adjusted + to account for this. See CASSANDRA-12256 for more details. + - Logs written to stdout are now consistent with logs written to files. + Time is now local (it was UTC on the console and local in files). Date, thread, file + and line info where added to stdout. (see CASSANDRA-12004) + - The 'clientutil' jar, which has been somewhat broken on the 3.x branch, is not longer provided. + The features provided by that jar are provided by any good java driver and we advise relying on drivers rather on + that jar, but if you need that jar for backward compatiblity until you do so, you should use the version provided + on previous Cassandra branch, like the 3.0 branch (by design, the functionality provided by that jar are stable + accross versions so using the 3.0 jar for a client connecting to 3.x should work without issues). + - (Tools development) DatabaseDescriptor no longer implicitly startups components/services like + commit log replay. This may break existing 3rd party tools and clients. In order to startup + a standalone tool or client application, use the DatabaseDescriptor.toolInitialization() or + DatabaseDescriptor.clientInitialization() methods. Tool initialization sets up partitioner, + snitch, encryption context. Client initialization just applies the configuration but does not + setup anything. Instead of using Config.setClientMode() or Config.isClientMode(), which are + deprecated now, use one of the appropiate new methods in DatabaseDescriptor. + - Application layer keep-alives were added to the streaming protocol to prevent idle incoming connections from + timing out and failing the stream session (CASSANDRA-11839). This effectively deprecates the streaming_socket_timeout_in_ms + property in favor of streaming_keep_alive_period_in_secs. See cassandra.yaml for more details about this property. + - Duration litterals support the ISO 8601 format. By consequence, identifiers matching that format + (e.g P2Y or P1MT6H) will not be supported anymore (CASSANDRA-11873). + +3.8 +=== -3.0.9 -===== +New features +------------ + - Shared pool threads are now named according to the stage they are executing + tasks for. Thread names mentioned in traced queries change accordingly. + - A new option has been added to cassandra-stress "-rate fixed={number}/s" + that forces a scheduled rate of operations/sec over time. Using this, stress can + accurately account for coordinated ommission from the stress process. + - The cassandra-stress "-rate limit=" option has been renamed to "-rate throttle=" + - hdr histograms have been added to stress runs, it's output can be saved to disk using: + "-log hdrfile=" option. This histogram includes response/service/wait times when used with the + fixed or throttle rate options. The histogram file can be plotted on + http://hdrhistogram.github.io/HdrHistogram/plotFiles.html + - TimeWindowCompactionStrategy has been added. This has proven to be a better approach + to time series compaction and new tables should use this instead of DTCS. See + CASSANDRA-9666 for details. + - Change-Data-Capture is now available. See cassandra.yaml and for cdc-specific flags and + a brief explanation of on-disk locations for archived data in CommitLog form. This can + be enabled via ALTER TABLE ... WITH cdc=true. + Upon flush, CommitLogSegments containing data for CDC-enabled tables are moved to + the data/cdc_raw directory until removed by the user and writes to CDC-enabled tables + will be rejected with a WriteTimeoutException once cdc_total_space_in_mb is reached + between unflushed CommitLogSegments and cdc_raw. + NOTE: CDC is disabled by default in the .yaml file. Do not enable CDC on a mixed-version + cluster as it will lead to exceptions which can interrupt traffic. Once all nodes + have been upgraded to 3.8 it is safe to enable this feature and restart the cluster. Upgrading --------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/eca51a03/doc/cql3/CQL.textile ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/eca51a03/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDFunction.java ---------------------------------------------------------------------- diff --cc src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDFunction.java index 3ad60d0,47deafa..c568972 --- a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDFunction.java +++ b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDFunction.java @@@ -27,7 -27,9 +27,10 @@@ import java.util.* import java.util.concurrent.ExecutorService; import javax.script.*; +import jdk.nashorn.api.scripting.AbstractJSObject; + import jdk.nashorn.api.scripting.ClassFilter; + import jdk.nashorn.api.scripting.NashornScriptEngine; + import jdk.nashorn.api.scripting.NashornScriptEngineFactory; import org.apache.cassandra.concurrent.NamedThreadFactory; import org.apache.cassandra.cql3.ColumnIdentifier; import org.apache.cassandra.db.marshal.AbstractType; @@@ -158,13 -149,6 +152,10 @@@ final class ScriptBasedUDFunction exten throw new InvalidRequestException( String.format("Failed to compile function '%s' for language %s: %s", name, language, e)); } + + // It's not always possible to simply pass a plain Java object as a binding to Nashorn and + // let the script execute methods on it. - udfContextBinding = - ("Oracle Nashorn".equals(((ScriptEngine) scriptEngine).getFactory().getEngineName())) - ? new UDFContextWrapper() - : udfContext; ++ udfContextBinding = new UDFContextWrapper(); } protected ExecutorService executor() http://git-wip-us.apache.org/repos/asf/cassandra/blob/eca51a03/test/unit/org/apache/cassandra/cql3/validation/entities/UFScriptTest.java ----------------------------------------------------------------------
