[jira] [Created] (CASSANDRA-6340) Provide a mechanism for retrieving all replicas
Ahmed Bashir created CASSANDRA-6340: --- Summary: Provide a mechanism for retrieving all replicas Key: CASSANDRA-6340 URL: https://issues.apache.org/jira/browse/CASSANDRA-6340 Project: Cassandra Issue Type: Improvement Environment: Production Reporter: Ahmed Bashir In order to facilitate problem diagnosis, there should exist some mechanism to retrieve all copies of specific columns -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CASSANDRA-6339) Provide mechanism for probing specific endpoints for a row key and/or column
Ahmed Bashir created CASSANDRA-6339: --- Summary: Provide mechanism for probing specific endpoints for a row key and/or column Key: CASSANDRA-6339 URL: https://issues.apache.org/jira/browse/CASSANDRA-6339 Project: Cassandra Issue Type: Improvement Components: Core Environment: Production Reporter: Ahmed Bashir In order to facilitate problem diagnosis, there should be a mechanism for retrieving row/column information from a specific node as opposed to having the said node request data from replicas with some CL and reconciling the data. This mechanism should also avoid read repair and so forth; just a simple way to probe an endpoint to see what it knows about a row and/or column(s). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6275) 2.0.x leaks file handles
[ https://issues.apache.org/jira/browse/CASSANDRA-6275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820875#comment-13820875 ] Mikhail Mazursky commented on CASSANDRA-6275: - [~krummas] sorry, I cannot post that code. The scenario is something like this: {code} TRUNCATE table1; TRUNCATE table2; TRUNCATE table3; loop { SELECT FROM table1 WHERE key='xxx' (quorum); INSERT INTO table2 (quorum); INSERT INTO table3 IF NOT EXISTS; (should sometimes fail) UPDATE table1 WHERE key = 'someid' IF column='zzz'; (should sometimes fail) } {code} As I said, if I remove TRUNCATEs it do not leak. > 2.0.x leaks file handles > > > Key: CASSANDRA-6275 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6275 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: java version "1.7.0_25" > Java(TM) SE Runtime Environment (build 1.7.0_25-b15) > Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode) > Linux cassandra-test1 2.6.32-279.el6.x86_64 #1 SMP Thu Jun 21 15:00:18 EDT > 2012 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Mikhail Mazursky >Assignee: Marcus Eriksson > Attachments: cassandra_jstack.txt, leak.log, slog.gz > > > Looks like C* is leaking file descriptors when doing lots of CAS operations. > {noformat} > $ sudo cat /proc/15455/limits > Limit Soft Limit Hard Limit Units > Max cpu time unlimitedunlimitedseconds > Max file size unlimitedunlimitedbytes > Max data size unlimitedunlimitedbytes > Max stack size10485760 unlimitedbytes > Max core file size00bytes > Max resident set unlimitedunlimitedbytes > Max processes 1024 unlimitedprocesses > Max open files4096 4096 files > Max locked memory unlimitedunlimitedbytes > Max address space unlimitedunlimitedbytes > Max file locksunlimitedunlimitedlocks > Max pending signals 1463314633signals > Max msgqueue size 819200 819200 bytes > Max nice priority 00 > Max realtime priority 00 > Max realtime timeout unlimitedunlimitedus > {noformat} > Looks like the problem is not in limits. > Before load test: > {noformat} > cassandra-test0 ~]$ lsof -n | grep java | wc -l > 166 > cassandra-test1 ~]$ lsof -n | grep java | wc -l > 164 > cassandra-test2 ~]$ lsof -n | grep java | wc -l > 180 > {noformat} > After load test: > {noformat} > cassandra-test0 ~]$ lsof -n | grep java | wc -l > 967 > cassandra-test1 ~]$ lsof -n | grep java | wc -l > 1766 > cassandra-test2 ~]$ lsof -n | grep java | wc -l > 2578 > {noformat} > Most opened files have names like: > {noformat} > java 16890 cassandra 1636r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1637r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1638r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1639r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1640r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1641r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1642r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1643r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1644r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1645r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1646r REG 202,17 88724987 > 655520 /var/lib/cassandra/da
[jira] [Commented] (CASSANDRA-6275) 2.0.x leaks file handles
[ https://issues.apache.org/jira/browse/CASSANDRA-6275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820839#comment-13820839 ] Robert Coli commented on CASSANDRA-6275: A brief note to mention that when durable_writes are disabled, handling of clean shutdown (via SIGTERM/StorageServiceShutdownHook) has additional blocking while waiting for drain. If one is investigating and/or modifying the behavior of the shutdown hook, they should be aware that there are two different cases to test. See CASSANDRA-2958. > 2.0.x leaks file handles > > > Key: CASSANDRA-6275 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6275 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: java version "1.7.0_25" > Java(TM) SE Runtime Environment (build 1.7.0_25-b15) > Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode) > Linux cassandra-test1 2.6.32-279.el6.x86_64 #1 SMP Thu Jun 21 15:00:18 EDT > 2012 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Mikhail Mazursky >Assignee: Marcus Eriksson > Attachments: cassandra_jstack.txt, leak.log, slog.gz > > > Looks like C* is leaking file descriptors when doing lots of CAS operations. > {noformat} > $ sudo cat /proc/15455/limits > Limit Soft Limit Hard Limit Units > Max cpu time unlimitedunlimitedseconds > Max file size unlimitedunlimitedbytes > Max data size unlimitedunlimitedbytes > Max stack size10485760 unlimitedbytes > Max core file size00bytes > Max resident set unlimitedunlimitedbytes > Max processes 1024 unlimitedprocesses > Max open files4096 4096 files > Max locked memory unlimitedunlimitedbytes > Max address space unlimitedunlimitedbytes > Max file locksunlimitedunlimitedlocks > Max pending signals 1463314633signals > Max msgqueue size 819200 819200 bytes > Max nice priority 00 > Max realtime priority 00 > Max realtime timeout unlimitedunlimitedus > {noformat} > Looks like the problem is not in limits. > Before load test: > {noformat} > cassandra-test0 ~]$ lsof -n | grep java | wc -l > 166 > cassandra-test1 ~]$ lsof -n | grep java | wc -l > 164 > cassandra-test2 ~]$ lsof -n | grep java | wc -l > 180 > {noformat} > After load test: > {noformat} > cassandra-test0 ~]$ lsof -n | grep java | wc -l > 967 > cassandra-test1 ~]$ lsof -n | grep java | wc -l > 1766 > cassandra-test2 ~]$ lsof -n | grep java | wc -l > 2578 > {noformat} > Most opened files have names like: > {noformat} > java 16890 cassandra 1636r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1637r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1638r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1639r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1640r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1641r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1642r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1643r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1644r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1645r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1646r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1647r REG
[Cassandra Wiki] Update of "CassandraHardware_JP" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "CassandraHardware_JP" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/CassandraHardware_JP?action=diff&rev1=15&rev2=16 Comment: statcounter === クラウド === Cassandraのヘビーユーザーの何社かはクラウド環境にデプロイしています。例えばCloudKickはRackspace Cloud Serversを、SimpleGeoはAmazon EC2を使用しています。コミュニティの一般的な総意としては、CPU Bursting、ローカルディスクのRAID、およびパブリック/プライベートインターフェースの分離といった機能を備えるRackspaceのVMの方がパフォーマンスが良いとしています。 + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "CassandraHardware" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "CassandraHardware" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/CassandraHardware?action=diff&rev1=18&rev2=19 Comment: statcounter On EC2, the best practice is to use L or XL instances with local storage. I/o performance is proportionately much worse on S and M sizes, and EBS is a bad fit for several reasons (see [[http://www.mail-archive.com/user@cassandra.apache.org/msg11022.html|Erik Onnen's excellent explanation]]). Put the Cassandra commitlog on the root volume, and the data directory on the raid0'd ephemeral disks. + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "CassandraCli0_7" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "CassandraCli0_7" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/CassandraCli0_7?action=diff&rev1=39&rev2=40 Comment: statcounter This has just been a brief introduction with a couple of examples. For more information on how things work, type `help;` on the cli by itself or with any of the commands you're interested in. + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "CassandraCli08" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "CassandraCli08" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/CassandraCli08?action=diff&rev1=1&rev2=2 Comment: statcounter use Keyspace1 user 'badpasswd'; }}} + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} -
[Cassandra Wiki] Update of "CassandraCli07" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "CassandraCli07" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/CassandraCli07?action=diff&rev1=3&rev2=4 Comment: statcounter See [[CassandraCli0_7]] + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "CassandraCli06" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "CassandraCli06" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/CassandraCli06?action=diff&rev1=23&rev2=24 Returned 3 results. }}} + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "CassandraCli" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "CassandraCli" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/CassandraCli?action=diff&rev1=43&rev2=44 Comment: statcounter bin/cassandra-cli -host localhost -port 9160 -f script.txt }}} + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "Cassandra2474" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "Cassandra2474" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/Cassandra2474?action=diff&rev1=8&rev2=9 Comment: statcounter This also allows supporting SuperColumns, should we choose to do so. + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "Cassandra EC2" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "Cassandra EC2" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/Cassandra%20EC2?action=diff&rev1=7&rev2=8 Comment: statcounter * The [[http://www.datastax.com/docs/1.0/install/install_ami|DataStax AMI]] provides a quick and easy way to get Cassandra 1.0 up and running in minutes. * The [[https://github.com/riptano/chef|DataStax Chef]] repository provides yet another set of scripts for installing Cassandra 07x and 08x. These are updated a bit behind release schedule. + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "CaseStudies" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "CaseStudies" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/CaseStudies?action=diff&rev1=5&rev2=6 Comment: statcounter '''More to come''' + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ByteOrderedPartitioner" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ByteOrderedPartitioner" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ByteOrderedPartitioner?action=diff&rev1=2&rev2=3 Comment: statcounter Note that even if your application currently uses random UUID row keys for all data, you may run into balancing issues later on if you add new data with non-uniform keys, or keys of a different size. This is why RP is recommended for most applications. + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "BulkLoading" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "BulkLoading" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/BulkLoading?action=diff&rev1=1&rev2=2 Comment: statcounter * '''[[https://github.com/Ganglion/mumakil|Mumakil]]''' - a solution based on hadoop/avro to bulk load data into Cassandra + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "BinaryMemtable" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "BinaryMemtable" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/BinaryMemtable?action=diff&rev1=3&rev2=4 Comment: statcounter There is an example of using Hadoop to load data through the Binary Memtable interface at https://svn.apache.org/repos/asf/cassandra/branches/cassandra-0.8/examples/bmt/ . + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "Avro" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "Avro" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/Avro?action=diff&rev1=5&rev2=6 Comment: statcounter ||[[https://issues.apache.org/jira/browse/AVRO-341|avro-341]]||specify avro transport in spec|| ||[[https://issues.apache.org/jira/browse/AVRO-495|avro-495]]||(resolved in 1.4) Support includes for genavro IDL files|| + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ArticlesAndPresentations_JP" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArticlesAndPresentations_JP" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ArticlesAndPresentations_JP?action=diff&rev1=66&rev2=67 Comment: statcounter * [[http://www.slideshare.net/tomitakazutaka/consistency-level|ConsistencyLevel]] * [[http://www.slideshare.net/yutuki/cassandrah-baseno-sql|CassandraとHBaseの比較をして入門するNoSQL]] (Sep 2010) + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ArticlesAndPresentations" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArticlesAndPresentations" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ArticlesAndPresentations?action=diff&rev1=137&rev2=138 Comment: statcounter * [[http://itc.conversationsnetwork.org/shows/detail4439.html|The Cassandra Project - Technometria]]: Phil Windley interviews Jonathan Ellis on Cassandra. Date: 2010-03-09. * [[http://techzinglive.com/?p=75|Dude, Where’s My Database?! - Techzing! #8]]: Justin & Jason talk to Jonathan Ellis about Cassandra. Date: 2009-07-01 + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ArchitectureSSTable" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArchitectureSSTable" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ArchitectureSSTable?action=diff&rev1=6&rev2=7 1. Write disk key to SSTable Index file (UTF) 1. Write file position before (Write to SSTable Data File) (int) + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ArchitectureOverview_JP" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArchitectureOverview_JP" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ArchitectureOverview_JP?action=diff&rev1=14&rev2=15 Comment: statcounter You get consistency if R + W > N, where R is the number of records to read, W is the number of records to write, and N is the replication factor. A !ConsistencyLevel of ONE means R or W is 1. A !ConsistencyLevel of QUORUM means R or W is ceiling((N+1)/2). A !ConsistencyLevel of ALL means R or W is N. So if you want to write with a !ConsistencyLevel of ONE and then get the same data when you read, you need to read with !ConsistencyLevel ALL. + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ArchitectureOverview" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArchitectureOverview" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ArchitectureOverview?action=diff&rev1=14&rev2=15 Comment: statcounter You get consistency if R + W > N, where R is the number of records to read, W is the number of records to write, and N is the replication factor. A !ConsistencyLevel of ONE means R or W is 1. A !ConsistencyLevel of QUORUM means R or W is ceiling((N+1)/2). A !ConsistencyLevel of ALL means R or W is N. So if you want to write with a !ConsistencyLevel of ONE and then get the same data when you read, you need to read with !ConsistencyLevel ALL. + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ArchitectureInternals_ZH" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArchitectureInternals_ZH" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ArchitectureInternals_ZH?action=diff&rev1=20&rev2=21 Comment: statcounter * Cassandra's on-disk storage model is loosely based on sections 5.3 and 5.4 of [[http://labs.google.com/papers/bigtable.html|the Bigtable paper]]. * Facebook's Cassandra team authored a paper on Cassandra for LADIS 09: http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf. Most of the information there is applicable to Apache Cassandra (the main exception is the integration of !ZooKeeper). + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ArchitectureInternals_JP" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArchitectureInternals_JP" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ArchitectureInternals_JP?action=diff&rev1=20&rev2=21 * Cassandraのディスク上のストレージモデルは[[http://labs.google.com/papers/bigtable.html|Bigtableの論文]]のセクション5.3と5.4にほぼ基づいています。 * FacebookのCassandraチームがLADIS09で[[http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf|Cassandraの論文(PDF)]]を発表しました。いまでは、ZooKeeperとの統合部分が主な違いとなっています。 + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ArchitectureInternals" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArchitectureInternals" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ArchitectureInternals?action=diff&rev1=29&rev2=30 Comment: statcounter * Cassandra's on-disk storage model is loosely based on sections 5.3 and 5.4 of [[http://labs.google.com/papers/bigtable.html|the Bigtable paper]]. * Facebook's Cassandra team authored a paper on Cassandra for LADIS 09: http://www.cs.cornell.edu/projects/ladis2009/papers/lakshman-ladis2009.pdf. Most of the information there is applicable to Apache Cassandra (the main exception is the integration of !ZooKeeper). + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ArchitectureGossip_JP" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArchitectureGossip_JP" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ArchitectureGossip_JP?action=diff&rev1=4&rev2=5 Comment: statcounter あとでちゃんと書く + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ArchitectureGossip" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArchitectureGossip" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ArchitectureGossip?action=diff&rev1=10&rev2=11 Comment: statcounter Simplified, this what 10.0.0.2 asked for in the previous !GossipDigestAckMessage: everything after version 324 for 10.0.0.1's current generation, and everything for both 10.0.0.3 and 10.0.0.4 at their current generations. After 10.0.0.2 applies this information, 10.0.0.1 and 10.0.0.2 will have synced their gossip information, and this gossip round is complete. + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}}
[Cassandra Wiki] Update of "ArchitectureCommitLog" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArchitectureCommitLog" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ArchitectureCommitLog?action=diff&rev1=7&rev2=8 Comment: statcounter * For each log entry read, the log is replayed for a !ColumnFamily CF if the position of the log entry is no less than the !ReplayPosition for CF in the most recent !SSTable metadata. * When log replay is done, all Memtables are force flushed to disk and all commitlog segments are recycled. + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ArchitectureAntiEntropy" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ArchitectureAntiEntropy" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/ArchitectureAntiEntropy?action=diff&rev1=5&rev2=6 Comment: statcounter One possible fix to this problem would be to use something like a [[http://comonad.com/reader/2008/linear-bloom-filters/|Linear Bloom Filter]] to store a summary of every SSTable on disk, where each sub-bloom is partitioned using 'midpoint()' like the current !MerkleTree. Then, to validate a column family, you could OR together the bloom filters for each SSTable, and send it to neighbors without performing a compaction. + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "API10" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "API10" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/API10?action=diff&rev1=33&rev2=34 Comment: statcounter == Examples == [[http://wiki.apache.org/cassandra/ClientExamples|There are a few examples on this page over here.]] + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "API07" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "API07" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/API07?action=diff&rev1=3&rev2=4 Comment: statcounter == Examples == [[http://wiki.apache.org/cassandra/ClientExamples|There are a few examples on this page over here.]] + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[jira] [Commented] (CASSANDRA-6127) vnodes don't scale to hundreds of nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820669#comment-13820669 ] Jonathan Ellis commented on CASSANDRA-6127: --- bq. ISTM that FD processing Gossip updates synchronously is a fundamental problem. Any hiccup in processing will cause FD false positives. I've pulled a fix for this out to CASSANDRA-6338. > vnodes don't scale to hundreds of nodes > --- > > Key: CASSANDRA-6127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6127 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Any cluster that has vnodes and consists of hundreds of > physical nodes. >Reporter: Tupshin Harper >Assignee: Jonathan Ellis > Attachments: 2013-11-05_18-04-03_no_compression_cpu_time.png, > 2013-11-05_18-09-38_compression_on_cpu_time.png, 6000vnodes.patch, > AdjustableGossipPeriod.patch, cpu-vs-token-graph.png, > delayEstimatorUntilStatisticallyValid.patch, flaps-vs-tokens.png > > > There are a lot of gossip-related issues related to very wide clusters that > also have vnodes enabled. Let's use this ticket as a master in case there are > sub-tickets. > The most obvious symptom I've seen is with 1000 nodes in EC2 with m1.xlarge > instances. Each node configured with 32 vnodes. > Without vnodes, cluster spins up fine and is ready to handle requests within > 30 minutes or less. > With vnodes, nodes are reporting constant up/down flapping messages with no > external load on the cluster. After a couple of hours, they were still > flapping, had very high cpu load, and the cluster never looked like it was > going to stabilize or be useful for traffic. -- This message was sent by Atlassian JIRA (v6.1#6144)
[Cassandra Wiki] Update of "API06" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "API06" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/API06?action=diff&rev1=59&rev2=60 Comment: statcounter == Examples == [[http://wiki.apache.org/cassandra/ClientExamples|There are a few examples on this page over here.]] + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "API0506" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "API0506" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/API0506?action=diff&rev1=76&rev2=77 Comment: statcounter == Examples == [[http://wiki.apache.org/cassandra/ClientExamples|There are a few examples on this page over here.]] + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[jira] [Updated] (CASSANDRA-6338) Make gossip tolerate slow Gossip tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-6338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6338: -- Attachment: 6338.txt Attached. Of course this doesn't help if we have too much for one thread to handle and it just gets farther and farther behind, but I don't think we've seen a failure scenario like that yet (at least post-CASSANDRA-6244). And even then, at least this gives users something obvious in the log to alert them as to the cause of the problem instead of "your cluster mysteriously marked everyone down and started serving up UAE." > Make gossip tolerate slow Gossip tasks > -- > > Key: CASSANDRA-6338 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6338 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Labels: gossip > Fix For: 2.0.3 > > Attachments: 6338.txt > > > Currently if a single gossip task bogs down the gossip Stage, Gossip will > mark everyone down because it hasn't seen updates from them (since they are > all queued behind the slow one). > This means that full GCs can cause gossip "flapping" as well as any actually > problematic tasks such as recomputing pending ranges. -- This message was sent by Atlassian JIRA (v6.1#6144)
[Cassandra Wiki] Update of "API05" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "API05" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/API05?action=diff&rev1=79&rev2=80 Comment: statcounter == Examples == [[http://wiki.apache.org/cassandra/ClientExamples|There are a few examples on this page over here.]] + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[jira] [Updated] (CASSANDRA-6338) Make gossip tolerate slow Gossip tasks
[ https://issues.apache.org/jira/browse/CASSANDRA-6338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6338: -- Description: Currently if a single gossip task bogs down the gossip Stage, Gossip will mark everyone down because it hasn't seen updates from them (since they are all queued behind the slow one). This means that full GCs can cause gossip "flapping" as well as any actually problematic tasks such as recomputing pending ranges. was:Currently if a single gossip task bogs down the gossip Stage, Gossip will mark everyone down because it hasn't seen updates from them (since they are all queued behind the slow one). > Make gossip tolerate slow Gossip tasks > -- > > Key: CASSANDRA-6338 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6338 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Labels: gossip > Fix For: 2.0.3 > > > Currently if a single gossip task bogs down the gossip Stage, Gossip will > mark everyone down because it hasn't seen updates from them (since they are > all queued behind the slow one). > This means that full GCs can cause gossip "flapping" as well as any actually > problematic tasks such as recomputing pending ranges. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CASSANDRA-6338) Make gossip tolerate slow Gossip tasks
Jonathan Ellis created CASSANDRA-6338: - Summary: Make gossip tolerate slow Gossip tasks Key: CASSANDRA-6338 URL: https://issues.apache.org/jira/browse/CASSANDRA-6338 Project: Cassandra Issue Type: Bug Components: Core Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 2.0.3 Currently if a single gossip task bogs down the gossip Stage, Gossip will mark everyone down because it hasn't seen updates from them (since they are all queued behind the slow one). -- This message was sent by Atlassian JIRA (v6.1#6144)
[Cassandra Wiki] Update of "API04" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "API04" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/API04?action=diff&rev1=1&rev2=2 Comment: statcounter == Examples == [[http://wiki.apache.org/cassandra/ClientExamples|There are a few examples on this page over here.]] + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[jira] [Commented] (CASSANDRA-5519) Reduce index summary memory use for cold sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-5519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820653#comment-13820653 ] Tyler Hobbs commented on CASSANDRA-5519: I need to put this through more thorough testing and benchmarking, but I think it's at a good point for a preliminary review: https://github.com/thobbs/cassandra/compare/CASSANDRA-5519 A few comments/questions: * I went with best-effort for the memory pool (if all summaries don't fit in the allotted space even at the minimum sampling level, there's nothing we can do about it). The amount of memory used may also temporarily exceed the limit while building new summaries. * There are two new cassandra.yaml options: one for controlling the memory pool size and one for regulating how frequently summaries are resized. These can also be set through JMX. We could conceivably also make the down/upsample thresholds and the minimum sampling level configurable. All of these default values are just guesses. * I went with a reference counting strategy for free'ing the IndexSummary's Memory. This makes the API a bit unpleasant (mostly in SSTR), but it should have low overhead. A ReadWriteLock might also work well instead of this with a cleaner API; let me know if I should benchmark the two for comparison. * I'm triggering the IndexSummaryManager singleton's initialization in DatabaseDescriptor; this feels wrong, so I'm open to suggestions. > Reduce index summary memory use for cold sstables > - > > Key: CASSANDRA-5519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5519 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jonathan Ellis >Assignee: Tyler Hobbs >Priority: Minor > Fix For: 2.1 > > Attachments: downsample.py > > -- This message was sent by Atlassian JIRA (v6.1#6144)
[Cassandra Wiki] Update of "API03" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "API03" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/API03?action=diff&rev1=4&rev2=5 Comment: statcounter oneway void touch(1:string key, 2:bool fData), }}} + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "API" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "API" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/API?action=diff&rev1=38&rev2=39 Comment: statcounter == Thrift == You can read about the legacy Thrift RPC API [[API10|here]]. + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "AntiEntropy_JP" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "AntiEntropy_JP" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/AntiEntropy_JP?action=diff&rev1=2&rev2=3 Comment: statcounter Cassandra の anti-entropy の実装の重要な差異は、Merkle 木が column family ごとに作成され、それらは近隣のノードに (Merkle 木を?) 送るのにかかる時間以上にはメンテナンスされないことです。代わりに、木はメジャーコンパクションの間のデータセットのスナップショットになっていることが保証されます。すなわち、超過データはネットワークを経由して送られるかも知れませんが、これは local disk の IO をセーブします。これは非常に大きなデータセットについては好ましいことです。 - {{http://c.statcounter.com/9397521/0/fe557aad/1/|stats}} + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}}
[Cassandra Wiki] Update of "AntiEntropy" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "AntiEntropy" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/AntiEntropy?action=diff&rev1=6&rev2=7 Comment: statcounter The key difference in Cassandra's implementation of anti-entropy is that the Merkle trees are built per column family, and they are not maintained for longer than it takes to send them to neighboring nodes. Instead, the trees are generated as snapshots of the dataset during major compactions: this means that excess data might be sent across the network, but it saves local disk IO, and is preferable for very large datasets. - {{http://c.statcounter.com/9397521/0/fe557aad/1/|stats}} + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}}
[Cassandra Wiki] Update of "Administration Tools" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "Administration Tools" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/Administration%20Tools?action=diff&rev1=9&rev2=10 Comment: statcounter * [[https://github.com/tomekkup/helenos]] Helenos is a web based GUI tool that helps you to explore data and manage schema. - {{http://c.statcounter.com/9397521/0/fe557aad/1/|stats}} + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}}
[Cassandra Wiki] Trivial Update of "FrontPage" by JonathanEllis
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "FrontPage" page has been changed by JonathanEllis: https://wiki.apache.org/cassandra/FrontPage?action=diff&rev1=98&rev2=99 * [[FrontPage_JP|Japanese 日本語]] * [[FrontPage_PT-BR|BrazilianPortuguese Português do Brasil]] - {{http://c.statcounter.com/9397521/0/fe557aad/1/|stats}} + {{https://c.statcounter.com/9397521/0/fe557aad/1/|stats}}
[Cassandra Wiki] Update of "AntiEntropy_JP" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "AntiEntropy_JP" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/AntiEntropy_JP?action=diff&rev1=1&rev2=2 Comment: statcounter Cassandra の anti-entropy の実装の重要な差異は、Merkle 木が column family ごとに作成され、それらは近隣のノードに (Merkle 木を?) 送るのにかかる時間以上にはメンテナンスされないことです。代わりに、木はメジャーコンパクションの間のデータセットのスナップショットになっていることが保証されます。すなわち、超過データはネットワークを経由して送られるかも知れませんが、これは local disk の IO をセーブします。これは非常に大きなデータセットについては好ましいことです。 + {{http://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "AntiEntropy" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "AntiEntropy" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/AntiEntropy?action=diff&rev1=5&rev2=6 Comment: statcounter The key difference in Cassandra's implementation of anti-entropy is that the Merkle trees are built per column family, and they are not maintained for longer than it takes to send them to neighboring nodes. Instead, the trees are generated as snapshots of the dataset during major compactions: this means that excess data might be sent across the network, but it saves local disk IO, and is preferable for very large datasets. + {{http://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "Administration Tools" by GehrigKunz
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "Administration Tools" page has been changed by GehrigKunz: https://wiki.apache.org/cassandra/Administration%20Tools?action=diff&rev1=8&rev2=9 Comment: statcounter * [[https://github.com/tomekkup/helenos]] Helenos is a web based GUI tool that helps you to explore data and manage schema. + {{http://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[Cassandra Wiki] Update of "ContributorsGroup" by JonathanEllis
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "ContributorsGroup" page has been changed by JonathanEllis: https://wiki.apache.org/cassandra/ContributorsGroup?action=diff&rev1=21&rev2=22 * ErnieHershey * FlipKromer * gdusbabek + * GehrigKunz * JakeLuciani * JasonBrown * JeremiahJordan
[6/6] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fa86ce88 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fa86ce88 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fa86ce88 Branch: refs/heads/trunk Commit: fa86ce88c481ee193be39d7a32f9b9536d171dfe Parents: c20dd29 91f976c Author: Jonathan Ellis Authored: Tue Nov 12 15:51:08 2013 -0600 Committer: Jonathan Ellis Committed: Tue Nov 12 15:51:08 2013 -0600 -- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fa86ce88/CHANGES.txt --
[2/6] git commit: Fix reading expired row value from row cache patch by jbellis; reviewed by Mikhail Stepura for CASSANDRA-6325
Fix reading expired row value from row cache patch by jbellis; reviewed by Mikhail Stepura for CASSANDRA-6325 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9a594c3a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9a594c3a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9a594c3a Branch: refs/heads/cassandra-2.0 Commit: 9a594c3a55a3b5b63eafe5c7a4a0b2fe26fd0cb1 Parents: 3cb5854 Author: Jonathan Ellis Authored: Tue Nov 12 15:50:53 2013 -0600 Committer: Jonathan Ellis Committed: Tue Nov 12 15:50:53 2013 -0600 -- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9a594c3a/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 7abf5d8..f186c37 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -16,6 +16,7 @@ * cqlsh: handle 'null' as session duration (CASSANDRA-6317) * Fix json2sstable handling of range tombstones (CASSANDRA-6316) * Fix missing one row in reverse query (CASSANDRA-6330) + * Fix reading expired row value from row cache (CASSANDRA-6325) 1.2.11
[3/6] git commit: Fix reading expired row value from row cache patch by jbellis; reviewed by Mikhail Stepura for CASSANDRA-6325
Fix reading expired row value from row cache patch by jbellis; reviewed by Mikhail Stepura for CASSANDRA-6325 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9a594c3a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9a594c3a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9a594c3a Branch: refs/heads/trunk Commit: 9a594c3a55a3b5b63eafe5c7a4a0b2fe26fd0cb1 Parents: 3cb5854 Author: Jonathan Ellis Authored: Tue Nov 12 15:50:53 2013 -0600 Committer: Jonathan Ellis Committed: Tue Nov 12 15:50:53 2013 -0600 -- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9a594c3a/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 7abf5d8..f186c37 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -16,6 +16,7 @@ * cqlsh: handle 'null' as session duration (CASSANDRA-6317) * Fix json2sstable handling of range tombstones (CASSANDRA-6316) * Fix missing one row in reverse query (CASSANDRA-6330) + * Fix reading expired row value from row cache (CASSANDRA-6325) 1.2.11
[1/6] git commit: Fix reading expired row value from row cache patch by jbellis; reviewed by Mikhail Stepura for CASSANDRA-6325
Updated Branches: refs/heads/cassandra-1.2 3cb5854e9 -> 9a594c3a5 refs/heads/cassandra-2.0 7fb2d24fa -> 91f976ce1 refs/heads/trunk c20dd2992 -> fa86ce88c Fix reading expired row value from row cache patch by jbellis; reviewed by Mikhail Stepura for CASSANDRA-6325 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9a594c3a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9a594c3a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9a594c3a Branch: refs/heads/cassandra-1.2 Commit: 9a594c3a55a3b5b63eafe5c7a4a0b2fe26fd0cb1 Parents: 3cb5854 Author: Jonathan Ellis Authored: Tue Nov 12 15:50:53 2013 -0600 Committer: Jonathan Ellis Committed: Tue Nov 12 15:50:53 2013 -0600 -- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9a594c3a/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 7abf5d8..f186c37 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -16,6 +16,7 @@ * cqlsh: handle 'null' as session duration (CASSANDRA-6317) * Fix json2sstable handling of range tombstones (CASSANDRA-6316) * Fix missing one row in reverse query (CASSANDRA-6330) + * Fix reading expired row value from row cache (CASSANDRA-6325) 1.2.11
[5/6] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/91f976ce Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/91f976ce Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/91f976ce Branch: refs/heads/cassandra-2.0 Commit: 91f976ce19bd22c5e80c38f9e81dacdabcd0e56e Parents: 7fb2d24 9a594c3 Author: Jonathan Ellis Authored: Tue Nov 12 15:50:59 2013 -0600 Committer: Jonathan Ellis Committed: Tue Nov 12 15:50:59 2013 -0600 -- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/91f976ce/CHANGES.txt -- diff --cc CHANGES.txt index 69b22a3,f186c37..3305739 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -37,42 -16,10 +37,43 @@@ Merged from 1.2 * cqlsh: handle 'null' as session duration (CASSANDRA-6317) * Fix json2sstable handling of range tombstones (CASSANDRA-6316) * Fix missing one row in reverse query (CASSANDRA-6330) + * Fix reading expired row value from row cache (CASSANDRA-6325) -1.2.11 +2.0.2 + * Update FailureDetector to use nanontime (CASSANDRA-4925) + * Fix FileCacheService regressions (CASSANDRA-6149) + * Never return WriteTimeout for CL.ANY (CASSANDRA-6032) + * Fix race conditions in bulk loader (CASSANDRA-6129) + * Add configurable metrics reporting (CASSANDRA-4430) + * drop queries exceeding a configurable number of tombstones (CASSANDRA-6117) + * Track and persist sstable read activity (CASSANDRA-5515) + * Fixes for speculative retry (CASSANDRA-5932, CASSANDRA-6194) + * Improve memory usage of metadata min/max column names (CASSANDRA-6077) + * Fix thrift validation refusing row markers on CQL3 tables (CASSANDRA-6081) + * Fix insertion of collections with CAS (CASSANDRA-6069) + * Correctly send metadata on SELECT COUNT (CASSANDRA-6080) + * Track clients' remote addresses in ClientState (CASSANDRA-6070) + * Create snapshot dir if it does not exist when migrating + leveled manifest (CASSANDRA-6093) + * make sequential nodetool repair the default (CASSANDRA-5950) + * Add more hooks for compaction strategy implementations (CASSANDRA-6111) + * Fix potential NPE on composite 2ndary indexes (CASSANDRA-6098) + * Delete can potentially be skipped in batch (CASSANDRA-6115) + * Allow alter keyspace on system_traces (CASSANDRA-6016) + * Disallow empty column names in cql (CASSANDRA-6136) + * Use Java7 file-handling APIs and fix file moving on Windows (CASSANDRA-5383) + * Save compaction history to system keyspace (CASSANDRA-5078) + * Fix NPE if StorageService.getOperationMode() is executed before full startup (CASSANDRA-6166) + * CQL3: support pre-epoch longs for TimestampType (CASSANDRA-6212) + * Add reloadtriggers command to nodetool (CASSANDRA-4949) + * cqlsh: ignore empty 'value alias' in DESCRIBE (CASSANDRA-6139) + * Fix sstable loader (CASSANDRA-6205) + * Reject bootstrapping if the node already exists in gossip (CASSANDRA-5571) + * Fix NPE while loading paxos state (CASSANDRA-6211) + * cqlsh: add SHOW SESSION command (CASSANDRA-6228) +Merged from 1.2: + * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114) * Add a warning for small LCS sstable size (CASSANDRA-6191) * Add ability to list specific KS/CF combinations in nodetool cfstats (CASSANDRA-4191) * Mark CF clean if a mutation raced the drop and got it marked dirty (CASSANDRA-5946)
[4/6] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/91f976ce Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/91f976ce Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/91f976ce Branch: refs/heads/trunk Commit: 91f976ce19bd22c5e80c38f9e81dacdabcd0e56e Parents: 7fb2d24 9a594c3 Author: Jonathan Ellis Authored: Tue Nov 12 15:50:59 2013 -0600 Committer: Jonathan Ellis Committed: Tue Nov 12 15:50:59 2013 -0600 -- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/91f976ce/CHANGES.txt -- diff --cc CHANGES.txt index 69b22a3,f186c37..3305739 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -37,42 -16,10 +37,43 @@@ Merged from 1.2 * cqlsh: handle 'null' as session duration (CASSANDRA-6317) * Fix json2sstable handling of range tombstones (CASSANDRA-6316) * Fix missing one row in reverse query (CASSANDRA-6330) + * Fix reading expired row value from row cache (CASSANDRA-6325) -1.2.11 +2.0.2 + * Update FailureDetector to use nanontime (CASSANDRA-4925) + * Fix FileCacheService regressions (CASSANDRA-6149) + * Never return WriteTimeout for CL.ANY (CASSANDRA-6032) + * Fix race conditions in bulk loader (CASSANDRA-6129) + * Add configurable metrics reporting (CASSANDRA-4430) + * drop queries exceeding a configurable number of tombstones (CASSANDRA-6117) + * Track and persist sstable read activity (CASSANDRA-5515) + * Fixes for speculative retry (CASSANDRA-5932, CASSANDRA-6194) + * Improve memory usage of metadata min/max column names (CASSANDRA-6077) + * Fix thrift validation refusing row markers on CQL3 tables (CASSANDRA-6081) + * Fix insertion of collections with CAS (CASSANDRA-6069) + * Correctly send metadata on SELECT COUNT (CASSANDRA-6080) + * Track clients' remote addresses in ClientState (CASSANDRA-6070) + * Create snapshot dir if it does not exist when migrating + leveled manifest (CASSANDRA-6093) + * make sequential nodetool repair the default (CASSANDRA-5950) + * Add more hooks for compaction strategy implementations (CASSANDRA-6111) + * Fix potential NPE on composite 2ndary indexes (CASSANDRA-6098) + * Delete can potentially be skipped in batch (CASSANDRA-6115) + * Allow alter keyspace on system_traces (CASSANDRA-6016) + * Disallow empty column names in cql (CASSANDRA-6136) + * Use Java7 file-handling APIs and fix file moving on Windows (CASSANDRA-5383) + * Save compaction history to system keyspace (CASSANDRA-5078) + * Fix NPE if StorageService.getOperationMode() is executed before full startup (CASSANDRA-6166) + * CQL3: support pre-epoch longs for TimestampType (CASSANDRA-6212) + * Add reloadtriggers command to nodetool (CASSANDRA-4949) + * cqlsh: ignore empty 'value alias' in DESCRIBE (CASSANDRA-6139) + * Fix sstable loader (CASSANDRA-6205) + * Reject bootstrapping if the node already exists in gossip (CASSANDRA-5571) + * Fix NPE while loading paxos state (CASSANDRA-6211) + * cqlsh: add SHOW SESSION command (CASSANDRA-6228) +Merged from 1.2: + * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114) * Add a warning for small LCS sstable size (CASSANDRA-6191) * Add ability to list specific KS/CF combinations in nodetool cfstats (CASSANDRA-4191) * Mark CF clean if a mutation raced the drop and got it marked dirty (CASSANDRA-5946)
[jira] [Comment Edited] (CASSANDRA-6127) vnodes don't scale to hundreds of nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820499#comment-13820499 ] Quentin Conner edited comment on CASSANDRA-6127 at 11/12/13 9:34 PM: - The num_tokens setting has a mild impact on average cpu utilization. Please see the graph below for the trend with 256 nodes. !cpu-vs-token-graph.png! This graph does not characterize the bursty nature of any given node's CPU utilization. It does average the utilization over a 200 second period, taken at 10 second intervals using "sar -u". Since gossip heartbeat destinations are random, "unlucky" nodes will sometimes receive twice the gossip traffic and (10 second basis) CPU utilization has been casually observed @25% for N=256 and 60% for N=512. was (Author: qconner): The num_tokens setting has a mild impact on average cpu utilization. Please see the graph for the trend with N=256, 256 nodes. This graph does not characterize the bursty nature of any given node's CPU utilization. It does average the utilization over a 200 second period, taken at 10 second intervals using "sar -u". Since gossip heartbeat destinations are random, "unlucky" nodes will sometimes receive twice the gossip traffic and (10 second basis) CPU utilization has been casually observed @25% for N=256 and 60% for N=512. > vnodes don't scale to hundreds of nodes > --- > > Key: CASSANDRA-6127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6127 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Any cluster that has vnodes and consists of hundreds of > physical nodes. >Reporter: Tupshin Harper >Assignee: Jonathan Ellis > Attachments: 2013-11-05_18-04-03_no_compression_cpu_time.png, > 2013-11-05_18-09-38_compression_on_cpu_time.png, 6000vnodes.patch, > AdjustableGossipPeriod.patch, cpu-vs-token-graph.png, > delayEstimatorUntilStatisticallyValid.patch, flaps-vs-tokens.png > > > There are a lot of gossip-related issues related to very wide clusters that > also have vnodes enabled. Let's use this ticket as a master in case there are > sub-tickets. > The most obvious symptom I've seen is with 1000 nodes in EC2 with m1.xlarge > instances. Each node configured with 32 vnodes. > Without vnodes, cluster spins up fine and is ready to handle requests within > 30 minutes or less. > With vnodes, nodes are reporting constant up/down flapping messages with no > external load on the cluster. After a couple of hours, they were still > flapping, had very high cpu load, and the cluster never looked like it was > going to stabilize or be useful for traffic. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Comment Edited] (CASSANDRA-6127) vnodes don't scale to hundreds of nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820488#comment-13820488 ] Quentin Conner edited comment on CASSANDRA-6127 at 11/12/13 9:33 PM: - Flapping occurs with vnodes or without. Please see below. !flaps-vs-tokens.png! Using vnodes appears to exacerbate, possibly with longer messages, probably with higher cpu utilization. Either would delay the timestamp for the Failure Detector interarrival time. was (Author: qconner): Flapping occurs with vnodes or without. Please see attached. Using vnodes appears to exacerbate, possibly with longer messages, probably with higher cpu utilization. Either would delay the timestamp for the Failure Detector interarrival time. > vnodes don't scale to hundreds of nodes > --- > > Key: CASSANDRA-6127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6127 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Any cluster that has vnodes and consists of hundreds of > physical nodes. >Reporter: Tupshin Harper >Assignee: Jonathan Ellis > Attachments: 2013-11-05_18-04-03_no_compression_cpu_time.png, > 2013-11-05_18-09-38_compression_on_cpu_time.png, 6000vnodes.patch, > AdjustableGossipPeriod.patch, cpu-vs-token-graph.png, > delayEstimatorUntilStatisticallyValid.patch, flaps-vs-tokens.png > > > There are a lot of gossip-related issues related to very wide clusters that > also have vnodes enabled. Let's use this ticket as a master in case there are > sub-tickets. > The most obvious symptom I've seen is with 1000 nodes in EC2 with m1.xlarge > instances. Each node configured with 32 vnodes. > Without vnodes, cluster spins up fine and is ready to handle requests within > 30 minutes or less. > With vnodes, nodes are reporting constant up/down flapping messages with no > external load on the cluster. After a couple of hours, they were still > flapping, had very high cpu load, and the cluster never looked like it was > going to stabilize or be useful for traffic. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6127) vnodes don't scale to hundreds of nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quentin Conner updated CASSANDRA-6127: -- Attachment: cpu-vs-token-graph.png The num_tokens setting has a mild impact on average cpu utilization. Please see the graph for the trend with N=256, 256 nodes. This graph does not characterize the bursty nature of any given node's CPU utilization. It does average the utilization over a 200 second period, taken at 10 second intervals using "sar -u". Since gossip heartbeat destinations are random, "unlucky" nodes will sometimes receive twice the gossip traffic and (10 second basis) CPU utilization has been casually observed @25% for N=256 and 60% for N=512. > vnodes don't scale to hundreds of nodes > --- > > Key: CASSANDRA-6127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6127 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Any cluster that has vnodes and consists of hundreds of > physical nodes. >Reporter: Tupshin Harper >Assignee: Jonathan Ellis > Attachments: 2013-11-05_18-04-03_no_compression_cpu_time.png, > 2013-11-05_18-09-38_compression_on_cpu_time.png, 6000vnodes.patch, > AdjustableGossipPeriod.patch, cpu-vs-token-graph.png, > delayEstimatorUntilStatisticallyValid.patch, flaps-vs-tokens.png > > > There are a lot of gossip-related issues related to very wide clusters that > also have vnodes enabled. Let's use this ticket as a master in case there are > sub-tickets. > The most obvious symptom I've seen is with 1000 nodes in EC2 with m1.xlarge > instances. Each node configured with 32 vnodes. > Without vnodes, cluster spins up fine and is ready to handle requests within > 30 minutes or less. > With vnodes, nodes are reporting constant up/down flapping messages with no > external load on the cluster. After a couple of hours, they were still > flapping, had very high cpu load, and the cluster never looked like it was > going to stabilize or be useful for traffic. -- This message was sent by Atlassian JIRA (v6.1#6144)
[Cassandra Wiki] Update of "FrontPage" by JonathanEllis
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "FrontPage" page has been changed by JonathanEllis: https://wiki.apache.org/cassandra/FrontPage?action=diff&rev1=97&rev2=98 Comment: statcounter * [[FrontPage_JP|Japanese 日本語]] * [[FrontPage_PT-BR|BrazilianPortuguese Português do Brasil]] + {{http://c.statcounter.com/9397521/0/fe557aad/1/|stats}} +
[jira] [Updated] (CASSANDRA-6127) vnodes don't scale to hundreds of nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-6127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quentin Conner updated CASSANDRA-6127: -- Attachment: flaps-vs-tokens.png Flapping occurs with vnodes or without. Please see attached. Using vnodes appears to exacerbate, possibly with longer messages, probably with higher cpu utilization. Either would delay the timestamp for the Failure Detector interarrival time. > vnodes don't scale to hundreds of nodes > --- > > Key: CASSANDRA-6127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6127 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Any cluster that has vnodes and consists of hundreds of > physical nodes. >Reporter: Tupshin Harper >Assignee: Jonathan Ellis > Attachments: 2013-11-05_18-04-03_no_compression_cpu_time.png, > 2013-11-05_18-09-38_compression_on_cpu_time.png, 6000vnodes.patch, > AdjustableGossipPeriod.patch, delayEstimatorUntilStatisticallyValid.patch, > flaps-vs-tokens.png > > > There are a lot of gossip-related issues related to very wide clusters that > also have vnodes enabled. Let's use this ticket as a master in case there are > sub-tickets. > The most obvious symptom I've seen is with 1000 nodes in EC2 with m1.xlarge > instances. Each node configured with 32 vnodes. > Without vnodes, cluster spins up fine and is ready to handle requests within > 30 minutes or less. > With vnodes, nodes are reporting constant up/down flapping messages with no > external load on the cluster. After a couple of hours, they were still > flapping, had very high cpu load, and the cluster never looked like it was > going to stabilize or be useful for traffic. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6337) Set minTimestamp correctly to be able to drop expired sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820436#comment-13820436 ] Jonathan Ellis commented on CASSANDRA-6337: --- +1 > Set minTimestamp correctly to be able to drop expired sstables > -- > > Key: CASSANDRA-6337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6337 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 2.0.3 > > Attachments: 0001-Set-minTimestamp-correctly.patch > > > When calculating which sstables we can drop we set minTimestamp to > Integer.MAX_VALUE, this is wrong since minTimestamp is a long, and in most > cases minTimestamp on sstables is larger than Integer.MAX_VALUE. > We should set it to Long.MAX_VALUE, patch does that. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Comment Edited] (CASSANDRA-6335) Hints broken for nodes that change broadcast address
[ https://issues.apache.org/jira/browse/CASSANDRA-6335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820415#comment-13820415 ] Rick Branson edited comment on CASSANDRA-6335 at 11/12/13 8:41 PM: --- It looks like it stalls the hint delivery thread pool indefinitely. Restarting the node frees it up and it starts delivering hints again. I'm rolling this cluster to workaround the issue. EDIT: disregard the "stalling" issue... there were just a large # of hints that needed to be delivered to a specific node (because it was down for a bit for some maintenance) clogging up the pool and confusing the issue. was (Author: rbranson): It looks like it stalls the hint delivery thread pool indefinitely. Restarting the node frees it up and it starts delivering hints again. I'm rolling this cluster to workaround the issue. > Hints broken for nodes that change broadcast address > > > Key: CASSANDRA-6335 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6335 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Rick Branson > > When a node changes it's broadcast address, the transition process works > properly, but hints that are destined for it can't be delivered because of > the address change. It produces an exception: > java.lang.AssertionError: Missing host ID for 10.1.60.22 > at > org.apache.cassandra.service.StorageProxy.writeHintForMutation(StorageProxy.java:598) > at > org.apache.cassandra.service.StorageProxy$5.runMayThrow(StorageProxy.java:567) > at > org.apache.cassandra.service.StorageProxy$HintRunnable.run(StorageProxy.java:1679) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6335) Hints broken for nodes that change broadcast address
[ https://issues.apache.org/jira/browse/CASSANDRA-6335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820415#comment-13820415 ] Rick Branson commented on CASSANDRA-6335: - It looks like it stalls the hint delivery thread pool indefinitely. Restarting the node frees it up and it starts delivering hints again. I'm rolling this cluster to workaround the issue. > Hints broken for nodes that change broadcast address > > > Key: CASSANDRA-6335 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6335 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Rick Branson > > When a node changes it's broadcast address, the transition process works > properly, but hints that are destined for it can't be delivered because of > the address change. It produces an exception: > java.lang.AssertionError: Missing host ID for 10.1.60.22 > at > org.apache.cassandra.service.StorageProxy.writeHintForMutation(StorageProxy.java:598) > at > org.apache.cassandra.service.StorageProxy$5.runMayThrow(StorageProxy.java:567) > at > org.apache.cassandra.service.StorageProxy$HintRunnable.run(StorageProxy.java:1679) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (CASSANDRA-6321) Unit Tests Failing in trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-6321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-6321. - Resolution: Fixed Turns out we were trying to load the user types a bit too early with triggered different races and circular dependencies. Took the liberty to ninja-fix with commit c20dd29923ea9e99122d75cb8f3b1d91e199fc15. > Unit Tests Failing in trunk > --- > > Key: CASSANDRA-6321 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6321 > Project: Cassandra > Issue Type: Test > Components: Tests > Environment: Debian Wheezy, Oracle Java 1.7.0_25 > (buildbot.datastax.com VM) >Reporter: Michael Shuler >Priority: Minor > > Many unit tests are failing in trunk [0] due to a timeout, but there are > relatively few in the cassandra-2.0 branch [1]. > I bisected the first test failure, CacheProviderTest, to commit: > {code} > ((6c9efb0...)|BISECTING)mshuler@mana:~/DataStax/repos/cassandra$ git bisect > good > a552b305f3d1b17e394744b18efd7f40599f3c2e is the first bad commit > commit a552b305f3d1b17e394744b18efd7f40599f3c2e > Author: Sylvain Lebresne > Date: Thu Sep 19 09:20:13 2013 +0200 > Add user-defined types to CQL3 > > patch by slebresne; reviewed by iamaleskey for CASSANDRA-5590 > :100644 100644 e55c8f79d66e91e896dc5ef1f0b7567b03e3910b > 4231b0b1a42a2d243fd6bcd9367ab7299f6ce680 M CHANGES.txt > :04 04 b838b9b052f19eafe2f927188f8c46a48feae01e > c44978f3a5e2c69333fb7036a32b5f8a81f5d28a M src > {code} > It is possible that the large number of subsequent failures may be due to > some of the substantial changes in the addition of user types (? that's a > guess). I will see if I can bisect and identify some other individual test > failures, but wanted to get this general ticket started. > Example trunk unit test: > [0] > http://buildbot.datastax.com:8020/builders/trunk/builds/2531/steps/shell/logs/stdio > Example 2.0 unit test: > [1] > http://buildbot.datastax.com:8020/builders/cassandra-2.0/builds/340/steps/shell/logs/stdio > Thanks! > Michael Shuler -- This message was sent by Atlassian JIRA (v6.1#6144)
git commit: Load user types at the right place
Updated Branches: refs/heads/trunk ed1fabb39 -> c20dd2992 Load user types at the right place Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c20dd299 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c20dd299 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c20dd299 Branch: refs/heads/trunk Commit: c20dd29923ea9e99122d75cb8f3b1d91e199fc15 Parents: ed1fabb Author: Sylvain Lebresne Authored: Tue Nov 12 20:28:23 2013 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 12 20:46:02 2013 +0100 -- src/java/org/apache/cassandra/config/DatabaseDescriptor.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c20dd299/src/java/org/apache/cassandra/config/DatabaseDescriptor.java -- diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java index 6abbd0c..d412032 100644 --- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java +++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java @@ -468,7 +468,6 @@ public class DatabaseDescriptor assert systemKeyspaces.size() == Schema.systemKeyspaceNames.size(); for (KSMetaData ksmd : systemKeyspaces) Schema.instance.load(ksmd); -Schema.instance.loadUserTypes(); /* Load the seeds for node contact points */ if (conf.seed_provider == null) @@ -502,6 +501,8 @@ public class DatabaseDescriptor /** load keyspace (keyspace) definitions, but do not initialize the keyspace instances. */ public static void loadSchemas() { +Schema.instance.loadUserTypes(); + ColumnFamilyStore schemaCFS = SystemKeyspace.schemaCFS(SystemKeyspace.SCHEMA_KEYSPACES_CF); // if keyspace with definitions is empty try loading the old way
[jira] [Commented] (CASSANDRA-6275) 2.0.x leaks file handles
[ https://issues.apache.org/jira/browse/CASSANDRA-6275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820376#comment-13820376 ] Marcus Eriksson commented on CASSANDRA-6275: been looking at this a bit and can't really reproduce, suspected CASSANDRA-5228 - but that seems to work (or, found a bug, but unrelated to this, CASSANDRA-6337) slog.gz looks a bit like what [~mkjellman] reported in CASSANDRA-5241 (looping flushing of system tables) but that was resolved a long time ago. does anyone have a way to reproduce? [~ash2k] would it be possible to post your load test? > 2.0.x leaks file handles > > > Key: CASSANDRA-6275 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6275 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: java version "1.7.0_25" > Java(TM) SE Runtime Environment (build 1.7.0_25-b15) > Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode) > Linux cassandra-test1 2.6.32-279.el6.x86_64 #1 SMP Thu Jun 21 15:00:18 EDT > 2012 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Mikhail Mazursky >Assignee: Marcus Eriksson > Attachments: cassandra_jstack.txt, leak.log, slog.gz > > > Looks like C* is leaking file descriptors when doing lots of CAS operations. > {noformat} > $ sudo cat /proc/15455/limits > Limit Soft Limit Hard Limit Units > Max cpu time unlimitedunlimitedseconds > Max file size unlimitedunlimitedbytes > Max data size unlimitedunlimitedbytes > Max stack size10485760 unlimitedbytes > Max core file size00bytes > Max resident set unlimitedunlimitedbytes > Max processes 1024 unlimitedprocesses > Max open files4096 4096 files > Max locked memory unlimitedunlimitedbytes > Max address space unlimitedunlimitedbytes > Max file locksunlimitedunlimitedlocks > Max pending signals 1463314633signals > Max msgqueue size 819200 819200 bytes > Max nice priority 00 > Max realtime priority 00 > Max realtime timeout unlimitedunlimitedus > {noformat} > Looks like the problem is not in limits. > Before load test: > {noformat} > cassandra-test0 ~]$ lsof -n | grep java | wc -l > 166 > cassandra-test1 ~]$ lsof -n | grep java | wc -l > 164 > cassandra-test2 ~]$ lsof -n | grep java | wc -l > 180 > {noformat} > After load test: > {noformat} > cassandra-test0 ~]$ lsof -n | grep java | wc -l > 967 > cassandra-test1 ~]$ lsof -n | grep java | wc -l > 1766 > cassandra-test2 ~]$ lsof -n | grep java | wc -l > 2578 > {noformat} > Most opened files have names like: > {noformat} > java 16890 cassandra 1636r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1637r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1638r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1639r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1640r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1641r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1642r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1643r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1644r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1645r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1646r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-pa
[jira] [Updated] (CASSANDRA-6337) Set minTimestamp correctly to be able to drop expired sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-6337: --- Attachment: 0001-Set-minTimestamp-correctly.patch > Set minTimestamp correctly to be able to drop expired sstables > -- > > Key: CASSANDRA-6337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6337 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 2.0.3 > > Attachments: 0001-Set-minTimestamp-correctly.patch > > > When calculating which sstables we can drop we set minTimestamp to > Integer.MAX_VALUE, this is wrong since minTimestamp is a long, and in most > cases minTimestamp on sstables is larger than Integer.MAX_VALUE. > We should set it to Long.MAX_VALUE, patch does that. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CASSANDRA-6337) Set minTimestamp correctly to be able to drop expired sstables
Marcus Eriksson created CASSANDRA-6337: -- Summary: Set minTimestamp correctly to be able to drop expired sstables Key: CASSANDRA-6337 URL: https://issues.apache.org/jira/browse/CASSANDRA-6337 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Priority: Minor Fix For: 2.0.3 When calculating which sstables we can drop we set minTimestamp to Integer.MAX_VALUE, this is wrong since minTimestamp is a long, and in most cases minTimestamp on sstables is larger than Integer.MAX_VALUE. We should set it to Long.MAX_VALUE, patch does that. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (CASSANDRA-6334) Option to not listen on Thrift/CQL if cluster is
[ https://issues.apache.org/jira/browse/CASSANDRA-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams resolved CASSANDRA-6334. - Resolution: Duplicate > Option to not listen on Thrift/CQL if cluster is - > > Key: CASSANDRA-6334 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6334 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Rick Branson >Priority: Minor > > Sometimes nodes get isolated because of bugs or misconfiguration on either > the server or the client side. A useful configuration option found in other > systems is to not listen for client requests until the cluster reaches a > minimum number of nodes. This would prevent "perfectly valid" reads & writes > from taking place on these isolated nodes. What do we think about this? -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6335) Hints broken for nodes that change broadcast address
[ https://issues.apache.org/jira/browse/CASSANDRA-6335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820364#comment-13820364 ] Brandon Williams commented on CASSANDRA-6335: - Hmm, I think CASSANDRA-6256 may be related here, since the hints are keyed by uuid, not ip. But that one has me stumped, since we set the host id before even starting the gossiper. :( > Hints broken for nodes that change broadcast address > > > Key: CASSANDRA-6335 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6335 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Rick Branson > > When a node changes it's broadcast address, the transition process works > properly, but hints that are destined for it can't be delivered because of > the address change. It produces an exception: > java.lang.AssertionError: Missing host ID for 10.1.60.22 > at > org.apache.cassandra.service.StorageProxy.writeHintForMutation(StorageProxy.java:598) > at > org.apache.cassandra.service.StorageProxy$5.runMayThrow(StorageProxy.java:567) > at > org.apache.cassandra.service.StorageProxy$HintRunnable.run(StorageProxy.java:1679) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CASSANDRA-6336) RejectedExecutionException thrown in Cassandra log
Jinder Aujla created CASSANDRA-6336: --- Summary: RejectedExecutionException thrown in Cassandra log Key: CASSANDRA-6336 URL: https://issues.apache.org/jira/browse/CASSANDRA-6336 Project: Cassandra Issue Type: Bug Components: Core Environment: Linux Centos 64 bit, Sun JDK 1.7 Reporter: Jinder Aujla Fix For: 2.0.2 java.util.concurrent.RejectedExecutionException: Worker has already been shutdown at org.jboss.netty.channel.socket.nio.AbstractNioSelector.registerTask(AbstractNioSelector.java:115) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:73) at org.jboss.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:57) at org.jboss.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36) at org.jboss.netty.channel.socket.nio.AbstractNioChannelSink.execute(AbstractNioChannelSink.java:34) at org.jboss.netty.channel.Channels.fireExceptionCaughtLater(Channels.java:496) at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:46) at org.jboss.netty.channel.Channels.write(Channels.java:725) at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:71) at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59) at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:60) at org.jboss.netty.channel.Channels.write(Channels.java:725) at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:71) at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59) at org.jboss.netty.handler.execution.ExecutionHandler.handleDownstream(ExecutionHandler.java:186) at org.jboss.netty.channel.Channels.write(Channels.java:704) at org.jboss.netty.channel.Channels.write(Channels.java:671) at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:248) at org.apache.cassandra.transport.Message$Dispatcher.messageReceived(Message.java:311) at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6322) Classcast Exception thrown when under load
[ https://issues.apache.org/jira/browse/CASSANDRA-6322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820329#comment-13820329 ] Jinder Aujla commented on CASSANDRA-6322: - Great thanks.. any ideas when you'll be cutting 2.0.3? > Classcast Exception thrown when under load > -- > > Key: CASSANDRA-6322 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6322 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Linux Centos 64 bit, Sun JDK 1.7 >Reporter: Jinder Aujla >Assignee: Jonathan Ellis > Fix For: 2.0.3 > > Attachments: 6322.txt > > > Saw this in the logs when running a load test: > ERROR [EXPIRING-MAP-REAPER:1] 2013-11-08 21:52:56,389 CassandraDaemon.java > (line 187) Exception in thread Thread[EXPIRING-MAP-REAPER:1,5,main] > java.lang.ClassCastException: org.apache.cassandra.db.CounterMutation cannot > be cast to org.apache.cassandra.db.RowMutation > at > org.apache.cassandra.net.MessagingService$5.apply(MessagingService.java:350) > at > org.apache.cassandra.net.MessagingService$5.apply(MessagingService.java:340) > at org.apache.cassandra.utils.ExpiringMap$1.run(ExpiringMap.java:97) > at > org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:75) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:722) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CASSANDRA-6335) Hints broken for nodes that change broadcast address
Rick Branson created CASSANDRA-6335: --- Summary: Hints broken for nodes that change broadcast address Key: CASSANDRA-6335 URL: https://issues.apache.org/jira/browse/CASSANDRA-6335 Project: Cassandra Issue Type: Bug Components: Core Reporter: Rick Branson When a node changes it's broadcast address, the transition process works properly, but hints that are destined for it can't be delivered because of the address change. It produces an exception: java.lang.AssertionError: Missing host ID for 10.1.60.22 at org.apache.cassandra.service.StorageProxy.writeHintForMutation(StorageProxy.java:598) at org.apache.cassandra.service.StorageProxy$5.runMayThrow(StorageProxy.java:567) at org.apache.cassandra.service.StorageProxy$HintRunnable.run(StorageProxy.java:1679) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) -- This message was sent by Atlassian JIRA (v6.1#6144)
git commit: Fix build
Updated Branches: refs/heads/cassandra-2.0 acea7160e -> 7fb2d24fa Fix build Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7fb2d24f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7fb2d24f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7fb2d24f Branch: refs/heads/cassandra-2.0 Commit: 7fb2d24fa35d01eab2ff6e5ebaa6885c795a80b8 Parents: acea716 Author: Sylvain Lebresne Authored: Tue Nov 12 19:25:26 2013 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 12 19:25:26 2013 +0100 -- src/java/org/apache/cassandra/tools/SSTableImport.java | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7fb2d24f/src/java/org/apache/cassandra/tools/SSTableImport.java -- diff --git a/src/java/org/apache/cassandra/tools/SSTableImport.java b/src/java/org/apache/cassandra/tools/SSTableImport.java index 0e8172b..7b65d26 100644 --- a/src/java/org/apache/cassandra/tools/SSTableImport.java +++ b/src/java/org/apache/cassandra/tools/SSTableImport.java @@ -44,9 +44,9 @@ import org.apache.cassandra.config.Schema; import org.apache.cassandra.db.marshal.AbstractType; import org.apache.cassandra.db.marshal.BytesType; import org.apache.cassandra.db.marshal.CompositeType; -import org.apache.cassandra.db.marshal.MarshalException; import org.apache.cassandra.dht.IPartitioner; import org.apache.cassandra.io.sstable.SSTableWriter; +import org.apache.cassandra.serializers.MarshalException; import org.apache.cassandra.utils.ByteBufferUtil; import org.codehaus.jackson.JsonFactory; import org.codehaus.jackson.JsonParser; @@ -158,8 +158,7 @@ public class SSTableImport } else if (isRangeTombstone()) { -AbstractType type = CompositeType.getInstance(Arrays.asList(new AbstractType[]{meta.getColumnDefinitionComparator(0)})); -value = type.fromString((String)fields.get(1)); +value = comparator.fromString((String)fields.get(1)); } else {
[1/2] git commit: Fix build
Updated Branches: refs/heads/trunk bfd0144a1 -> ed1fabb39 Fix build Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7fb2d24f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7fb2d24f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7fb2d24f Branch: refs/heads/trunk Commit: 7fb2d24fa35d01eab2ff6e5ebaa6885c795a80b8 Parents: acea716 Author: Sylvain Lebresne Authored: Tue Nov 12 19:25:26 2013 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 12 19:25:26 2013 +0100 -- src/java/org/apache/cassandra/tools/SSTableImport.java | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7fb2d24f/src/java/org/apache/cassandra/tools/SSTableImport.java -- diff --git a/src/java/org/apache/cassandra/tools/SSTableImport.java b/src/java/org/apache/cassandra/tools/SSTableImport.java index 0e8172b..7b65d26 100644 --- a/src/java/org/apache/cassandra/tools/SSTableImport.java +++ b/src/java/org/apache/cassandra/tools/SSTableImport.java @@ -44,9 +44,9 @@ import org.apache.cassandra.config.Schema; import org.apache.cassandra.db.marshal.AbstractType; import org.apache.cassandra.db.marshal.BytesType; import org.apache.cassandra.db.marshal.CompositeType; -import org.apache.cassandra.db.marshal.MarshalException; import org.apache.cassandra.dht.IPartitioner; import org.apache.cassandra.io.sstable.SSTableWriter; +import org.apache.cassandra.serializers.MarshalException; import org.apache.cassandra.utils.ByteBufferUtil; import org.codehaus.jackson.JsonFactory; import org.codehaus.jackson.JsonParser; @@ -158,8 +158,7 @@ public class SSTableImport } else if (isRangeTombstone()) { -AbstractType type = CompositeType.getInstance(Arrays.asList(new AbstractType[]{meta.getColumnDefinitionComparator(0)})); -value = type.fromString((String)fields.get(1)); +value = comparator.fromString((String)fields.get(1)); } else {
[2/2] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed1fabb3 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed1fabb3 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed1fabb3 Branch: refs/heads/trunk Commit: ed1fabb395d02849e06e7b54713e9d099571b4df Parents: bfd0144 7fb2d24 Author: Sylvain Lebresne Authored: Tue Nov 12 19:25:38 2013 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 12 19:25:38 2013 +0100 -- --
[5/5] git commit: Merge branch 'cassandra-2.0' into trunk
Merge branch 'cassandra-2.0' into trunk Conflicts: src/java/org/apache/cassandra/tools/SSTableImport.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bfd0144a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bfd0144a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bfd0144a Branch: refs/heads/trunk Commit: bfd0144a189c4326c987d91cc20e66e47236dc1a Parents: 9a59d25 acea716 Author: Sylvain Lebresne Authored: Tue Nov 12 19:23:12 2013 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 12 19:23:12 2013 +0100 -- CHANGES.txt| 3 +++ .../cassandra/cql3/statements/SelectStatement.java | 2 +- .../apache/cassandra/db/marshal/CompositeType.java | 6 +++--- .../org/apache/cassandra/tools/SSTableImport.java | 17 ++--- 4 files changed, 21 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/bfd0144a/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/bfd0144a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/bfd0144a/src/java/org/apache/cassandra/db/marshal/CompositeType.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/bfd0144a/src/java/org/apache/cassandra/tools/SSTableImport.java -- diff --cc src/java/org/apache/cassandra/tools/SSTableImport.java index 2e9c4ed,0e8172b..3dcb851 --- a/src/java/org/apache/cassandra/tools/SSTableImport.java +++ b/src/java/org/apache/cassandra/tools/SSTableImport.java @@@ -44,8 -44,9 +44,9 @@@ import org.apache.cassandra.config.Sche import org.apache.cassandra.db.marshal.AbstractType; import org.apache.cassandra.db.marshal.BytesType; import org.apache.cassandra.db.marshal.CompositeType; -import org.apache.cassandra.db.marshal.MarshalException; import org.apache.cassandra.dht.IPartitioner; import org.apache.cassandra.io.sstable.SSTableWriter; ++import org.apache.cassandra.serializers.MarshalException; import org.apache.cassandra.utils.ByteBufferUtil; import org.codehaus.jackson.JsonFactory; import org.codehaus.jackson.JsonParser; @@@ -151,8 -152,19 +152,18 @@@ public class SSTableImpor } } - value = isDeleted() ? ByteBufferUtil.hexToBytes((String) fields.get(1)) - : stringAsType((String) fields.get(1), meta.getValueValidator(meta.getColumnDefinitionFromCellName(name))); + if (isDeleted()) + { + value = ByteBufferUtil.hexToBytes((String) fields.get(1)); + } + else if (isRangeTombstone()) + { -AbstractType type = CompositeType.getInstance(Arrays.asList(new AbstractType[]{meta.getColumnDefinitionComparator(0)})); -value = type.fromString((String)fields.get(1)); ++value = comparator.fromString((String)fields.get(1)); + } + else + { -value = stringAsType((String) fields.get(1), meta.getValueValidator(meta.getColumnDefinitionFromColumnName(name))); ++value = stringAsType((String) fields.get(1), meta.getValueValidator(meta.getColumnDefinitionFromCellName(name))); + } } }
[3/5] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0 Conflicts: src/java/org/apache/cassandra/cql3/statements/SelectStatement.java src/java/org/apache/cassandra/tools/SSTableImport.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c6af75a1 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c6af75a1 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c6af75a1 Branch: refs/heads/trunk Commit: c6af75a19b3d19fdb3aa6040b655793b58b0db97 Parents: 2cc4670 3cb5854 Author: Sylvain Lebresne Authored: Tue Nov 12 19:13:00 2013 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 12 19:13:00 2013 +0100 -- CHANGES.txt | 2 ++ .../cql3/statements/SelectStatement.java | 2 +- .../org/apache/cassandra/tools/SSTableImport.java | 18 +++--- 3 files changed, 18 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c6af75a1/CHANGES.txt -- diff --cc CHANGES.txt index cc5fb9e,7abf5d8..424a9f4 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -34,42 -14,11 +34,44 @@@ Merged from 1.2 * Fix potential socket leak in connectionpool creation (CASSANDRA-6308) * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy (CASSANDRA-6238) * cqlsh: handle 'null' as session duration (CASSANDRA-6317) + * Fix json2sstable handling of range tombstones (CASSANDRA-6316) + * Fix missing one row in reverse query (CASSANDRA-6330) -1.2.11 +2.0.2 + * Update FailureDetector to use nanontime (CASSANDRA-4925) + * Fix FileCacheService regressions (CASSANDRA-6149) + * Never return WriteTimeout for CL.ANY (CASSANDRA-6032) + * Fix race conditions in bulk loader (CASSANDRA-6129) + * Add configurable metrics reporting (CASSANDRA-4430) + * drop queries exceeding a configurable number of tombstones (CASSANDRA-6117) + * Track and persist sstable read activity (CASSANDRA-5515) + * Fixes for speculative retry (CASSANDRA-5932, CASSANDRA-6194) + * Improve memory usage of metadata min/max column names (CASSANDRA-6077) + * Fix thrift validation refusing row markers on CQL3 tables (CASSANDRA-6081) + * Fix insertion of collections with CAS (CASSANDRA-6069) + * Correctly send metadata on SELECT COUNT (CASSANDRA-6080) + * Track clients' remote addresses in ClientState (CASSANDRA-6070) + * Create snapshot dir if it does not exist when migrating + leveled manifest (CASSANDRA-6093) + * make sequential nodetool repair the default (CASSANDRA-5950) + * Add more hooks for compaction strategy implementations (CASSANDRA-6111) + * Fix potential NPE on composite 2ndary indexes (CASSANDRA-6098) + * Delete can potentially be skipped in batch (CASSANDRA-6115) + * Allow alter keyspace on system_traces (CASSANDRA-6016) + * Disallow empty column names in cql (CASSANDRA-6136) + * Use Java7 file-handling APIs and fix file moving on Windows (CASSANDRA-5383) + * Save compaction history to system keyspace (CASSANDRA-5078) + * Fix NPE if StorageService.getOperationMode() is executed before full startup (CASSANDRA-6166) + * CQL3: support pre-epoch longs for TimestampType (CASSANDRA-6212) + * Add reloadtriggers command to nodetool (CASSANDRA-4949) + * cqlsh: ignore empty 'value alias' in DESCRIBE (CASSANDRA-6139) + * Fix sstable loader (CASSANDRA-6205) + * Reject bootstrapping if the node already exists in gossip (CASSANDRA-5571) + * Fix NPE while loading paxos state (CASSANDRA-6211) + * cqlsh: add SHOW SESSION command (CASSANDRA-6228) +Merged from 1.2: + * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114) * Add a warning for small LCS sstable size (CASSANDRA-6191) * Add ability to list specific KS/CF combinations in nodetool cfstats (CASSANDRA-4191) * Mark CF clean if a mutation raced the drop and got it marked dirty (CASSANDRA-5946) http://git-wip-us.apache.org/repos/asf/cassandra/blob/c6af75a1/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java -- diff --cc src/java/org/apache/cassandra/cql3/statements/SelectStatement.java index 14400d5,c1c88ba..4b9a334 --- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java @@@ -405,35 -374,17 +405,35 @@@ public class SelectStatement implement } } -private int getLimit() +private int getLimit(List variables) throws InvalidRequestException { -// Internally, we don't support exclusive bounds for COMPACT slices. Instead, when we know we have an exlcusive -// slice on a COMPACT table, we query one more element (to make sure we don't return less results than
[2/5] git commit: Fixing missing one row during reverse query on compact tables
Fixing missing one row during reverse query on compact tables patch by slebresne; reviewed by iamaleksey for CASSANDRA-6330 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3cb5854e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3cb5854e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3cb5854e Branch: refs/heads/trunk Commit: 3cb5854e9271ffc4e338841a49dd0b11d18e8c4f Parents: 159744f Author: Sylvain Lebresne Authored: Tue Nov 12 19:09:46 2013 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 12 19:09:46 2013 +0100 -- CHANGES.txt | 1 + .../apache/cassandra/cql3/statements/SelectStatement.java | 10 +++--- 2 files changed, 8 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3cb5854e/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index e0a2320..7abf5d8 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -15,6 +15,7 @@ * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy (CASSANDRA-6238) * cqlsh: handle 'null' as session duration (CASSANDRA-6317) * Fix json2sstable handling of range tombstones (CASSANDRA-6316) + * Fix missing one row in reverse query (CASSANDRA-6330) 1.2.11 http://git-wip-us.apache.org/repos/asf/cassandra/blob/3cb5854e/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java -- diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java index 44a1e64..c1c88ba 100644 --- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java @@ -376,9 +376,13 @@ public class SelectStatement implements CQLStatement private int getLimit() { -// Internally, we don't support exclusive bounds for slices. Instead, -// we query one more element if necessary and exclude -return sliceRestriction != null && !sliceRestriction.isInclusive(Bound.START) && parameters.limit != Integer.MAX_VALUE +// Internally, we don't support exclusive bounds for COMPACT slices. Instead, when we know we have an exlcusive +// slice on a COMPACT table, we query one more element (to make sure we don't return less results than asked post-exclusion) +// and exclude the post-query. Note that while we might excluse both the START and END bound, there is no reason to +// ask for limit + 2 since if we exlude both bound from the result it means we can't have missed non-fetched results. +return (sliceRestriction != null +&& parameters.limit != Integer.MAX_VALUE +&& (!sliceRestriction.isInclusive(Bound.START) || !sliceRestriction.isInclusive(Bound.END))) ? parameters.limit + 1 : parameters.limit; }
[1/5] git commit: Fix json2sstable handling of range tombstones
Updated Branches: refs/heads/trunk 9a59d250a -> bfd0144a1 Fix json2sstable handling of range tombstones patch by Lyuben Todorov; reviewed by Aleksey Yeschenko for CASSANDRA-6316 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/159744f2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/159744f2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/159744f2 Branch: refs/heads/trunk Commit: 159744f244125af84f4e819f45d2e2ebef2b369a Parents: 512ea5a Author: Aleksey Yeschenko Authored: Tue Nov 12 20:09:02 2013 +0300 Committer: Aleksey Yeschenko Committed: Tue Nov 12 20:09:02 2013 +0300 -- CHANGES.txt| 1 + .../org/apache/cassandra/tools/SSTableImport.java | 17 +++-- 2 files changed, 16 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/159744f2/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index bc0d65b..e0a2320 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -14,6 +14,7 @@ * Fix potential socket leak in connectionpool creation (CASSANDRA-6308) * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy (CASSANDRA-6238) * cqlsh: handle 'null' as session duration (CASSANDRA-6317) + * Fix json2sstable handling of range tombstones (CASSANDRA-6316) 1.2.11 http://git-wip-us.apache.org/repos/asf/cassandra/blob/159744f2/src/java/org/apache/cassandra/tools/SSTableImport.java -- diff --git a/src/java/org/apache/cassandra/tools/SSTableImport.java b/src/java/org/apache/cassandra/tools/SSTableImport.java index 68d4397..80d1aad 100644 --- a/src/java/org/apache/cassandra/tools/SSTableImport.java +++ b/src/java/org/apache/cassandra/tools/SSTableImport.java @@ -22,6 +22,7 @@ import static org.apache.cassandra.utils.ByteBufferUtil.hexToBytes; import java.io.File; import java.io.IOException; import java.nio.ByteBuffer; +import java.util.Arrays; import java.util.List; import java.util.Map; import java.util.SortedMap; @@ -50,6 +51,7 @@ import org.apache.cassandra.db.SuperColumn; import org.apache.cassandra.db.filter.QueryPath; import org.apache.cassandra.db.marshal.AbstractType; import org.apache.cassandra.db.marshal.BytesType; +import org.apache.cassandra.db.marshal.CompositeType; import org.apache.cassandra.db.marshal.MarshalException; import org.apache.cassandra.dht.IPartitioner; import org.apache.cassandra.io.sstable.SSTableWriter; @@ -155,8 +157,19 @@ public class SSTableImport } } -value = isDeleted() ? ByteBufferUtil.hexToBytes((String) fields.get(1)) -: stringAsType((String) fields.get(1), meta.getValueValidator(meta.getColumnDefinitionFromColumnName(name))); +if (isDeleted()) +{ +value = ByteBufferUtil.hexToBytes((String) fields.get(1)); +} +else if (isRangeTombstone()) +{ +AbstractType type = CompositeType.getInstance(Arrays.asList(new AbstractType[]{meta.getColumnDefinitionComparator(0)})); +value = type.fromString((String)fields.get(1)); +} +else +{ +value = stringAsType((String) fields.get(1), meta.getValueValidator(meta.getColumnDefinitionFromColumnName(name))); +} } }
[4/5] git commit: Fix missing results on query with IN clauses
Fix missing results on query with IN clauses patch by slebresne; reviewed by iamaleksey for CASSANDRA-6327 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/acea7160 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/acea7160 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/acea7160 Branch: refs/heads/trunk Commit: acea7160ed8eb1779b9830821ea5efe89040769c Parents: c6af75a Author: Sylvain Lebresne Authored: Tue Nov 12 19:14:28 2013 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 12 19:14:28 2013 +0100 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/marshal/CompositeType.java | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/acea7160/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 424a9f4..69b22a3 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -18,6 +18,7 @@ * Correctly throw error when frame too large (CASSANDRA-5981) * Fix serialization bug in PagedRange with 2ndary indexes (CASSANDRA-6299) * Fix CQL3 table validation in Thrift (CASSANDRA-6140) + * Fix bug missing results with IN clauses (CASSANDRA-6327) Merged from 1.2: * add non-jamm path for cached statements (CASSANDRA-6293) * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114) http://git-wip-us.apache.org/repos/asf/cassandra/blob/acea7160/src/java/org/apache/cassandra/db/marshal/CompositeType.java -- diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java b/src/java/org/apache/cassandra/db/marshal/CompositeType.java index 3110049..c9e2ab9 100644 --- a/src/java/org/apache/cassandra/db/marshal/CompositeType.java +++ b/src/java/org/apache/cassandra/db/marshal/CompositeType.java @@ -229,11 +229,11 @@ public class CompositeType extends AbstractCompositeType AbstractType t = types.get(i); ByteBuffer s = i < start.length ? start[i] : ByteBufferUtil.EMPTY_BYTE_BUFFER; ByteBuffer f = i < finish.length ? finish[i] : ByteBufferUtil.EMPTY_BYTE_BUFFER; -if (!t.intersects(minColumnNames.get(i), maxColumnNames.get(i), s, f)) -return false; +if (t.intersects(minColumnNames.get(i), maxColumnNames.get(i), s, f)) +return true; } } -return true; +return false; } private static class StaticParsedComparator implements ParsedComparator
git commit: Fix missing results on query with IN clauses
Updated Branches: refs/heads/cassandra-2.0 c6af75a19 -> acea7160e Fix missing results on query with IN clauses patch by slebresne; reviewed by iamaleksey for CASSANDRA-6327 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/acea7160 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/acea7160 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/acea7160 Branch: refs/heads/cassandra-2.0 Commit: acea7160ed8eb1779b9830821ea5efe89040769c Parents: c6af75a Author: Sylvain Lebresne Authored: Tue Nov 12 19:14:28 2013 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 12 19:14:28 2013 +0100 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/db/marshal/CompositeType.java | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/acea7160/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 424a9f4..69b22a3 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -18,6 +18,7 @@ * Correctly throw error when frame too large (CASSANDRA-5981) * Fix serialization bug in PagedRange with 2ndary indexes (CASSANDRA-6299) * Fix CQL3 table validation in Thrift (CASSANDRA-6140) + * Fix bug missing results with IN clauses (CASSANDRA-6327) Merged from 1.2: * add non-jamm path for cached statements (CASSANDRA-6293) * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114) http://git-wip-us.apache.org/repos/asf/cassandra/blob/acea7160/src/java/org/apache/cassandra/db/marshal/CompositeType.java -- diff --git a/src/java/org/apache/cassandra/db/marshal/CompositeType.java b/src/java/org/apache/cassandra/db/marshal/CompositeType.java index 3110049..c9e2ab9 100644 --- a/src/java/org/apache/cassandra/db/marshal/CompositeType.java +++ b/src/java/org/apache/cassandra/db/marshal/CompositeType.java @@ -229,11 +229,11 @@ public class CompositeType extends AbstractCompositeType AbstractType t = types.get(i); ByteBuffer s = i < start.length ? start[i] : ByteBufferUtil.EMPTY_BYTE_BUFFER; ByteBuffer f = i < finish.length ? finish[i] : ByteBufferUtil.EMPTY_BYTE_BUFFER; -if (!t.intersects(minColumnNames.get(i), maxColumnNames.get(i), s, f)) -return false; +if (t.intersects(minColumnNames.get(i), maxColumnNames.get(i), s, f)) +return true; } } -return true; +return false; } private static class StaticParsedComparator implements ParsedComparator
[2/3] git commit: Fixing missing one row during reverse query on compact tables
Fixing missing one row during reverse query on compact tables patch by slebresne; reviewed by iamaleksey for CASSANDRA-6330 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3cb5854e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3cb5854e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3cb5854e Branch: refs/heads/cassandra-2.0 Commit: 3cb5854e9271ffc4e338841a49dd0b11d18e8c4f Parents: 159744f Author: Sylvain Lebresne Authored: Tue Nov 12 19:09:46 2013 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 12 19:09:46 2013 +0100 -- CHANGES.txt | 1 + .../apache/cassandra/cql3/statements/SelectStatement.java | 10 +++--- 2 files changed, 8 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3cb5854e/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index e0a2320..7abf5d8 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -15,6 +15,7 @@ * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy (CASSANDRA-6238) * cqlsh: handle 'null' as session duration (CASSANDRA-6317) * Fix json2sstable handling of range tombstones (CASSANDRA-6316) + * Fix missing one row in reverse query (CASSANDRA-6330) 1.2.11 http://git-wip-us.apache.org/repos/asf/cassandra/blob/3cb5854e/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java -- diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java index 44a1e64..c1c88ba 100644 --- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java @@ -376,9 +376,13 @@ public class SelectStatement implements CQLStatement private int getLimit() { -// Internally, we don't support exclusive bounds for slices. Instead, -// we query one more element if necessary and exclude -return sliceRestriction != null && !sliceRestriction.isInclusive(Bound.START) && parameters.limit != Integer.MAX_VALUE +// Internally, we don't support exclusive bounds for COMPACT slices. Instead, when we know we have an exlcusive +// slice on a COMPACT table, we query one more element (to make sure we don't return less results than asked post-exclusion) +// and exclude the post-query. Note that while we might excluse both the START and END bound, there is no reason to +// ask for limit + 2 since if we exlude both bound from the result it means we can't have missed non-fetched results. +return (sliceRestriction != null +&& parameters.limit != Integer.MAX_VALUE +&& (!sliceRestriction.isInclusive(Bound.START) || !sliceRestriction.isInclusive(Bound.END))) ? parameters.limit + 1 : parameters.limit; }
[3/3] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0 Conflicts: src/java/org/apache/cassandra/cql3/statements/SelectStatement.java src/java/org/apache/cassandra/tools/SSTableImport.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c6af75a1 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c6af75a1 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c6af75a1 Branch: refs/heads/cassandra-2.0 Commit: c6af75a19b3d19fdb3aa6040b655793b58b0db97 Parents: 2cc4670 3cb5854 Author: Sylvain Lebresne Authored: Tue Nov 12 19:13:00 2013 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 12 19:13:00 2013 +0100 -- CHANGES.txt | 2 ++ .../cql3/statements/SelectStatement.java | 2 +- .../org/apache/cassandra/tools/SSTableImport.java | 18 +++--- 3 files changed, 18 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c6af75a1/CHANGES.txt -- diff --cc CHANGES.txt index cc5fb9e,7abf5d8..424a9f4 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -34,42 -14,11 +34,44 @@@ Merged from 1.2 * Fix potential socket leak in connectionpool creation (CASSANDRA-6308) * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy (CASSANDRA-6238) * cqlsh: handle 'null' as session duration (CASSANDRA-6317) + * Fix json2sstable handling of range tombstones (CASSANDRA-6316) + * Fix missing one row in reverse query (CASSANDRA-6330) -1.2.11 +2.0.2 + * Update FailureDetector to use nanontime (CASSANDRA-4925) + * Fix FileCacheService regressions (CASSANDRA-6149) + * Never return WriteTimeout for CL.ANY (CASSANDRA-6032) + * Fix race conditions in bulk loader (CASSANDRA-6129) + * Add configurable metrics reporting (CASSANDRA-4430) + * drop queries exceeding a configurable number of tombstones (CASSANDRA-6117) + * Track and persist sstable read activity (CASSANDRA-5515) + * Fixes for speculative retry (CASSANDRA-5932, CASSANDRA-6194) + * Improve memory usage of metadata min/max column names (CASSANDRA-6077) + * Fix thrift validation refusing row markers on CQL3 tables (CASSANDRA-6081) + * Fix insertion of collections with CAS (CASSANDRA-6069) + * Correctly send metadata on SELECT COUNT (CASSANDRA-6080) + * Track clients' remote addresses in ClientState (CASSANDRA-6070) + * Create snapshot dir if it does not exist when migrating + leveled manifest (CASSANDRA-6093) + * make sequential nodetool repair the default (CASSANDRA-5950) + * Add more hooks for compaction strategy implementations (CASSANDRA-6111) + * Fix potential NPE on composite 2ndary indexes (CASSANDRA-6098) + * Delete can potentially be skipped in batch (CASSANDRA-6115) + * Allow alter keyspace on system_traces (CASSANDRA-6016) + * Disallow empty column names in cql (CASSANDRA-6136) + * Use Java7 file-handling APIs and fix file moving on Windows (CASSANDRA-5383) + * Save compaction history to system keyspace (CASSANDRA-5078) + * Fix NPE if StorageService.getOperationMode() is executed before full startup (CASSANDRA-6166) + * CQL3: support pre-epoch longs for TimestampType (CASSANDRA-6212) + * Add reloadtriggers command to nodetool (CASSANDRA-4949) + * cqlsh: ignore empty 'value alias' in DESCRIBE (CASSANDRA-6139) + * Fix sstable loader (CASSANDRA-6205) + * Reject bootstrapping if the node already exists in gossip (CASSANDRA-5571) + * Fix NPE while loading paxos state (CASSANDRA-6211) + * cqlsh: add SHOW SESSION command (CASSANDRA-6228) +Merged from 1.2: + * (Hadoop) Require CFRR batchSize to be at least 2 (CASSANDRA-6114) * Add a warning for small LCS sstable size (CASSANDRA-6191) * Add ability to list specific KS/CF combinations in nodetool cfstats (CASSANDRA-4191) * Mark CF clean if a mutation raced the drop and got it marked dirty (CASSANDRA-5946) http://git-wip-us.apache.org/repos/asf/cassandra/blob/c6af75a1/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java -- diff --cc src/java/org/apache/cassandra/cql3/statements/SelectStatement.java index 14400d5,c1c88ba..4b9a334 --- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java @@@ -405,35 -374,17 +405,35 @@@ public class SelectStatement implement } } -private int getLimit() +private int getLimit(List variables) throws InvalidRequestException { -// Internally, we don't support exclusive bounds for COMPACT slices. Instead, when we know we have an exlcusive -// slice on a COMPACT table, we query one more element (to make sure we don't return less resul
[1/3] git commit: Fix json2sstable handling of range tombstones
Updated Branches: refs/heads/cassandra-2.0 2cc4670ac -> c6af75a19 Fix json2sstable handling of range tombstones patch by Lyuben Todorov; reviewed by Aleksey Yeschenko for CASSANDRA-6316 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/159744f2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/159744f2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/159744f2 Branch: refs/heads/cassandra-2.0 Commit: 159744f244125af84f4e819f45d2e2ebef2b369a Parents: 512ea5a Author: Aleksey Yeschenko Authored: Tue Nov 12 20:09:02 2013 +0300 Committer: Aleksey Yeschenko Committed: Tue Nov 12 20:09:02 2013 +0300 -- CHANGES.txt| 1 + .../org/apache/cassandra/tools/SSTableImport.java | 17 +++-- 2 files changed, 16 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/159744f2/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index bc0d65b..e0a2320 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -14,6 +14,7 @@ * Fix potential socket leak in connectionpool creation (CASSANDRA-6308) * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy (CASSANDRA-6238) * cqlsh: handle 'null' as session duration (CASSANDRA-6317) + * Fix json2sstable handling of range tombstones (CASSANDRA-6316) 1.2.11 http://git-wip-us.apache.org/repos/asf/cassandra/blob/159744f2/src/java/org/apache/cassandra/tools/SSTableImport.java -- diff --git a/src/java/org/apache/cassandra/tools/SSTableImport.java b/src/java/org/apache/cassandra/tools/SSTableImport.java index 68d4397..80d1aad 100644 --- a/src/java/org/apache/cassandra/tools/SSTableImport.java +++ b/src/java/org/apache/cassandra/tools/SSTableImport.java @@ -22,6 +22,7 @@ import static org.apache.cassandra.utils.ByteBufferUtil.hexToBytes; import java.io.File; import java.io.IOException; import java.nio.ByteBuffer; +import java.util.Arrays; import java.util.List; import java.util.Map; import java.util.SortedMap; @@ -50,6 +51,7 @@ import org.apache.cassandra.db.SuperColumn; import org.apache.cassandra.db.filter.QueryPath; import org.apache.cassandra.db.marshal.AbstractType; import org.apache.cassandra.db.marshal.BytesType; +import org.apache.cassandra.db.marshal.CompositeType; import org.apache.cassandra.db.marshal.MarshalException; import org.apache.cassandra.dht.IPartitioner; import org.apache.cassandra.io.sstable.SSTableWriter; @@ -155,8 +157,19 @@ public class SSTableImport } } -value = isDeleted() ? ByteBufferUtil.hexToBytes((String) fields.get(1)) -: stringAsType((String) fields.get(1), meta.getValueValidator(meta.getColumnDefinitionFromColumnName(name))); +if (isDeleted()) +{ +value = ByteBufferUtil.hexToBytes((String) fields.get(1)); +} +else if (isRangeTombstone()) +{ +AbstractType type = CompositeType.getInstance(Arrays.asList(new AbstractType[]{meta.getColumnDefinitionComparator(0)})); +value = type.fromString((String)fields.get(1)); +} +else +{ +value = stringAsType((String) fields.get(1), meta.getValueValidator(meta.getColumnDefinitionFromColumnName(name))); +} } }
git commit: Fixing missing one row during reverse query on compact tables
Updated Branches: refs/heads/cassandra-1.2 159744f24 -> 3cb5854e9 Fixing missing one row during reverse query on compact tables patch by slebresne; reviewed by iamaleksey for CASSANDRA-6330 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3cb5854e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3cb5854e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3cb5854e Branch: refs/heads/cassandra-1.2 Commit: 3cb5854e9271ffc4e338841a49dd0b11d18e8c4f Parents: 159744f Author: Sylvain Lebresne Authored: Tue Nov 12 19:09:46 2013 +0100 Committer: Sylvain Lebresne Committed: Tue Nov 12 19:09:46 2013 +0100 -- CHANGES.txt | 1 + .../apache/cassandra/cql3/statements/SelectStatement.java | 10 +++--- 2 files changed, 8 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3cb5854e/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index e0a2320..7abf5d8 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -15,6 +15,7 @@ * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy (CASSANDRA-6238) * cqlsh: handle 'null' as session duration (CASSANDRA-6317) * Fix json2sstable handling of range tombstones (CASSANDRA-6316) + * Fix missing one row in reverse query (CASSANDRA-6330) 1.2.11 http://git-wip-us.apache.org/repos/asf/cassandra/blob/3cb5854e/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java -- diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java index 44a1e64..c1c88ba 100644 --- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java @@ -376,9 +376,13 @@ public class SelectStatement implements CQLStatement private int getLimit() { -// Internally, we don't support exclusive bounds for slices. Instead, -// we query one more element if necessary and exclude -return sliceRestriction != null && !sliceRestriction.isInclusive(Bound.START) && parameters.limit != Integer.MAX_VALUE +// Internally, we don't support exclusive bounds for COMPACT slices. Instead, when we know we have an exlcusive +// slice on a COMPACT table, we query one more element (to make sure we don't return less results than asked post-exclusion) +// and exclude the post-query. Note that while we might excluse both the START and END bound, there is no reason to +// ask for limit + 2 since if we exlude both bound from the result it means we can't have missed non-fetched results. +return (sliceRestriction != null +&& parameters.limit != Integer.MAX_VALUE +&& (!sliceRestriction.isInclusive(Bound.START) || !sliceRestriction.isInclusive(Bound.END))) ? parameters.limit + 1 : parameters.limit; }
[jira] [Commented] (CASSANDRA-6325) AssertionError on startup reading saved Serializing row cache
[ https://issues.apache.org/jira/browse/CASSANDRA-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820280#comment-13820280 ] Mikhail Stepura commented on CASSANDRA-6325: +1 > AssertionError on startup reading saved Serializing row cache > - > > Key: CASSANDRA-6325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6325 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: upgrade from 1.2.9ish to 1.2.11ish >Reporter: Chris Burroughs >Assignee: Mikhail Stepura >Priority: Minor > Fix For: 1.2.12 > > Attachments: 6325-v2.txt, CASSANDRA-1.2-6325.patch > > > I don't see any reason what this could have to do with the upgrade, but don't > have a large enough non-prod cluster to just keep restarting on. Occurred on > roughly 2 out of 100 restarted nodes. > {noformat} > ERROR [main] 2013-11-08 14:40:13,535 CassandraDaemon.java (line 482) > Exception encountered during startup > java.lang.AssertionError > at > org.apache.cassandra.cache.SerializingCacheProvider$RowCacheSerializer.serialize(SerializingCacheProvider.java:41) > at > org.apache.cassandra.cache.SerializingCacheProvider$RowCacheSerializer.serialize(SerializingCacheProvider.java:37) > at > org.apache.cassandra.cache.SerializingCache.serialize(SerializingCache.java:118) > at > org.apache.cassandra.cache.SerializingCache.put(SerializingCache.java:176) > at > org.apache.cassandra.cache.InstrumentingCache.put(InstrumentingCache.java:44) > at > org.apache.cassandra.cache.AutoSavingCache.loadSaved(AutoSavingCache.java:156) > at > org.apache.cassandra.db.ColumnFamilyStore.initRowCache(ColumnFamilyStore.java:444) > at org.apache.cassandra.db.Table.open(Table.java:114) > at org.apache.cassandra.db.Table.open(Table.java:87) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:278) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465) > {noformat} > I have the files if there is any useful analysis that can be run. Looked > 'normal' to a cursory `less` inspection. > Possibly related: CASSANDRA-4463 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6316) json2sstable breaks on RangeTombstone
[ https://issues.apache.org/jira/browse/CASSANDRA-6316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820276#comment-13820276 ] Aleksey Yeschenko commented on CASSANDRA-6316: -- [~lyubent] Could you attach a patch for 2.0? Keep in mind that CASSANDRA-5435 exists. > json2sstable breaks on RangeTombstone > - > > Key: CASSANDRA-6316 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6316 > Project: Cassandra > Issue Type: Bug > Environment: 1.2.10 >Reporter: Matt Jurik >Assignee: Lyuben Todorov >Priority: Minor > Attachments: 6316.diff > > > It seems that sstable2json writes out json which json2sstable has trouble > reading. > {code} > java.lang.NumberFormatException: An hex string representing bytes must have > an even length > at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:52) > at > org.apache.cassandra.utils.ByteBufferUtil.hexToBytes(ByteBufferUtil.java:503) > at > org.apache.cassandra.tools.SSTableImport.stringAsType(SSTableImport.java:572) > at > org.apache.cassandra.tools.SSTableImport.access$000(SSTableImport.java:66) > at > org.apache.cassandra.tools.SSTableImport$JsonColumn.(SSTableImport.java:158) > at > org.apache.cassandra.tools.SSTableImport.addColumnsToCF(SSTableImport.java:229) > at > org.apache.cassandra.tools.SSTableImport.addToStandardCF(SSTableImport.java:212) > at > org.apache.cassandra.tools.SSTableImport.importUnsorted(SSTableImport.java:361) > at > org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:318) > at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:537) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120) > ERROR: An hex string representing bytes must have an even length > 12345:2:! > {code} -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (CASSANDRA-6334) Option to not listen on Thrift/CQL if cluster is
Rick Branson created CASSANDRA-6334: --- Summary: Option to not listen on Thrift/CQL if cluster is https://issues.apache.org/jira/browse/CASSANDRA-6334 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Rick Branson Priority: Minor Sometimes nodes get isolated because of bugs or misconfiguration on either the server or the client side. A useful configuration option found in other systems is to not listen for client requests until the cluster reaches a minimum number of nodes. This would prevent "perfectly valid" reads & writes from taking place on these isolated nodes. What do we think about this? -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (CASSANDRA-6333) ArrayIndexOutOfBound when using count(*) with over 10,000 rows
[ https://issues.apache.org/jira/browse/CASSANDRA-6333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-6333: - Assignee: Sylvain Lebresne > ArrayIndexOutOfBound when using count(*) with over 10,000 rows > -- > > Key: CASSANDRA-6333 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6333 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Cassandra 2.0.2, Ubuntu 12.04.3 LTS, Oracle Java 1.7.0_21 >Reporter: Tyler Tolley >Assignee: Sylvain Lebresne > > We've been getting a TSocket read 0 bytes error when we try and run SELECT > count(*) FROM if the table has over 10,000 rows. > I've been able to reproduce the problem by using cassandra-stress to insert > different number of rows. When I insert under 10,000, the count is returned. > When I insert exactly 10,000, I get a message that my results were limited to > 10,000 by default. If insert 10,001, I get the exception below. > {code} > ERROR [Thrift:4] 2013-11-12 09:54:04,850 CustomTThreadPoolServer.java (line > 212) Error occurred during processing of message. > java.lang.ArrayIndexOutOfBoundsException: -1 > at java.util.ArrayList.elementData(ArrayList.java:371) > at java.util.ArrayList.remove(ArrayList.java:448) > at org.apache.cassandra.cql3.ResultSet.trim(ResultSet.java:92) > at > org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:848) > at > org.apache.cassandra.cql3.statements.SelectStatement.pageCountQuery(SelectStatement.java:196) > at > org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:163) > at > org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:57) > at > org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:129) > at > org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:145) > at > org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:136) > at > org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1936) > at > org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4394) > at > org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4378) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:194) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:722) > {code} -- This message was sent by Atlassian JIRA (v6.1#6144)
git commit: Fix json2sstable handling of range tombstones
Updated Branches: refs/heads/cassandra-1.2 512ea5a76 -> 159744f24 Fix json2sstable handling of range tombstones patch by Lyuben Todorov; reviewed by Aleksey Yeschenko for CASSANDRA-6316 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/159744f2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/159744f2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/159744f2 Branch: refs/heads/cassandra-1.2 Commit: 159744f244125af84f4e819f45d2e2ebef2b369a Parents: 512ea5a Author: Aleksey Yeschenko Authored: Tue Nov 12 20:09:02 2013 +0300 Committer: Aleksey Yeschenko Committed: Tue Nov 12 20:09:02 2013 +0300 -- CHANGES.txt| 1 + .../org/apache/cassandra/tools/SSTableImport.java | 17 +++-- 2 files changed, 16 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/159744f2/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index bc0d65b..e0a2320 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -14,6 +14,7 @@ * Fix potential socket leak in connectionpool creation (CASSANDRA-6308) * Allow LOCAL_ONE/LOCAL_QUORUM to work with SimpleStrategy (CASSANDRA-6238) * cqlsh: handle 'null' as session duration (CASSANDRA-6317) + * Fix json2sstable handling of range tombstones (CASSANDRA-6316) 1.2.11 http://git-wip-us.apache.org/repos/asf/cassandra/blob/159744f2/src/java/org/apache/cassandra/tools/SSTableImport.java -- diff --git a/src/java/org/apache/cassandra/tools/SSTableImport.java b/src/java/org/apache/cassandra/tools/SSTableImport.java index 68d4397..80d1aad 100644 --- a/src/java/org/apache/cassandra/tools/SSTableImport.java +++ b/src/java/org/apache/cassandra/tools/SSTableImport.java @@ -22,6 +22,7 @@ import static org.apache.cassandra.utils.ByteBufferUtil.hexToBytes; import java.io.File; import java.io.IOException; import java.nio.ByteBuffer; +import java.util.Arrays; import java.util.List; import java.util.Map; import java.util.SortedMap; @@ -50,6 +51,7 @@ import org.apache.cassandra.db.SuperColumn; import org.apache.cassandra.db.filter.QueryPath; import org.apache.cassandra.db.marshal.AbstractType; import org.apache.cassandra.db.marshal.BytesType; +import org.apache.cassandra.db.marshal.CompositeType; import org.apache.cassandra.db.marshal.MarshalException; import org.apache.cassandra.dht.IPartitioner; import org.apache.cassandra.io.sstable.SSTableWriter; @@ -155,8 +157,19 @@ public class SSTableImport } } -value = isDeleted() ? ByteBufferUtil.hexToBytes((String) fields.get(1)) -: stringAsType((String) fields.get(1), meta.getValueValidator(meta.getColumnDefinitionFromColumnName(name))); +if (isDeleted()) +{ +value = ByteBufferUtil.hexToBytes((String) fields.get(1)); +} +else if (isRangeTombstone()) +{ +AbstractType type = CompositeType.getInstance(Arrays.asList(new AbstractType[]{meta.getColumnDefinitionComparator(0)})); +value = type.fromString((String)fields.get(1)); +} +else +{ +value = stringAsType((String) fields.get(1), meta.getValueValidator(meta.getColumnDefinitionFromColumnName(name))); +} } }
[jira] [Created] (CASSANDRA-6333) ArrayIndexOutOfBound when using count(*) with over 10,000 rows
Tyler Tolley created CASSANDRA-6333: --- Summary: ArrayIndexOutOfBound when using count(*) with over 10,000 rows Key: CASSANDRA-6333 URL: https://issues.apache.org/jira/browse/CASSANDRA-6333 Project: Cassandra Issue Type: Bug Components: Core Environment: Cassandra 2.0.2, Ubuntu 12.04.3 LTS, Oracle Java 1.7.0_21 Reporter: Tyler Tolley We've been getting a TSocket read 0 bytes error when we try and run SELECT count(*) FROM if the table has over 10,000 rows. I've been able to reproduce the problem by using cassandra-stress to insert different number of rows. When I insert under 10,000, the count is returned. When I insert exactly 10,000, I get a message that my results were limited to 10,000 by default. If insert 10,001, I get the exception below. {code} ERROR [Thrift:4] 2013-11-12 09:54:04,850 CustomTThreadPoolServer.java (line 212) Error occurred during processing of message. java.lang.ArrayIndexOutOfBoundsException: -1 at java.util.ArrayList.elementData(ArrayList.java:371) at java.util.ArrayList.remove(ArrayList.java:448) at org.apache.cassandra.cql3.ResultSet.trim(ResultSet.java:92) at org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:848) at org.apache.cassandra.cql3.statements.SelectStatement.pageCountQuery(SelectStatement.java:196) at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:163) at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:57) at org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:129) at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:145) at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:136) at org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1936) at org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4394) at org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4378) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:194) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) {code} -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (CASSANDRA-6275) 2.0.x leaks file handles
[ https://issues.apache.org/jira/browse/CASSANDRA-6275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson reassigned CASSANDRA-6275: -- Assignee: Marcus Eriksson > 2.0.x leaks file handles > > > Key: CASSANDRA-6275 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6275 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: java version "1.7.0_25" > Java(TM) SE Runtime Environment (build 1.7.0_25-b15) > Java HotSpot(TM) 64-Bit Server VM (build 23.25-b01, mixed mode) > Linux cassandra-test1 2.6.32-279.el6.x86_64 #1 SMP Thu Jun 21 15:00:18 EDT > 2012 x86_64 x86_64 x86_64 GNU/Linux >Reporter: Mikhail Mazursky >Assignee: Marcus Eriksson > Attachments: cassandra_jstack.txt, leak.log, slog.gz > > > Looks like C* is leaking file descriptors when doing lots of CAS operations. > {noformat} > $ sudo cat /proc/15455/limits > Limit Soft Limit Hard Limit Units > Max cpu time unlimitedunlimitedseconds > Max file size unlimitedunlimitedbytes > Max data size unlimitedunlimitedbytes > Max stack size10485760 unlimitedbytes > Max core file size00bytes > Max resident set unlimitedunlimitedbytes > Max processes 1024 unlimitedprocesses > Max open files4096 4096 files > Max locked memory unlimitedunlimitedbytes > Max address space unlimitedunlimitedbytes > Max file locksunlimitedunlimitedlocks > Max pending signals 1463314633signals > Max msgqueue size 819200 819200 bytes > Max nice priority 00 > Max realtime priority 00 > Max realtime timeout unlimitedunlimitedus > {noformat} > Looks like the problem is not in limits. > Before load test: > {noformat} > cassandra-test0 ~]$ lsof -n | grep java | wc -l > 166 > cassandra-test1 ~]$ lsof -n | grep java | wc -l > 164 > cassandra-test2 ~]$ lsof -n | grep java | wc -l > 180 > {noformat} > After load test: > {noformat} > cassandra-test0 ~]$ lsof -n | grep java | wc -l > 967 > cassandra-test1 ~]$ lsof -n | grep java | wc -l > 1766 > cassandra-test2 ~]$ lsof -n | grep java | wc -l > 2578 > {noformat} > Most opened files have names like: > {noformat} > java 16890 cassandra 1636r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1637r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1638r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1639r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1640r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1641r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1642r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1643r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1644r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1645r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1646r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1647r REG 202,17 161158485 > 655420 /var/lib/cassandra/data/system/paxos/system-paxos-jb-255-Data.db > java 16890 cassandra 1648r REG 202,17 88724987 > 655520 /var/lib/cassandra/data/system/paxos/system-paxos-jb-644-Data.db > java 16890 cassandra 1649r REG 202,17 161158485 > 655420 /var/lib/cassandra/
[jira] [Updated] (CASSANDRA-6329) example pig script doesn't run
[ https://issues.apache.org/jira/browse/CASSANDRA-6329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-6329: Reviewer: Brandon Williams > example pig script doesn't run > -- > > Key: CASSANDRA-6329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6329 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Jeremy Hanna >Assignee: Jeremy Hanna >Priority: Trivial > Labels: pig > Attachments: CASSANDRA-6329.txt > > > The following line in the examples/pig/example-script.pig will not run (using > Pig 0.9.2): > {code} > state_footage = FOREACH state_grouped GENERATE GROUP AS State, > SUM(state_flat.SquareFeet) AS TotalFeet:int; > {code} > It needs to have a lowercase 'group' because that's the variable name after > doing the GROUP operation in the previous line: > {code} > state_footage = FOREACH state_grouped GENERATE group AS State, > SUM(state_flat.SquareFeet) AS TotalFeet:int; > {code} > Also, I wonder if it would be good to separate the CassandraStorage and > CqlStorage into two different scripts, just so you can run the CqlStorage > example out of the box without having to comment out the CassandraStorage > example. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6319) cql limit ?
[ https://issues.apache.org/jira/browse/CASSANDRA-6319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820173#comment-13820173 ] Aleksey Yeschenko commented on CASSANDRA-6319: -- What should it be saying? > cql limit ? > --- > > Key: CASSANDRA-6319 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6319 > Project: Cassandra > Issue Type: New Feature >Reporter: Michael Brackx > > allow limit with bind variable in cql -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[ https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820163#comment-13820163 ] Andreas Schnitzerling commented on CASSANDRA-6283: -- Another relying critical issue: Cassandra can't delete cache files, system.log: "WARN [CompactionExecutor:53] 2013-11-10 21:41:59,862 AutoSavingCache.java (line 277) Failed to delete D:\Programme\cassandra\saved_caches\system-schema_columns-KeyCache-b.db" That leads to a crash on next start, system.log: "ERROR [main] 2013-11-12 11:02:43,654 CassandraDaemon.java (line 478) Exception encountered during startup java.lang.OutOfMemoryError: Java heap space at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:394) at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355) at org.apache.cassandra.service.CacheService$KeyCacheSerializer.deserialize(CacheService.java:352) at org.apache.cassandra.cache.AutoSavingCache.loadSaved(AutoSavingCache.java:119) at org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:264) at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:409) at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:386) at org.apache.cassandra.db.index.AbstractSimplePerColumnSecondaryIndex.init(AbstractSimplePerColumnSecondaryIndex.java:52) at org.apache.cassandra.db.index.SecondaryIndexManager.addIndexedColumn(SecondaryIndexManager.java:274) at org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:279) at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:409) at org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:381) at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:314) at org.apache.cassandra.db.Keyspace.(Keyspace.java:268) at org.apache.cassandra.db.Keyspace.open(Keyspace.java:110) at org.apache.cassandra.db.Keyspace.open(Keyspace.java:88) at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:274) at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:461) at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:504)" > Windows 7 data files keept open / can't be deleted after compaction. > > > Key: CASSANDRA-6283 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6283 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Windows 7 (32) / Java 1.7.0.45 >Reporter: Andreas Schnitzerling >Priority: Critical > Labels: newbie, patch, test > Fix For: 2.0.3, 2.1 > > Attachments: screenshot-1.jpg, system.log > > > Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't > help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause > is: Opened file handles seem to be lost and not closed properly. Win 7 > blames, that another process is still using the file (but its obviously > cassandra). Only restart of the server makes the files deleted. But after > heavy using (changes) of tables, there are about 24K files in the data folder > (instead of 35 after every restart) and Cassandra crashes. I experiminted and > I found out, that a finalizer fixes the problem. So after GC the files will > be deleted (not optimal, but working fine). It runs now 2 days continously > without problem. Possible fix/test: > I wrote the following finalizer at the end of class > org.apache.cassandra.io.util.RandomAccessReader: > @Override > protected void finalize() throws Throwable { > deallocate(); > super.finalize(); > } > Can somebody test / develop / patch it? Thx. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (CASSANDRA-6325) AssertionError on startup reading saved Serializing row cache
[ https://issues.apache.org/jira/browse/CASSANDRA-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820156#comment-13820156 ] Jonathan Ellis commented on CASSANDRA-6325: --- WDYT, Mikhail? > AssertionError on startup reading saved Serializing row cache > - > > Key: CASSANDRA-6325 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6325 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: upgrade from 1.2.9ish to 1.2.11ish >Reporter: Chris Burroughs >Assignee: Mikhail Stepura >Priority: Minor > Fix For: 1.2.12 > > Attachments: 6325-v2.txt, CASSANDRA-1.2-6325.patch > > > I don't see any reason what this could have to do with the upgrade, but don't > have a large enough non-prod cluster to just keep restarting on. Occurred on > roughly 2 out of 100 restarted nodes. > {noformat} > ERROR [main] 2013-11-08 14:40:13,535 CassandraDaemon.java (line 482) > Exception encountered during startup > java.lang.AssertionError > at > org.apache.cassandra.cache.SerializingCacheProvider$RowCacheSerializer.serialize(SerializingCacheProvider.java:41) > at > org.apache.cassandra.cache.SerializingCacheProvider$RowCacheSerializer.serialize(SerializingCacheProvider.java:37) > at > org.apache.cassandra.cache.SerializingCache.serialize(SerializingCache.java:118) > at > org.apache.cassandra.cache.SerializingCache.put(SerializingCache.java:176) > at > org.apache.cassandra.cache.InstrumentingCache.put(InstrumentingCache.java:44) > at > org.apache.cassandra.cache.AutoSavingCache.loadSaved(AutoSavingCache.java:156) > at > org.apache.cassandra.db.ColumnFamilyStore.initRowCache(ColumnFamilyStore.java:444) > at org.apache.cassandra.db.Table.open(Table.java:114) > at org.apache.cassandra.db.Table.open(Table.java:87) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:278) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465) > {noformat} > I have the files if there is any useful analysis that can be run. Looked > 'normal' to a cursory `less` inspection. > Possibly related: CASSANDRA-4463 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (CASSANDRA-6327) select with "in" clause wrongly returns empty result
[ https://issues.apache.org/jira/browse/CASSANDRA-6327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-6327: - Reviewer: Aleksey Yeschenko +1 > select with "in" clause wrongly returns empty result > > > Key: CASSANDRA-6327 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6327 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Cassandra 2.0.2, x86-64 Ubuntu 13.10 >Reporter: Duncan Sands >Assignee: Sylvain Lebresne > Fix For: 2.0.3 > > Attachments: 6327.txt, current_prices.tar > > > This query returns no result: > cqlsh:tick_data> select syd from current_prices where shard = 1 and syd in > (1, 556129); > (0 rows) > However this query does return a result, showing that the previous query was > wrong to return no result: > cqlsh:tick_data> select syd from current_prices where shard = 1 and syd in > (556129); > syd > > 556129 > (1 rows) > This can be reproduced as follows: > (a) Create a keyspace tick_data: > create keyspace tick_data WITH replication = {'class': 'SimpleStrategy', > 'replication_factor': 1}; > (b) Create a table current_prices: > CREATE TABLE current_prices ( > shard int, > syd int, > ask decimal, > bid decimal, > currency ascii, > when timeuuid, > PRIMARY KEY (shard, syd) > ); > (c) Stop Cassandra and untar the attached tar file in > /var/lib/cassandra/data/tick_data/. It populates the current_prices table. > (d) Restart Cassandra and perform the above selects. -- This message was sent by Atlassian JIRA (v6.1#6144)