[jira] [Created] (CASSANDRA-13561) Purge TTL on expiration

2017-05-29 Thread Andrew Whang (JIRA) 
Andrew Whang created CASSANDRA-13561:


 Summary: Purge TTL on expiration
 Key: CASSANDRA-13561
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13561
 Project: Cassandra
  Issue Type: New Feature
Reporter: Andrew Whang
Priority: Minor


Tables with mostly TTL columns tend to suffer from high droppable tombstone 
ratio, which results in higher read latency, cpu utilization, and disk usage. 
Expired TTL data become tombstones, and the nature of purging tombstones during 
compaction (due to checking for overlapping SSTables) make them susceptible to 
surviving much longer than expected. A table option to purge TTL on expiration 
would address this issue, by preventing them from becoming tombstones. A 
boolean purge_ttl_on_expiration table setting would allow users to easily turn 
the feature on or off. 

Being more aggressive with gc_grace could also address the problem of long 
lasting tombstones, but that would affect tombstones from deletes as well. 

Even if a purged [expired] cell is revived via repair from a node that hasn't 
yet compacted away the cell, it would be revived as an expiring cell with the 
same localDeletionTime, so reads should properly handle them. As well, it would 
be purged in the next compaction. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13068) Fully expired sstable not dropped when running out of disk space

2017-05-29 Thread Lerh Chuan Low (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028709#comment-16028709
 ] 

Lerh Chuan Low commented on CASSANDRA-13068:


Thanks for the quick reply. I'll look into writing some tests (I was initially 
wondering how that would be done without actually filling up the disk of the 
machine). I didn't know about Byteman, so going to do some digging into it and 
also thanks for providing an example. 

Re: {{Sets.difference}} - you're right, just checked the java doc. Wasn't aware 
it returned an unmodifiable view of a set. 

> Fully expired sstable not dropped when running out of disk space
> 
>
> Key: CASSANDRA-13068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13068
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>  Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> If a fully expired sstable is larger than the remaining disk space we won't 
> run the compaction that can drop the sstable (ie, in our disk space check 
> should not include the fully expired sstables)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13549) Cqlsh throws and error when querying a duration data type

2017-05-29 Thread Benjamin Lerer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13549:
---
   Resolution: Fixed
Fix Version/s: 4.0
   3.11.0
   Status: Resolved  (was: Ready to Commit)

Committed into 3.11 at 5a860a70f28b7756e0073d2d5d239b5e748a0b73 and merged into 
trunk

> Cqlsh throws and error when querying a duration data type
> -
>
> Key: CASSANDRA-13549
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13549
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.10 dev environment running on a MacOS Sierra
>Reporter: Akhil Mehra
>Assignee: Akhil Mehra
> Fix For: 3.11.0, 4.0
>
>
> h3. Overview
> Querying duration related data from the cqlsh prompt results in an error.
> Consider the following create table and insert statement.
> {code:title=Table and insert statement with duration data 
> type|borderStyle=solid}
> CREATE TABLE duration_test (
>   primary_key text,
>   col20 duration,
>   PRIMARY KEY (primary_key)
> );
> INSERT INTO duration_test (primary_key, col20) VALUES ('primary_key_example', 
> 1y5mo89h4m48s);
> {code}
> On executing a select query on col20 in cqlsh I get an error "Failed to 
> format value '"\x00\xfe\x02GS\xfc\xa5\xc0\x00' : 'ascii' codec can't decode 
> byte 0xfe in position 2: ordinal not in range(128)"
> {code:title=Duration Query|borderStyle=solid}
> Select  col20 from duration_test;
> {code}
> h3. Investigation
> On investigating this further I found that the current python Cassandra 
> driver used found in 
> lib/cassandra-driver-internal-only-3.7.0.post0-2481531.zip does not seem to 
> support duration data type. This was added in Jan this year 
> https://github.com/datastax/python-driver/pull/689.
> So I downloaded the latest driver release 
> https://github.com/datastax/python-driver/releases/tag/3.9.0. I embedded the 
> latest driver into cassandra-driver-internal-only-3.7.0.post0-2481531.zip. 
> This fixed the driver related issue but there was still a formatting issue. 
> I then went on to modify the format_value_duration methos in the 
> pylib/cqlshlib/formatting.py. Diff posted below
> {code}
>  @formatter_for('Duration')
>  def format_value_duration(val, colormap, **_):
> -buf = six.iterbytes(val)
> -months = decode_vint(buf)
> -days = decode_vint(buf)
> -nanoseconds = decode_vint(buf)
> -return format_python_formatted_type(duration_as_str(months, days, 
> nanoseconds), colormap, 'duration')
> +return format_python_formatted_type(duration_as_str(val.months, 
> val.days, val.nanoseconds), colormap, 'duration')
> {code}
> This resulted in fixing the issue and duration types are now correctly 
> displayed.
> Happy to fix the issue if I can get some guidance on:
> # If this is a valid issue. Tried searching JIRA but did not find anything 
> reported. 
> # If my assumptions are correct i.e. this is actually a bug
> # how to package the new driver into the source code. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[3/3] cassandra git commit: Merge branch cassandra-3.11 into trunk

2017-05-29 Thread blerer 
Merge branch cassandra-3.11 into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7b149ce4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7b149ce4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7b149ce4

Branch: refs/heads/trunk
Commit: 7b149ce4a42d4831426ccd18dfe4fe7d004ea80b
Parents: 710720d 5a860a7
Author: Benjamin Lerer 
Authored: Mon May 29 17:43:33 2017 +0200
Committer: Benjamin Lerer 
Committed: Mon May 29 17:43:42 2017 +0200

--
 CHANGES.txt   |   1 +
 lib/cassandra-driver-internal-only-3.10.zip   | Bin 0 -> 256997 bytes
 ...a-driver-internal-only-3.7.1.post0-19c1603.zip | Bin 252027 -> 0 bytes
 pylib/cqlshlib/formatting.py  |   6 +-
 4 files changed, 2 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7b149ce4/CHANGES.txt
--
diff --cc CHANGES.txt
index ebc819a,c006b50..d81d2c5
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,77 -1,5 +1,78 @@@
 +4.0
 + * Update repairTime for keyspaces on completion (CASSANDRA-13539)
 + * Add configurable upper bound for validation executor threads 
(CASSANDRA-13521)
 + * Bring back maxHintTTL propery (CASSANDRA-12982)
 + * Add testing guidelines (CASSANDRA-13497)
 + * Add more repair metrics (CASSANDRA-13531)
 + * RangeStreamer should be smarter when picking endpoints for streaming 
(CASSANDRA-4650)
 + * Avoid rewrapping an exception thrown for cache load functions 
(CASSANDRA-13367)
 + * Log time elapsed for each incremental repair phase (CASSANDRA-13498)
 + * Add multiple table operation support to cassandra-stress (CASSANDRA-8780)
 + * Fix incorrect cqlsh results when selecting same columns multiple times 
(CASSANDRA-13262)
 + * Fix WriteResponseHandlerTest is sensitive to test execution order 
(CASSANDRA-13421)
 + * Improve incremental repair logging (CASSANDRA-13468)
 + * Start compaction when incremental repair finishes (CASSANDRA-13454)
 + * Add repair streaming preview (CASSANDRA-13257)
 + * Cleanup isIncremental/repairedAt usage (CASSANDRA-13430)
 + * Change protocol to allow sending key space independent of query string 
(CASSANDRA-10145)
 + * Make gc_log and gc_warn settable at runtime (CASSANDRA-12661)
 + * Take number of files in L0 in account when estimating remaining compaction 
tasks (CASSANDRA-13354)
 + * Skip building views during base table streams on range movements 
(CASSANDRA-13065)
 + * Improve error messages for +/- operations on maps and tuples 
(CASSANDRA-13197)
 + * Remove deprecated repair JMX APIs (CASSANDRA-11530)
 + * Fix version check to enable streaming keep-alive (CASSANDRA-12929)
 + * Make it possible to monitor an ideal consistency level separate from 
actual consistency level (CASSANDRA-13289)
 + * Outbound TCP connections ignore internode authenticator (CASSANDRA-13324)
 + * Upgrade junit from 4.6 to 4.12 (CASSANDRA-13360)
 + * Cleanup ParentRepairSession after repairs (CASSANDRA-13359)
 + * Incremental repair not streaming correct sstables (CASSANDRA-13328)
 + * Upgrade the jna version to 4.3.0 (CASSANDRA-13300)
 + * Add the currentTimestamp, currentDate, currentTime and currentTimeUUID 
functions (CASSANDRA-13132)
 + * Remove config option index_interval (CASSANDRA-10671)
 + * Reduce lock contention for collection types and serializers 
(CASSANDRA-13271)
 + * Make it possible to override MessagingService.Verb ids (CASSANDRA-13283)
 + * Avoid synchronized on prepareForRepair in ActiveRepairService 
(CASSANDRA-9292)
 + * Adds the ability to use uncompressed chunks in compressed files 
(CASSANDRA-10520)
 + * Don't flush sstables when streaming for incremental repair 
(CASSANDRA-13226)
 + * Remove unused method (CASSANDRA-13227)
 + * Fix minor bugs related to #9143 (CASSANDRA-13217)
 + * Output warning if user increases RF (CASSANDRA-13079)
 + * Remove pre-3.0 streaming compatibility code for 4.0 (CASSANDRA-13081)
 + * Add support for + and - operations on dates (CASSANDRA-11936)
 + * Fix consistency of incrementally repaired data (CASSANDRA-9143)
 + * Increase commitlog version (CASSANDRA-13161)
 + * Make TableMetadata immutable, optimize Schema (CASSANDRA-9425)
 + * Refactor ColumnCondition (CASSANDRA-12981)
 + * Parallelize streaming of different keyspaces (CASSANDRA-4663)
 + * Improved compactions metrics (CASSANDRA-13015)
 + * Speed-up start-up sequence by avoiding un-needed flushes (CASSANDRA-13031)
 + * Use Caffeine (W-TinyLFU) for on-heap caches (CASSANDRA-10855)
 + * Thrift removal (CASSANDRA-5)
 + * Remove pre-3.0 compatibility code for 4.0 (CASSANDRA-12716)
 + * Add column definition kind to dropped columns in schema (CASSANDRA-12705)
 + * Add (automate) Nodetool

[2/3] cassandra git commit: Fix formatting of duration columns in CQLSH

2017-05-29 Thread blerer 
Fix formatting of duration columns in CQLSH

patch by Akhil Mehra; reviewed by Benjamin Lerer for CASSANDRA-13549


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a860a70
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a860a70
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a860a70

Branch: refs/heads/trunk
Commit: 5a860a70f28b7756e0073d2d5d239b5e748a0b73
Parents: 60c2004
Author: Akhil Mehra 
Authored: Mon May 29 17:31:57 2017 +0200
Committer: Benjamin Lerer 
Committed: Mon May 29 17:42:29 2017 +0200

--
 CHANGES.txt   |   1 +
 lib/cassandra-driver-internal-only-3.10.zip   | Bin 0 -> 256997 bytes
 ...a-driver-internal-only-3.7.1.post0-19c1603.zip | Bin 252027 -> 0 bytes
 pylib/cqlshlib/formatting.py  |   6 +-
 4 files changed, 2 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a860a70/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a8e172e..c006b50 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.11.0
+ * Fix formatting of duration columns in CQLSH (CASSANDRA-13549)
  * Fix the problem with duplicated rows when using paging with SASI 
(CASSANDRA-13302)
  * Allow CONTAINS statements filtering on the partition key and it’s parts 
(CASSANDRA-13275)
  * Fall back to even ranges calculation in clusters with vnodes when tokens 
are distributed unevenly (CASSANDRA-13229)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a860a70/lib/cassandra-driver-internal-only-3.10.zip
--
diff --git a/lib/cassandra-driver-internal-only-3.10.zip 
b/lib/cassandra-driver-internal-only-3.10.zip
new file mode 100644
index 000..22b877c
Binary files /dev/null and b/lib/cassandra-driver-internal-only-3.10.zip differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a860a70/lib/cassandra-driver-internal-only-3.7.1.post0-19c1603.zip
--
diff --git a/lib/cassandra-driver-internal-only-3.7.1.post0-19c1603.zip 
b/lib/cassandra-driver-internal-only-3.7.1.post0-19c1603.zip
deleted file mode 100644
index 900d64d..000
Binary files a/lib/cassandra-driver-internal-only-3.7.1.post0-19c1603.zip and 
/dev/null differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a860a70/pylib/cqlshlib/formatting.py
--
diff --git a/pylib/cqlshlib/formatting.py b/pylib/cqlshlib/formatting.py
index 08665fd..cf3b32d 100644
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@ -396,11 +396,7 @@ def format_value_time(val, colormap, **_):
 
 @formatter_for('Duration')
 def format_value_duration(val, colormap, **_):
-buf = six.iterbytes(val)
-months = decode_vint(buf)
-days = decode_vint(buf)
-nanoseconds = decode_vint(buf)
-return format_python_formatted_type(duration_as_str(months, days, 
nanoseconds), colormap, 'duration')
+return format_python_formatted_type(duration_as_str(val.months, val.days, 
val.nanoseconds), colormap, 'duration')
 
 
 def duration_as_str(months, days, nanoseconds):


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[1/3] cassandra git commit: Fix formatting of duration columns in CQLSH

2017-05-29 Thread blerer 
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-3.11 60c200496 -> 5a860a70f
  refs/heads/trunk 710720d98 -> 7b149ce4a


Fix formatting of duration columns in CQLSH

patch by Akhil Mehra; reviewed by Benjamin Lerer for CASSANDRA-13549


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a860a70
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a860a70
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a860a70

Branch: refs/heads/cassandra-3.11
Commit: 5a860a70f28b7756e0073d2d5d239b5e748a0b73
Parents: 60c2004
Author: Akhil Mehra 
Authored: Mon May 29 17:31:57 2017 +0200
Committer: Benjamin Lerer 
Committed: Mon May 29 17:42:29 2017 +0200

--
 CHANGES.txt   |   1 +
 lib/cassandra-driver-internal-only-3.10.zip   | Bin 0 -> 256997 bytes
 ...a-driver-internal-only-3.7.1.post0-19c1603.zip | Bin 252027 -> 0 bytes
 pylib/cqlshlib/formatting.py  |   6 +-
 4 files changed, 2 insertions(+), 5 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a860a70/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a8e172e..c006b50 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.11.0
+ * Fix formatting of duration columns in CQLSH (CASSANDRA-13549)
  * Fix the problem with duplicated rows when using paging with SASI 
(CASSANDRA-13302)
  * Allow CONTAINS statements filtering on the partition key and it’s parts 
(CASSANDRA-13275)
  * Fall back to even ranges calculation in clusters with vnodes when tokens 
are distributed unevenly (CASSANDRA-13229)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a860a70/lib/cassandra-driver-internal-only-3.10.zip
--
diff --git a/lib/cassandra-driver-internal-only-3.10.zip 
b/lib/cassandra-driver-internal-only-3.10.zip
new file mode 100644
index 000..22b877c
Binary files /dev/null and b/lib/cassandra-driver-internal-only-3.10.zip differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a860a70/lib/cassandra-driver-internal-only-3.7.1.post0-19c1603.zip
--
diff --git a/lib/cassandra-driver-internal-only-3.7.1.post0-19c1603.zip 
b/lib/cassandra-driver-internal-only-3.7.1.post0-19c1603.zip
deleted file mode 100644
index 900d64d..000
Binary files a/lib/cassandra-driver-internal-only-3.7.1.post0-19c1603.zip and 
/dev/null differ

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a860a70/pylib/cqlshlib/formatting.py
--
diff --git a/pylib/cqlshlib/formatting.py b/pylib/cqlshlib/formatting.py
index 08665fd..cf3b32d 100644
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@ -396,11 +396,7 @@ def format_value_time(val, colormap, **_):
 
 @formatter_for('Duration')
 def format_value_duration(val, colormap, **_):
-buf = six.iterbytes(val)
-months = decode_vint(buf)
-days = decode_vint(buf)
-nanoseconds = decode_vint(buf)
-return format_python_formatted_type(duration_as_str(months, days, 
nanoseconds), colormap, 'duration')
+return format_python_formatted_type(duration_as_str(val.months, val.days, 
val.nanoseconds), colormap, 'duration')
 
 
 def duration_as_str(months, days, nanoseconds):


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13549) Cqlsh throws and error when querying a duration data type

2017-05-29 Thread Benjamin Lerer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13549:
---
Status: Ready to Commit  (was: Patch Available)

> Cqlsh throws and error when querying a duration data type
> -
>
> Key: CASSANDRA-13549
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13549
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.10 dev environment running on a MacOS Sierra
>Reporter: Akhil Mehra
>Assignee: Akhil Mehra
>
> h3. Overview
> Querying duration related data from the cqlsh prompt results in an error.
> Consider the following create table and insert statement.
> {code:title=Table and insert statement with duration data 
> type|borderStyle=solid}
> CREATE TABLE duration_test (
>   primary_key text,
>   col20 duration,
>   PRIMARY KEY (primary_key)
> );
> INSERT INTO duration_test (primary_key, col20) VALUES ('primary_key_example', 
> 1y5mo89h4m48s);
> {code}
> On executing a select query on col20 in cqlsh I get an error "Failed to 
> format value '"\x00\xfe\x02GS\xfc\xa5\xc0\x00' : 'ascii' codec can't decode 
> byte 0xfe in position 2: ordinal not in range(128)"
> {code:title=Duration Query|borderStyle=solid}
> Select  col20 from duration_test;
> {code}
> h3. Investigation
> On investigating this further I found that the current python Cassandra 
> driver used found in 
> lib/cassandra-driver-internal-only-3.7.0.post0-2481531.zip does not seem to 
> support duration data type. This was added in Jan this year 
> https://github.com/datastax/python-driver/pull/689.
> So I downloaded the latest driver release 
> https://github.com/datastax/python-driver/releases/tag/3.9.0. I embedded the 
> latest driver into cassandra-driver-internal-only-3.7.0.post0-2481531.zip. 
> This fixed the driver related issue but there was still a formatting issue. 
> I then went on to modify the format_value_duration methos in the 
> pylib/cqlshlib/formatting.py. Diff posted below
> {code}
>  @formatter_for('Duration')
>  def format_value_duration(val, colormap, **_):
> -buf = six.iterbytes(val)
> -months = decode_vint(buf)
> -days = decode_vint(buf)
> -nanoseconds = decode_vint(buf)
> -return format_python_formatted_type(duration_as_str(months, days, 
> nanoseconds), colormap, 'duration')
> +return format_python_formatted_type(duration_as_str(val.months, 
> val.days, val.nanoseconds), colormap, 'duration')
> {code}
> This resulted in fixing the issue and duration types are now correctly 
> displayed.
> Happy to fix the issue if I can get some guidance on:
> # If this is a valid issue. Tried searching JIRA but did not find anything 
> reported. 
> # If my assumptions are correct i.e. this is actually a bug
> # how to package the new driver into the source code. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13549) Cqlsh throws and error when querying a duration data type

2017-05-29 Thread Benjamin Lerer (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028456#comment-16028456
 ] 

Benjamin Lerer commented on CASSANDRA-13549:


I ran the test for 3.11 and trunk. There are few failling CQLSH tests but they 
were failing before the patch so I am +1 for the patch.

Thanks for the work.

> Cqlsh throws and error when querying a duration data type
> -
>
> Key: CASSANDRA-13549
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13549
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.10 dev environment running on a MacOS Sierra
>Reporter: Akhil Mehra
>Assignee: Akhil Mehra
>
> h3. Overview
> Querying duration related data from the cqlsh prompt results in an error.
> Consider the following create table and insert statement.
> {code:title=Table and insert statement with duration data 
> type|borderStyle=solid}
> CREATE TABLE duration_test (
>   primary_key text,
>   col20 duration,
>   PRIMARY KEY (primary_key)
> );
> INSERT INTO duration_test (primary_key, col20) VALUES ('primary_key_example', 
> 1y5mo89h4m48s);
> {code}
> On executing a select query on col20 in cqlsh I get an error "Failed to 
> format value '"\x00\xfe\x02GS\xfc\xa5\xc0\x00' : 'ascii' codec can't decode 
> byte 0xfe in position 2: ordinal not in range(128)"
> {code:title=Duration Query|borderStyle=solid}
> Select  col20 from duration_test;
> {code}
> h3. Investigation
> On investigating this further I found that the current python Cassandra 
> driver used found in 
> lib/cassandra-driver-internal-only-3.7.0.post0-2481531.zip does not seem to 
> support duration data type. This was added in Jan this year 
> https://github.com/datastax/python-driver/pull/689.
> So I downloaded the latest driver release 
> https://github.com/datastax/python-driver/releases/tag/3.9.0. I embedded the 
> latest driver into cassandra-driver-internal-only-3.7.0.post0-2481531.zip. 
> This fixed the driver related issue but there was still a formatting issue. 
> I then went on to modify the format_value_duration methos in the 
> pylib/cqlshlib/formatting.py. Diff posted below
> {code}
>  @formatter_for('Duration')
>  def format_value_duration(val, colormap, **_):
> -buf = six.iterbytes(val)
> -months = decode_vint(buf)
> -days = decode_vint(buf)
> -nanoseconds = decode_vint(buf)
> -return format_python_formatted_type(duration_as_str(months, days, 
> nanoseconds), colormap, 'duration')
> +return format_python_formatted_type(duration_as_str(val.months, 
> val.days, val.nanoseconds), colormap, 'duration')
> {code}
> This resulted in fixing the issue and duration types are now correctly 
> displayed.
> Happy to fix the issue if I can get some guidance on:
> # If this is a valid issue. Tried searching JIRA but did not find anything 
> reported. 
> # If my assumptions are correct i.e. this is actually a bug
> # how to package the new driver into the source code. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13272) "nodetool bootstrap resume" does not exit

2017-05-29 Thread Tim Lamballais (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028423#comment-16028423
 ] 

Tim Lamballais edited comment on CASSANDRA-13272 at 5/29/17 3:07 PM:
-

{noformat}
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 9bc046f..d7c1aa5 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1287,8 +1287,9 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onFailure(Throwable e)
 {
-String message = "Error during bootstrap: " + 
e.getCause().getMessage();
-logger.error(message, e.getCause());
+Throwable cause = Throwables.getRootCause(e);
+String message = "Error during bootstrap: " + 
cause.getMessage();
+logger.error(message, cause);
 progressSupport.progress("bootstrap", new 
ProgressEvent(ProgressEventType.ERROR, 1, 1, message));
 progressSupport.progress("bootstrap", new 
ProgressEvent(ProgressEventType.COMPLETE, 1, 1, "Resume bootstrap complete"));
 }
diff --git a/src/java/org/apache/cassandra/utils/Throwables.java 
b/src/java/org/apache/cassandra/utils/Throwables.java
index 5ad9686..30fc9f4 100644
--- a/src/java/org/apache/cassandra/utils/Throwables.java
+++ b/src/java/org/apache/cassandra/utils/Throwables.java
@@ -30,6 +30,18 @@ import org.apache.cassandra.io.FSWriteError;

 public final class Throwables
 {
+public static Throwable getRootCause(Throwable t)
+{
+Throwable cause = t.getCause();
+if (cause == null) {
+return t;
+}
+while (cause.getCause() != null) {
+cause = cause.getCause();
+}
+return cause;
+}
+
 public enum FileOpType { READ, WRITE }

 public interface DiscreteAction
{noformat}


was (Author: wimtie):
{noformat}
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 9bc046f..3bd2a0c 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1287,8 +1287,9 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onFailure(Throwable e)
 {
-String message = "Error during bootstrap: " + 
e.getCause().getMessage();
-logger.error(message, e.getCause());
+Throwable cause = Throwables.getCause(e);
+String message = "Error during bootstrap: " + 
cause.getMessage();
+logger.error(message, cause);
 progressSupport.progress("bootstrap", new 
ProgressEvent(ProgressEventType.ERROR, 1, 1, message));
 progressSupport.progress("bootstrap", new 
ProgressEvent(ProgressEventType.COMPLETE, 1, 1, "Resume bootstrap complete"));
 }
diff --git a/src/java/org/apache/cassandra/utils/Throwables.java 
b/src/java/org/apache/cassandra/utils/Throwables.java
index 5ad9686..0936fd2 100644
--- a/src/java/org/apache/cassandra/utils/Throwables.java
+++ b/src/java/org/apache/cassandra/utils/Throwables.java
@@ -30,6 +30,12 @@ import org.apache.cassandra.io.FSWriteError;

 public final class Throwables
 {
+public static Throwable getCause(Throwable t)
+{
+Throwable cause = t.getCause();
+return cause != null ? cause : t;
+}
+
 public enum FileOpType { READ, WRITE }

 public interface DiscreteAction
{noformat}

> "nodetool bootstrap resume" does not exit
> -
>
> Key: CASSANDRA-13272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13272
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle, Streaming and Messaging
>Reporter: Tom van der Woerdt
>  Labels: lhf
>
> I have a script that calls "nodetool bootstrap resume" after a failed join 
> (in my environment some streams sometimes fail due to mis-tuning of stream 
> bandwidth settings). However, if the streams fail again, nodetool won't exit.
> Last lines before it just hangs forever :
> {noformat}
> [2017-02-26 07:02:42,287] received file 
> /var/lib/cassandra/data/keyspace/table-63d5d42009fa11e5879ebd9463bffdac/mc-12670-big-Data.db
>  (progress: 1112%)
> [2017-02-26 07:02:42,287] received file 
> /var/lib/cassandra/data/keyspace/table-63d5d42009fa11e5879ebd9463bffdac/mc-12670-big-Data.db
>  (progress: 1112%)
> [2017-02-26

[jira] [Created] (CASSANDRA-13560) Improved cleanup performance

2017-05-29 Thread Brian Gallew (JIRA) 
Brian Gallew created CASSANDRA-13560:


 Summary: Improved cleanup performance
 Key: CASSANDRA-13560
 URL: https://issues.apache.org/jira/browse/CASSANDRA-13560
 Project: Cassandra
  Issue Type: Improvement
  Components: Compaction
Reporter: Brian Gallew


I've been thinking about sstables.  One of their properties is that they are 
sorted.  In the face of that property, it would seem that the cleanup 
functionality *should* be very fast as all of the partitions which no longer 
belong to a given node should be in either one or two contiguous blocks of 
space.  Perhaps this is naive, but I would think the index should clearly 
indicate what needs to be retained versus what can be disposed of, and thus a 
cleanup should be able to start reading with the first valid partition, stop 
with the last, and skip the bulk of loading/unloading that seems to be 
happening.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-13272) "nodetool bootstrap resume" does not exit

2017-05-29 Thread Tim Lamballais (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028423#comment-16028423
 ] 

Tim Lamballais edited comment on CASSANDRA-13272 at 5/29/17 2:41 PM:
-

{noformat}
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 9bc046f..3bd2a0c 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1287,8 +1287,9 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onFailure(Throwable e)
 {
-String message = "Error during bootstrap: " + 
e.getCause().getMessage();
-logger.error(message, e.getCause());
+Throwable cause = Throwables.getCause(e);
+String message = "Error during bootstrap: " + 
cause.getMessage();
+logger.error(message, cause);
 progressSupport.progress("bootstrap", new 
ProgressEvent(ProgressEventType.ERROR, 1, 1, message));
 progressSupport.progress("bootstrap", new 
ProgressEvent(ProgressEventType.COMPLETE, 1, 1, "Resume bootstrap complete"));
 }
diff --git a/src/java/org/apache/cassandra/utils/Throwables.java 
b/src/java/org/apache/cassandra/utils/Throwables.java
index 5ad9686..0936fd2 100644
--- a/src/java/org/apache/cassandra/utils/Throwables.java
+++ b/src/java/org/apache/cassandra/utils/Throwables.java
@@ -30,6 +30,12 @@ import org.apache.cassandra.io.FSWriteError;

 public final class Throwables
 {
+public static Throwable getCause(Throwable t)
+{
+Throwable cause = t.getCause();
+return cause != null ? cause : t;
+}
+
 public enum FileOpType { READ, WRITE }

 public interface DiscreteAction
{noformat}


was (Author: wimtie):
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 9bc046f..3bd2a0c 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1287,8 +1287,9 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onFailure(Throwable e)
 {
-String message = "Error during bootstrap: " + 
e.getCause().getMessage();
-logger.error(message, e.getCause());
+Throwable cause = Throwables.getCause(e);
+String message = "Error during bootstrap: " + 
cause.getMessage();
+logger.error(message, cause);
 progressSupport.progress("bootstrap", new 
ProgressEvent(ProgressEventType.ERROR, 1, 1, message));
 progressSupport.progress("bootstrap", new 
ProgressEvent(ProgressEventType.COMPLETE, 1, 1, "Resume bootstrap complete"));
 }
diff --git a/src/java/org/apache/cassandra/utils/Throwables.java 
b/src/java/org/apache/cassandra/utils/Throwables.java
index 5ad9686..0936fd2 100644
--- a/src/java/org/apache/cassandra/utils/Throwables.java
+++ b/src/java/org/apache/cassandra/utils/Throwables.java
@@ -30,6 +30,12 @@ import org.apache.cassandra.io.FSWriteError;

 public final class Throwables
 {
+public static Throwable getCause(Throwable t)
+{
+Throwable cause = t.getCause();
+return cause != null ? cause : t;
+}
+
 public enum FileOpType { READ, WRITE }

 public interface DiscreteAction

> "nodetool bootstrap resume" does not exit
> -
>
> Key: CASSANDRA-13272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13272
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle, Streaming and Messaging
>Reporter: Tom van der Woerdt
>  Labels: lhf
>
> I have a script that calls "nodetool bootstrap resume" after a failed join 
> (in my environment some streams sometimes fail due to mis-tuning of stream 
> bandwidth settings). However, if the streams fail again, nodetool won't exit.
> Last lines before it just hangs forever :
> {noformat}
> [2017-02-26 07:02:42,287] received file 
> /var/lib/cassandra/data/keyspace/table-63d5d42009fa11e5879ebd9463bffdac/mc-12670-big-Data.db
>  (progress: 1112%)
> [2017-02-26 07:02:42,287] received file 
> /var/lib/cassandra/data/keyspace/table-63d5d42009fa11e5879ebd9463bffdac/mc-12670-big-Data.db
>  (progress: 1112%)
> [2017-02-26 07:02:59,843] received file 
> /var/lib/cassandra/data/keyspace/table-63d5d42009fa11e5879ebd9463bffdac/mc-12671-big-Data.db
>  (progress: 1112%)
> [2017-02-26 09:25:51,000]

[jira] [Updated] (CASSANDRA-13272) "nodetool bootstrap resume" does not exit

2017-05-29 Thread Tim Lamballais (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Lamballais updated CASSANDRA-13272:
---
Status: Patch Available  (was: Open)

diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 9bc046f..3bd2a0c 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1287,8 +1287,9 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 @Override
 public void onFailure(Throwable e)
 {
-String message = "Error during bootstrap: " + 
e.getCause().getMessage();
-logger.error(message, e.getCause());
+Throwable cause = Throwables.getCause(e);
+String message = "Error during bootstrap: " + 
cause.getMessage();
+logger.error(message, cause);
 progressSupport.progress("bootstrap", new 
ProgressEvent(ProgressEventType.ERROR, 1, 1, message));
 progressSupport.progress("bootstrap", new 
ProgressEvent(ProgressEventType.COMPLETE, 1, 1, "Resume bootstrap complete"));
 }
diff --git a/src/java/org/apache/cassandra/utils/Throwables.java 
b/src/java/org/apache/cassandra/utils/Throwables.java
index 5ad9686..0936fd2 100644
--- a/src/java/org/apache/cassandra/utils/Throwables.java
+++ b/src/java/org/apache/cassandra/utils/Throwables.java
@@ -30,6 +30,12 @@ import org.apache.cassandra.io.FSWriteError;

 public final class Throwables
 {
+public static Throwable getCause(Throwable t)
+{
+Throwable cause = t.getCause();
+return cause != null ? cause : t;
+}
+
 public enum FileOpType { READ, WRITE }

 public interface DiscreteAction

> "nodetool bootstrap resume" does not exit
> -
>
> Key: CASSANDRA-13272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13272
> Project: Cassandra
>  Issue Type: Bug
>  Components: Lifecycle, Streaming and Messaging
>Reporter: Tom van der Woerdt
>  Labels: lhf
>
> I have a script that calls "nodetool bootstrap resume" after a failed join 
> (in my environment some streams sometimes fail due to mis-tuning of stream 
> bandwidth settings). However, if the streams fail again, nodetool won't exit.
> Last lines before it just hangs forever :
> {noformat}
> [2017-02-26 07:02:42,287] received file 
> /var/lib/cassandra/data/keyspace/table-63d5d42009fa11e5879ebd9463bffdac/mc-12670-big-Data.db
>  (progress: 1112%)
> [2017-02-26 07:02:42,287] received file 
> /var/lib/cassandra/data/keyspace/table-63d5d42009fa11e5879ebd9463bffdac/mc-12670-big-Data.db
>  (progress: 1112%)
> [2017-02-26 07:02:59,843] received file 
> /var/lib/cassandra/data/keyspace/table-63d5d42009fa11e5879ebd9463bffdac/mc-12671-big-Data.db
>  (progress: 1112%)
> [2017-02-26 09:25:51,000] session with /10.x.y.z complete (progress: 1112%)
> [2017-02-26 09:33:45,017] session with /10.x.y.z complete (progress: 1112%)
> [2017-02-26 09:39:27,216] session with /10.x.y.z complete (progress: 1112%)
> [2017-02-26 09:53:33,084] session with /10.x.y.z complete (progress: 1112%)
> [2017-02-26 09:55:07,115] session with /10.x.y.z complete (progress: 1112%)
> [2017-02-26 10:06:49,557] session with /10.x.y.z complete (progress: 1112%)
> [2017-02-26 10:40:55,880] session with /10.x.y.z complete (progress: 1112%)
> [2017-02-26 11:09:21,025] session with /10.x.y.z complete (progress: 1112%)
> [2017-02-26 12:44:35,755] session with /10.x.y.z complete (progress: 1112%)
> [2017-02-26 12:49:18,867] session with /10.x.y.z complete (progress: 1112%)
> [2017-02-26 13:23:50,611] session with /10.x.y.z complete (progress: 1112%)
> [2017-02-26 13:23:50,612] Stream failed
> {noformat}
> At that point ("Stream failed") I would expect nodetool to exit with a 
> non-zero exit code. Instead, it just wants me to ^C it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13549) Cqlsh throws and error when querying a duration data type

2017-05-29 Thread Benjamin Lerer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13549:
---
Status: Patch Available  (was: Open)

> Cqlsh throws and error when querying a duration data type
> -
>
> Key: CASSANDRA-13549
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13549
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.10 dev environment running on a MacOS Sierra
>Reporter: Akhil Mehra
>Assignee: Akhil Mehra
>
> h3. Overview
> Querying duration related data from the cqlsh prompt results in an error.
> Consider the following create table and insert statement.
> {code:title=Table and insert statement with duration data 
> type|borderStyle=solid}
> CREATE TABLE duration_test (
>   primary_key text,
>   col20 duration,
>   PRIMARY KEY (primary_key)
> );
> INSERT INTO duration_test (primary_key, col20) VALUES ('primary_key_example', 
> 1y5mo89h4m48s);
> {code}
> On executing a select query on col20 in cqlsh I get an error "Failed to 
> format value '"\x00\xfe\x02GS\xfc\xa5\xc0\x00' : 'ascii' codec can't decode 
> byte 0xfe in position 2: ordinal not in range(128)"
> {code:title=Duration Query|borderStyle=solid}
> Select  col20 from duration_test;
> {code}
> h3. Investigation
> On investigating this further I found that the current python Cassandra 
> driver used found in 
> lib/cassandra-driver-internal-only-3.7.0.post0-2481531.zip does not seem to 
> support duration data type. This was added in Jan this year 
> https://github.com/datastax/python-driver/pull/689.
> So I downloaded the latest driver release 
> https://github.com/datastax/python-driver/releases/tag/3.9.0. I embedded the 
> latest driver into cassandra-driver-internal-only-3.7.0.post0-2481531.zip. 
> This fixed the driver related issue but there was still a formatting issue. 
> I then went on to modify the format_value_duration methos in the 
> pylib/cqlshlib/formatting.py. Diff posted below
> {code}
>  @formatter_for('Duration')
>  def format_value_duration(val, colormap, **_):
> -buf = six.iterbytes(val)
> -months = decode_vint(buf)
> -days = decode_vint(buf)
> -nanoseconds = decode_vint(buf)
> -return format_python_formatted_type(duration_as_str(months, days, 
> nanoseconds), colormap, 'duration')
> +return format_python_formatted_type(duration_as_str(val.months, 
> val.days, val.nanoseconds), colormap, 'duration')
> {code}
> This resulted in fixing the issue and duration types are now correctly 
> displayed.
> Happy to fix the issue if I can get some guidance on:
> # If this is a valid issue. Tried searching JIRA but did not find anything 
> reported. 
> # If my assumptions are correct i.e. this is actually a bug
> # how to package the new driver into the source code. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-13549) Cqlsh throws and error when querying a duration data type

2017-05-29 Thread Benjamin Lerer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer reassigned CASSANDRA-13549:
--

Assignee: Akhil Mehra  (was: Benjamin Lerer)

> Cqlsh throws and error when querying a duration data type
> -
>
> Key: CASSANDRA-13549
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13549
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.10 dev environment running on a MacOS Sierra
>Reporter: Akhil Mehra
>Assignee: Akhil Mehra
>
> h3. Overview
> Querying duration related data from the cqlsh prompt results in an error.
> Consider the following create table and insert statement.
> {code:title=Table and insert statement with duration data 
> type|borderStyle=solid}
> CREATE TABLE duration_test (
>   primary_key text,
>   col20 duration,
>   PRIMARY KEY (primary_key)
> );
> INSERT INTO duration_test (primary_key, col20) VALUES ('primary_key_example', 
> 1y5mo89h4m48s);
> {code}
> On executing a select query on col20 in cqlsh I get an error "Failed to 
> format value '"\x00\xfe\x02GS\xfc\xa5\xc0\x00' : 'ascii' codec can't decode 
> byte 0xfe in position 2: ordinal not in range(128)"
> {code:title=Duration Query|borderStyle=solid}
> Select  col20 from duration_test;
> {code}
> h3. Investigation
> On investigating this further I found that the current python Cassandra 
> driver used found in 
> lib/cassandra-driver-internal-only-3.7.0.post0-2481531.zip does not seem to 
> support duration data type. This was added in Jan this year 
> https://github.com/datastax/python-driver/pull/689.
> So I downloaded the latest driver release 
> https://github.com/datastax/python-driver/releases/tag/3.9.0. I embedded the 
> latest driver into cassandra-driver-internal-only-3.7.0.post0-2481531.zip. 
> This fixed the driver related issue but there was still a formatting issue. 
> I then went on to modify the format_value_duration methos in the 
> pylib/cqlshlib/formatting.py. Diff posted below
> {code}
>  @formatter_for('Duration')
>  def format_value_duration(val, colormap, **_):
> -buf = six.iterbytes(val)
> -months = decode_vint(buf)
> -days = decode_vint(buf)
> -nanoseconds = decode_vint(buf)
> -return format_python_formatted_type(duration_as_str(months, days, 
> nanoseconds), colormap, 'duration')
> +return format_python_formatted_type(duration_as_str(val.months, 
> val.days, val.nanoseconds), colormap, 'duration')
> {code}
> This resulted in fixing the issue and duration types are now correctly 
> displayed.
> Happy to fix the issue if I can get some guidance on:
> # If this is a valid issue. Tried searching JIRA but did not find anything 
> reported. 
> # If my assumptions are correct i.e. this is actually a bug
> # how to package the new driver into the source code. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-12922) Bloom filter miss counts are not measured correctly

2017-05-29 Thread Benjamin Lerer (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028387#comment-16028387
 ] 

Benjamin Lerer commented on CASSANDRA-12922:


[~mahdi] Are you still working on that issue or should we reassign it?

> Bloom filter miss counts are not measured correctly
> ---
>
> Key: CASSANDRA-12922
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12922
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local Write-Read Paths
>Reporter: Branimir Lambov
>Assignee: Mahdi Mohammadi
>  Labels: lhf
>
> Bloom filter hits and misses are evaluated incorrectly in 
> {{BigTableReader.getPosition}}: we properly record hits, but not misses. In 
> particular, if we don't find a match for a key in the index, which is where 
> almost all non-matches will be rejected, [we don't record a bloom filter 
> false 
> positive|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java#L228].
> This leads to very misleading output from e.g. {{nodetool tablestats}}.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13095) Timeouts between nodes

2017-05-29 Thread Benjamin Lerer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13095:
---
Assignee: Danil Smirnov
  Status: Patch Available  (was: Open)

> Timeouts between nodes
> --
>
> Key: CASSANDRA-13095
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13095
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Danil Smirnov
>Assignee: Danil Smirnov
>Priority: Minor
> Attachments: 13095-2.1.patch
>
>
> Recently I've run into a problem with heavily loaded cluster when sometimes 
> messages between certain nodes become blocked with no reason.
> It looks like the same situation that described here 
> https://issues.apache.org/jira/browse/CASSANDRA-12676?focusedCommentId=15736166=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15736166
> Thread dump showed infinite loop here: 
> https://github.com/apache/cassandra/blob/a8a43dd32eb92406d7d8b105e08c68b3d5c7df49/src/java/org/apache/cassandra/utils/CoalescingStrategies.java#L109
> Apparently the problem is in the initial value of epoch filed in 
> TimeHorizonMovingAverageCoalescingStrategy class. When it's value is not 
> evenly divisible by BUCKET_INTERVAL, ix(epoch-1) does not point to the 
> correct bucket. As a result, sum gradually increases and, upon reaching 
> MEASURED_INTERVAL, averageGap becomes 0 and thread blocks.
> It's hard to reproduce because it takes a long time for sum to grow and when 
> no messages are send for some time, sum becomes 0 
> https://github.com/apache/cassandra/blob/a8a43dd32eb92406d7d8b105e08c68b3d5c7df49/src/java/org/apache/cassandra/utils/CoalescingStrategies.java#L301
>  and bug is no longer reproducible (until connection between nodes is 
> re-created).
> I've added a patch which should fix the problem. Don't know if it would be of 
> any help since CASSANDRA-12676 will apparently disable this behaviour. One 
> note about performance regressions though. There is a small chance it being 
> result of the bug described here, so it might be worth testing performance 
> after fixes and/or tuning the algorithm.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13067) Integer overflows with file system size reported by Amazon Elastic File System (EFS)

2017-05-29 Thread Benjamin Lerer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13067:
---
Reviewer: Benjamin Lerer

> Integer overflows with file system size reported by Amazon Elastic File 
> System (EFS)
> 
>
> Key: CASSANDRA-13067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13067
> Project: Cassandra
>  Issue Type: Bug
> Environment: Cassandra in OpenShift running on Amazon EC2 instance 
> with EFS mounted for data
>Reporter: Michael Hanselmann
>Assignee: Matt Wringe
> Attachments: 0001-Handle-exabyte-sized-filesystems.patch
>
>
> When not explicitly configured Cassandra uses 
> [{{nio.FileStore.getTotalSpace}}|https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileStore.html]
>  to determine the total amount of available space in order to [calculate the 
> preferred commit log 
> size|https://github.com/apache/cassandra/blob/cassandra-3.9/src/java/org/apache/cassandra/config/DatabaseDescriptor.java#L553].
>  [Amazon EFS|https://aws.amazon.com/efs/] instances report a filesystem size 
> of 8 EiB when empty. [{{getTotalSpace}} causes an integer overflow 
> (JDK-8162520)|https://bugs.openjdk.java.net/browse/JDK-8162520] and returns a 
> negative number, resulting in a negative preferred size and causing the 
> checked integer to throw.
> Overriding {{commitlog_total_space_in_mb}} is not sufficient as 
> [{{DataDirectory.getAvailableSpace}}|https://github.com/apache/cassandra/blob/cassandra-3.9/src/java/org/apache/cassandra/db/Directories.java#L550]
>  makes use of {{nio.FileStore.getUsableSpace}}.
> [AMQ-6441] is a comparable issue in ActiveMQ.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13047) Point cqlsh help to the new doc

2017-05-29 Thread Benjamin Lerer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13047:
---
Component/s: CQL

> Point cqlsh help to the new doc
> ---
>
> Key: CASSANDRA-13047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13047
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
>Reporter: Sylvain Lebresne
> Fix For: 4.0
>
>
> Cqlsh still points to the "old" textile CQL doc, but that's not really 
> maintain anymore now that we have the new doc (which include everything the 
> old doc had and more). We should update cqlsh to point to the new doc so we 
> can remove the old one completely.
> Any takers?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13554) SASI Index not working with Secondary Index

2017-05-29 Thread Joshua McKenzie (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joshua McKenzie updated CASSANDRA-13554:

Priority: Major  (was: Critical)

> SASI Index not working with Secondary Index
> ---
>
> Key: CASSANDRA-13554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13554
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Karan
>
> I have a table with 1 SASI Index and 1 secondary index. If using both the 
> indices in the same query, the query doesn't return any data. 
> However, when changing the secondary index into SASI Index and using both the 
> indices in the same query, expected data is returned by the query.
> Not sure if this is the expected behaviour or a bug.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13554) SASI Index not working with Secondary Index

2017-05-29 Thread Joshua McKenzie (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028328#comment-16028328
 ] 

Joshua McKenzie commented on CASSANDRA-13554:
-

Dropping priority as SASI is a "beta" feature / custom index, and we expect 
there to be bugs in it in its current form.

> SASI Index not working with Secondary Index
> ---
>
> Key: CASSANDRA-13554
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13554
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Karan
>
> I have a table with 1 SASI Index and 1 secondary index. If using both the 
> indices in the same query, the query doesn't return any data. 
> However, when changing the secondary index into SASI Index and using both the 
> indices in the same query, expected data is returned by the query.
> Not sure if this is the expected behaviour or a bug.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-13072) Cassandra failed to run on Linux-aarch64

2017-05-29 Thread Benjamin Lerer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer reassigned CASSANDRA-13072:
--

Assignee: Benjamin Lerer

> Cassandra failed to run on Linux-aarch64
> 
>
> Key: CASSANDRA-13072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13072
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Hardware: ARM aarch64
> OS: Ubuntu 16.04.1 LTS
>Reporter: Jun He
>Assignee: Benjamin Lerer
>  Labels: incompatible
> Attachments: compat_report.html
>
>
> Steps to reproduce:
> 1. Download cassandra latest source
> 2. Build it with "ant"
> 3. Run with "./bin/cassandra". Daemon is crashed with following error message:
> {quote}
> INFO  05:30:21 Initializing system.schema_functions
> INFO  05:30:21 Initializing system.schema_aggregates
> ERROR 05:30:22 Exception in thread Thread[MemtableFlushWriter:1,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
> at 
> org.apache.cassandra.utils.memory.MemoryUtil.allocate(MemoryUtil.java:97) 
> ~[main/:na]
> at org.apache.cassandra.io.util.Memory.(Memory.java:74) 
> ~[main/:na]
> at org.apache.cassandra.io.util.SafeMemory.(SafeMemory.java:32) 
> ~[main/:na]
> at 
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.(CompressionMetadata.java:316)
>  ~[main/:na]
> at 
> org.apache.cassandra.io.compress.CompressionMetadata$Writer.open(CompressionMetadata.java:330)
>  ~[main/:na]
> at 
> org.apache.cassandra.io.compress.CompressedSequentialWriter.(CompressedSequentialWriter.java:76)
>  ~[main/:na]
> at 
> org.apache.cassandra.io.util.SequentialWriter.open(SequentialWriter.java:163) 
> ~[main/:na]
> at 
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.(BigTableWriter.java:73)
>  ~[main/:na]
> at 
> org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:93)
>  ~[main/:na]
> at 
> org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:96)
>  ~[main/:na]
> at 
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:114)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:519)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:497)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:480)
>  ~[main/:na]
> at 
> org.apache.cassandra.db.Memtable.createFlushWriter(Memtable.java:439) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:371) 
> ~[main/:na]
> at org.apache.cassandra.db.Memtable.flush(Memtable.java:332) 
> ~[main/:na]
> at 
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
>  ~[main/:na]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  ~[na:1.8.0_111]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  ~[na:1.8.0_111]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
> {quote}
> Analyze:
> This issue is caused by bundled jna-4.0.0.jar which doesn't come with aarch64 
> native support. Replace lib/jna-4.0.0.jar with jna-4.2.0.jar from 
> http://central.maven.org/maven2/net/java/dev/jna/jna/4.2.0/ can fix this 
> problem.
> Attached is the binary compatibility report of jna.jar between 4.0 and 4.2. 
> The result is good (97.4%). So is there possibility to upgrade jna to 4.2.0 
> in upstream? Should there be any kind of tests to execute, please kindly 
> point me. Thanks a lot.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13149) AssertionError prepending to a list

2017-05-29 Thread Benjamin Lerer (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028270#comment-16028270
 ] 

Benjamin Lerer commented on CASSANDRA-13149:


I had a quick look at the code and it seems to me that the code can trigger the 
assertion even with  {{System#currentTimeMillis}} being monotonic.
As the time is being retrieved before the start of the loop adding the values 
[here| 
https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/cql3/Lists.java#L425]
 another query might end up being faster and change the last time before all 
the value got added.
Even if the {{PrecisionTime#getNext}} method was used only once per update we 
could have a race condition and trigger the assertion.

The best solution would be in my opinion to avoid sharing {{PrecisionTime}} and 
create one instance per operation.

[~slebresne] does my analysis and my proposed solution make sense to you?

> AssertionError prepending to a list
> ---
>
> Key: CASSANDRA-13149
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13149
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: 3.0.8
>Reporter: Steven Warren
>
> Prepending to a list produces the following AssertionError randomly. Changing 
> the update to append (and sort in the client) works around the issue.
> {code}
> java.lang.AssertionError: null
>   at 
> org.apache.cassandra.cql3.Lists$PrecisionTime.getNext(Lists.java:275) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.cql3.Lists$Prepender.execute(Lists.java:430) 
> ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.UpdateStatement.addUpdateForKey(UpdateStatement.java:94)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:682)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:613)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:420)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:408)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:487)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:464)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:130)
>  ~[apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:507)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:401)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at 
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
>  [netty-all-4.0.23.Final.jar:4.0.23.Final]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_101]
>   at 
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
>  [apache-cassandra-3.0.8.jar:3.0.8]
>   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
> [apache-cassandra-3.0.8.jar:3.0.8]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-13004) Corruption while adding a column to a table

2017-05-29 Thread Benjamin Lerer (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-13004:
---
Component/s: Distributed Metadata

> Corruption while adding a column to a table
> ---
>
> Key: CASSANDRA-13004
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13004
> Project: Cassandra
>  Issue Type: Bug
>  Components: Distributed Metadata
>Reporter: Stanislav Vishnevskiy
>Priority: Blocker
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> We had the following schema in production. 
> {code:none}
> CREATE TYPE IF NOT EXISTS discord_channels.channel_recipient (
> nick text
> );
> CREATE TYPE IF NOT EXISTS discord_channels.channel_permission_overwrite (
> id bigint,
> type int,
> allow_ int,
> deny int
> );
> CREATE TABLE IF NOT EXISTS discord_channels.channels (
> id bigint,
> guild_id bigint,
> type tinyint,
> name text,
> topic text,
> position int,
> owner_id bigint,
> icon_hash text,
> recipients map,
> permission_overwrites map,
> bitrate int,
> user_limit int,
> last_pin_timestamp timestamp,
> last_message_id bigint,
> PRIMARY KEY (id)
> );
> {code}
> And then we executed the following alter.
> {code:none}
> ALTER TABLE discord_channels.channels ADD application_id bigint;
> {code}
> And one row (that we can tell) got corrupted at the same time and could no 
> longer be read from the Python driver. 
> {code:none}
> [E 161206 01:56:58 geventreactor:141] Error decoding response from Cassandra. 
> ver(4); flags(); stream(27); op(8); offset(9); len(887); buffer: 
> '\x84\x00\x00\x1b\x08\x00\x00\x03w\x00\x00\x00\x02\x00\x00\x00\x01\x00\x00\x00\x0f\x00\x10discord_channels\x00\x08channels\x00\x02id\x00\x02\x00\x0eapplication_id\x00\x02\x00\x07bitrate\x00\t\x00\x08guild_id\x00\x02\x00\ticon_hash\x00\r\x00\x0flast_message_id\x00\x02\x00\x12last_pin_timestamp\x00\x0b\x00\x04name\x00\r\x00\x08owner_id\x00\x02\x00\x15permission_overwrites\x00!\x00\x02\x000\x00\x10discord_channels\x00\x1cchannel_permission_overwrite\x00\x04\x00\x02id\x00\x02\x00\x04type\x00\t\x00\x06allow_\x00\t\x00\x04deny\x00\t\x00\x08position\x00\t\x00\nrecipients\x00!\x00\x02\x000\x00\x10discord_channels\x00\x11channel_recipient\x00\x01\x00\x04nick\x00\r\x00\x05topic\x00\r\x00\x04type\x00\x14\x00\nuser_limit\x00\t\x00\x00\x00\x01\x00\x00\x00\x08\x03\x8a\x19\x8e\xf8\x82\x00\x01\xff\xff\xff\xff\x00\x00\x00\x04\x00\x00\xfa\x00\x00\x00\x00\x08\x00\x00\xfa\x00\x00\xf8G\xc5\x00\x00\x00\x00\x00\x00\x00\x08\x03\x8b\xc0\xb5nB\x00\x02\x00\x00\x00\x08G\xc5\xffI\x98\xc4\xb4(\x00\x00\x00\x03\x8b\xc0\xa8\xff\xff\xff\xff\x00\x00\x01<\x00\x00\x00\x06\x00\x00\x00\x08\x03\x81L\xea\xfc\x82\x00\n\x00\x00\x00$\x00\x00\x00\x08\x03\x81L\xea\xfc\x82\x00\n\x00\x00\x00\x04\x00\x00\x00\x01\x00\x00\x00\x04\x00\x00\x08\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x08\x03\x8a\x1e\xe6\x8b\x80\x00\n\x00\x00\x00$\x00\x00\x00\x08\x03\x8a\x1e\xe6\x8b\x80\x00\n\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x040\x07\xf8Q\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x08\x03\x8a\x1f\x1b{\x82\x00\x00\x00\x00\x00$\x00\x00\x00\x08\x03\x8a\x1f\x1b{\x82\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x04\x00\x07\xf8Q\x00\x00\x00\x04\x10\x00\x00\x00\x00\x00\x00\x08\x03\x8a\x1fH6\x82\x00\x01\x00\x00\x00$\x00\x00\x00\x08\x03\x8a\x1fH6\x82\x00\x01\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x04\x00\x05\xe8A\x00\x00\x00\x04\x10\x02\x00\x00\x00\x00\x00\x08\x03\x8a+=\xca\xc0\x00\n\x00\x00\x00$\x00\x00\x00\x08\x03\x8a+=\xca\xc0\x00\n\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x08\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x08\x03\x8a\x8f\x979\x80\x00\n\x00\x00\x00$\x00\x00\x00\x08\x03\x8a\x8f\x979\x80\x00\n\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x04\x00
>  
> \x08\x01\x00\x00\x00\x04\xc4\xb4(\x00\xff\xff\xff\xff\x00\x00\x00O[f\x80Q\x07general\x05\xf8G\xc5\xffI\x98\xc4\xb4(\x00\xf8O[f\x80Q\x00\x00\x00\x02\x04\xf8O[f\x80Q\x00\xf8G\xc5\xffI\x98\x01\x00\x00\xf8O[f\x80Q\x00\x00\x00\x00\xf8G\xc5\xffI\x97\xc4\xb4(\x06\x00\xf8O\x7fe\x1fm\x08\x03\x00\x00\x00\x01\x00\x00\x00\x00\x04\x00\x00\x00\x00'
> {code}
> And then in cqlsh when trying to read the row we got this. 
> {code:none}
> /usr/bin/cqlsh.py:632: DateOverFlowWarning: Some timestamps are larger than 
> Python datetime can represent. Timestamps are displayed in milliseconds from 
> epoch.
> Traceback (most recent call last):
>   File "/usr/bin/cqlsh.py", line 1301, in perform_simple_statement
> result = future.result()
>   File 
> "/usr/share/cassandra/lib/cassandra-driver-internal-only-3.5.0.post0-d8d0456.zip/cassandra-driver-3.5.0.post0-d8d0456/cassandra/cluster.py",
>  line 3650, in result
> raise self._final_exception
>

[jira] [Assigned] (CASSANDRA-12368) Background read repair not fixing replicas

2017-05-29 Thread Alex Petrov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov reassigned CASSANDRA-12368:
---

Assignee: Alex Petrov

> Background read repair not fixing replicas
> --
>
> Key: CASSANDRA-12368
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12368
> Project: Cassandra
>  Issue Type: Bug
>  Components: Coordination, Local Write-Read Paths
>Reporter: Paulo Motta
>Assignee: Alex Petrov
> Fix For: 3.11.x
>
>
> On 3.x background read repair is detecting mismatch and performing background 
> reads, but for some reason it's not actually repairing replicas. The problem 
> apparently does not occur on 2.2-.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-13543) Cassandra SASI index gives unexpected number of results

2017-05-29 Thread Alex Petrov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov reassigned CASSANDRA-13543:
---

Assignee: Alex Petrov

> Cassandra SASI index gives unexpected number of results
> ---
>
> Key: CASSANDRA-13543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13543
> Project: Cassandra
>  Issue Type: Bug
>  Components: sasi
>Reporter: Alexander Nabatchikov
>Assignee: Alex Petrov
>
> I've faced the issue with LIKE query to the column indexed by SASI index. 
> Cassandra can return different number of rows when the data stays immutable.
> {code}
> CREATE TABLE idx_test
> (
>   id int,
>   str text,
>   i int,
>   PRIMARY KEY (id)
> );
> CREATE CUSTOM INDEX idx_test_idx ON idx_test (str)
> USING 'org.apache.cassandra.index.sasi.SASIIndex'
> WITH OPTIONS = { 
>   'mode': 'CONTAINS',
>   'analyzer_class': 
> 'org.apache.cassandra.index.sasi.analyzer.StandardAnalyzer',
>   'tokenization_enable_stemming': 'true',
>   'tokenization_normalize_lowercase': 'true'
> };
> INSERT INTO idx_test (id, str, i) VALUES (1, 'a b c d', 10);
> INSERT INTO idx_test (id, str, i) VALUES (2, 'a b c d', 10);
> INSERT INTO idx_test (id, str, i) VALUES (3, 'a b c d', 10);
> INSERT INTO idx_test (id, str, i) VALUES (4, 'a b c d', 10);
> INSERT INTO idx_test (id, str, i) VALUES (5, 'a b c d', 10);
> INSERT INTO idx_test (id, str, i) VALUES (6, 'a b c d', 10);
> INSERT INTO idx_test (id, str, i) VALUES (7, 'a b c d', 10);
> INSERT INTO idx_test (id, str, i) VALUES (8, 'a b c d', 10);
> INSERT INTO idx_test (id, str, i) VALUES (9, 'a b c d', 10);
> INSERT INTO idx_test (id, str, i) VALUES (10, 'a b c d', 10);
> {code}
> Query:
> {code}
> SELECT * FROM idx_test WHERE str LIKE 'b' 
> AND i = 10
> ALLOW FILTERING;
> {code}
> This query mostly returns 0 rows, but sometimes 1 row appears in result row 
> set as:
> {code}
> id |  i  |  str
> 10 |  10 |  a b c d
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-12435) Cassandra timeout when using secondary index in cluster mode.

2017-05-29 Thread Alex Petrov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-12435:

Description: 
I have a 3-node cluster with a keyspace with RF=3, then I created a table like 
below and inserted some data(about 10MB).

{code}
CREATE TABLE vehicle_fuzzy_plate (
day int,
hour int,
repo int,
sensor_id int,
ts timestamp,
id timeuuid,
plate_text_pattern_70 text,
plate_text_pattern_10 text,
plate_text_pattern_11 text,
plate_text_pattern_12 text,
plate_text_pattern_13 text,
plate_text_pattern_14 text,
plate_text_pattern_15 text,
plate_text_pattern_16 text,
plate_text_pattern_20 text,
plate_text_pattern_21 text,
plate_text_pattern_22 text,
plate_text_pattern_23 text,
plate_text_pattern_24 text,
plate_text_pattern_25 text,
plate_text_pattern_30 text,
plate_text_pattern_31 text,
plate_text_pattern_32 text,
plate_text_pattern_33 text,
plate_text_pattern_34 text,
plate_text_pattern_40 text,
plate_text_pattern_41 text,
plate_text_pattern_42 text,
plate_text_pattern_43 text,
plate_text_pattern_50 text,
plate_text_pattern_51 text,
plate_text_pattern_52 text,
plate_text_pattern_60 text,
plate_text_pattern_61 text,
PRIMARY KEY (day, repo, sensor_id, ts, id)
) WITH CLUSTERING ORDER BY (repo ASC, sensor_id ASC, ts ASC, id ASC);
CREATE INDEX plate_text_pattern_10_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_10);
CREATE INDEX plate_text_pattern_11_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_11);
CREATE INDEX plate_text_pattern_12_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_12);
CREATE INDEX plate_text_pattern_13_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_13);
CREATE INDEX plate_text_pattern_14_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_14);
CREATE INDEX plate_text_pattern_15_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_15);
CREATE INDEX plate_text_pattern_16_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_16);
CREATE INDEX plate_text_pattern_20_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_20);
CREATE INDEX plate_text_pattern_21_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_21);
CREATE INDEX plate_text_pattern_22_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_22);
CREATE INDEX plate_text_pattern_23_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_23);
CREATE INDEX plate_text_pattern_24_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_24);
CREATE INDEX plate_text_pattern_25_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_25);
CREATE INDEX plate_text_pattern_30_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_30);
CREATE INDEX plate_text_pattern_31_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_31);
CREATE INDEX plate_text_pattern_32_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_32);
CREATE INDEX plate_text_pattern_33_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_33);
CREATE INDEX plate_text_pattern_34_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_34);
CREATE INDEX plate_text_pattern_40_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_40);
CREATE INDEX plate_text_pattern_41_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_41);
CREATE INDEX plate_text_pattern_42_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_42);
CREATE INDEX plate_text_pattern_43_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_43);
CREATE INDEX plate_text_pattern_50_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_50);
CREATE INDEX plate_text_pattern_51_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_51);
CREATE INDEX plate_text_pattern_52_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_52);
CREATE INDEX plate_text_pattern_60_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_60);
CREATE INDEX plate_text_pattern_61_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_61);
CREATE INDEX plate_text_pattern_70_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_70);
{code}

After that, I send some queries through Java driver with Consistency=Quorum one 
by one, for example,

{code}
SELECT id,sensor_id,ts FROM vehicle_fuzzy_plate WHERE day IN 
(16993,16994,16995,16996,16997,16998,16999,17000,17001,17002,17003,17004,17005,17006,17007,17008,17009,17010,17011,17012,17013,17014,17015,17016,17017,17018,17019,17020,17021,17022,17023)
 AND repo IN (234) AND sensor_id IN (123,16) AND ts>=1468226847000 AND 
ts<=1470818847000 AND plate_text_pattern_70 IN ('70贵JBP378') ALLOW FILTERING;
{code}

The weird part is that some queries will response timeout (about 30%). I 
checked the debug.log and found these exception,

{code}
DEBUG [SharedPool-Worker-1] 2016-08-11 14:09:00,885 ReadCallback.java:126 - 
Timed out; received 1 of 2 responses (including data)
ERROR [MessagingService-Incoming-/192.168.2.122] 2016-08-11 14:09:05,954 
CassandraDaemon.java:217 - Exception in thread 
Thread[MessagingService-Incoming-/192.168.2.122,5,main]
java.lang.ArrayIndexOutOfBoundsException: 74
at

[jira] [Updated] (CASSANDRA-12435) Cassandra timeout when using secondary index in cluster mode.

2017-05-29 Thread Alex Petrov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov updated CASSANDRA-12435:

Description: 
I have a 3-node cluster with a keyspace with RF=3, then I created a table like 
below and inserted some data(about 10MB).

{code{
CREATE TABLE vehicle_fuzzy_plate (
day int,
hour int,
repo int,
sensor_id int,
ts timestamp,
id timeuuid,
plate_text_pattern_70 text,
plate_text_pattern_10 text,
plate_text_pattern_11 text,
plate_text_pattern_12 text,
plate_text_pattern_13 text,
plate_text_pattern_14 text,
plate_text_pattern_15 text,
plate_text_pattern_16 text,
plate_text_pattern_20 text,
plate_text_pattern_21 text,
plate_text_pattern_22 text,
plate_text_pattern_23 text,
plate_text_pattern_24 text,
plate_text_pattern_25 text,
plate_text_pattern_30 text,
plate_text_pattern_31 text,
plate_text_pattern_32 text,
plate_text_pattern_33 text,
plate_text_pattern_34 text,
plate_text_pattern_40 text,
plate_text_pattern_41 text,
plate_text_pattern_42 text,
plate_text_pattern_43 text,
plate_text_pattern_50 text,
plate_text_pattern_51 text,
plate_text_pattern_52 text,
plate_text_pattern_60 text,
plate_text_pattern_61 text,
PRIMARY KEY (day, repo, sensor_id, ts, id)
) WITH CLUSTERING ORDER BY (repo ASC, sensor_id ASC, ts ASC, id ASC);
CREATE INDEX plate_text_pattern_10_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_10);
CREATE INDEX plate_text_pattern_11_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_11);
CREATE INDEX plate_text_pattern_12_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_12);
CREATE INDEX plate_text_pattern_13_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_13);
CREATE INDEX plate_text_pattern_14_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_14);
CREATE INDEX plate_text_pattern_15_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_15);
CREATE INDEX plate_text_pattern_16_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_16);
CREATE INDEX plate_text_pattern_20_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_20);
CREATE INDEX plate_text_pattern_21_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_21);
CREATE INDEX plate_text_pattern_22_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_22);
CREATE INDEX plate_text_pattern_23_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_23);
CREATE INDEX plate_text_pattern_24_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_24);
CREATE INDEX plate_text_pattern_25_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_25);
CREATE INDEX plate_text_pattern_30_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_30);
CREATE INDEX plate_text_pattern_31_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_31);
CREATE INDEX plate_text_pattern_32_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_32);
CREATE INDEX plate_text_pattern_33_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_33);
CREATE INDEX plate_text_pattern_34_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_34);
CREATE INDEX plate_text_pattern_40_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_40);
CREATE INDEX plate_text_pattern_41_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_41);
CREATE INDEX plate_text_pattern_42_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_42);
CREATE INDEX plate_text_pattern_43_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_43);
CREATE INDEX plate_text_pattern_50_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_50);
CREATE INDEX plate_text_pattern_51_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_51);
CREATE INDEX plate_text_pattern_52_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_52);
CREATE INDEX plate_text_pattern_60_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_60);
CREATE INDEX plate_text_pattern_61_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_61);
CREATE INDEX plate_text_pattern_70_idx ON vehicle_fuzzy_plate 
(plate_text_pattern_70);
{code}

After that, I send some queries through Java driver with Consistency=Quorum one 
by one, for example,

{code}
SELECT id,sensor_id,ts FROM vehicle_fuzzy_plate WHERE day IN 
(16993,16994,16995,16996,16997,16998,16999,17000,17001,17002,17003,17004,17005,17006,17007,17008,17009,17010,17011,17012,17013,17014,17015,17016,17017,17018,17019,17020,17021,17022,17023)
 AND repo IN (234) AND sensor_id IN (123,16) AND ts>=1468226847000 AND 
ts<=1470818847000 AND plate_text_pattern_70 IN ('70贵JBP378') ALLOW FILTERING;
{code}

The weird part is that some queries will response timeout (about 30%). I 
checked the debug.log and found these exception,

{code}
DEBUG [SharedPool-Worker-1] 2016-08-11 14:09:00,885 ReadCallback.java:126 - 
Timed out; received 1 of 2 responses (including data)
ERROR [MessagingService-Incoming-/192.168.2.122] 2016-08-11 14:09:05,954 
CassandraDaemon.java:217 - Exception in thread 
Thread[MessagingService-Incoming-/192.168.2.122,5,main]
java.lang.ArrayIndexOutOfBoundsException: 74
at

[jira] [Commented] (CASSANDRA-8457) nio MessagingService

2017-05-29 Thread Sylvain Lebresne (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028159#comment-16028159
 ] 

Sylvain Lebresne commented on CASSANDRA-8457:
-

Had a quick look at the last changes. A few remaining questions/remarks:
* There is still a few {{TODO:JEB}} in the code: would be good to resolve/clear 
them if we're going to a more final branch.
* MessageOutHandler.AUTO_FLUSH_THRESHOLD feels like a magic constants. At least 
a comment on why it's a reasonable value would be nice.
* Largely a nit, but its a tad confusing that {{OutboundConnectionParams}} 
contains a {{OutboundMessagingConnection}}. It feels like saying "The 
parameters you need for an outbound connection is ... an outbound connection". 
Maybe a simple renaming would make this clearer, though it feels maybe this 
could be cleanup up a bit further.
* {{OutboundHandshakeHandler.WRITE_IDLE_MS}}: what's the rational behind 10 
seconds? Not saying it feels wrong per-se, but wondering if there is more than 
a gut-feeling to it, and I'd kind of suggest exposing a system property to 
change that default.
* The handling of "backlogged" messages and channel writability feels a bit 
complex. For instance, it looks like 
{{MessageOutHandler.channelWritabilityChanged}} can potentially silently drop a 
message every time it's called, and I'll admit not being sure if this can have 
unintened consequence in overload situations. In general, it would be really 
great to have some testing of this patch under a fair amount of load (and 
overload) to make sure things work as intended.
* Nit: there has been a few Netty minor released since the one in the branch, 
maybe worth upgrading (since we change it anyway).

Other than that, and related to one comment above, it would be nice to see some 
tests run for this (including upgrade tests as this is particularly sensible to 
any MessagingService changes). The CI links listed with the branch a bunch of 
comments ago are completely out of dates. Some basic benchmark results would 
hurt either since that completely change a fairly sensible part of the code. Of 
course, it's not that meaningful without CASSANDRA-12229, which makes this a 
bit awkward. Maybe you could create a parent ticket of which this and 
CASSANDRA-12229 would be sub-tasks where we could focus the 
testing/benchmarking/final discussions on the whole thing?


> nio MessagingService
> 
>
> Key: CASSANDRA-8457
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8457
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Jonathan Ellis
>Assignee: Jason Brown
>Priority: Minor
>  Labels: netty, performance
> Fix For: 4.x
>
>
> Thread-per-peer (actually two each incoming and outbound) is a big 
> contributor to context switching, especially for larger clusters.  Let's look 
> at switching to nio, possibly via Netty.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-12561) LCS compaction going into infinite loop due to non-existent sstables

2017-05-29 Thread Alex Petrov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov reassigned CASSANDRA-12561:
---

Assignee: Alex Petrov

> LCS compaction going into infinite loop due to non-existent sstables
> 
>
> Key: CASSANDRA-12561
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12561
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Nimi Wariboko Jr.
>Assignee: Alex Petrov
>  Labels: lcs
>
> I believe this is related/similar to CASSANDRA-11373, but I'm running 3.5 and 
> I still have this issue.
> AFAICT, this happens when getCompactionCandidates in LeveledManifest.java 
> returns a candidate that does not exist on disk. 
> Eventually, all the compaction threads back up, garbage collections start 
> taking an upwards of 20 seconds and messages start being dropped.
> To get around this, I patched my instance with the following code in 
> LeveledManifest.java
> {code:java}
> Set removeCandidates = new HashSet<>();
> for (SSTableReader sstable : candidates)
> {
> if (!(new java.io.File(sstable.getFilename())).exists()) {
> removeCandidates.add(sstable);
> logger.warn("Not compating candidate {} because it does 
> not exist ({}).", sstable.getFilename(), sstable.openReason);
> }
> }
> candidates.removeAll(removeCandidates);
> if (candidates.size() < 2)
> return Collections.emptyList();
> else
> return candidates;
> {code}
> This just removes any candidate that doesn't exist on disk - however I'm not 
> sure what the side effects of this are.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-12734) Materialized View schema file for snapshots created as tables

2017-05-29 Thread Alex Petrov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-12734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Petrov reassigned CASSANDRA-12734:
---

Assignee: Alex Petrov

> Materialized View schema file for snapshots created as tables
> -
>
> Key: CASSANDRA-12734
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12734
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tools
>Reporter: Hau Phan
>Assignee: Alex Petrov
> Fix For: 3.0.9
>
>
> The materialized view schema file that gets created and stored with the 
> sstables is created as a table instead of a materialized view.  
> Can the materialized view be created and added to the corresponding table's  
> schema file?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13549) Cqlsh throws and error when querying a duration data type

2017-05-29 Thread Benjamin Lerer (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028135#comment-16028135
 ] 

Benjamin Lerer commented on CASSANDRA-13549:


Thanks for the patches :-)

bq. If you want me to drop it back to 3.9.0 python driver please let me know. 

According to the driver 
[CHANGELOGS|https://github.com/datastax/python-driver/blob/master/CHANGELOG.rst]
 the support for the {{Duration}} type has been added to the {{3.10}} version 
so we should use that one. It also not make a lot of sense to build a 
pre-release of {{3.11}}.

bq. Is there any way I can run the dtests on these two branches on 
http://cassci.datastax.com/ ?

{{cassci}} will soon be shutdown so we should not run build there anymore. I 
will run the tests for the 2 branches on our internal CI.

  

> Cqlsh throws and error when querying a duration data type
> -
>
> Key: CASSANDRA-13549
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13549
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL
> Environment: Cassandra 3.10 dev environment running on a MacOS Sierra
>Reporter: Akhil Mehra
>Assignee: Benjamin Lerer
>
> h3. Overview
> Querying duration related data from the cqlsh prompt results in an error.
> Consider the following create table and insert statement.
> {code:title=Table and insert statement with duration data 
> type|borderStyle=solid}
> CREATE TABLE duration_test (
>   primary_key text,
>   col20 duration,
>   PRIMARY KEY (primary_key)
> );
> INSERT INTO duration_test (primary_key, col20) VALUES ('primary_key_example', 
> 1y5mo89h4m48s);
> {code}
> On executing a select query on col20 in cqlsh I get an error "Failed to 
> format value '"\x00\xfe\x02GS\xfc\xa5\xc0\x00' : 'ascii' codec can't decode 
> byte 0xfe in position 2: ordinal not in range(128)"
> {code:title=Duration Query|borderStyle=solid}
> Select  col20 from duration_test;
> {code}
> h3. Investigation
> On investigating this further I found that the current python Cassandra 
> driver used found in 
> lib/cassandra-driver-internal-only-3.7.0.post0-2481531.zip does not seem to 
> support duration data type. This was added in Jan this year 
> https://github.com/datastax/python-driver/pull/689.
> So I downloaded the latest driver release 
> https://github.com/datastax/python-driver/releases/tag/3.9.0. I embedded the 
> latest driver into cassandra-driver-internal-only-3.7.0.post0-2481531.zip. 
> This fixed the driver related issue but there was still a formatting issue. 
> I then went on to modify the format_value_duration methos in the 
> pylib/cqlshlib/formatting.py. Diff posted below
> {code}
>  @formatter_for('Duration')
>  def format_value_duration(val, colormap, **_):
> -buf = six.iterbytes(val)
> -months = decode_vint(buf)
> -days = decode_vint(buf)
> -nanoseconds = decode_vint(buf)
> -return format_python_formatted_type(duration_as_str(months, days, 
> nanoseconds), colormap, 'duration')
> +return format_python_formatted_type(duration_as_str(val.months, 
> val.days, val.nanoseconds), colormap, 'duration')
> {code}
> This resulted in fixing the issue and duration types are now correctly 
> displayed.
> Happy to fix the issue if I can get some guidance on:
> # If this is a valid issue. Tried searching JIRA but did not find anything 
> reported. 
> # If my assumptions are correct i.e. this is actually a bug
> # how to package the new driver into the source code. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-8272) 2ndary indexes can return stale data

2017-05-29 Thread Sylvain Lebresne (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028128#comment-16028128
 ] 

Sylvain Lebresne commented on CASSANDRA-8272:
-

Yes, that's generally what I had in mind, thanks!

For the row-filter aware counter however, I don't think we can aford to have it 
not be a stopping transformation: we very much rely on that stopping to not OOM 
nodes (and generally read the whole database on a read), whether it be for user 
limits or paging. I'm not sure I understand why stopping it a concern in this 
case however?

As an aside, had a very very quick scan of the patch, and I'll also note that 
in {{StorageProxy}} and {{SinglePartitionReadCommand.Group.executeInternal}}, 
using only the post-processor of the 1st command would break if the index 
actually makes assumption based on the command it's passed on, so it feels 
dodgy and I think we sould make sure it's applied to each command result 
individually.


> 2ndary indexes can return stale data
> 
>
> Key: CASSANDRA-8272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8272
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Andrés de la Peña
> Fix For: 3.0.x
>
>
> When replica return 2ndary index results, it's possible for a single replica 
> to return a stale result and that result will be sent back to the user, 
> potentially failing the CL contract.
> For instance, consider 3 replicas A, B and C, and the following situation:
> {noformat}
> CREATE TABLE test (k int PRIMARY KEY, v text);
> CREATE INDEX ON test(v);
> INSERT INTO test(k, v) VALUES (0, 'foo');
> {noformat}
> with every replica up to date. Now, suppose that the following queries are 
> done at {{QUORUM}}:
> {noformat}
> UPDATE test SET v = 'bar' WHERE k = 0;
> SELECT * FROM test WHERE v = 'foo';
> {noformat}
> then, if A and B acknowledge the insert but C respond to the read before 
> having applied the insert, then the now stale result will be returned (since 
> C will return it and A or B will return nothing).
> A potential solution would be that when we read a tombstone in the index (and 
> provided we make the index inherit the gcGrace of it's parent CF), instead of 
> skipping that tombstone, we'd insert in the result a corresponding range 
> tombstone.  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-7396) Allow selecting Map key, List index

2017-05-29 Thread Benjamin Lerer (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028127#comment-16028127
 ] 

Benjamin Lerer commented on CASSANDRA-7396:
---

Forbidding these selectors is, in my opinion, different from what is done for 
clustering keys and inequality predicates. Before to {{3.0}} we were preventing 
any clustering restrictions after an inequality predicate, simply because we 
had not put in place a mechanism to handle those queries. Since {{3.0}}, we 
allow them as long as {{ALLOW FILTERING}} is specified.

CQL allow things like {{SELECT v + 1 - 1 FROM myTable}}, even if it is 
inefficient. One of the problem of CQL is that it has a lot of corner cases, 
that makes the language difficult to use and I have seen a lot of complains 
about it. Which is why I am not in favor rejecting some specific mix of 
selectors.

Moreover, you do not really want to forbid further operations after a slice as 
a query like {{SELECT filterMap(m\[1..\])\[..2\] FROM ...}} can totally make 
sense. And blocking a query like {{SELECT m\[1..\]\[..2\] FROM ...}} but not a 
query like {{SELECT filterMap(m\[1..\])\[..2\] FROM ...}}, even if the 
{{filterMap}} function is not doing any filtering, is also inconsistent.

So, from consistency point of view, I would argue that we should keep the 
current behavior and not forbid further restrictions after a slice.
The operations perfomed by the selectors are straight forward:
* {{\[?\]}}: returns a collection element
* {{\[?..?\]}}: returns a slice of the collection

There is no notion of {{inner}} or {{outer}} collections because the selectors 
can be used on a collection that does not contains another collection.
  
 

> Allow selecting Map key, List index
> ---
>
> Key: CASSANDRA-7396
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7396
> Project: Cassandra
>  Issue Type: New Feature
>  Components: CQL
>Reporter: Jonathan Ellis
>Assignee: Sylvain Lebresne
>  Labels: cql, docs-impacting
> Fix For: 4.x
>
> Attachments: 7396_unit_tests.txt
>
>
> Allow "SELECT map['key]" and "SELECT list[index]."  (Selecting a UDT subfield 
> is already supported.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13068) Fully expired sstable not dropped when running out of disk space

2017-05-29 Thread Marcus Eriksson (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028122#comment-16028122
 ] 

Marcus Eriksson commented on CASSANDRA-13068:
-

Had a quick read through, and we really need tests for the disk space stuff, I 
added one here: https://github.com/krummas/cassandra/commits/lerh/13068 - maybe 
you could add a few more testing this feature?

just a single code comment so far, in 
{{CompactionTask#reduceScopeForLimitedSpace}} - no need to do 
{{nonExpiredSSTables.remove(removedSSTable);}} - {{nonExpiredSSTables}} is 
{{Sets.difference(transaction.originals(), 
controller.getFullyExpiredSSTables());}} so when you cancel the sstable from 
the transaction, it gets removed from {{nonExpiredSSTables}} (otherwise it 
would have thrown an exception since {{Sets.difference}} returns an 
unmodifiable set) 

I'll do a deeper code review once the tests are in

> Fully expired sstable not dropped when running out of disk space
> 
>
> Key: CASSANDRA-13068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13068
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>  Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> If a fully expired sstable is larger than the remaining disk space we won't 
> run the compaction that can drop the sstable (ie, in our disk space check 
> should not include the fully expired sstables)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-13556) Corrupted SSTables

2017-05-29 Thread Ihor Prokopov (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-13556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16028086#comment-16028086
 ] 

Ihor Prokopov commented on CASSANDRA-13556:
---

Hi Jeff, thanks for your response a lot!
Here is stacktrace:
{code}
ERROR [CompactionExecutor:2116] 2017-05-26 13:10:00,784 
CassandraDaemon.java:226 - Exception in thread 
Thread[CompactionExecutor:2116,1,main]
org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted: 
/var/cassandra/apache-cassandra-3.9/data/data/toweya/fetcher-8a86a1b0f4f611e6a6db373124acc314/mc-178837-big-Data.db
at 
org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:112)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.io.sstable.SSTableIdentityIterator.computeNext(SSTableIdentityIterator.java:30)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:500)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:360)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) 
~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133) 
~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.db.ColumnIndex.buildRowIndex(ColumnIndex.java:106) 
~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:169)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:135)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter.realAppend(MaxSSTableSizeWriter.java:98)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:141)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:189)
 ~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
~[apache-cassandra-3.9.jar:3.9]
at 
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82)
 ~[apache-cassandra-3.9.jar:3.9]
{code}

Also we noticed, that some rows returns without any errors, but fealds' values 
were shuffled (I mean that all fealds contains values from other fields).

Best regards, Ihor

> Corrupted SSTables
> --
>
> Key: CASSANDRA-13556
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13556
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
> Environment: CentOS Linux release 7.3.1611 (Core)
> openjdk version "1.8.0_121"
> OpenJDK Runtime Environment (build 1.8.0_121-b13)
> OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)
> Python cassandra (DataStax) driver v3.6.0
>Reporter: Ihor Prokopov
>Priority: Critical
> Fix For: 3.9
>
>
> After 3 month of working, we noticed that number of compaction tasks were 
> growing (~600 pending tasks). SStables verification shows that some of them 
> were corrupted. Repairing didn't help (it was crashing with error). 
> Also some of requests (f.e. select * from fetcher where 
> domain=8289511971670945261 and uri=-5417197141545933706; ) fails with next 
> error:
> {color:red}
> Traceback (most recent call last):
>   File "/var/cassandra/apache-cassandra-3.9/bin/cqlsh.py", line 1264, in 
> perform_simple_statement
>

[jira] [Assigned] (CASSANDRA-13068) Fully expired sstable not dropped when running out of disk space

2017-05-29 Thread Marcus Eriksson (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-13068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson reassigned CASSANDRA-13068:
---

 Assignee: Lerh Chuan Low
 Reviewer: Marcus Eriksson
Fix Version/s: 4.x
   3.11.x
   3.0.x

> Fully expired sstable not dropped when running out of disk space
> 
>
> Key: CASSANDRA-13068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13068
> Project: Cassandra
>  Issue Type: Bug
>  Components: Compaction
>Reporter: Marcus Eriksson
>Assignee: Lerh Chuan Low
>  Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.x
>
>
> If a fully expired sstable is larger than the remaining disk space we won't 
> run the compaction that can drop the sstable (ie, in our disk space check 
> should not include the fully expired sstables)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org