[jira] [Commented] (CASSANDRA-5576) CREATE/DROP TRIGGER in CQL

2013-06-06 Thread Aleksey Yeschenko (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13676755#comment-13676755
 ] 

Aleksey Yeschenko commented on CASSANDRA-5576:
--

{quote} 
-0 – I see the value of options when we introduce parameterization, but 
having the actual trigger class be part of that seems odd, since that's 
something every trigger needs (i.e. a requirement, not an option or parameter).

So I think I'd prefer to have the EXECUTE syntax, with a single target. We can 
add WITH options later. (But I agree that it's a good idea to split the 
triggers into a separate system table to make that easier.)
{quote}

I would agree with you if there were no precedent. But that's the syntax we 
already use for CREATE CUSTOM INDEX (CASSANDRA-5484), and it's worth reusing it 
just for consistency's sake, IMO.

 CREATE/DROP TRIGGER in CQL
 --

 Key: CASSANDRA-5576
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5576
 Project: Cassandra
  Issue Type: Bug
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Vijay
 Fix For: 2.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5599) Intermittently, CQL SELECT with WHERE on secondary indexed field value returns null when there are rows

2013-06-06 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13676854#comment-13676854
 ] 

Michał Michalski commented on CASSANDRA-5599:
-

To make sure: does it work after recreating index or the problem still happens 
sometimes?
If it's still failing sometimes:
* Did you try running repair?
* What consistency level do you use for writes and reads?
* How many nodes do you have? 
* Does it happen when you SELECT with CL.CL.ALL?

 Intermittently, CQL SELECT  with WHERE on secondary indexed field value 
 returns null when there are rows
 

 Key: CASSANDRA-5599
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5599
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.4, 1.2.5
 Environment: x86_64 / RedHat Enterprise Linux 4.x
Reporter: Ravi Basawa
Priority: Minor

 Intermittently, CQL SELECT  with WHERE on secondary indexed field value 
 returns null when there are rows.
 As it happens intermittently, it is difficult to replicate. To resolve we 
 have had to recreate the index. Also using the nodetool to reindex did not 
 help us either.
 We would create a table, create a secondary index for that table on a field, 
 import data then when we try to select rows from that table with where on 
 said field which should return results, we get a null back ...intermittently. 
  Sometimes works, sometimes not.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-5544) Hadoop jobs assigns only one mapper in task

2013-06-06 Thread Cyril Scetbon (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13675919#comment-13675919
 ] 

Cyril Scetbon edited comment on CASSANDRA-5544 at 6/6/13 9:25 AM:
--

Yes. I used Pig 0.11.1, Hadoop 1.1.2 (as newer versions are not supported 
[CASSANDRA-5201|https://issues.apache.org/jira/browse/CASSANDRA-5201]) and 
cassandra 1.2.3 (I added the current patch from git commits and built sources) 

  was (Author: cscetbon):
Yes. I used Pig 0.11.1, Hadoop 1.1.2 (as newer versions are not supported 
[https://issues.apache.org/jira/browse/CASSANDRA-5201]) and cassandra 1.2.3 (I 
added the current patch from git commits and built sources) 
  
 Hadoop jobs assigns only one mapper in task 
 

 Key: CASSANDRA-5544
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5544
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Fix For: 1.2.6

 Attachments: 5544-1.txt, 5544-2.txt, 5544-3.txt, 5544.txt, Screen 
 Shot 2013-05-26 at 4.49.48 PM.png


 We have got very strange beheviour of hadoop cluster after upgrading 
 Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of 
 Cassandra, where three of them are hodoop slaves. Now when we are submitting 
 job through Pig script, only one map assigns in task running on one of the 
 hadoop slaves regardless of 
 volume of data (already tried with more than million rows).
 Configure of pig as follows:
 export PIG_HOME=/oracle/pig-0.10.0
 export PIG_CONF_DIR=${HADOOP_HOME}/conf
 export PIG_INITIAL_ADDRESS=192.168.157.103
 export PIG_RPC_PORT=9160
 export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner
 Also we have these following properties in hadoop:
  property
  namemapred.tasktracker.map.tasks.maximum/name
  value10/value
  /property
  property
  namemapred.map.tasks/name
  value4/value
  /property

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-5544) Hadoop jobs assigns only one mapper in task

2013-06-06 Thread Cyril Scetbon (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13675919#comment-13675919
 ] 

Cyril Scetbon edited comment on CASSANDRA-5544 at 6/6/13 9:24 AM:
--

Yes. I used Pig 0.11.1, Hadoop 1.1.2 (as newer versions are not supported 
[https://issues.apache.org/jira/browse/CASSANDRA-5201]) and cassandra 1.2.3 (I 
added the current patch from git commits and built sources) 

  was (Author: cscetbon):
yes
  
 Hadoop jobs assigns only one mapper in task 
 

 Key: CASSANDRA-5544
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5544
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
Affects Versions: 1.2.1
 Environment: Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1
Reporter: Shamim Ahmed
Assignee: Alex Liu
 Fix For: 1.2.6

 Attachments: 5544-1.txt, 5544-2.txt, 5544-3.txt, 5544.txt, Screen 
 Shot 2013-05-26 at 4.49.48 PM.png


 We have got very strange beheviour of hadoop cluster after upgrading 
 Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of 
 Cassandra, where three of them are hodoop slaves. Now when we are submitting 
 job through Pig script, only one map assigns in task running on one of the 
 hadoop slaves regardless of 
 volume of data (already tried with more than million rows).
 Configure of pig as follows:
 export PIG_HOME=/oracle/pig-0.10.0
 export PIG_CONF_DIR=${HADOOP_HOME}/conf
 export PIG_INITIAL_ADDRESS=192.168.157.103
 export PIG_RPC_PORT=9160
 export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner
 Also we have these following properties in hadoop:
  property
  namemapred.tasktracker.map.tasks.maximum/name
  value10/value
  /property
  property
  namemapred.map.tasks/name
  value4/value
  /property

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Fix typo in CQL doc

2013-06-06 Thread slebresne 
Updated Branches:
  refs/heads/cassandra-1.2 e301c38d3 - 6850cdc3e


Fix typo in CQL doc


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6850cdc3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6850cdc3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6850cdc3

Branch: refs/heads/cassandra-1.2
Commit: 6850cdc3ed9ea15fecc13beea093012c124f8b8c
Parents: e301c38
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Thu Jun 6 12:16:28 2013 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Thu Jun 6 12:16:28 2013 +0200

--
 doc/cql3/CQL.textile |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6850cdc3/doc/cql3/CQL.textile
--
diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile
index 059342e..13b6f70 100644
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@ -976,9 +976,9 @@ CQL distinguishes between _reserved_ and _non-reserved_ 
keywords. Reserved keywo
 | @BOOLEAN@  | no  |
 | @BY@   | yes |
 | @CLUSTERING@   | no  |
-| @COLUMNFAMLY@  | yes |
+| @COLUMNFAMILY@ | yes |
 | @COMPACT@  | no  |
-| @CONSITENCY@   | no  |
+| @CONSISTENCY@  | no  |
 | @COUNT@| no  |
 | @COUNTER@  | no  |
 | @CREATE@   | yes |

[jira] [Commented] (CASSANDRA-5617) Typo in cql3 keywords

2013-06-06 Thread Sylvain Lebresne (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13676896#comment-13676896
 ] 

Sylvain Lebresne commented on CASSANDRA-5617:
-

Fixed in 6850cdc3ed9ea15fecc13beea093012c124f8b8c and pushed to the website. 
Thanks!

 Typo in cql3 keywords
 -

 Key: CASSANDRA-5617
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5617
 Project: Cassandra
  Issue Type: Improvement
  Components: Documentation  website
Reporter: tariq rahiman
Priority: Minor

 http://cassandra.apache.org/doc/cql3/CQL.html#appendixA
 COLUMNFAMLY and CONSITENCY listed in appendix-A probably are typos, should be 
 COLUMNFAMILY and CONSISTENCY respectively.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

svn commit: r1490231 - /cassandra/site/publish/doc/cql3/CQL.html

2013-06-06 Thread slebresne 
Author: slebresne
Date: Thu Jun  6 10:39:20 2013
New Revision: 1490231

URL: http://svn.apache.org/r1490231
Log:
Update typo in CQL doc

Modified:
cassandra/site/publish/doc/cql3/CQL.html

Modified: cassandra/site/publish/doc/cql3/CQL.html
URL: 
http://svn.apache.org/viewvc/cassandra/site/publish/doc/cql3/CQL.html?rev=1490231r1=1490230r2=1490231view=diff
==
--- cassandra/site/publish/doc/cql3/CQL.html (original)
+++ cassandra/site/publish/doc/cql3/CQL.html Thu Jun  6 10:39:20 2013
@@ -1,4 +1,4 @@
-?xml version='1.0' encoding='utf-8' ?!DOCTYPE html PUBLIC -//W3C//DTD 
XHTML 1.0 Transitional//EN 
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;html 
xmlns=http://www.w3.org/1999/xhtml;headmeta http-equiv=Content-Type 
content=text/html; charset=utf-8/titleCQL/title/headbodyplink 
rel=StyleSheet href=CQL.css type=text/css media=screen/ph1 
id=CassandraQueryLanguageCQLv3.0.2Cassandra Query Language (CQL) 
v3.0.2/h1span id=tableOfContentsol style=list-style: none;lia 
href=CQL.html#CassandraQueryLanguageCQLv3.0.2Cassandra Query Language (CQL) 
v3.0.2/aol style=list-style: none;lia href=CQL.html#CQLSyntaxCQL 
Syntax/aol style=list-style: none;lia 
href=CQL.html#PreamblePreamble/a/lilia 
href=CQL.html#ConventionsConventions/a/lilia 
href=CQL.html#identifiersIdentifiers and keywords/a/lilia 
href=CQL.html#constantsConstants/a/lilia 
href=CQL.html#CommentsComment
 s/a/lilia href=CQL.html#statementsStatements/a/lilia 
href=CQL.html#preparedStatementPrepared Statement/a/li/ol/lilia 
href=CQL.html#dataDefinitionData Definition/aol style=list-style: 
none;lia href=CQL.html#createKeyspaceStmtCREATE KEYSPACE/a/lilia 
href=CQL.html#useStmtUSE/a/lilia 
href=CQL.html#alterKeyspaceStmtALTER KEYSPACE/a/lilia 
href=CQL.html#dropKeyspaceStmtDROP KEYSPACE/a/lilia 
href=CQL.html#createTableStmtCREATE TABLE/a/lilia 
href=CQL.html#alterTableStmtALTER TABLE/a/lilia 
href=CQL.html#dropTableStmtDROP TABLE/a/lilia 
href=CQL.html#truncateStmtTRUNCATE/a/lilia 
href=CQL.html#createIndexStmtCREATE INDEX/a/lilia 
href=CQL.html#dropIndexStmtDROP INDEX/a/li/ol/lilia 
href=CQL.html#dataManipulationData Manipulation/aol style=list-style: 
none;lia href=CQL.html#insertStmtINSERT/a/lilia 
href=CQL.html#updateStmt
 UPDATE/a/lilia href=CQL.html#deleteStmtDELETE/a/lilia 
 href=CQL.html#batchStmtBATCH/a/li/ol/lilia 
 href=CQL.html#queriesQueries/aol style=list-style: none;lia 
 href=CQL.html#selectStmtSELECT/a/li/ol/lilia 
 href=CQL.html#typesData Types/aol style=list-style: none;lia 
 href=CQL.html#usingdatesWorking with dates/a/lilia 
 href=CQL.html#countersCounters/a/lilia 
 href=CQL.html#collectionsWorking with collections/a/li/ol/lilia 
 href=CQL.html#functionsFunctions/aol style=list-style: none;lia 
 href=CQL.html#tokenFunToken/a/lilia 
 href=CQL.html#timeuuidFunTimeuuid functions/a/lilia 
 href=CQL.html#blobFunBlob conversion functions/a/li/ol/lilia 
 href=CQL.html#appendixAAppendix A: CQL Keywords/a/lilia 
 href=CQL.html#changesChanges/aol style=list-style: none;lia 
 href=CQL.html#a3.0.23.0.2/a/lilia href=CQL.html#a3.0.13.0.
 1/a/li/ol/lilia 
href=CQL.html#VersioningVersioning/a/li/ol/li/ol/spanh2 
id=CQLSyntaxCQL Syntax/h2h3 id=PreamblePreamble/h3pThis document 
describes the Cassandra Query Language (CQL) version 3. CQL v3 is not backward 
compatible with CQL v2 and differs from it in numerous ways. Note that this 
document describes the last version of the languages. However, the a 
href=#changeschanges/a section provides the diff between the different 
versions of CQL v3./ppCQL v3 offers a model very close to SQL in the sense 
that data is put in emtables/em containing emrows/em of 
emcolumns/em. For that reason, when used in this document, these terms 
(tables, rows and columns) have the same definition than they have in SQL. But 
please note that as such, they do strongnot/strong refer to the concept of 
rows and columns found in the internal implementation of Cassandra and in the 
thrift and CQL v2 API./ph3 id=ConventionsConvention
 s/h3pTo aid in specifying the CQL syntax, we will use the following 
conventions in this document:/pulliLanguage rules will be given in a a 
href=http://en.wikipedia.org/wiki/Backus%E2%80%93Naur_Form;BNF/a -like 
notation:/li/ulpre class=syntaxprelt;start ::= TERMINAL 
lt;non-terminal1 lt;non-terminal1
+?xml version='1.0' encoding='utf-8' ?!DOCTYPE html PUBLIC -//W3C//DTD 
XHTML 1.0 Transitional//EN 
http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd;html 
xmlns=http://www.w3.org/1999/xhtml;headmeta http-equiv=Content-Type 
content=text/html; charset=utf-8/titleCQL/title/headbodyplink 
rel=StyleSheet href=CQL.css type=text/css media=screen/ph1 
id=CassandraQueryLanguageCQLv3.0.3Cassandra Query Language (CQL) 
v3.0.3/h1span id=tableOfContentsol style=list-style: none;lia 
href=CQL.html#CassandraQueryLanguageCQLv3.0.3Cassandra Query Language (CQL) 
v3.0.3/aol style=list-style: none;lia href=CQL.html#CQLSyntaxCQL

[jira] [Commented] (CASSANDRA-5576) CREATE/DROP TRIGGER in CQL

2013-06-06 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677023#comment-13677023
 ] 

Jonathan Ellis commented on CASSANDRA-5576:
---

I don't buy we made a mistake in 5484 so we should keep perpetuating the 
mistake in other places argument.  We should do it right here and update 
CREATE CUSTOM INDEX syntax for 2.0 to be consistent with this, instead of the 
other way around.

 CREATE/DROP TRIGGER in CQL
 --

 Key: CASSANDRA-5576
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5576
 Project: Cassandra
  Issue Type: Bug
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Vijay
 Fix For: 2.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-4926) CQL3: IN relation does not work for last part of composite key

2013-06-06 Thread Aleksey Yeschenko (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-4926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-4926.
--

Resolution: Won't Fix

This query works properly in CQL3 in Cassandra 1.2. During 1.1, however, CQL3 
was in beta, and it's not going to be fixed.

 CQL3: IN relation does not work for last part of composite key
 --

 Key: CASSANDRA-4926
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4926
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.1.6
Reporter: Roland Mechler

 Everything below was done using cqlsh -3, against single node Cassandra 1.1.6.
 I created a test table with the following definition:
 CREATE TABLE testtable (
   a text,
   b text,
   c text,
   data text,
   PRIMARY KEY (a, b, c)
 ) WITH
   comment='' AND
   caching='KEYS_ONLY' AND
   read_repair_chance=0.10 AND
   gc_grace_seconds=864000 AND
   replicate_on_write='true' AND
   compaction_strategy_class='SizeTieredCompactionStrategy' AND
   compression_parameters:sstable_compression='SnappyCompressor';
 and inserted some data:
 INSERT INTO testtable (a, b, c, data) VALUES ('1', '1', '1', 'data 1');
 INSERT INTO testtable (a, b, c, data) VALUES ('1', '1', '2', 'data 2');
 INSERT INTO testtable (a, b, c, data) VALUES ('1', '1', '3', 'data 3');
 so a full table query produced:
 cqlsh:testkeyspace select * from testtable;
  a | b | c | data
 ---+---+---+
  1 | 1 | 1 | data 1
  1 | 1 | 2 | data 2
  1 | 1 | 3 | data 3
 Next I tried the following query with an IN relation on the second part of 
 the key:
 cqlsh:testkeyspace select * from testtable where a = '1' and b in ('1', '2');
 Bad Request: PRIMARY KEY part b cannot be restricted by IN relation (only the 
 first and last parts can)
 which is apparently not supported, but the error message suggests an IN 
 relation in the last part is supported, so I tried:
 cqlsh:testkeyspace select * from testtable where a = '1' and b = '1' and c 
 in ('1', '2');
 cqlsh:testkeyspace 
 which did not produce an error, but also returned no results. I was expecting 
 2 rows. This seems to be a bug?
 Note that I get the expected result if the IN relation contains only 1 value:
 cqlsh:testkeyspace select * from testtable where a = '1' and b = '1' and c 
 in ('1');
  a | b | c | data
 ---+---+---+
  1 | 1 | 1 | data 1

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5576) CREATE/DROP TRIGGER in CQL

2013-06-06 Thread Aleksey Yeschenko (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677161#comment-13677161
 ] 

Aleksey Yeschenko commented on CASSANDRA-5576:
--

I don't think it was a mistake in case of 5484. Do you have better suggestions?

But, all right, I'm okay with your suggestion for CREATE TRIGGER.

[~slebresne] any ideas re: changing 5484 and syntax for CREATE TRIGGER?

 CREATE/DROP TRIGGER in CQL
 --

 Key: CASSANDRA-5576
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5576
 Project: Cassandra
  Issue Type: Bug
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Vijay
 Fix For: 2.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5576) CREATE/DROP TRIGGER in CQL

2013-06-06 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677175#comment-13677175
 ] 

Jonathan Ellis commented on CASSANDRA-5576:
---

bq. I don't think it was a mistake in case of 5484.

Hmm.  I see what you mean wrt compaction, compression configuration.  Damn it.  
All right, I guess I'm sold on everything in options map.

 CREATE/DROP TRIGGER in CQL
 --

 Key: CASSANDRA-5576
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5576
 Project: Cassandra
  Issue Type: Bug
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Vijay
 Fix For: 2.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5576) CREATE/DROP TRIGGER in CQL

2013-06-06 Thread Sylvain Lebresne (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677186#comment-13677186
 ] 

Sylvain Lebresne commented on CASSANDRA-5576:
-

I don't think I care too much which syntax we pick, though I'm definitely in 
favor of more consistency over less.

It does slightly bug me to change the syntax for custom indexes after it's been 
release. But, given that nobody is probably using the custom index syntax at 
this point (it's not really properly documented yet typically), I'm fine with 
it if you guys think it measurably improve the syntax (I have no real opinion 
for once on that question). But I'd *really* prefer doing it now rather than 
leaving the current syntax in 1.2 and changing it in 2.0.  

Though if we do change the custom index syntax, what are we talking about? 
Something like:
{noformat}
CREATE CUSTOM INDEX ON table(column) USING classname
{noformat}
with a trigger syntax that would be
{noformat}
CREATE TRIGGER name ON table EXECUTE classname
{noformat}
Knowing that for both we could add support for {{WITH options=\{...\} }} 
whenever we actually have options?



 CREATE/DROP TRIGGER in CQL
 --

 Key: CASSANDRA-5576
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5576
 Project: Cassandra
  Issue Type: Bug
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Vijay
 Fix For: 2.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5576) CREATE/DROP TRIGGER in CQL

2013-06-06 Thread Aleksey Yeschenko (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677200#comment-13677200
 ] 

Aleksey Yeschenko commented on CASSANDRA-5576:
--

btw, if we do go with USING for CREATE CUSTOM INDEX, a more consistent
{noformat}
CREATE TRIGGER name on table USING class WITH options
{noformat}
also sounds okay to me (and avoids adding an extra keyword, too).

But I'm equally fine with 'everything in options map'. As long as we are 
consistent, at least betweetn CREATE CUSTOM INDEX and CREATE TRIGGER.

 CREATE/DROP TRIGGER in CQL
 --

 Key: CASSANDRA-5576
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5576
 Project: Cassandra
  Issue Type: Bug
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Vijay
 Fix For: 2.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (CASSANDRA-5617) Typo in cql3 keywords

2013-06-06 Thread tariq rahiman (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

tariq rahiman resolved CASSANDRA-5617.
--

Resolution: Fixed

Verified in website - Fixed

 Typo in cql3 keywords
 -

 Key: CASSANDRA-5617
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5617
 Project: Cassandra
  Issue Type: Improvement
  Components: Documentation  website
Reporter: tariq rahiman
Priority: Minor

 http://cassandra.apache.org/doc/cql3/CQL.html#appendixA
 COLUMNFAMLY and CONSITENCY listed in appendix-A probably are typos, should be 
 COLUMNFAMILY and CONSISTENCY respectively.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5619) CAS UPDATE for a lost race: save round trip by returning column values

2013-06-06 Thread Blair Zajac (JIRA) 
Blair Zajac created CASSANDRA-5619:
--

 Summary: CAS UPDATE for a lost race: save round trip by returning 
column values
 Key: CASSANDRA-5619
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5619
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 2.0
Reporter: Blair Zajac


Looking at the new CAS CQL3 support examples [1], if one lost a race for an 
UPDATE, to save a round trip to get the current values to decide if you need to 
perform your work, could the columns that were used in the IF clause also be 
returned to the caller?  Maybe the columns values as part of the SET part could 
also be returned.

I don't know if this is generally useful though.

In the case of creating a new user account with a given username which is the 
partition key, if one lost the race to another person creating an account with 
the same username, it doesn't matter to the loser what the column values are, 
just that they lost.

I'm new to Cassandra, so maybe there's other use cases, such as doing 
incremental amount of work on a row.  In pure Java projects I've done while 
loops around AtomicReference.html#compareAndSet() until the work was done on 
the referenced object to handle multiple threads each making forward progress 
in updating the references object.

[1] https://github.com/riptano/cassandra-dtest/blob/master/cql_tests.py#L3044

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5619) CAS UPDATE for a lost race: save round trip by returning column values

2013-06-06 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677313#comment-13677313
 ] 

Jonathan Ellis commented on CASSANDRA-5619:
---

It sounds reasonable in principle but what does the API look like for Thrift 
and CQL?

 CAS UPDATE for a lost race: save round trip by returning column values
 --

 Key: CASSANDRA-5619
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5619
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 2.0
Reporter: Blair Zajac

 Looking at the new CAS CQL3 support examples [1], if one lost a race for an 
 UPDATE, to save a round trip to get the current values to decide if you need 
 to perform your work, could the columns that were used in the IF clause also 
 be returned to the caller?  Maybe the columns values as part of the SET part 
 could also be returned.
 I don't know if this is generally useful though.
 In the case of creating a new user account with a given username which is the 
 partition key, if one lost the race to another person creating an account 
 with the same username, it doesn't matter to the loser what the column values 
 are, just that they lost.
 I'm new to Cassandra, so maybe there's other use cases, such as doing 
 incremental amount of work on a row.  In pure Java projects I've done while 
 loops around AtomicReference.html#compareAndSet() until the work was done on 
 the referenced object to handle multiple threads each making forward progress 
 in updating the references object.
 [1] https://github.com/riptano/cassandra-dtest/blob/master/cql_tests.py#L3044

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5582) Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor

2013-06-06 Thread Pavel Yaskevich (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677317#comment-13677317
 ] 

Pavel Yaskevich commented on CASSANDRA-5582:


Is now available at 
http://search.maven.org/#artifactdetails%7Ccom.thinkaurelius.thrift%7Cthrift-server%7C0.1%7Cjar
 , i will prepare changes to build.xml

 Replace CustomHsHaServer with better optimized solution based on LMAX 
 Disruptor
 ---

 Key: CASSANDRA-5582
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5582
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 2.0

 Attachments: CASSANDRA-5530-invoker-fix.patch, CASSANDRA-5582.patch, 
 disruptor-3.0.1.jar, disruptor-thrift-0.1-SNAPSHOT.jar, Pavel's Patch.rtf


 I have been working on https://github.com/xedin/disruptor_thrift_server and 
 consider it as stable and performant enough for integration with Cassandra. 
 Proposed replacement can work in both on/off Heap modes (depending if JNA is 
 available) and doesn't blindly reallocate things, which allows to resolve 
 CASSANDRA-4265 as Won't Fix.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5619) CAS UPDATE for a lost race: save round trip by returning column values

2013-06-06 Thread Blair Zajac (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677332#comment-13677332
 ] 

Blair Zajac commented on CASSANDRA-5619:


I'm new to Cassandra and haven't looked at the Thrift or CQL2 API, I've been 
doing everything on CQL3 where possible, as suggested by your blogs ;)

Given that the new CAS support returns a Boolean on the success or failure 
where the non-CAS UPDATE returned nothing, couldn't the additional columns be 
added on the return in addition to the Boolean?  I was looking at the CQL3 Java 
Driver, and there, it seems pretty straight forward.

 CAS UPDATE for a lost race: save round trip by returning column values
 --

 Key: CASSANDRA-5619
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5619
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 2.0
Reporter: Blair Zajac

 Looking at the new CAS CQL3 support examples [1], if one lost a race for an 
 UPDATE, to save a round trip to get the current values to decide if you need 
 to perform your work, could the columns that were used in the IF clause also 
 be returned to the caller?  Maybe the columns values as part of the SET part 
 could also be returned.
 I don't know if this is generally useful though.
 In the case of creating a new user account with a given username which is the 
 partition key, if one lost the race to another person creating an account 
 with the same username, it doesn't matter to the loser what the column values 
 are, just that they lost.
 I'm new to Cassandra, so maybe there's other use cases, such as doing 
 incremental amount of work on a row.  In pure Java projects I've done while 
 loops around AtomicReference.html#compareAndSet() until the work was done on 
 the referenced object to handle multiple threads each making forward progress 
 in updating the references object.
 [1] https://github.com/riptano/cassandra-dtest/blob/master/cql_tests.py#L3044

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5542) BulkLoader is broken in trunk

2013-06-06 Thread Yuki Morishita (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-5542:
--

Attachment: 5542-fix-NPE.txt

Attaching patch to switch using indexSummary instead of CFMetaData where 
affected.

 BulkLoader is broken in trunk
 -

 Key: CASSANDRA-5542
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 2.0
Reporter: Yuki Morishita
Assignee: Yuki Morishita
 Fix For: 2.0

 Attachments: 5542-fix-NPE.txt, 5542.txt


 After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open 
 SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval.
 When using bulkloader, CFMetaData is not available, so we cannot open SSTable 
 to be streamed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Reopened] (CASSANDRA-5542) BulkLoader is broken in trunk

2013-06-06 Thread Yuki Morishita (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita reopened CASSANDRA-5542:
---


Still, sstableloader throws NPE on the execution path at SSTableReader.

 BulkLoader is broken in trunk
 -

 Key: CASSANDRA-5542
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 2.0
Reporter: Yuki Morishita
Assignee: Yuki Morishita
 Fix For: 2.0

 Attachments: 5542-fix-NPE.txt, 5542.txt


 After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open 
 SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval.
 When using bulkloader, CFMetaData is not available, so we cannot open SSTable 
 to be streamed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5620) Incorrect Syntax for Timestamp in Select query

2013-06-06 Thread tariq rahiman (JIRA) 
tariq rahiman created CASSANDRA-5620:


 Summary: Incorrect Syntax for Timestamp in Select query
 Key: CASSANDRA-5620
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5620
 Project: Cassandra
  Issue Type: Bug
  Components: Documentation  website
Reporter: tariq rahiman
Priority: Minor


In the page http://cassandra.apache.org/doc/cql3/CQL.html under the heading 
Queries -- SELECT -- where-clause  We have a table posts followed by a 
Query which doesnt seem to work against the Cassandra Server. 
Timestamp requires quotes in the WHERE clause.

CREATE TABLE posts (
userid text,
blog_title text,
posted_at timestamp,
entry_title text,
content text,
category int,
PRIMARY KEY (userid, blog_title, posted_at)
)

Incorrect
SELECT entry_title, content FROM posts WHERE userid='john doe' AND 
blog_title='John's Blog' AND posted_at = 2012-01-01 AND posted_at  2012-01-31

I get the the error mismatched character 'EOF' expecting '''

If I correc the error in Johns''Blog and proceed I get missing EOF at '-01'

Correct
SELECT entry_title, content FROM posts WHERE userid='john doe' AND 
blog_title='John''s Blog' AND posted_at = '2012-01-01' AND posted_at  
'2012-01-31'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5542) BulkLoader is broken in trunk

2013-06-06 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677355#comment-13677355
 ] 

Jonathan Ellis commented on CASSANDRA-5542:
---

sample key count is going to be nonsense, will that break anything?

 BulkLoader is broken in trunk
 -

 Key: CASSANDRA-5542
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 2.0
Reporter: Yuki Morishita
Assignee: Yuki Morishita
 Fix For: 2.0

 Attachments: 5542-fix-NPE.txt, 5542.txt


 After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open 
 SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval.
 When using bulkloader, CFMetaData is not available, so we cannot open SSTable 
 to be streamed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5542) BulkLoader is broken in trunk

2013-06-06 Thread Yuki Morishita (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677362#comment-13677362
 ] 

Yuki Morishita commented on CASSANDRA-5542:
---

hmm, you are right, but it is used to tell the receiving node the estimated key 
count for creating BF.
Maybe we should query schema first, then bulk load.
I need that anyway if we switch to use cf ID for new streaming 
protocol(https://issues.apache.org/jira/browse/CASSANDRA-5286?focusedCommentId=13671413page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13671413).

 BulkLoader is broken in trunk
 -

 Key: CASSANDRA-5542
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 2.0
Reporter: Yuki Morishita
Assignee: Yuki Morishita
 Fix For: 2.0

 Attachments: 5542-fix-NPE.txt, 5542.txt


 After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open 
 SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval.
 When using bulkloader, CFMetaData is not available, so we cannot open SSTable 
 to be streamed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5576) CREATE/DROP TRIGGER in CQL

2013-06-06 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677372#comment-13677372
 ] 

Jonathan Ellis commented on CASSANDRA-5576:
---

If everyone is fine with both options then I move we leave the final decision 
to Vijay who is coding it up. :)

 CREATE/DROP TRIGGER in CQL
 --

 Key: CASSANDRA-5576
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5576
 Project: Cassandra
  Issue Type: Bug
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Vijay
 Fix For: 2.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5542) BulkLoader is broken in trunk

2013-06-06 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677374#comment-13677374
 ] 

Jonathan Ellis commented on CASSANDRA-5542:
---

It makes me sad to introduce a schema dependency, since that means you can't 
run sstableloader except on a cluster member.

What if we added key count and cfid to the stats/metadata sstable component?

 BulkLoader is broken in trunk
 -

 Key: CASSANDRA-5542
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 2.0
Reporter: Yuki Morishita
Assignee: Yuki Morishita
 Fix For: 2.0

 Attachments: 5542-fix-NPE.txt, 5542.txt


 After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open 
 SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval.
 When using bulkloader, CFMetaData is not available, so we cannot open SSTable 
 to be streamed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[2/4] git commit: Fix typo in CQL doc

2013-06-06 Thread brandonwilliams 
Fix typo in CQL doc


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6850cdc3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6850cdc3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6850cdc3

Branch: refs/heads/trunk
Commit: 6850cdc3ed9ea15fecc13beea093012c124f8b8c
Parents: e301c38
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Thu Jun 6 12:16:28 2013 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Thu Jun 6 12:16:28 2013 +0200

--
 doc/cql3/CQL.textile |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6850cdc3/doc/cql3/CQL.textile
--
diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile
index 059342e..13b6f70 100644
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@ -976,9 +976,9 @@ CQL distinguishes between _reserved_ and _non-reserved_ 
keywords. Reserved keywo
 | @BOOLEAN@  | no  |
 | @BY@   | yes |
 | @CLUSTERING@   | no  |
-| @COLUMNFAMLY@  | yes |
+| @COLUMNFAMILY@ | yes |
 | @COMPACT@  | no  |
-| @CONSITENCY@   | no  |
+| @CONSISTENCY@  | no  |
 | @COUNT@| no  |
 | @COUNTER@  | no  |
 | @CREATE@   | yes |

[3/4] git commit: Fix typo in CQL doc

2013-06-06 Thread brandonwilliams 
Fix typo in CQL doc


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/72ca5e6b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/72ca5e6b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/72ca5e6b

Branch: refs/heads/trunk
Commit: 72ca5e6bfb8d52f8f66b992b6a562ae63f01d897
Parents: 4c0be33
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Thu Jun 6 12:16:28 2013 +0200
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Jun 6 13:59:14 2013 -0500

--
 doc/cql3/CQL.textile |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/72ca5e6b/doc/cql3/CQL.textile
--
diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile
index 4708644..169cf2b 100644
--- a/doc/cql3/CQL.textile
+++ b/doc/cql3/CQL.textile
@@ -984,9 +984,9 @@ CQL distinguishes between _reserved_ and _non-reserved_ 
keywords. Reserved keywo
 | @BOOLEAN@  | no  |
 | @BY@   | yes |
 | @CLUSTERING@   | no  |
-| @COLUMNFAMLY@  | yes |
+| @COLUMNFAMILY@ | yes |
 | @COMPACT@  | no  |
-| @CONSITENCY@   | no  |
+| @CONSISTENCY@  | no  |
 | @COUNT@| no  |
 | @COUNTER@  | no  |
 | @CREATE@   | yes |

[4/4] git commit: Merge branch 'cassandra-1.2' into trunk

2013-06-06 Thread brandonwilliams 
Merge branch 'cassandra-1.2' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/deac4cc1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/deac4cc1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/deac4cc1

Branch: refs/heads/trunk
Commit: deac4cc17bed2df56b2d0b452dc52b693a65e9cf
Parents: 72ca5e6 6850cdc
Author: Brandon Williams brandonwilli...@apache.org
Authored: Thu Jun 6 13:59:41 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Thu Jun 6 13:59:41 2013 -0500

--

--

[1/4] git commit: More detailed read repair metrics. Patch by Jingsi Zhu, reviewed by brandonwilliams for CASSANDRA-5618

2013-06-06 Thread brandonwilliams 
Updated Branches:
  refs/heads/trunk 4c0be33ea - deac4cc17


More detailed read repair metrics.
Patch by Jingsi Zhu, reviewed by brandonwilliams for CASSANDRA-5618


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e301c38d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e301c38d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e301c38d

Branch: refs/heads/trunk
Commit: e301c38d3a592ff943ea0703403542dd2b7f499d
Parents: b853630
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Jun 5 15:11:56 2013 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Jun 5 15:11:56 2013 -0500

--
 CHANGES.txt|1 +
 .../cassandra/metrics/ReadRepairMetrics.java   |   25 +++
 .../org/apache/cassandra/service/ReadCallback.java |5 ++-
 .../org/apache/cassandra/service/StorageProxy.java |   25 ++-
 .../cassandra/service/StorageProxyMBean.java   |6 +++
 src/java/org/apache/cassandra/tools/NodeCmd.java   |2 +
 src/java/org/apache/cassandra/tools/NodeProbe.java |   15 +
 7 files changed, 77 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e301c38d/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index edbb94d..75b3173 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -19,6 +19,7 @@
  * cqlsh: fix COPY FROM with ReversedType (CASSANDRA-5610)
  * Allow creating CUSTOM indexes on collections (CASSANDRA-5615)
  * Evaluate now() function at execution time (CASSANDRA-5616)
+ * Expose detailed read repair metrics (CASSANDRA-5618)
 Merged from 1.1:
  * Remove buggy thrift max message length option (CASSANDRA-5529)
  * Fix NPE in Pig's widerow mode (CASSANDRA-5488)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e301c38d/src/java/org/apache/cassandra/metrics/ReadRepairMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/ReadRepairMetrics.java 
b/src/java/org/apache/cassandra/metrics/ReadRepairMetrics.java
new file mode 100644
index 000..3f48fee
--- /dev/null
+++ b/src/java/org/apache/cassandra/metrics/ReadRepairMetrics.java
@@ -0,0 +1,25 @@
+package org.apache.cassandra.metrics;
+
+import java.util.concurrent.TimeUnit;
+
+import com.yammer.metrics.Metrics;
+import com.yammer.metrics.core.Counter;
+import com.yammer.metrics.core.Gauge;
+import com.yammer.metrics.core.Meter;
+import com.yammer.metrics.core.MetricName;
+import com.yammer.metrics.util.RatioGauge;
+
+/**
+ * Metrics related to Read Repair.
+ */
+public class ReadRepairMetrics {
+public static final String GROUP_NAME = org.apache.cassandra.metrics;
+public static final String TYPE_NAME = ReadRepair;
+
+public static final Meter repairedBlocking =
+Metrics.newMeter(new MetricName(GROUP_NAME, TYPE_NAME, 
RepairedBlocking), RepairedBlocking, TimeUnit.SECONDS);
+public static final Meter repairedBackground =
+Metrics.newMeter(new MetricName(GROUP_NAME, TYPE_NAME, 
RepairedBackground), RepairedBackground, TimeUnit.SECONDS);
+public static final Meter attempted = 
+Metrics.newMeter(new MetricName(GROUP_NAME, TYPE_NAME, 
Attempted), Attempted, TimeUnit.SECONDS);
+}

http://git-wip-us.apache.org/repos/asf/cassandra/blob/e301c38d/src/java/org/apache/cassandra/service/ReadCallback.java
--
diff --git a/src/java/org/apache/cassandra/service/ReadCallback.java 
b/src/java/org/apache/cassandra/service/ReadCallback.java
index a19df5f..92032f2 100644
--- a/src/java/org/apache/cassandra/service/ReadCallback.java
+++ b/src/java/org/apache/cassandra/service/ReadCallback.java
@@ -34,6 +34,7 @@ import org.apache.cassandra.db.ReadCommand;
 import org.apache.cassandra.db.Table;
 import org.apache.cassandra.exceptions.ReadTimeoutException;
 import org.apache.cassandra.exceptions.UnavailableException;
+import org.apache.cassandra.metrics.ReadRepairMetrics;
 import org.apache.cassandra.net.IAsyncCallback;
 import org.apache.cassandra.net.MessageIn;
 import org.apache.cassandra.net.MessageOut;
@@ -174,7 +175,9 @@ public class ReadCallbackTMessage, TResolved implements 
IAsyncCallbackTMessag
 
 if (logger.isDebugEnabled())
 logger.debug(Digest mismatch:, e);
-
+
+ReadRepairMetrics.repairedBackground.mark();
+
 ReadCommand readCommand = (ReadCommand) command;
 final RowDataResolver repairResolver = new 
RowDataResolver(readCommand.table, readCommand.key, readCommand.filter());
 IAsyncCallback

[jira] [Commented] (CASSANDRA-5542) BulkLoader is broken in trunk

2013-06-06 Thread Yuki Morishita (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677387#comment-13677387
 ] 

Yuki Morishita commented on CASSANDRA-5542:
---

bq. It makes me sad to introduce a schema dependency, since that means you 
can't run sstableloader except on a cluster member.

It does query schema even on 1.2(CASSANDRA-4755), so it is fairly easy to fetch 
info we need.

bq. What if we added key count and cfid to the stats/metadata sstable component?

This can also be done, but the change requires stats.db to be always present 
when using bulkloader, which we don't today.

 BulkLoader is broken in trunk
 -

 Key: CASSANDRA-5542
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 2.0
Reporter: Yuki Morishita
Assignee: Yuki Morishita
 Fix For: 2.0

 Attachments: 5542-fix-NPE.txt, 5542.txt


 After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open 
 SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval.
 When using bulkloader, CFMetaData is not available, so we cannot open SSTable 
 to be streamed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[08/10] merge from 1.2

2013-06-06 Thread jbellis 
http://git-wip-us.apache.org/repos/asf/cassandra/blob/90052d5a/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
--
diff --git a/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java 
b/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
index 792618d..5905f4a 100644
--- a/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
+++ b/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
@@ -21,6 +21,10 @@ package org.apache.cassandra.thrift;
  *
  */
 
+import java.util.Collections;
+import java.util.Map;
+import java.util.Set;
+
 import org.apache.cassandra.hadoop.ConfigHelper;
 import org.apache.thrift.transport.TFramedTransport;
 import org.apache.thrift.transport.TSocket;
@@ -29,7 +33,7 @@ import org.apache.thrift.transport.TTransportException;
 
 import org.apache.hadoop.conf.Configuration;
 
-public class TFramedTransportFactory implements ITransportFactory
+public class TFramedTransportFactory implements ITransportFactory, 
TClientTransportFactory
 {
 public TTransport openTransport(TSocket socket, Configuration conf) throws 
TTransportException
 {
@@ -37,4 +41,22 @@ public class TFramedTransportFactory implements 
ITransportFactory
 transport.open();
 return transport;
 }
+
+//
+public TTransport openTransport(String host, int port, Configuration conf) 
throws TTransportException
+{
+TSocket socket = new TSocket(host, port);
+TTransport transport = new TFramedTransport(socket, 
ConfigHelper.getThriftFramedTransportSize(conf));
+transport.open();
+return transport;
+}
+
+public void setOptions(MapString, String options)
+{
+}
+
+public SetString supportedOptions()
+{
+return Collections.emptySet();
+}
 }

[04/10] Add CQL3 input/output formats patch by Alex Liu; reviewed by jbellis and Mike Schrag for CASSANDRA-4421

2013-06-06 Thread jbellis 
http://git-wip-us.apache.org/repos/asf/cassandra/blob/56e0ad1b/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java 
b/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
new file mode 100644
index 000..3939e0b
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
@@ -0,0 +1,386 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.hadoop.cql3;
+
+import java.io.IOException;
+import java.net.InetAddress;
+import java.nio.ByteBuffer;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.apache.cassandra.thrift.*;
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.marshal.CompositeType;
+import org.apache.cassandra.db.marshal.LongType;
+import org.apache.cassandra.db.marshal.TypeParser;
+import org.apache.cassandra.dht.Range;
+import org.apache.cassandra.dht.Token;
+import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.exceptions.SyntaxException;
+import org.apache.cassandra.hadoop.AbstractColumnFamilyRecordWriter;
+import org.apache.cassandra.hadoop.ConfigHelper;
+import org.apache.cassandra.hadoop.Progressable;
+import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.FBUtilities;
+import org.apache.cassandra.utils.Pair;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.mapreduce.TaskAttemptContext;
+import org.apache.thrift.TException;
+import org.apache.thrift.transport.TTransport;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * The codeColumnFamilyRecordWriter/code maps the output lt;key, valuegt;
+ * pairs to a Cassandra column family. In particular, it applies the binded 
variables
+ * in the value to the prepared statement, which it associates with the key, 
and in 
+ * turn the responsible endpoint.
+ *
+ * p
+ * Furthermore, this writer groups the cql queries by the endpoint responsible 
for
+ * the rows being affected. This allows the cql queries to be executed in 
parallel,
+ * directly to a responsible endpoint.
+ * /p
+ *
+ * @see ColumnFamilyOutputFormat
+ */
+final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMapString, ByteBuffer, ListByteBuffer
+{
+private static final Logger logger = 
LoggerFactory.getLogger(ColumnFamilyRecordWriter.class);
+
+// handles for clients for each range running in the threadpool
+private final MapRange, RangeClient clients;
+
+// host to prepared statement id mappings
+private ConcurrentHashMapCassandra.Client, Integer preparedStatements = 
new ConcurrentHashMapCassandra.Client, Integer();
+
+private final String cql;
+
+private AbstractType? keyValidator;
+private String [] partitionkeys;
+
+/**
+ * Upon construction, obtain the map that this writer will use to collect
+ * mutations, and the ring cache for the given keyspace.
+ *
+ * @param context the task attempt context
+ * @throws IOException
+ */
+ColumnFamilyRecordWriter(TaskAttemptContext context) throws IOException
+{
+this(context.getConfiguration());
+this.progressable = new Progressable(context);
+}
+
+ColumnFamilyRecordWriter(Configuration conf, Progressable progressable) 
throws IOException
+{
+this(conf);
+this.progressable = progressable;
+}
+
+ColumnFamilyRecordWriter(Configuration conf) throws IOException
+{
+super(conf);
+this.clients = new HashMapRange, RangeClient();
+cql = CQLConfigHelper.getOutputCql(conf);
+
+try
+{
+String host = getAnyHost();
+int port = ConfigHelper.getOutputRpcPort(conf);
+Cassandra.Client client = 
ColumnFamilyOutputFormat.createAuthenticatedClient(host, port, conf);
+retrievePartitionKeyValidator(client);
+
+if (client != null)
+

[07/10] git commit: Merge branch 'cassandra-1.2' into trunk

2013-06-06 Thread jbellis 
Merge branch 'cassandra-1.2' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6a9e6d59
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6a9e6d59
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6a9e6d59

Branch: refs/heads/trunk
Commit: 6a9e6d598d92a7e9d2e1aca63cea6ead16115d19
Parents: deac4cc 56e0ad1
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Jun 6 14:18:50 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Jun 6 14:18:50 2013 -0500

--

--

[01/10] Add CQL3 input/output formats patch by Alex Liu; reviewed by jbellis and Mike Schrag for CASSANDRA-4421

2013-06-06 Thread jbellis 
Updated Branches:
  refs/heads/cassandra-1.2 6850cdc3e - 56e0ad1be
  refs/heads/trunk deac4cc17 - 90052d5a4


http://git-wip-us.apache.org/repos/asf/cassandra/blob/56e0ad1b/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java 
b/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
new file mode 100644
index 000..3939e0b
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
@@ -0,0 +1,386 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.hadoop.cql3;
+
+import java.io.IOException;
+import java.net.InetAddress;
+import java.nio.ByteBuffer;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import org.apache.cassandra.thrift.*;
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.marshal.CompositeType;
+import org.apache.cassandra.db.marshal.LongType;
+import org.apache.cassandra.db.marshal.TypeParser;
+import org.apache.cassandra.dht.Range;
+import org.apache.cassandra.dht.Token;
+import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.exceptions.SyntaxException;
+import org.apache.cassandra.hadoop.AbstractColumnFamilyRecordWriter;
+import org.apache.cassandra.hadoop.ConfigHelper;
+import org.apache.cassandra.hadoop.Progressable;
+import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.FBUtilities;
+import org.apache.cassandra.utils.Pair;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.mapreduce.TaskAttemptContext;
+import org.apache.thrift.TException;
+import org.apache.thrift.transport.TTransport;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * The codeColumnFamilyRecordWriter/code maps the output lt;key, valuegt;
+ * pairs to a Cassandra column family. In particular, it applies the binded 
variables
+ * in the value to the prepared statement, which it associates with the key, 
and in 
+ * turn the responsible endpoint.
+ *
+ * p
+ * Furthermore, this writer groups the cql queries by the endpoint responsible 
for
+ * the rows being affected. This allows the cql queries to be executed in 
parallel,
+ * directly to a responsible endpoint.
+ * /p
+ *
+ * @see ColumnFamilyOutputFormat
+ */
+final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMapString, ByteBuffer, ListByteBuffer
+{
+private static final Logger logger = 
LoggerFactory.getLogger(ColumnFamilyRecordWriter.class);
+
+// handles for clients for each range running in the threadpool
+private final MapRange, RangeClient clients;
+
+// host to prepared statement id mappings
+private ConcurrentHashMapCassandra.Client, Integer preparedStatements = 
new ConcurrentHashMapCassandra.Client, Integer();
+
+private final String cql;
+
+private AbstractType? keyValidator;
+private String [] partitionkeys;
+
+/**
+ * Upon construction, obtain the map that this writer will use to collect
+ * mutations, and the ring cache for the given keyspace.
+ *
+ * @param context the task attempt context
+ * @throws IOException
+ */
+ColumnFamilyRecordWriter(TaskAttemptContext context) throws IOException
+{
+this(context.getConfiguration());
+this.progressable = new Progressable(context);
+}
+
+ColumnFamilyRecordWriter(Configuration conf, Progressable progressable) 
throws IOException
+{
+this(conf);
+this.progressable = progressable;
+}
+
+ColumnFamilyRecordWriter(Configuration conf) throws IOException
+{
+super(conf);
+this.clients = new HashMapRange, RangeClient();
+cql = CQLConfigHelper.getOutputCql(conf);
+
+try
+{
+String host = getAnyHost();
+int port = ConfigHelper.getOutputRpcPort(conf);
+Cassandra.Client client = 
ColumnFamilyOutputFormat.createAuthenticatedClient(host, port, conf);
+

[jira] [Commented] (CASSANDRA-4421) Support cql3 table definitions in Hadoop InputFormat

2013-06-06 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677400#comment-13677400
 ] 

Jonathan Ellis commented on CASSANDRA-4421:
---

Committed!

 Support cql3 table definitions in Hadoop InputFormat
 

 Key: CASSANDRA-4421
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4421
 Project: Cassandra
  Issue Type: Improvement
  Components: API
Affects Versions: 1.1.0
 Environment: Debian Squeeze
Reporter: bert Passek
Assignee: Alex Liu
  Labels: cql3
 Fix For: 1.2.6

 Attachments: 4421-10-patch-for-1.2.txt, 4421-10-patch-for-trunk.txt, 
 4421-1.txt, 4421-2.txt, 4421-3.txt, 4421-4.txt, 4421-5.txt, 4421-6.cb.txt, 
 4421-6-je.txt, 4421-7-je.txt, 4421-8-cb.txt, 4421-8-je.txt, 4421-9-je.txt, 
 4421.txt


 Hello,
 i faced a bug while writing composite column values and following validation 
 on server side.
 This is the setup for reproduction:
 1. create a keyspace
 create keyspace test with strategy_class = 'SimpleStrategy' and 
 strategy_options:replication_factor = 1;
 2. create a cf via cql (3.0)
 create table test1 (
 a int,
 b int,
 c int,
 primary key (a, b)
 );
 If i have a look at the schema in cli i noticed that there is no column 
 metadata for columns not part of primary key.
 create column family test1
   with column_type = 'Standard'
   and comparator = 
 'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
   and default_validation_class = 'UTF8Type'
   and key_validation_class = 'Int32Type'
   and read_repair_chance = 0.1
   and dclocal_read_repair_chance = 0.0
   and gc_grace = 864000
   and min_compaction_threshold = 4
   and max_compaction_threshold = 32
   and replicate_on_write = true
   and compaction_strategy = 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
   and caching = 'KEYS_ONLY'
   and compression_options = {'sstable_compression' : 
 'org.apache.cassandra.io.compress.SnappyCompressor'};
 Please notice the default validation class: UTF8Type
 Now i would like to insert value  127 via cassandra client (no cql, part of 
 mr-jobs). Have a look at the attachement.
 Batch mutate fails:
 InvalidRequestException(why:(String didn't validate.) [test][test1][1:c] 
 failed validation)
 A validator for column value is fetched in 
 ThriftValidation::validateColumnData which returns always the default 
 validator which is UTF8Type as described above (The ColumnDefinition for 
 given column name c is always null)
 In UTF8Type there is a check for
 if (b  127)
return false;
 Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table 
 creation. I assigned data types to all columns, but i can not set values for 
 a composite column because the default validation class is used.
 I think the schema should know the correct validator even for composite 
 columns. The usage of the default validation class does not make sense.
 Best Regards 
 Bert Passek

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[2/3] git commit: r/m unused ITransportFactory and rename TClientTransportFactory - ITransportFactory

2013-06-06 Thread jbellis 
r/m unused ITransportFactory and rename TClientTransportFactory - 
ITransportFactory


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b973a333
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b973a333
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b973a333

Branch: refs/heads/trunk
Commit: b973a3336949be81dc10ca16dca29e4b2fb237ec
Parents: 56e0ad1
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Jun 6 14:24:27 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Jun 6 14:24:27 2013 -0500

--
 .../org/apache/cassandra/hadoop/ConfigHelper.java  |   10 +-
 .../apache/cassandra/thrift/ITransportFactory.java |   43 -
 .../cassandra/thrift/TClientTransportFactory.java  |   70 ---
 .../cassandra/thrift/TFramedTransportFactory.java  |   11 +--
 4 files changed, 46 insertions(+), 88 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b973a333/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/ConfigHelper.java 
b/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
index 3dcfdd7..36228cf 100644
--- a/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
+++ b/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
@@ -566,21 +566,21 @@ public class ConfigHelper
 }
 }
 
-public static TClientTransportFactory 
getClientTransportFactory(Configuration conf)
+public static ITransportFactory getClientTransportFactory(Configuration 
conf)
 {
 String factoryClassName = conf.get(
-TClientTransportFactory.PROPERTY_KEY,
+ITransportFactory.PROPERTY_KEY,
 TFramedTransportFactory.class.getName());
-TClientTransportFactory factory = 
getClientTransportFactory(factoryClassName);
+ITransportFactory factory = 
getClientTransportFactory(factoryClassName);
 MapString, String options = getOptions(conf, 
factory.supportedOptions());
 factory.setOptions(options);
 return factory;
 }
 
-private static TClientTransportFactory getClientTransportFactory(String 
factoryClassName) {
+private static ITransportFactory getClientTransportFactory(String 
factoryClassName) {
 try
 {
-return (TClientTransportFactory) 
Class.forName(factoryClassName).newInstance();
+return (ITransportFactory) 
Class.forName(factoryClassName).newInstance();
 }
 catch (Exception e)
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b973a333/src/java/org/apache/cassandra/thrift/ITransportFactory.java
--
diff --git a/src/java/org/apache/cassandra/thrift/ITransportFactory.java 
b/src/java/org/apache/cassandra/thrift/ITransportFactory.java
index e3e87c2..98b1c03 100644
--- a/src/java/org/apache/cassandra/thrift/ITransportFactory.java
+++ b/src/java/org/apache/cassandra/thrift/ITransportFactory.java
@@ -22,14 +22,49 @@ package org.apache.cassandra.thrift;
  */
 
 import org.apache.hadoop.conf.Configuration;
-import org.apache.thrift.transport.TSocket;
 import org.apache.thrift.transport.TTransport;
-import org.apache.thrift.transport.TTransportException;
 
-import javax.security.auth.login.LoginException;
+import java.util.Map;
+import java.util.Set;
 
 
+/**
+ * Transport factory for establishing thrift connections from clients to a 
remote server.
+ */
 public interface ITransportFactory
 {
-TTransport openTransport(TSocket socket, Configuration conf) throws 
LoginException, TTransportException;
+static final String PROPERTY_KEY = cassandra.client.transport.factory;
+static final String LONG_OPTION = transport-factory;
+static final String SHORT_OPTION = tr;
+
+/**
+ * Opens a client transport to a thrift server.
+ * Example:
+ *
+ * pre
+ * TTransport transport = clientTransportFactory.openTransport(address, 
port);
+ * Cassandra.Iface client = new Cassandra.Client(new 
BinaryProtocol(transport));
+ * /pre
+ *
+ * @param host fully qualified hostname of the server
+ * @param port RPC port of the server
+ * @param conf Hadoop configuration
+ * @return open and ready to use transport
+ * @throws Exception implementation defined; usually throws 
TTransportException or IOException
+ * if the connection cannot be established
+ */
+TTransport openTransport(String host, int port, Configuration conf) throws 
Exception;
+
+/**
+ * Sets an implementation defined set of options.
+ * Keys in this map must conform to the set set returned by 
TClientTransportFactory#supportedOptions.
+ *

[3/3] git commit: Merge branch 'cassandra-1.2' into trunk

2013-06-06 Thread jbellis 
Merge branch 'cassandra-1.2' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0eb048f2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0eb048f2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0eb048f2

Branch: refs/heads/trunk
Commit: 0eb048f2358f37565703fb70152fc066aed8cc73
Parents: 90052d5 b973a33
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Jun 6 14:24:32 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Jun 6 14:24:32 2013 -0500

--
 .../org/apache/cassandra/hadoop/ConfigHelper.java  |   10 +-
 .../apache/cassandra/thrift/ITransportFactory.java |   43 -
 .../cassandra/thrift/TClientTransportFactory.java  |   70 ---
 .../cassandra/thrift/TFramedTransportFactory.java  |   11 +--
 4 files changed, 46 insertions(+), 88 deletions(-)
--

[1/3] git commit: r/m unused ITransportFactory and rename TClientTransportFactory - ITransportFactory

2013-06-06 Thread jbellis 
Updated Branches:
  refs/heads/cassandra-1.2 56e0ad1be - b973a3336
  refs/heads/trunk 90052d5a4 - 0eb048f23


r/m unused ITransportFactory and rename TClientTransportFactory - 
ITransportFactory


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b973a333
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b973a333
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b973a333

Branch: refs/heads/cassandra-1.2
Commit: b973a3336949be81dc10ca16dca29e4b2fb237ec
Parents: 56e0ad1
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Jun 6 14:24:27 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Jun 6 14:24:27 2013 -0500

--
 .../org/apache/cassandra/hadoop/ConfigHelper.java  |   10 +-
 .../apache/cassandra/thrift/ITransportFactory.java |   43 -
 .../cassandra/thrift/TClientTransportFactory.java  |   70 ---
 .../cassandra/thrift/TFramedTransportFactory.java  |   11 +--
 4 files changed, 46 insertions(+), 88 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b973a333/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/ConfigHelper.java 
b/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
index 3dcfdd7..36228cf 100644
--- a/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
+++ b/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
@@ -566,21 +566,21 @@ public class ConfigHelper
 }
 }
 
-public static TClientTransportFactory 
getClientTransportFactory(Configuration conf)
+public static ITransportFactory getClientTransportFactory(Configuration 
conf)
 {
 String factoryClassName = conf.get(
-TClientTransportFactory.PROPERTY_KEY,
+ITransportFactory.PROPERTY_KEY,
 TFramedTransportFactory.class.getName());
-TClientTransportFactory factory = 
getClientTransportFactory(factoryClassName);
+ITransportFactory factory = 
getClientTransportFactory(factoryClassName);
 MapString, String options = getOptions(conf, 
factory.supportedOptions());
 factory.setOptions(options);
 return factory;
 }
 
-private static TClientTransportFactory getClientTransportFactory(String 
factoryClassName) {
+private static ITransportFactory getClientTransportFactory(String 
factoryClassName) {
 try
 {
-return (TClientTransportFactory) 
Class.forName(factoryClassName).newInstance();
+return (ITransportFactory) 
Class.forName(factoryClassName).newInstance();
 }
 catch (Exception e)
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b973a333/src/java/org/apache/cassandra/thrift/ITransportFactory.java
--
diff --git a/src/java/org/apache/cassandra/thrift/ITransportFactory.java 
b/src/java/org/apache/cassandra/thrift/ITransportFactory.java
index e3e87c2..98b1c03 100644
--- a/src/java/org/apache/cassandra/thrift/ITransportFactory.java
+++ b/src/java/org/apache/cassandra/thrift/ITransportFactory.java
@@ -22,14 +22,49 @@ package org.apache.cassandra.thrift;
  */
 
 import org.apache.hadoop.conf.Configuration;
-import org.apache.thrift.transport.TSocket;
 import org.apache.thrift.transport.TTransport;
-import org.apache.thrift.transport.TTransportException;
 
-import javax.security.auth.login.LoginException;
+import java.util.Map;
+import java.util.Set;
 
 
+/**
+ * Transport factory for establishing thrift connections from clients to a 
remote server.
+ */
 public interface ITransportFactory
 {
-TTransport openTransport(TSocket socket, Configuration conf) throws 
LoginException, TTransportException;
+static final String PROPERTY_KEY = cassandra.client.transport.factory;
+static final String LONG_OPTION = transport-factory;
+static final String SHORT_OPTION = tr;
+
+/**
+ * Opens a client transport to a thrift server.
+ * Example:
+ *
+ * pre
+ * TTransport transport = clientTransportFactory.openTransport(address, 
port);
+ * Cassandra.Iface client = new Cassandra.Client(new 
BinaryProtocol(transport));
+ * /pre
+ *
+ * @param host fully qualified hostname of the server
+ * @param port RPC port of the server
+ * @param conf Hadoop configuration
+ * @return open and ready to use transport
+ * @throws Exception implementation defined; usually throws 
TTransportException or IOException
+ * if the connection cannot be established
+ */
+TTransport openTransport(String host, int port, Configuration conf) throws 
Exception;
+
+/**
+ * Sets an implementation defined set of options.

[jira] [Commented] (CASSANDRA-4421) Support cql3 table definitions in Hadoop InputFormat

2013-06-06 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677404#comment-13677404
 ] 

Jonathan Ellis commented on CASSANDRA-4421:
---

(Post-commit, I renamed TClientTransportFactory to ITransportFactory.)

 Support cql3 table definitions in Hadoop InputFormat
 

 Key: CASSANDRA-4421
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4421
 Project: Cassandra
  Issue Type: Improvement
  Components: API
Affects Versions: 1.1.0
 Environment: Debian Squeeze
Reporter: bert Passek
Assignee: Alex Liu
  Labels: cql3
 Fix For: 1.2.6

 Attachments: 4421-10-patch-for-1.2.txt, 4421-10-patch-for-trunk.txt, 
 4421-1.txt, 4421-2.txt, 4421-3.txt, 4421-4.txt, 4421-5.txt, 4421-6.cb.txt, 
 4421-6-je.txt, 4421-7-je.txt, 4421-8-cb.txt, 4421-8-je.txt, 4421-9-je.txt, 
 4421.txt


 Hello,
 i faced a bug while writing composite column values and following validation 
 on server side.
 This is the setup for reproduction:
 1. create a keyspace
 create keyspace test with strategy_class = 'SimpleStrategy' and 
 strategy_options:replication_factor = 1;
 2. create a cf via cql (3.0)
 create table test1 (
 a int,
 b int,
 c int,
 primary key (a, b)
 );
 If i have a look at the schema in cli i noticed that there is no column 
 metadata for columns not part of primary key.
 create column family test1
   with column_type = 'Standard'
   and comparator = 
 'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
   and default_validation_class = 'UTF8Type'
   and key_validation_class = 'Int32Type'
   and read_repair_chance = 0.1
   and dclocal_read_repair_chance = 0.0
   and gc_grace = 864000
   and min_compaction_threshold = 4
   and max_compaction_threshold = 32
   and replicate_on_write = true
   and compaction_strategy = 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
   and caching = 'KEYS_ONLY'
   and compression_options = {'sstable_compression' : 
 'org.apache.cassandra.io.compress.SnappyCompressor'};
 Please notice the default validation class: UTF8Type
 Now i would like to insert value  127 via cassandra client (no cql, part of 
 mr-jobs). Have a look at the attachement.
 Batch mutate fails:
 InvalidRequestException(why:(String didn't validate.) [test][test1][1:c] 
 failed validation)
 A validator for column value is fetched in 
 ThriftValidation::validateColumnData which returns always the default 
 validator which is UTF8Type as described above (The ColumnDefinition for 
 given column name c is always null)
 In UTF8Type there is a check for
 if (b  127)
return false;
 Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table 
 creation. I assigned data types to all columns, but i can not set values for 
 a composite column because the default validation class is used.
 I think the schema should know the correct validator even for composite 
 columns. The usage of the default validation class does not make sense.
 Best Regards 
 Bert Passek

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5542) BulkLoader is broken in trunk

2013-06-06 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677407#comment-13677407
 ] 

Jonathan Ellis commented on CASSANDRA-5542:
---

Oh, I'm fine with querying remotely a la 4755.  I thought we were talking about 
opening up the system keyspace locally.

 BulkLoader is broken in trunk
 -

 Key: CASSANDRA-5542
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5542
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 2.0
Reporter: Yuki Morishita
Assignee: Yuki Morishita
 Fix For: 2.0

 Attachments: 5542-fix-NPE.txt, 5542.txt


 After CASSANDRA-5015 and CASSANDRA-5521, we need CFMetaData to open 
 SSTable(Reader), especially to get bloom_filter_fp_chance and index_interval.
 When using bulkloader, CFMetaData is not available, so we cannot open SSTable 
 to be streamed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: remove bad import

2013-06-06 Thread dbrosius 
Updated Branches:
  refs/heads/trunk 0eb048f23 - 9b0c3de85


remove bad import


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9b0c3de8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9b0c3de8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9b0c3de8

Branch: refs/heads/trunk
Commit: 9b0c3de85c37ebde3d0130eb29d000702ad509dc
Parents: 0eb048f
Author: Dave Brosius dbros...@apache.org
Authored: Thu Jun 6 15:41:04 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Thu Jun 6 15:41:04 2013 -0400

--
 .../cassandra/thrift/TFramedTransportFactory.java  |1 -
 1 files changed, 0 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b0c3de8/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
--
diff --git a/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java 
b/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
index 6907900..7d2d89e 100644
--- a/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
+++ b/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
@@ -32,7 +32,6 @@ import org.apache.thrift.transport.TTransport;
 import org.apache.thrift.transport.TTransportException;
 
 import org.apache.hadoop.conf.Configuration;
-import sun.plugin2.message.transport.TransportFactory;
 
 public class TFramedTransportFactory implements ITransportFactory
 {

git commit: remove bad import

2013-06-06 Thread dbrosius 
Updated Branches:
  refs/heads/cassandra-1.2 b973a3336 - fd6d19bd5


remove bad import


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fd6d19bd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fd6d19bd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fd6d19bd

Branch: refs/heads/cassandra-1.2
Commit: fd6d19bd5cdf764ce6667d1bd974795bb1d871fc
Parents: b973a33
Author: Dave Brosius dbros...@apache.org
Authored: Thu Jun 6 16:00:42 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Thu Jun 6 16:00:42 2013 -0400

--
 .../cassandra/thrift/TFramedTransportFactory.java  |1 -
 1 files changed, 0 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fd6d19bd/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
--
diff --git a/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java 
b/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
index 6907900..7d2d89e 100644
--- a/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
+++ b/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
@@ -32,7 +32,6 @@ import org.apache.thrift.transport.TTransport;
 import org.apache.thrift.transport.TTransportException;
 
 import org.apache.hadoop.conf.Configuration;
-import sun.plugin2.message.transport.TransportFactory;
 
 public class TFramedTransportFactory implements ITransportFactory
 {

[2/2] git commit: Merge branch 'cassandra-1.2' into trunk

2013-06-06 Thread dbrosius 
Merge branch 'cassandra-1.2' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6da1cbbc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6da1cbbc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6da1cbbc

Branch: refs/heads/trunk
Commit: 6da1cbbce740fd05c528ee1e627f7d8a89699f49
Parents: 9b0c3de fd6d19b
Author: Dave Brosius dbros...@apache.org
Authored: Thu Jun 6 16:01:24 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Thu Jun 6 16:01:24 2013 -0400

--

--

[1/2] git commit: remove bad import

2013-06-06 Thread dbrosius 
Updated Branches:
  refs/heads/trunk 9b0c3de85 - 6da1cbbce


remove bad import


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fd6d19bd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fd6d19bd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fd6d19bd

Branch: refs/heads/trunk
Commit: fd6d19bd5cdf764ce6667d1bd974795bb1d871fc
Parents: b973a33
Author: Dave Brosius dbros...@apache.org
Authored: Thu Jun 6 16:00:42 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Thu Jun 6 16:00:42 2013 -0400

--
 .../cassandra/thrift/TFramedTransportFactory.java  |1 -
 1 files changed, 0 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fd6d19bd/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
--
diff --git a/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java 
b/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
index 6907900..7d2d89e 100644
--- a/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
+++ b/src/java/org/apache/cassandra/thrift/TFramedTransportFactory.java
@@ -32,7 +32,6 @@ import org.apache.thrift.transport.TTransport;
 import org.apache.thrift.transport.TTransportException;
 
 import org.apache.hadoop.conf.Configuration;
-import sun.plugin2.message.transport.TransportFactory;
 
 public class TFramedTransportFactory implements ITransportFactory
 {

[jira] [Updated] (CASSANDRA-5383) Windows 7 deleting/renaming files problem

2013-06-06 Thread Ryan McGuire (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire updated CASSANDRA-5383:


Attachment: v2+cant_move_file_patch.log

[~krummas] Is the new patch 
(0001-CASSANDRA-5383-cant-move-a-file-on-top-of-another-fi.patch) meant to be 
applied on top of the v2 patch? 

I ran it that way and attached v2+cant-move-file-patch.log. I'm still seeing 
numberous FSWriteErrors.

 Windows 7 deleting/renaming files problem
 -

 Key: CASSANDRA-5383
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5383
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Affects Versions: 2.0
Reporter: Ryan McGuire
Assignee: Marcus Eriksson
 Fix For: 2.0.1

 Attachments: 
 0001-CASSANDRA-5383-cant-move-a-file-on-top-of-another-fi.patch, 
 0001-CASSANDRA-5383-v2.patch, 
 0001-use-Java7-apis-for-deleting-and-moving-files-and-cre.patch, 
 5383_patch_v2_system.log, test_log.5383.patch_v2.log.txt, 
 v2+cant_move_file_patch.log


 Two unit tests are failing on Windows 7 due to errors in renaming/deleting 
 files:
 org.apache.cassandra.db.ColumnFamilyStoreTest: 
 {code}
 [junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
 [junit] Tests run: 27, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 
 13.904 sec
 [junit] 
 [junit] - Standard Error -
 [junit] ERROR 13:06:46,058 Unable to delete 
 build\test\cassandra\data\Keyspace1\Indexed2\Keyspace1-Indexed2.birthdate_index-ja-1-Data.db
  (it will be removed on server restart; we'll also retry after GC)
 [junit] ERROR 13:06:48,508 Fatal exception in thread 
 Thread[NonPeriodicTasks:1,5,main]
 [junit] java.lang.RuntimeException: Tried to hard link to file that does 
 not exist 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-7-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:72)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1057)
 [junit]   at 
 org.apache.cassandra.db.DataTracker$1.run(DataTracker.java:168)
 [junit]   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 [junit]   at 
 java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 [junit]   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
 [junit]   at java.lang.Thread.run(Thread.java:662)
 [junit] -  ---
 [junit] Testcase: 
 testSliceByNamesCommandOldMetatada(org.apache.cassandra.db.ColumnFamilyStoreTest):
   Caused an ERROR
 [junit] Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit] java.lang.RuntimeException: Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:133)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:122)
 [junit]   at 
 org.apache.cassandra.db.compaction.LeveledManifest.mutateLevel(LeveledManifest.java:575)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:589)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetatada(ColumnFamilyStoreTest.java:885)
 [junit] 
 [junit] 
 [junit] Testcase: 
 testRemoveUnifinishedCompactionLeftovers(org.apache.cassandra.db.ColumnFamilyStoreTest):
 Caused an ERROR
 [junit] java.io.IOException: Failed to delete 
 c:\Users\Ryan\git\cassandra\build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit] FSWriteError in 
 build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:112)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:103)
 [junit]   at

[jira] [Created] (CASSANDRA-5621) Link for latest bleeding edge releases is broken

2013-06-06 Thread E O (JIRA) 
E O created CASSANDRA-5621:
--

 Summary: Link for latest bleeding edge releases is broken
 Key: CASSANDRA-5621
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5621
 Project: Cassandra
  Issue Type: Bug
  Components: Documentation  website
Reporter: E O
Priority: Trivial


This issue is just like 
[CASSANDRA-2736|https://issues.apache.org/jira/browse/CASSANDRA-2736]. The 
builds link on this [page|https://cassandra.apache.org/download/] points to 
[Jenkins continuous 
integration|https://builds.apache.org/job/Cassandra/lastSuccessfulBuild/artifact/cassandra/build/]
 which reports a 404 status.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5383) Windows 7 deleting/renaming files problem

2013-06-06 Thread Marcus Eriksson (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677521#comment-13677521
 ] 

Marcus Eriksson commented on CASSANDRA-5383:


[~enigmacurry] no the last patch should be applied alone, it is just a hack 
around the problem, not being able to replace files in windows

 Windows 7 deleting/renaming files problem
 -

 Key: CASSANDRA-5383
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5383
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Affects Versions: 2.0
Reporter: Ryan McGuire
Assignee: Marcus Eriksson
 Fix For: 2.0.1

 Attachments: 
 0001-CASSANDRA-5383-cant-move-a-file-on-top-of-another-fi.patch, 
 0001-CASSANDRA-5383-v2.patch, 
 0001-use-Java7-apis-for-deleting-and-moving-files-and-cre.patch, 
 5383_patch_v2_system.log, test_log.5383.patch_v2.log.txt, 
 v2+cant_move_file_patch.log


 Two unit tests are failing on Windows 7 due to errors in renaming/deleting 
 files:
 org.apache.cassandra.db.ColumnFamilyStoreTest: 
 {code}
 [junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
 [junit] Tests run: 27, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 
 13.904 sec
 [junit] 
 [junit] - Standard Error -
 [junit] ERROR 13:06:46,058 Unable to delete 
 build\test\cassandra\data\Keyspace1\Indexed2\Keyspace1-Indexed2.birthdate_index-ja-1-Data.db
  (it will be removed on server restart; we'll also retry after GC)
 [junit] ERROR 13:06:48,508 Fatal exception in thread 
 Thread[NonPeriodicTasks:1,5,main]
 [junit] java.lang.RuntimeException: Tried to hard link to file that does 
 not exist 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-7-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:72)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1057)
 [junit]   at 
 org.apache.cassandra.db.DataTracker$1.run(DataTracker.java:168)
 [junit]   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 [junit]   at 
 java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 [junit]   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
 [junit]   at java.lang.Thread.run(Thread.java:662)
 [junit] -  ---
 [junit] Testcase: 
 testSliceByNamesCommandOldMetatada(org.apache.cassandra.db.ColumnFamilyStoreTest):
   Caused an ERROR
 [junit] Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit] java.lang.RuntimeException: Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:133)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:122)
 [junit]   at 
 org.apache.cassandra.db.compaction.LeveledManifest.mutateLevel(LeveledManifest.java:575)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:589)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetatada(ColumnFamilyStoreTest.java:885)
 [junit] 
 [junit] 
 [junit] Testcase: 
 testRemoveUnifinishedCompactionLeftovers(org.apache.cassandra.db.ColumnFamilyStoreTest):
 Caused an ERROR
 [junit] java.io.IOException: Failed to delete 
 c:\Users\Ryan\git\cassandra\build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit] FSWriteError in 
 build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:112)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:103)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:139)
 [junit]   at

[jira] [Updated] (CASSANDRA-5383) Windows 7 deleting/renaming files problem

2013-06-06 Thread Ryan McGuire (JIRA) 

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire updated CASSANDRA-5383:


Attachment: cant_move_file_patch.log

OK, I reran it with just the 
0001-CASSANDRA-5383-cant-move-a-file-on-top-of-another-fi.patch applied to 
trunk. Attached cant_move_file_patch.log. I'm still seeing FSWriteErrors.

 Windows 7 deleting/renaming files problem
 -

 Key: CASSANDRA-5383
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5383
 Project: Cassandra
  Issue Type: Bug
  Components: Tests
Affects Versions: 2.0
Reporter: Ryan McGuire
Assignee: Marcus Eriksson
 Fix For: 2.0.1

 Attachments: 
 0001-CASSANDRA-5383-cant-move-a-file-on-top-of-another-fi.patch, 
 0001-CASSANDRA-5383-v2.patch, 
 0001-use-Java7-apis-for-deleting-and-moving-files-and-cre.patch, 
 5383_patch_v2_system.log, cant_move_file_patch.log, 
 test_log.5383.patch_v2.log.txt, v2+cant_move_file_patch.log


 Two unit tests are failing on Windows 7 due to errors in renaming/deleting 
 files:
 org.apache.cassandra.db.ColumnFamilyStoreTest: 
 {code}
 [junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
 [junit] Tests run: 27, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 
 13.904 sec
 [junit] 
 [junit] - Standard Error -
 [junit] ERROR 13:06:46,058 Unable to delete 
 build\test\cassandra\data\Keyspace1\Indexed2\Keyspace1-Indexed2.birthdate_index-ja-1-Data.db
  (it will be removed on server restart; we'll also retry after GC)
 [junit] ERROR 13:06:48,508 Fatal exception in thread 
 Thread[NonPeriodicTasks:1,5,main]
 [junit] java.lang.RuntimeException: Tried to hard link to file that does 
 not exist 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-7-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:72)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1057)
 [junit]   at 
 org.apache.cassandra.db.DataTracker$1.run(DataTracker.java:168)
 [junit]   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
 [junit]   at 
 java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 [junit]   at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
 [junit]   at 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
 [junit]   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
 [junit]   at java.lang.Thread.run(Thread.java:662)
 [junit] -  ---
 [junit] Testcase: 
 testSliceByNamesCommandOldMetatada(org.apache.cassandra.db.ColumnFamilyStoreTest):
   Caused an ERROR
 [junit] Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit] java.lang.RuntimeException: Failed to rename 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
  to 
 build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:133)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:122)
 [junit]   at 
 org.apache.cassandra.db.compaction.LeveledManifest.mutateLevel(LeveledManifest.java:575)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:589)
 [junit]   at 
 org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetatada(ColumnFamilyStoreTest.java:885)
 [junit] 
 [junit] 
 [junit] Testcase: 
 testRemoveUnifinishedCompactionLeftovers(org.apache.cassandra.db.ColumnFamilyStoreTest):
 Caused an ERROR
 [junit] java.io.IOException: Failed to delete 
 c:\Users\Ryan\git\cassandra\build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit] FSWriteError in 
 build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:112)
 [junit]   at 
 org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:103)
 [junit]   at 
 org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:139)

[3/9] rename

2013-06-06 Thread jbellis 
http://git-wip-us.apache.org/repos/asf/cassandra/blob/02a7ba81/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
--
diff --git 
a/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
new file mode 100644
index 000..3a0f628
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
@@ -0,0 +1,763 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.hadoop.cql3;
+
+import java.io.IOException;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.ByteBuffer;
+import java.nio.charset.CharacterCodingException;
+import java.util.*;
+
+import com.google.common.collect.AbstractIterator;
+import com.google.common.collect.Iterables;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.marshal.CompositeType;
+import org.apache.cassandra.db.marshal.LongType;
+import org.apache.cassandra.db.marshal.TypeParser;
+import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.exceptions.SyntaxException;
+import org.apache.cassandra.hadoop.ColumnFamilySplit;
+import org.apache.cassandra.hadoop.ConfigHelper;
+import org.apache.cassandra.thrift.*;
+import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.FBUtilities;
+import org.apache.cassandra.utils.Pair;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.mapreduce.InputSplit;
+import org.apache.hadoop.mapreduce.RecordReader;
+import org.apache.hadoop.mapreduce.TaskAttemptContext;
+import org.apache.thrift.TException;
+import org.apache.thrift.transport.TTransport;
+
+/**
+ * Hadoop RecordReader read the values return from the CQL query
+ * It use CQL key range query to page through the wide rows.
+ * p/
+ * Return ListIColumn as keys columns
+ * p/
+ * MapByteBuffer, IColumn as column name to columns mappings
+ */
+public class CqlPagingRecordReader extends RecordReaderMapString, 
ByteBuffer, MapString, ByteBuffer
+implements org.apache.hadoop.mapred.RecordReaderMapString, 
ByteBuffer, MapString, ByteBuffer
+{
+private static final Logger logger = 
LoggerFactory.getLogger(CqlPagingRecordReader.class);
+
+public static final int DEFAULT_CQL_PAGE_LIMIT = 1000; // TODO: find the 
number large enough but not OOM
+
+private ColumnFamilySplit split;
+private RowIterator rowIterator;
+
+private PairMapString, ByteBuffer, MapString, ByteBuffer currentRow;
+private int totalRowCount; // total number of rows to fetch
+private String keyspace;
+private String cfName;
+private Cassandra.Client client;
+private ConsistencyLevel consistencyLevel;
+
+// partition keys -- key aliases
+private ListBoundColumn partitionBoundColumns = new 
ArrayListBoundColumn();
+
+// cluster keys -- column aliases
+private ListBoundColumn clusterColumns = new ArrayListBoundColumn();
+
+// map prepared query type to item id
+private MapInteger, Integer preparedQueryIds = new HashMapInteger, 
Integer();
+
+// cql query select columns
+private String columns;
+
+// the number of cql rows per page
+private int pageRowSize;
+
+// user defined where clauses
+private String userDefinedWhereClauses;
+
+private IPartitioner partitioner;
+
+private AbstractType? keyValidator;
+
+public CqlPagingRecordReader()
+{
+super();
+}
+
+public void initialize(InputSplit split, TaskAttemptContext context) 
throws IOException
+{
+this.split = (ColumnFamilySplit) split;
+Configuration conf = context.getConfiguration();
+totalRowCount = (this.split.getLength()  Long.MAX_VALUE)
+  ? (int) this.split.getLength()
+  : ConfigHelper.getInputSplitSize(conf);
+cfName = ConfigHelper.getInputColumnFamily(conf);
+consistencyLevel =

[9/9] git commit: Merge remote-tracking branch 'origin/trunk' into trunk

2013-06-06 Thread jbellis 
Merge remote-tracking branch 'origin/trunk' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2e9f51d1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2e9f51d1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2e9f51d1

Branch: refs/heads/trunk
Commit: 2e9f51d16d29782b55aa2b9a10b23849c01b9a61
Parents: c1dfc8f 6da1cbb
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Jun 6 17:01:02 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Jun 6 17:01:02 2013 -0500

--
 .../cassandra/thrift/TFramedTransportFactory.java  |1 -
 1 files changed, 0 insertions(+), 1 deletions(-)
--

[6/9] git commit: cleanup

2013-06-06 Thread jbellis 
cleanup


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ff5b8304
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ff5b8304
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ff5b8304

Branch: refs/heads/cassandra-1.2
Commit: ff5b8304431477eb82fc13ab1344a2fc92c66a57
Parents: fd6d19b
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Jun 6 16:47:03 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Jun 6 17:00:55 2013 -0500

--
 .../hadoop/cql3/ColumnFamilyRecordWriter.java  |   43 +++
 1 files changed, 20 insertions(+), 23 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff5b8304/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java 
b/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
index 3939e0b..c32d9ef 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
@@ -152,16 +152,16 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
  * (i.e., null), then the entire key is marked for {@link Deletion}.
  * /p
  *
- * @param keybuff
+ * @param keyColumns
  *the key to write.
  * @param values
  *the values to write.
  * @throws IOException
  */
 @Override
-public void write(MapString, ByteBuffer keys, ListByteBuffer values) 
throws IOException
+public void write(MapString, ByteBuffer keyColumns, ListByteBuffer 
values) throws IOException
 {
-ByteBuffer rowKey = getRowKey(keys);
+ByteBuffer rowKey = getRowKey(keyColumns);
 RangeToken range = ringCache.getRange(rowKey);
 
 // get the client for the given range, or create a new one
@@ -180,7 +180,7 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 
 /**
  * A client that runs in a threadpool and connects to the list of 
endpoints for a particular
- * range. Binded variable values for keys in that range are sent to this 
client via a queue.
+ * range. Bound variables for keys in that range are sent to this client 
via a queue.
  */
 public class RangeClient extends AbstractRangeClientListByteBuffer
 {
@@ -201,10 +201,10 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 outer:
 while (run || !queue.isEmpty())
 {
-PairByteBuffer, ListByteBuffer bindVariables;
+PairByteBuffer, ListByteBuffer item;
 try
 {
-bindVariables = queue.take();
+item = queue.take();
 }
 catch (InterruptedException e)
 {
@@ -220,15 +220,16 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 {
 int i = 0;
 int itemId = preparedStatement(client);
-while (bindVariables != null)
+while (item != null)
 {
-client.execute_prepared_cql3_query(itemId, 
bindVariables.right, ConsistencyLevel.ONE);
+ListByteBuffer bindVariables = item.right;
+client.execute_prepared_cql3_query(itemId, 
bindVariables, ConsistencyLevel.ONE);
 i++;
 
 if (i = batchThreshold)
 break;
 
-bindVariables = queue.poll();
+item = queue.poll();
 }
 
 break;
@@ -293,7 +294,7 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 }
 }
 
-private ByteBuffer getRowKey(MapString, ByteBuffer keysMap)
+private ByteBuffer getRowKey(MapString, ByteBuffer keyColumns)
 {
 //current row key
 ByteBuffer rowKey;
@@ -301,13 +302,13 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 {
 ByteBuffer[] keys = new ByteBuffer[partitionkeys.length];
 for (int i = 0; i keys.length; i++)
-keys[i] = keysMap.get(partitionkeys[i]);
+keys[i] = keyColumns.get(partitionkeys[i]);
 
 rowKey = ((CompositeType) keyValidator).build(keys);
 }

[2/9] git commit: Merge branch 'cassandra-1.2' into trunk

2013-06-06 Thread jbellis 
Merge branch 'cassandra-1.2' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6a360a06
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6a360a06
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6a360a06

Branch: refs/heads/trunk
Commit: 6a360a06fe2d49ea165768f9406ab277de88a968
Parents: 0eb048f a2436f0
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Jun 6 16:47:08 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Jun 6 16:47:08 2013 -0500

--
 .../hadoop/cql3/ColumnFamilyRecordWriter.java  |   43 +++
 1 files changed, 20 insertions(+), 23 deletions(-)
--

[5/9] git commit: Merge branch 'cassandra-1.2' into trunk

2013-06-06 Thread jbellis 
Merge branch 'cassandra-1.2' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c1dfc8fc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c1dfc8fc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c1dfc8fc

Branch: refs/heads/trunk
Commit: c1dfc8fc53f65da5c9d9c7b4a3234200a3616d78
Parents: 6a360a0 02a7ba8
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Jun 6 17:00:28 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Jun 6 17:00:28 2013 -0500

--
 examples/hadoop_cql3_word_count/src/WordCount.java |   18 +-
 .../src/WordCountCounters.java |   11 +-
 .../cassandra/hadoop/cql3/CQLConfigHelper.java |2 +-
 .../hadoop/cql3/ColumnFamilyInputFormat.java   |   83 --
 .../hadoop/cql3/ColumnFamilyOutputFormat.java  |   78 --
 .../hadoop/cql3/ColumnFamilyRecordReader.java  |  763 ---
 .../hadoop/cql3/ColumnFamilyRecordWriter.java  |  383 
 .../cassandra/hadoop/cql3/CqlOutputFormat.java |   78 ++
 .../hadoop/cql3/CqlPagingInputFormat.java  |   83 ++
 .../hadoop/cql3/CqlPagingRecordReader.java |  763 +++
 .../cassandra/hadoop/cql3/CqlRecordWriter.java |  383 
 11 files changed, 1320 insertions(+), 1325 deletions(-)
--

[7/9] rename

2013-06-06 Thread jbellis 
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a0047797/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
--
diff --git 
a/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
new file mode 100644
index 000..3a0f628
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
@@ -0,0 +1,763 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.hadoop.cql3;
+
+import java.io.IOException;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.ByteBuffer;
+import java.nio.charset.CharacterCodingException;
+import java.util.*;
+
+import com.google.common.collect.AbstractIterator;
+import com.google.common.collect.Iterables;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.marshal.CompositeType;
+import org.apache.cassandra.db.marshal.LongType;
+import org.apache.cassandra.db.marshal.TypeParser;
+import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.exceptions.SyntaxException;
+import org.apache.cassandra.hadoop.ColumnFamilySplit;
+import org.apache.cassandra.hadoop.ConfigHelper;
+import org.apache.cassandra.thrift.*;
+import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.FBUtilities;
+import org.apache.cassandra.utils.Pair;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.mapreduce.InputSplit;
+import org.apache.hadoop.mapreduce.RecordReader;
+import org.apache.hadoop.mapreduce.TaskAttemptContext;
+import org.apache.thrift.TException;
+import org.apache.thrift.transport.TTransport;
+
+/**
+ * Hadoop RecordReader read the values return from the CQL query
+ * It use CQL key range query to page through the wide rows.
+ * p/
+ * Return ListIColumn as keys columns
+ * p/
+ * MapByteBuffer, IColumn as column name to columns mappings
+ */
+public class CqlPagingRecordReader extends RecordReaderMapString, 
ByteBuffer, MapString, ByteBuffer
+implements org.apache.hadoop.mapred.RecordReaderMapString, 
ByteBuffer, MapString, ByteBuffer
+{
+private static final Logger logger = 
LoggerFactory.getLogger(CqlPagingRecordReader.class);
+
+public static final int DEFAULT_CQL_PAGE_LIMIT = 1000; // TODO: find the 
number large enough but not OOM
+
+private ColumnFamilySplit split;
+private RowIterator rowIterator;
+
+private PairMapString, ByteBuffer, MapString, ByteBuffer currentRow;
+private int totalRowCount; // total number of rows to fetch
+private String keyspace;
+private String cfName;
+private Cassandra.Client client;
+private ConsistencyLevel consistencyLevel;
+
+// partition keys -- key aliases
+private ListBoundColumn partitionBoundColumns = new 
ArrayListBoundColumn();
+
+// cluster keys -- column aliases
+private ListBoundColumn clusterColumns = new ArrayListBoundColumn();
+
+// map prepared query type to item id
+private MapInteger, Integer preparedQueryIds = new HashMapInteger, 
Integer();
+
+// cql query select columns
+private String columns;
+
+// the number of cql rows per page
+private int pageRowSize;
+
+// user defined where clauses
+private String userDefinedWhereClauses;
+
+private IPartitioner partitioner;
+
+private AbstractType? keyValidator;
+
+public CqlPagingRecordReader()
+{
+super();
+}
+
+public void initialize(InputSplit split, TaskAttemptContext context) 
throws IOException
+{
+this.split = (ColumnFamilySplit) split;
+Configuration conf = context.getConfiguration();
+totalRowCount = (this.split.getLength()  Long.MAX_VALUE)
+  ? (int) this.split.getLength()
+  : ConfigHelper.getInputSplitSize(conf);
+cfName = ConfigHelper.getInputColumnFamily(conf);
+consistencyLevel =

[1/9] git commit: cleanup

2013-06-06 Thread jbellis 
Updated Branches:
  refs/heads/cassandra-1.2 fd6d19bd5 - a00477976
  refs/heads/trunk 6da1cbbce - 2e9f51d16


cleanup


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a2436f07
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a2436f07
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a2436f07

Branch: refs/heads/trunk
Commit: a2436f07a45ba81cd32b0f915a32d0f83f19538e
Parents: b973a33
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Jun 6 16:47:03 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Jun 6 16:47:03 2013 -0500

--
 .../hadoop/cql3/ColumnFamilyRecordWriter.java  |   43 +++
 1 files changed, 20 insertions(+), 23 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a2436f07/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java 
b/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
index 3939e0b..c32d9ef 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
@@ -152,16 +152,16 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
  * (i.e., null), then the entire key is marked for {@link Deletion}.
  * /p
  *
- * @param keybuff
+ * @param keyColumns
  *the key to write.
  * @param values
  *the values to write.
  * @throws IOException
  */
 @Override
-public void write(MapString, ByteBuffer keys, ListByteBuffer values) 
throws IOException
+public void write(MapString, ByteBuffer keyColumns, ListByteBuffer 
values) throws IOException
 {
-ByteBuffer rowKey = getRowKey(keys);
+ByteBuffer rowKey = getRowKey(keyColumns);
 RangeToken range = ringCache.getRange(rowKey);
 
 // get the client for the given range, or create a new one
@@ -180,7 +180,7 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 
 /**
  * A client that runs in a threadpool and connects to the list of 
endpoints for a particular
- * range. Binded variable values for keys in that range are sent to this 
client via a queue.
+ * range. Bound variables for keys in that range are sent to this client 
via a queue.
  */
 public class RangeClient extends AbstractRangeClientListByteBuffer
 {
@@ -201,10 +201,10 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 outer:
 while (run || !queue.isEmpty())
 {
-PairByteBuffer, ListByteBuffer bindVariables;
+PairByteBuffer, ListByteBuffer item;
 try
 {
-bindVariables = queue.take();
+item = queue.take();
 }
 catch (InterruptedException e)
 {
@@ -220,15 +220,16 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 {
 int i = 0;
 int itemId = preparedStatement(client);
-while (bindVariables != null)
+while (item != null)
 {
-client.execute_prepared_cql3_query(itemId, 
bindVariables.right, ConsistencyLevel.ONE);
+ListByteBuffer bindVariables = item.right;
+client.execute_prepared_cql3_query(itemId, 
bindVariables, ConsistencyLevel.ONE);
 i++;
 
 if (i = batchThreshold)
 break;
 
-bindVariables = queue.poll();
+item = queue.poll();
 }
 
 break;
@@ -293,7 +294,7 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 }
 }
 
-private ByteBuffer getRowKey(MapString, ByteBuffer keysMap)
+private ByteBuffer getRowKey(MapString, ByteBuffer keyColumns)
 {
 //current row key
 ByteBuffer rowKey;
@@ -301,13 +302,13 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 {
 ByteBuffer[] keys = new ByteBuffer[partitionkeys.length];
 for (int i = 0; i keys.length; i++)
-keys[i] = keysMap.get(partitionkeys[i]);
+keys[i] =

[jira] [Created] (CASSANDRA-5622) Update CqlRecordWriter to conform to RecordWriter API

2013-06-06 Thread Jonathan Ellis (JIRA) 
Jonathan Ellis created CASSANDRA-5622:
-

 Summary: Update CqlRecordWriter to conform to RecordWriter API
 Key: CASSANDRA-5622
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5622
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Jonathan Ellis
Assignee: Alex Liu
 Fix For: 1.2.6


{{RecordWriterK, V}} is supposed to write values V that can be uniquely 
identified by keys K.

Currently CqlRW requires the user to give it all the bind variables for a 
complete statement in V, and effectively ignores K.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: fix capitalization of file name

2013-06-06 Thread dbrosius 
Updated Branches:
  refs/heads/cassandra-1.2 a00477976 - 60b3584e0


fix capitalization of file name


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/60b3584e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/60b3584e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/60b3584e

Branch: refs/heads/cassandra-1.2
Commit: 60b3584e01773a14984a8e4433307205c4e32ddc
Parents: a004779
Author: Dave Brosius dbros...@apache.org
Authored: Thu Jun 6 18:15:08 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Thu Jun 6 18:15:08 2013 -0400

--
 .../cassandra/hadoop/cql3/CQLConfigHelper.java |  109 ---
 .../cassandra/hadoop/cql3/CqlConfigHelper.java |  109 +++
 2 files changed, 109 insertions(+), 109 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/60b3584e/src/java/org/apache/cassandra/hadoop/cql3/CQLConfigHelper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CQLConfigHelper.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CQLConfigHelper.java
deleted file mode 100644
index cb61d05..000
--- a/src/java/org/apache/cassandra/hadoop/cql3/CQLConfigHelper.java
+++ /dev/null
@@ -1,109 +0,0 @@
-package org.apache.cassandra.hadoop.cql3;
-/*
-*
-* Licensed to the Apache Software Foundation (ASF) under one
-* or more contributor license agreements.  See the NOTICE file
-* distributed with this work for additional information
-* regarding copyright ownership.  The ASF licenses this file
-* to you under the Apache License, Version 2.0 (the
-* License); you may not use this file except in compliance
-* with the License.  You may obtain a copy of the License at
-*
-*   http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing,
-* software distributed under the License is distributed on an
-* AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-* KIND, either express or implied.  See the License for the
-* specific language governing permissions and limitations
-* under the License.
-*
-*/
-import org.apache.hadoop.conf.Configuration;
-
-public class CqlConfigHelper
-{
-private static final String INPUT_CQL_COLUMNS_CONFIG = 
cassandra.input.columnfamily.columns; // separate by colon ,
-private static final String INPUT_CQL_PAGE_ROW_SIZE_CONFIG = 
cassandra.input.page.row.size;
-private static final String INPUT_CQL_WHERE_CLAUSE_CONFIG = 
cassandra.input.where.clause;
-private static final String OUTPUT_CQL = cassandra.output.cql;
-
-/**
- * Set the CQL columns for the input of this job.
- *
- * @param conf Job configuration you are about to run
- * @param columns
- */
-public static void setInputColumns(Configuration conf, String columns)
-{
-if (columns == null || columns.isEmpty())
-return;
-
-conf.set(INPUT_CQL_COLUMNS_CONFIG, columns);
-}
-
-/**
- * Set the CQL query Limit for the input of this job.
- *
- * @param conf Job configuration you are about to run
- * @param cqlPageRowSize
- */
-public static void setInputCQLPageRowSize(Configuration conf, String 
cqlPageRowSize)
-{
-if (cqlPageRowSize == null)
-{
-throw new UnsupportedOperationException(cql page row size may not 
be null);
-}
-
-conf.set(INPUT_CQL_PAGE_ROW_SIZE_CONFIG, cqlPageRowSize);
-}
-
-/**
- * Set the CQL user defined where clauses for the input of this job.
- *
- * @param conf Job configuration you are about to run
- * @param clauses
- */
-public static void setInputWhereClauses(Configuration conf, String clauses)
-{
-if (clauses == null || clauses.isEmpty())
-return;
-
-conf.set(INPUT_CQL_WHERE_CLAUSE_CONFIG, clauses);
-}
-  
-/**
- * Set the CQL prepared statement for the output of this job.
- *
- * @param conf Job configuration you are about to run
- * @param cql
- */
-public static void setOutputCql(Configuration conf, String cql)
-{
-if (cql == null || cql.isEmpty())
-return;
-
-conf.set(OUTPUT_CQL, cql);
-}
-
-
-public static String getInputcolumns(Configuration conf)
-{
-return conf.get(INPUT_CQL_COLUMNS_CONFIG);
-}
-
-public static String getInputPageRowSize(Configuration conf)
-{
-return conf.get(INPUT_CQL_PAGE_ROW_SIZE_CONFIG);
-}
-
-public static String getInputWhereClauses(Configuration conf)
-{
-return conf.get(INPUT_CQL_WHERE_CLAUSE_CONFIG);
-}
-
-public static String getOutputCql(Configuration conf)
-{
-return

[jira] [Comment Edited] (CASSANDRA-5622) Update CqlRecordWriter to conform to RecordWriter API

2013-06-06 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677593#comment-13677593
 ] 

Jonathan Ellis edited comment on CASSANDRA-5622 at 6/6/13 10:16 PM:


Two solutions suggest themselves:
# Embrace ignoring of K and just pass new Object()
# Treat K as designed and accept responsibility for generating appropriate bind 
variables

I'm not a huge fan of #1 since it forces the user to do more work to generate 
bind variables for the PK values.  I'd rather automate that.

Let me give an example of what I mean in #2.  Suppose we have word counts as 
follows:
{code}
CREATE TABLE word_counts (
  file_name text,
  counted_at timestamp,
  word text,
  count int,
  PRIMARY KEY ((file_name, counted_at), word)
);
{code}

Then the user would configure CQL such as UPDATE word_counts SET count = ?.  
CqlRW would inspect the table definition and realize it needs to add  WHERE 
file_name = ? AND counted_at = ? AND word = ?.

Then when it gets a MapString, BB as K, it combines those with the ListBB 
variables.  It knows the ordering of the pk columns since it generated it 
itself.

One downside: this doesn't lend itself to INSERT statements, since that does 
not separate SET/WHERE the way UPDATE does.  This is a limitation I can live 
with.

  was (Author: jbellis):
Two solutions suggest themselves:
# Embrace ignoring of K and just pass new Object()
# Treat K as designed and accept responsibility for generating appropriate bind 
variables

I'm not a huge fan of #1 since it forces the user to do more work to generate 
bind variables for the PK values.  I'd rather automate that.

Let me give an example of what I mean in #2.  Suppose we have word counts as 
follows:
{code}
CREATE TABLE word_counts (
  file_name text,
  counted_at timestamp,
  word text,
  count int,
  PRIMARY KEY ((file_name, counted_at), word)
);
{code}

Then the user would configuer CQL such as UPDATE word_counts SET count = ?.  
CqlRW would inspect the table definition and realize it needs to add  WHERE 
file_name = ? AND counted_at = ? AND word = ?.

Then when it gets a MapString, BB as K, it combines those with the ListBB 
variables.  It knows the ordering of the pk columns since it generated it 
itself.

One downside: this doesn't lend itself to INSERT statements, since that does 
not separate SET/WHERE the way UPDATE does.  This is a limitation I can live 
with.
  
 Update CqlRecordWriter to conform to RecordWriter API
 -

 Key: CASSANDRA-5622
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5622
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Jonathan Ellis
Assignee: Alex Liu
 Fix For: 1.2.6


 {{RecordWriterK, V}} is supposed to write values V that can be uniquely 
 identified by keys K.
 Currently CqlRW requires the user to give it all the bind variables for a 
 complete statement in V, and effectively ignores K.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5622) Update CqlRecordWriter to conform to RecordWriter API

2013-06-06 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677593#comment-13677593
 ] 

Jonathan Ellis commented on CASSANDRA-5622:
---

Two solutions suggest themselves:
# Embrace ignoring of K and just pass new Object()
# Treat K as designed and accept responsibility for generating appropriate bind 
variables

I'm not a huge fan of #1 since it forces the user to do more work to generate 
bind variables for the PK values.  I'd rather automate that.

Let me give an example of what I mean in #2.  Suppose we have word counts as 
follows:
{code}
CREATE TABLE word_counts (
  file_name text,
  counted_at timestamp,
  word text,
  count int,
  PRIMARY KEY ((file_name, counted_at), word)
);
{code}

Then the user would configuer CQL such as UPDATE word_counts SET count = ?.  
CqlRW would inspect the table definition and realize it needs to add  WHERE 
file_name = ? AND counted_at = ? AND word = ?.

Then when it gets a MapString, BB as K, it combines those with the ListBB 
variables.  It knows the ordering of the pk columns since it generated it 
itself.

One downside: this doesn't lend itself to INSERT statements, since that does 
not separate SET/WHERE the way UPDATE does.  This is a limitation I can live 
with.

 Update CqlRecordWriter to conform to RecordWriter API
 -

 Key: CASSANDRA-5622
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5622
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Jonathan Ellis
Assignee: Alex Liu
 Fix For: 1.2.6


 {{RecordWriterK, V}} is supposed to write values V that can be uniquely 
 identified by keys K.
 Currently CqlRW requires the user to give it all the bind variables for a 
 complete statement in V, and effectively ignores K.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[2/5] rename

2013-06-06 Thread dbrosius 
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a0047797/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
--
diff --git 
a/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
new file mode 100644
index 000..3a0f628
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlPagingRecordReader.java
@@ -0,0 +1,763 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.hadoop.cql3;
+
+import java.io.IOException;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.ByteBuffer;
+import java.nio.charset.CharacterCodingException;
+import java.util.*;
+
+import com.google.common.collect.AbstractIterator;
+import com.google.common.collect.Iterables;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.marshal.CompositeType;
+import org.apache.cassandra.db.marshal.LongType;
+import org.apache.cassandra.db.marshal.TypeParser;
+import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.exceptions.SyntaxException;
+import org.apache.cassandra.hadoop.ColumnFamilySplit;
+import org.apache.cassandra.hadoop.ConfigHelper;
+import org.apache.cassandra.thrift.*;
+import org.apache.cassandra.utils.ByteBufferUtil;
+import org.apache.cassandra.utils.FBUtilities;
+import org.apache.cassandra.utils.Pair;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.mapreduce.InputSplit;
+import org.apache.hadoop.mapreduce.RecordReader;
+import org.apache.hadoop.mapreduce.TaskAttemptContext;
+import org.apache.thrift.TException;
+import org.apache.thrift.transport.TTransport;
+
+/**
+ * Hadoop RecordReader read the values return from the CQL query
+ * It use CQL key range query to page through the wide rows.
+ * p/
+ * Return ListIColumn as keys columns
+ * p/
+ * MapByteBuffer, IColumn as column name to columns mappings
+ */
+public class CqlPagingRecordReader extends RecordReaderMapString, 
ByteBuffer, MapString, ByteBuffer
+implements org.apache.hadoop.mapred.RecordReaderMapString, 
ByteBuffer, MapString, ByteBuffer
+{
+private static final Logger logger = 
LoggerFactory.getLogger(CqlPagingRecordReader.class);
+
+public static final int DEFAULT_CQL_PAGE_LIMIT = 1000; // TODO: find the 
number large enough but not OOM
+
+private ColumnFamilySplit split;
+private RowIterator rowIterator;
+
+private PairMapString, ByteBuffer, MapString, ByteBuffer currentRow;
+private int totalRowCount; // total number of rows to fetch
+private String keyspace;
+private String cfName;
+private Cassandra.Client client;
+private ConsistencyLevel consistencyLevel;
+
+// partition keys -- key aliases
+private ListBoundColumn partitionBoundColumns = new 
ArrayListBoundColumn();
+
+// cluster keys -- column aliases
+private ListBoundColumn clusterColumns = new ArrayListBoundColumn();
+
+// map prepared query type to item id
+private MapInteger, Integer preparedQueryIds = new HashMapInteger, 
Integer();
+
+// cql query select columns
+private String columns;
+
+// the number of cql rows per page
+private int pageRowSize;
+
+// user defined where clauses
+private String userDefinedWhereClauses;
+
+private IPartitioner partitioner;
+
+private AbstractType? keyValidator;
+
+public CqlPagingRecordReader()
+{
+super();
+}
+
+public void initialize(InputSplit split, TaskAttemptContext context) 
throws IOException
+{
+this.split = (ColumnFamilySplit) split;
+Configuration conf = context.getConfiguration();
+totalRowCount = (this.split.getLength()  Long.MAX_VALUE)
+  ? (int) this.split.getLength()
+  : ConfigHelper.getInputSplitSize(conf);
+cfName = ConfigHelper.getInputColumnFamily(conf);
+consistencyLevel =

[1/5] git commit: cleanup

2013-06-06 Thread dbrosius 
Updated Branches:
  refs/heads/trunk 2e9f51d16 - 6b16c1a57


cleanup


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ff5b8304
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ff5b8304
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ff5b8304

Branch: refs/heads/trunk
Commit: ff5b8304431477eb82fc13ab1344a2fc92c66a57
Parents: fd6d19b
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Jun 6 16:47:03 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Jun 6 17:00:55 2013 -0500

--
 .../hadoop/cql3/ColumnFamilyRecordWriter.java  |   43 +++
 1 files changed, 20 insertions(+), 23 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff5b8304/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java 
b/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
index 3939e0b..c32d9ef 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/ColumnFamilyRecordWriter.java
@@ -152,16 +152,16 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
  * (i.e., null), then the entire key is marked for {@link Deletion}.
  * /p
  *
- * @param keybuff
+ * @param keyColumns
  *the key to write.
  * @param values
  *the values to write.
  * @throws IOException
  */
 @Override
-public void write(MapString, ByteBuffer keys, ListByteBuffer values) 
throws IOException
+public void write(MapString, ByteBuffer keyColumns, ListByteBuffer 
values) throws IOException
 {
-ByteBuffer rowKey = getRowKey(keys);
+ByteBuffer rowKey = getRowKey(keyColumns);
 RangeToken range = ringCache.getRange(rowKey);
 
 // get the client for the given range, or create a new one
@@ -180,7 +180,7 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 
 /**
  * A client that runs in a threadpool and connects to the list of 
endpoints for a particular
- * range. Binded variable values for keys in that range are sent to this 
client via a queue.
+ * range. Bound variables for keys in that range are sent to this client 
via a queue.
  */
 public class RangeClient extends AbstractRangeClientListByteBuffer
 {
@@ -201,10 +201,10 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 outer:
 while (run || !queue.isEmpty())
 {
-PairByteBuffer, ListByteBuffer bindVariables;
+PairByteBuffer, ListByteBuffer item;
 try
 {
-bindVariables = queue.take();
+item = queue.take();
 }
 catch (InterruptedException e)
 {
@@ -220,15 +220,16 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 {
 int i = 0;
 int itemId = preparedStatement(client);
-while (bindVariables != null)
+while (item != null)
 {
-client.execute_prepared_cql3_query(itemId, 
bindVariables.right, ConsistencyLevel.ONE);
+ListByteBuffer bindVariables = item.right;
+client.execute_prepared_cql3_query(itemId, 
bindVariables, ConsistencyLevel.ONE);
 i++;
 
 if (i = batchThreshold)
 break;
 
-bindVariables = queue.poll();
+item = queue.poll();
 }
 
 break;
@@ -293,7 +294,7 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 }
 }
 
-private ByteBuffer getRowKey(MapString, ByteBuffer keysMap)
+private ByteBuffer getRowKey(MapString, ByteBuffer keyColumns)
 {
 //current row key
 ByteBuffer rowKey;
@@ -301,13 +302,13 @@ final class ColumnFamilyRecordWriter extends 
AbstractColumnFamilyRecordWriterMa
 {
 ByteBuffer[] keys = new ByteBuffer[partitionkeys.length];
 for (int i = 0; i keys.length; i++)
-keys[i] = keysMap.get(partitionkeys[i]);
+keys[i] = keyColumns.get(partitionkeys[i]);
 
 rowKey =

[5/5] git commit: Merge branch 'cassandra-1.2' into trunk

2013-06-06 Thread dbrosius 
Merge branch 'cassandra-1.2' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b16c1a5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b16c1a5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b16c1a5

Branch: refs/heads/trunk
Commit: 6b16c1a570c608ba1e1dd15a3213e42f21b58c96
Parents: 2e9f51d 60b3584
Author: Dave Brosius dbros...@apache.org
Authored: Thu Jun 6 18:15:40 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Thu Jun 6 18:15:40 2013 -0400

--
 .../cassandra/hadoop/cql3/CQLConfigHelper.java |  109 ---
 .../cassandra/hadoop/cql3/CqlConfigHelper.java |  109 +++
 2 files changed, 109 insertions(+), 109 deletions(-)
--

[4/5] git commit: fix capitalization of file name

2013-06-06 Thread dbrosius 
fix capitalization of file name


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/60b3584e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/60b3584e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/60b3584e

Branch: refs/heads/trunk
Commit: 60b3584e01773a14984a8e4433307205c4e32ddc
Parents: a004779
Author: Dave Brosius dbros...@apache.org
Authored: Thu Jun 6 18:15:08 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Thu Jun 6 18:15:08 2013 -0400

--
 .../cassandra/hadoop/cql3/CQLConfigHelper.java |  109 ---
 .../cassandra/hadoop/cql3/CqlConfigHelper.java |  109 +++
 2 files changed, 109 insertions(+), 109 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/60b3584e/src/java/org/apache/cassandra/hadoop/cql3/CQLConfigHelper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CQLConfigHelper.java 
b/src/java/org/apache/cassandra/hadoop/cql3/CQLConfigHelper.java
deleted file mode 100644
index cb61d05..000
--- a/src/java/org/apache/cassandra/hadoop/cql3/CQLConfigHelper.java
+++ /dev/null
@@ -1,109 +0,0 @@
-package org.apache.cassandra.hadoop.cql3;
-/*
-*
-* Licensed to the Apache Software Foundation (ASF) under one
-* or more contributor license agreements.  See the NOTICE file
-* distributed with this work for additional information
-* regarding copyright ownership.  The ASF licenses this file
-* to you under the Apache License, Version 2.0 (the
-* License); you may not use this file except in compliance
-* with the License.  You may obtain a copy of the License at
-*
-*   http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing,
-* software distributed under the License is distributed on an
-* AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-* KIND, either express or implied.  See the License for the
-* specific language governing permissions and limitations
-* under the License.
-*
-*/
-import org.apache.hadoop.conf.Configuration;
-
-public class CqlConfigHelper
-{
-private static final String INPUT_CQL_COLUMNS_CONFIG = 
cassandra.input.columnfamily.columns; // separate by colon ,
-private static final String INPUT_CQL_PAGE_ROW_SIZE_CONFIG = 
cassandra.input.page.row.size;
-private static final String INPUT_CQL_WHERE_CLAUSE_CONFIG = 
cassandra.input.where.clause;
-private static final String OUTPUT_CQL = cassandra.output.cql;
-
-/**
- * Set the CQL columns for the input of this job.
- *
- * @param conf Job configuration you are about to run
- * @param columns
- */
-public static void setInputColumns(Configuration conf, String columns)
-{
-if (columns == null || columns.isEmpty())
-return;
-
-conf.set(INPUT_CQL_COLUMNS_CONFIG, columns);
-}
-
-/**
- * Set the CQL query Limit for the input of this job.
- *
- * @param conf Job configuration you are about to run
- * @param cqlPageRowSize
- */
-public static void setInputCQLPageRowSize(Configuration conf, String 
cqlPageRowSize)
-{
-if (cqlPageRowSize == null)
-{
-throw new UnsupportedOperationException(cql page row size may not 
be null);
-}
-
-conf.set(INPUT_CQL_PAGE_ROW_SIZE_CONFIG, cqlPageRowSize);
-}
-
-/**
- * Set the CQL user defined where clauses for the input of this job.
- *
- * @param conf Job configuration you are about to run
- * @param clauses
- */
-public static void setInputWhereClauses(Configuration conf, String clauses)
-{
-if (clauses == null || clauses.isEmpty())
-return;
-
-conf.set(INPUT_CQL_WHERE_CLAUSE_CONFIG, clauses);
-}
-  
-/**
- * Set the CQL prepared statement for the output of this job.
- *
- * @param conf Job configuration you are about to run
- * @param cql
- */
-public static void setOutputCql(Configuration conf, String cql)
-{
-if (cql == null || cql.isEmpty())
-return;
-
-conf.set(OUTPUT_CQL, cql);
-}
-
-
-public static String getInputcolumns(Configuration conf)
-{
-return conf.get(INPUT_CQL_COLUMNS_CONFIG);
-}
-
-public static String getInputPageRowSize(Configuration conf)
-{
-return conf.get(INPUT_CQL_PAGE_ROW_SIZE_CONFIG);
-}
-
-public static String getInputWhereClauses(Configuration conf)
-{
-return conf.get(INPUT_CQL_WHERE_CLAUSE_CONFIG);
-}
-
-public static String getOutputCql(Configuration conf)
-{
-return conf.get(OUTPUT_CQL);
-}
-}

[jira] [Commented] (CASSANDRA-5622) Update CqlRecordWriter to conform to RecordWriter API

2013-06-06 Thread Alex Liu (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13677687#comment-13677687
 ] 

Alex Liu commented on CASSANDRA-5622:
-

Agreed, #2 is more concise and easy to use.

 Update CqlRecordWriter to conform to RecordWriter API
 -

 Key: CASSANDRA-5622
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5622
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Jonathan Ellis
Assignee: Alex Liu
 Fix For: 1.2.6


 {{RecordWriterK, V}} is supposed to write values V that can be uniquely 
 identified by keys K.
 Currently CqlRW requires the user to give it all the bind variables for a 
 complete statement in V, and effectively ignores K.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira