[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779047#comment-13779047
]
Alex Liu commented on CASSANDRA-6102:
-
[~jalkanen] Can you try it with CqlStorage which should work? We recommend to
use CqlStorage unless you have to use CassandraStorage. There is some issue
with CassandraStorage which can't get the right validator type for the columns
based on system tables.
We may needs fall back to thrift api to get the metadata for COMPACT STORAGE
cql table.
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779083#comment-13779083
]
Alex Liu commented on CASSANDRA-6102:
-
I propose that we implement the following
{code}
CqlStorage supports all kind of tables/column families including old thrift
column families, new Cql tables with/without Compact storage. (this is already
done)
CassandraStorage supports only old thrift column families PLUS Cql tables with
Compact storage. It DOES NOT support other Cql tables. (I am changing code for
this)
{code}
Any objection/thought?
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779106#comment-13779106
]
Brandon Williams commented on CASSANDRA-6102:
-
I'm mostly ok with it, except I really want to get away from the cli for
CASSANDRA-5709, and long-term I think we need to. On the other hand, if we had
CASSANDRA-5695 we could just get rid of those tests, so I'm still undecided.
Can you explain what the problem is here?
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13779296#comment-13779296
]
Alex Liu commented on CASSANDRA-6102:
-
Let me double check it, I may fix the issue.
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13780701#comment-13780701
]
Alex Liu commented on CASSANDRA-6102:
-
For regular Cql table e.g.
{code}
CREATE TABLE testm (
m text,
n text,
o text,
p text,
q text,
r text,
PRIMARY KEY (m, n, o)
)
cqlsh:test> select * from testm;
m | n | o | p | q | r
+++++
m | n | o | p | q | r
m1 | n1 | o1 | p1 | q1 | r1
{code}
the schema and result is
{code}
(m,{((n,o,),),((n,o,p),p),((n,o,q),q),((n,o,r),r)})
(m1,{((n1,o1,),),((n1,o1,p),p1),((n1,o1,q),q1),((n1,o1,r),r1)})
grunt> describe test3;
test3: {key: chararray,columns: {(name: (),value: bytearray)}}
{code}
System.schema_columns
{code}
qlsh:test> select * from system.schema_columns where keyspace_name='test' and
columnfamily_name='testm';
keyspace_name | columnfamily_name | column_name | component_index | index_name
| index_options | index_type | validator
---+---+-+-++---++--
test | testm | p | 2 | null
| null | null | org.apache.cassandra.db.marshal.UTF8Type
test | testm | q | 2 | null
| null | null | org.apache.cassandra.db.marshal.UTF8Type
test | testm | r | 2 | null
| null | null | org.apache.cassandra.db.marshal.UTF8Type
{code}
Because the column is composite, so we need use the last component of the
composite columns to find the validator
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Attachments: 6102-1.2-branch.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13780702#comment-13780702
]
Alex Liu commented on CASSANDRA-6102:
-
For compact storage cql table, we just need treat it as thrift table.
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Attachments: 6102-1.2-branch.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13780708#comment-13780708
]
Janne Jalkanen commented on CASSANDRA-6102:
---
What is the patch against? Trying it on both trunk and cassandra-1.2 branches;
on cassandra-1.2 it fails because of
[javac]
/Users/jalkanen/Eclipse/cassandra/src/java/org/apache/cassandra/hadoop/pig/CqlStorage.java:57:
error: CqlStorage is not abstract and does not override abstract method
getColumnMetadata(Client) in AbstractCassandraStorage
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Attachments: 6102-1.2-branch.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13780719#comment-13780719
]
Janne Jalkanen commented on CASSANDRA-6102:
---
Confirmed working; result for the test case above is now (Pig 0.11.1 and
patched 1.2 latest)
grunt> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
grunt> dump testc;
(foo,(ivalue,10),(svalue,bar),(value,65),{})
Thank you!
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Attachments: 6102-1.2-branch.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13788529#comment-13788529
]
Brandon Williams commented on CASSANDRA-6102:
-
This part:
{noformat}
+// Don't want to create another TBase class, so use
CfDef.populate_io_cache_on_flush
+// to store flag of compact storage cql table.
+if (cql3Table && !(parseType(cfDef.comparator_type) instanceof
AbstractCompositeType))
+cfDef.setPopulate_io_cache_on_flush(true);
+
+// Don't want to create another TBase class, so use
CfDef.replicate_on_write
+// to store flag of cql table.
+if (cql3Table)
+cfDef.setReplicate_on_write(true);
{noformat}
Feels like a hack that is going to bite us down the road when those options
really do get removed.
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Attachments: 6102-1.2-branch.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13788596#comment-13788596
]
Alex Liu commented on CASSANDRA-6102:
-
Since thrift structs do not support inheritance, that is, a struct may not
extend other structs, so the choice is to create a new struct CfInfo which has
the following properties
{code}
struct CfInfo {
1: required string keyspace,
2: required string name,
3: optional string column_type="Standard",
4: optional string comparator_type="BytesType",
5: optional string subcomparator_type,
6: optional list column_metadata,
7: optional string default_validation_class,
8: optional string key_validation_class,
9: optional boolean compact_cql_table,
10: optional boolean cql3_table
}
{code}
CfInfo is a new struct, so it doesn't affect the other existing code.
or add two additional properties to CfDef
{code}
39: optional boolean compact_cql_table,
40: optional boolean cql3_table
{code}
It adds additional properties only used by Pig, so a little overhead.
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Attachments: 6102-1.2-branch.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13788599#comment-13788599
]
Alex Liu commented on CASSANDRA-6102:
-
[~brandon.williams] which one is better?
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Attachments: 6102-1.2-branch.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13789389#comment-13789389
]
Alex Liu commented on CASSANDRA-6102:
-
It needs be serialized so that it can be transferred to the task nodes. We use
Thrift serialization so the class need be generated by thrift interface(thrift
structs). If we change to other serialization mechanism, we don't need TBase
class.
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Attachments: 6102-1.2-branch.txt, 6102-v2.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13789401#comment-13789401
]
Brandon Williams commented on CASSANDRA-6102:
-
Ah, right. Well, I'm open to other options so we don't chain ourselves to
thrift more than we have to, but between modifying CfDef just for this or
creating CfInfo, I'd prefer CfInfo.
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Attachments: 6102-1.2-branch.txt, 6102-v2.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13789955#comment-13789955
]
Alex Liu commented on CASSANDRA-6102:
-
A custom serialization format is implemented as followings
{code}
{code}
e.g. 11 as compact cql3 table
01 as none compact cql3 table
where 1 as true, 0 as false.
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Attachments: 6102-1.2-branch.txt, 6102-v2.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13789964#comment-13789964
]
Alex Liu commented on CASSANDRA-6102:
-
6102-v3.txt patch is attached which implemnts the new custom serialization
format.
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Attachments: 6102-1.2-branch.txt, 6102-v2.txt, 6102-v3.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13792987#comment-13792987
]
Alex Liu commented on CASSANDRA-6102:
-
6102-v3.txt patch is attached which reverts back the UUIDType mappings
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Attachments: 6102-1.2-branch.txt, 6102-v2.txt, 6102-v3.txt,
> 6102-v4.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13793138#comment-13793138
]
Alex Liu commented on CASSANDRA-6102:
-
sure
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Fix For: 1.2.11
>
> Attachments: 6102-1.2-branch.txt, 6102-v2.txt, 6102-v3.txt,
> 6102-v4.txt, 6102-v5.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (CASSANDRA-6102) CassandraStorage broken for bigints and ints
[
https://issues.apache.org/jira/browse/CASSANDRA-6102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13793265#comment-13793265
]
Mikhail Stepura commented on CASSANDRA-6102:
It looks like
https://github.com/apache/cassandra/commit/bc8e2475fa71f4bbbf95d4294d78b96a1aa1211c
broke the trunk
{noformat}
[javac]
C:\Users\mishail\workspace\cassandra\src\java\org\apache\cassandra\hadoop\pig\AbstractCassandraStorage.java:135:
error: variable validators is already defined in method
columnToTuple(IColumn,AbstractCassandraStorage.CfInfo,AbstractType)
[javac] Map validators =
getValidatorMap(cfDef);
[javac] ^
[javac]
C:\Users\mishail\workspace\cassandra\src\java\org\apache\cassandra\hadoop\pig\AbstractCassandraStorage.java:157:
error: cannot find symbol
[javac] for (IColumn subcol : col.getSubColumns())
[javac] ^
[javac] symbol: class IColumn
[javac] location: class AbstractCassandraStorage
{noformat}
> CassandraStorage broken for bigints and ints
>
>
> Key: CASSANDRA-6102
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6102
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Environment: Cassandra 1.2.9 & 1.2.10, Pig 0.11.1, OSX 10.8.x
>Reporter: Janne Jalkanen
>Assignee: Alex Liu
> Fix For: 1.2.11
>
> Attachments: 6102-1.2-branch.txt, 6102-2.0-branch.txt, 6102-v2.txt,
> 6102-v3.txt, 6102-v4.txt, 6102-v5.txt
>
>
> I am seeing something rather strange in the way Cass 1.2 + Pig seem to handle
> integer values.
> Setup: Cassandra 1.2.10, OSX 10.8, JDK 1.7u40, Pig 0.11.1. Single node for
> testing this.
> First a table:
> {noformat}
> > CREATE TABLE testc (
> key text PRIMARY KEY,
> ivalue int,
> svalue text,
> value bigint
> ) WITH COMPACT STORAGE;
> > insert into testc (key,ivalue,svalue,value) values ('foo',10,'bar',65);
> > select * from testc;
> key | ivalue | svalue | value
> -+++---
> foo | 10 |bar | 65
> {noformat}
> For my Pig setup, I then use libraries from different C* versions to actually
> talk to my database (which stays on 1.2.10 all the time).
> Cassandra 1.0.12 (using cassandra_storage.jar):
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.1.10:
> {noformat}
> testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> (foo,(svalue,bar),(ivalue,10),(value,65),{})
> {noformat}
> Cassandra 1.2.10:
> {noformat}
> (testc = LOAD 'cassandra://keyspace/testc' USING CassandraStorage();
> dump testc
> foo,{(ivalue,
> ),(svalue,bar),(value,A)})
> {noformat}
> To me it appears that ints and bigints are interpreted as ascii values in
> cass 1.2.10. Did something change for CassandraStorage, is there a
> regression, or am I doing something wrong? Quick perusal of the JIRA didn't
> reveal anything that I could directly pin on this.
> Note that using compact storage does not seem to affect the issue, though it
> obviously changes the resulting pig format.
> In addition, trying to use Pygmalion
> {noformat}
> tf = foreach testc generate key,
> flatten(FromCassandraBag('ivalue,svalue,value',columns)) as
> (ivalue:int,svalue:chararray,lvalue:long);
> dump tf
> (foo,
> ,bar,A)
> {noformat}
> So no help there. Explicitly casting the values to (long) or (int) just
> results in a ClassCastException.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
