JIRA filed, with a messy patch too: https://issues.apache.org/jira/browse/CASSANDRA-3371
cheers, Pete On Mon, Oct 17, 2011 at 2:27 AM, Pete Warden <p...@jetpac.com> wrote: > I've dug deeper into this, since this got my script running but still left > me at sea when dealing with the actual data. It's looking like there may be > a mismatch between the schema that's being reported by > CassandraStorage.java, and the data that's actually returned. Here's an > example: > > rows = LOAD 'cassandra://Frap/PhotoVotes' USING CassandraStorage(); > DESCRIBE rows; > rows: {key: chararray,columns: {(name: chararray,value: > bytearray,photo_owner: chararray,value_photo_owner: bytearray,pid: > chararray,value_pid: bytearray,matched_string: > chararray,value_matched_string: bytearray,src_big: chararray,value_src_big: > bytearray,time: chararray,value_time: bytearray,vote_type: > chararray,value_vote_type: bytearray,voter: chararray,value_voter: > bytearray)}} > DUMP rows; > (691831038_1317937188.48955,{(photo_owner,1596090180),(pid,6855155124568798560),(matched_string,),(src_big,),(time,Thu > Oct 06 14:39:48 -0700 2011),(vote_type,album_dislike),(voter,691831038)}) > > getSchema() is reporting the columns as an inner bag of tuples, each of > which contains 16 values. In fact, getNext() seems to return an inner bag > containing 7 tuples, each of which contains two values. > > I'll file a JIRA and do my best to create a patch to do the right thing, > but I wanted to sanity check what I'm seeing here since I'm a Pig newbie. Am > I missing something? It appears that things got out of sync with this > change: > > http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java?r1=1177083&r2=1177082&pathrev=1177083 > > While I'm in there, is there a reason for using an inner bag to hold the > columns? Do we ever have more than one set of the same columns for a given > key in Cassandra? I'm thinking of tweaking things to look like this for my > example, since it would make my processing code easier: > rows: {cassandra_key: chararray, photo_owner: chararray, pid: chararray, > matched_string: chararray, src_big: chararray, time: chararray,vote_type: > chararray,voter: chararray} > The main downside I can see is the possible clash between the cassandra_key > value and a column with the same name. > > cheers, > Pete > > On Tue, Oct 11, 2011 at 11:59 PM, Pete Warden <p...@jetpac.com> wrote: > >> For posterity, I ended up hacking around this by renaming the repeated >> 'value' alias in CassandraStorage and rebuilding it. Here's the patch: >> >> --- >> src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java.original >> 2011-10-11 >> 23:42:19.000000000 -0700 >> +++ src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java 2011-10-11 >> 23:44:26.000000000 -0700 >> @@ -357,7 +357,7 @@ >> validator = validators.get(cdef.getName()); >> if (validator == null) >> validator = marshallers.get(1); >> - valSchema.setName("value"); >> + valSchema.setName("value_"+new String(cdef.getName())); >> valSchema.setType(getPigType(validator)); >> tupleFields.add(valSchema); >> } >> >> I'm not suggesting this is a correct fix, but it does allow me to move >> forward. Another suggestion was to try Pig 0.8.1 instead, but I ran into >> https://cwiki.apache.org/confluence/display/PIG/FAQ#FAQ-Q%3AWhatshallIdoifIsaw%22FailedtocreateDataStorage%22%3F >> >> On Tue, Oct 11, 2011 at 10:34 PM, Pete Warden <p...@jetpac.com> wrote: >> >>> Thanks for all your help Brandon and Jeremy, that got me to the point >>> where I could load data. >>> >>> I'm now hitting a new issue that seems like it could possibly be related. >>> When I try to access the data like this: >>> >>> grunt> rows = LOAD 'cassandra://Frap/FriendsAlreadyRanked' USING >>> CassandraStorage(); >>> grunt> parts = FOREACH rows GENERATE key, >>> FromCassandraBag('time_last_ranked', columns); >>> >>> I see the following error: >>> >>> 2011-10-11 22:23:43,877 [main] ERROR org.apache.pig.tools.grunt.Grunt - >>> ERROR 1108: >>> <line 4, column 71> Duplicate schema alias: value in "columns" >>> >>> At first I thought it might be related to the Pygmalion helper functions, >>> so I tried to strip it back to basics using this second line instead: >>> >>> parts = FOREACH rows GENERATE key,$1; >>> >>> and I still get an identical error. >>> >>> Any further thoughts on how I can dig into this? >>> >>> Thanks again, >>> Pete >>> >>> On Tue, Oct 11, 2011 at 3:37 PM, Brandon Williams <dri...@gmail.com>wrote: >>> >>>> On Tue, Oct 11, 2011 at 4:24 PM, Pete Warden <p...@petewarden.com> >>>> wrote: >>>> > I'm trying to run the most basic example for pig_cassandra, counting >>>> the >>>> > number of rows in a column family, and I'm hitting the following >>>> error: >>>> > 2011-10-11 14:13:32,321 [main] ERROR org.apache.pig.tools.grunt.Grunt >>>> - >>>> > ERROR 1031: Incompatable field schema: left is >>>> > "columns:bag{:tuple(name:bytearray,value:bytearray)}", right is >>>> > >>>> "columns:bag{:tuple(name:chararray,value:bytearray,time_last_ranked:chararray,value:bytearray)}" >>>> >>>> After https://issues.apache.org/jira/browse/CASSANDRA-2777 you need to >>>> remove the 'AS' and everything after it; your schema definition >>>> conflicts with what was inferred. >>>> >>>> -Brandon >>>> >>> >>> >> >