Hi All, I have some question about using EB's VectorWritableConverter in my Pig script for data vectorization. I am generating the tuples using a UDF, however for simplicity I am loading the data from a file in the following code. My UDF returns tuples of the form (1,0,1,1...) etc.
My map.dat file has the following format: 1,0,1,1 0,1,1,1, 0,0,1,1, 1,1,0,0, ....... ....... ........ I register the necessary jar files. %declare SEQFILE_LOADER 'com.twitter.elephantbird.pig.load.SequenceFileLoader'; %declare TEXT_CONVERTER 'com.twitter.elephantbird.pig.util.TextConverter'; %declare LONG_CONVERTER 'com.twitter.elephantbird.pig.util.LongWritableConverter'; %declare VECTOR_CONVERTER 'com.twitter.elephantbird.pig.mahout.VectorWritableConverter'; /* Loading from a file instead of UDF for simplicity */ A = LOAD 'map.dat'; /* I am not sure how to use the VectorWritableConverter to convert tuple in the relation A to a vector using VectorWritableConverter */ B = FOREACH A GENERATE $VECTOR_CONVERTER(); DUMP B;