----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/16533/#review31537 -----------------------------------------------------------
build.xml <https://reviews.apache.org/r/16533/#comment60056> Remove this line src/org/apache/pig/backend/hadoop/accumulo/AccumuloStorage.java <https://reviews.apache.org/r/16533/#comment60058> We can remove this line That much better, thanks. Caster is added. But for AccumuloBinaryConverter, we shall retain the data in binary style. Eg, toBytes(Integer), we shall not convert the integer into String, then get bytes from string, but retain in 4 bytes integer style (like Bytes.toBytes(Integer) in hbase, not sure what's best in Accumulo). Also in addition to specify caster in construct parameter, better to have a configuration entry for that. I assume user prefer BinaryConverter will always use BinaryConverter, they don't want to specify the option every time. I still feel the syntax for AccumuloStorage is not straightforward. Can user use the following style (similar to HBaseStorage)? a = load 'accumulo://....' using AccumuloStorage('info:age info2:address', 'other options'); -- produce a 3 item tuple (key, info:age, info2:address). info/info2 is colFam, age/address is colQual -- user might optionally specify info:*, which will produce a map including every colQual in this colFam -- I see aggregate flag in the code, but I cannot think of much use case that will be useful, how do you feel? store x into 'accumulo://....' using AccumuloStorage('info:age info2:address', 'other options'); -- input tuple contains 3 items: (key, info:age, info2:address) -- mirror what's in input side, user can pass a map instead, then he needs to use wild card in construct AccumuloStorage('info:*'); - Daniel Dai On Jan. 10, 2014, 7:20 p.m., Josh Elser wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/16533/ > ----------------------------------------------------------- > > (Updated Jan. 10, 2014, 7:20 p.m.) > > > Review request for pig. > > > Bugs: PIG-3573 > https://issues.apache.org/jira/browse/PIG-3573 > > > Repository: pig-git > > > Description > ------- > > Provides basic StoreFunc and LoadFunc implementations. Based off of code that > was in an Accumulo contrib project. > > > Diffs > ----- > > build.xml 575c9ae > ivy.xml 180eb2c > ivy/libraries.properties 14abdf8 > src/org/apache/pig/backend/hadoop/accumulo/AbstractAccumuloStorage.java > PRE-CREATION > src/org/apache/pig/backend/hadoop/accumulo/AccumuloBinaryConverter.java > PRE-CREATION > src/org/apache/pig/backend/hadoop/accumulo/AccumuloStorage.java > PRE-CREATION > src/org/apache/pig/backend/hadoop/accumulo/AccumuloStorageOptions.java > PRE-CREATION > src/org/apache/pig/backend/hadoop/accumulo/FixedByteArrayOutputStream.java > PRE-CREATION > src/org/apache/pig/backend/hadoop/accumulo/Utils.java PRE-CREATION > > test/org/apache/pig/backend/hadoop/accumulo/AbstractAccumuloStorageTest.java > PRE-CREATION > test/org/apache/pig/backend/hadoop/accumulo/AccumuloPigClusterTest.java > PRE-CREATION > > test/org/apache/pig/backend/hadoop/accumulo/AccumuloStorageConfigurationTest.java > PRE-CREATION > test/org/apache/pig/backend/hadoop/accumulo/AccumuloStorageOptionsTest.java > PRE-CREATION > test/org/apache/pig/backend/hadoop/accumulo/AccumuloStorageTest.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/16533/diff/ > > > Testing > ------- > > Local tests reading, writing and JOIN'ing Accumulo tables. Tested against > Hadoop-1.0.4 and 2.2.0, with Accumulo 1.5.0 > > > Thanks, > > Josh Elser > >
