An empty value seems to imply that you wrote some unexpected data to
your table. The following code does work correctly (using 1.7.0).
byte[] bytes = new LongLexicoder().encode(1l);
System.out.println(Arrays.toString(bytes));
System.out.println(new LongLexicoder().decode(bytes));
Can you check the code you're using to create Mutations and verify that
you're passing in the bytes from the LongLexicoder encode() method as
the Value?
You can try removing the SummingCombiner from your table which should
let you scan the table to verify the records (and also let your
compactions happen). Because you removed the VersioningIterator, it
should preserve all of the entries you have in the table.
z11373 wrote:
Hi,
I attach a summing combiner into a newly created table. The snippet code is
something like:
EnumSet<IteratorScope> iteratorScopes =
EnumSet.allOf(IteratorScope.class);
// first, remove versioning iterator since it will not
work with combiner
conn.tableOperations().removeIterator(tableName,
VERS_ITERATOR_NAME,
iteratorScopes);
// create the combiner setting, in this case it is
SummingCombiner, which
will
// sum value of all rows with same key (different
timestamp is considered
same)
// and result in single row with that key and aggregate
value
IteratorSetting setting = new IteratorSetting(
COMBINERS_PRIORITY,
SUM_COMBINERS_NAME,
SummingCombiner.class);
// set the combiner to apply to all columns
SummingCombiner.setCombineAllColumns(setting, true);
// need to set encoding type, otherwise exception will
be thrown during
scan
SummingCombiner.setEncodingType(setting,
LongLexicoder.class);
// attach the combiner to the table
conn.tableOperations().attachIterator(tableName,
setting,
iteratorScopes);
As you see from the code above, I use LongLexicoder class as the encoding
type.
The mutation I add for that table will be unique row id, a string for column
family, empty column qualifier, and the value is "new
LongLexicoder().encode(1L)", so basically the value is 1.
It runs fine until at one point (and I can see rows are inserted into the
table), but it hung then.
Looking at the tablet server logs I found:
2015-08-31 17:59:42,371 [tserver.MinorCompactor] WARN : MinC failed (0) to
create
hdfs://<machine>:<port>/accumulo/tables/l/default_tablet/F00009gp.rf_tmp
retrying ...
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.accumulo.core.client.lexicoder.ULongLexicoder.decode(ULongLexicoder.java:60)
at
org.apache.accumulo.core.client.lexicoder.LongLexicoder.decode(LongLexicoder.java:33)
at
org.apache.accumulo.core.client.lexicoder.LongLexicoder.decode(LongLexicoder.java:25)
at
org.apache.accumulo.core.iterators.TypedValueCombiner$VIterator.hasNext(TypedValueCombiner.java:82)
at
org.apache.accumulo.core.iterators.user.SummingCombiner.typedReduce(SummingCombiner.java:31)
at
org.apache.accumulo.core.iterators.user.SummingCombiner.typedReduce(SummingCombiner.java:27)
at
org.apache.accumulo.core.iterators.TypedValueCombiner.reduce(TypedValueCombiner.java:182)
at
org.apache.accumulo.core.iterators.Combiner.findTop(Combiner.java:166)
at
org.apache.accumulo.core.iterators.Combiner.next(Combiner.java:147)
at
org.apache.accumulo.tserver.Compactor.compactLocalityGroup(Compactor.java:505)
at org.apache.accumulo.tserver.Compactor.call(Compactor.java:362)
at
org.apache.accumulo.tserver.MinorCompactor.call(MinorCompactor.java:96)
at org.apache.accumulo.tserver.Tablet.minorCompact(Tablet.java:2072)
at org.apache.accumulo.tserver.Tablet.access$4400(Tablet.java:172)
at
org.apache.accumulo.tserver.Tablet$MinorCompactionTask.run(Tablet.java:2159)
at
org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
at
org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
at java.lang.Thread.run(Thread.java:722)
Looking at Accumulo's source code in ULongLexicoder.java, it looks like the
array is empty, hence it throws exception, which is still happening even
after I killed my app. I am thinking to stop Accumulo.
Do you have any idea why the array is empty? I am thinking to experiment
with String encoder instead of using LongLexicoder.
public Long decode(byte[] data) {
long l = 0;
int shift = 0;
if (data[0]< 0 || data[0]> 16)
throw new IllegalArgumentException("Unexpected length " + (0xff&
data[0]));
...
}
Thanks,
zainal
--
View this message in context:
http://apache-accumulo.1065345.n5.nabble.com/exception-thrown-during-minor-compaction-tp15010.html
Sent from the Developers mailing list archive at Nabble.com.
