Hi,
I attach a summing combiner into a newly created table. The snippet code is
something like:
EnumSet<IteratorScope> iteratorScopes =
EnumSet.allOf(IteratorScope.class);
// first, remove versioning iterator since it will not
work with combiner
conn.tableOperations().removeIterator(tableName,
VERS_ITERATOR_NAME,
iteratorScopes);
// create the combiner setting, in this case it is
SummingCombiner, which
will
// sum value of all rows with same key (different
timestamp is considered
same)
// and result in single row with that key and aggregate
value
IteratorSetting setting = new IteratorSetting(
COMBINERS_PRIORITY,
SUM_COMBINERS_NAME,
SummingCombiner.class);
// set the combiner to apply to all columns
SummingCombiner.setCombineAllColumns(setting, true);
// need to set encoding type, otherwise exception will
be thrown during
scan
SummingCombiner.setEncodingType(setting,
LongLexicoder.class);
// attach the combiner to the table
conn.tableOperations().attachIterator(tableName,
setting,
iteratorScopes);
As you see from the code above, I use LongLexicoder class as the encoding
type.
The mutation I add for that table will be unique row id, a string for column
family, empty column qualifier, and the value is "new
LongLexicoder().encode(1L)", so basically the value is 1.
It runs fine until at one point (and I can see rows are inserted into the
table), but it hung then.
Looking at the tablet server logs I found:
2015-08-31 17:59:42,371 [tserver.MinorCompactor] WARN : MinC failed (0) to
create
hdfs://<machine>:<port>/accumulo/tables/l/default_tablet/F00009gp.rf_tmp
retrying ...
java.lang.ArrayIndexOutOfBoundsException: 0
at
org.apache.accumulo.core.client.lexicoder.ULongLexicoder.decode(ULongLexicoder.java:60)
at
org.apache.accumulo.core.client.lexicoder.LongLexicoder.decode(LongLexicoder.java:33)
at
org.apache.accumulo.core.client.lexicoder.LongLexicoder.decode(LongLexicoder.java:25)
at
org.apache.accumulo.core.iterators.TypedValueCombiner$VIterator.hasNext(TypedValueCombiner.java:82)
at
org.apache.accumulo.core.iterators.user.SummingCombiner.typedReduce(SummingCombiner.java:31)
at
org.apache.accumulo.core.iterators.user.SummingCombiner.typedReduce(SummingCombiner.java:27)
at
org.apache.accumulo.core.iterators.TypedValueCombiner.reduce(TypedValueCombiner.java:182)
at
org.apache.accumulo.core.iterators.Combiner.findTop(Combiner.java:166)
at
org.apache.accumulo.core.iterators.Combiner.next(Combiner.java:147)
at
org.apache.accumulo.tserver.Compactor.compactLocalityGroup(Compactor.java:505)
at org.apache.accumulo.tserver.Compactor.call(Compactor.java:362)
at
org.apache.accumulo.tserver.MinorCompactor.call(MinorCompactor.java:96)
at org.apache.accumulo.tserver.Tablet.minorCompact(Tablet.java:2072)
at org.apache.accumulo.tserver.Tablet.access$4400(Tablet.java:172)
at
org.apache.accumulo.tserver.Tablet$MinorCompactionTask.run(Tablet.java:2159)
at
org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at
org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
at
org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
at java.lang.Thread.run(Thread.java:722)
Looking at Accumulo's source code in ULongLexicoder.java, it looks like the
array is empty, hence it throws exception, which is still happening even
after I killed my app. I am thinking to stop Accumulo.
Do you have any idea why the array is empty? I am thinking to experiment
with String encoder instead of using LongLexicoder.
public Long decode(byte[] data) {
long l = 0;
int shift = 0;
if (data[0] < 0 || data[0] > 16)
throw new IllegalArgumentException("Unexpected length " + (0xff &
data[0]));
...
}
Thanks,
zainal
--
View this message in context:
http://apache-accumulo.1065345.n5.nabble.com/exception-thrown-during-minor-compaction-tp15010.html
Sent from the Developers mailing list archive at Nabble.com.
