[
https://issues.apache.org/jira/browse/HBASE-6028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524252#comment-16524252
]
Mohit Goel commented on HBASE-6028:
-----------------------------------
Proposed command line usage is :
{code}
hbase(main):023:0> help "compaction_switch"
Turn the compaction on or off on regionservers. Disabling compactions will also
interrupt any
currently ongoing
compactions. It is ephemeral. This setting will be lost on restart of the
server. Compaction
can also be enabled/disabled by modifying configuration
hbase.regionserver.compaction.enabled in
hbase-site.xml.
Examples:
To enable compactions on all region servers
hbase> compaction_switch true
To disable compactions on all region servers
hbase> compaction_switch false
To enable compactions on specific region servers
hbase> compaction_switch true 'server2','server1'
To disable compactions on specific region servers
hbase> compaction_switch false 'server2','server1'
NOTE: A server name is its host, port plus startcode. For example:
host187.example.com,60020,1289493121758
hbase(main):020:0> compaction_switch "false"
SERVER PREV_STATE
dhcp-10-16-2-111.pa.cloudera.com,16022,153004290209 true
2
dhcp-10-16-2-111.pa.cloudera.com,16023,153004290320 true
3
2 row(s)
Took 0.0214 seconds
hbase(main):021:0> compaction_switch "true"
SERVER PREV_STATE
dhcp-10-16-2-111.pa.cloudera.com,16022,153004290209 false
2
dhcp-10-16-2-111.pa.cloudera.com,16023,153004290320 false
3
2 row(s)
Took 0.0217 seconds
{code}
Logs from RS when switch off compaction is run :
{code}
2018-06-26 13:37:20,000 WARN [regionserver/dhcp-10-16-2-111:16022]
compactions.CompactionProgress: totalCompactingKVs=2277787 less than
currentCompactedKVs=3146342
2018-06-26 13:37:22,003 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16022]
regionserver.CompactSplit: Interrupting running compactions because user
switched off compactions
2018-06-26 13:37:22,005 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16022-shortCompactions-1530045419380]
throttle.PressureAwareThroughputController:
10fffe4a4c251917d4422e917a3f114d#info0#compaction#26 average throughput is
175.87 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active
operations remaining, total limit is unlimited
2018-06-26 13:37:22,005 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16022-longCompactions-1530045363453]
throttle.PressureAwareThroughputController:
10fffe4a4c251917d4422e917a3f114d#info0#compaction#24 average throughput is
176.70 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active
operations remaining, total limit is unlimited
2018-06-26 13:37:22,042 ERROR
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16022-shortCompactions-1530045419380]
regionserver.CompactSplit: Compaction failed
region=TestTable,00000000000000000000299441,1529703743138.10fffe4a4c251917d4422e917a3f114d.,
storeName=info0, priority=0, startTime=1530045430730
java.io.IOException: Could not iterate StoreFileScanner[HFileScanner for reader
reader=file:/Users/mohit.goel/srcdir/distributed/data/data/default/TestTable/10fffe4a4c251917d4422e917a3f114d/info0/1eef927ebb0d47d59196229e3ce566bd,
compression=none, cacheConf=blockCache=LruBlockCache{blockCount=7,
currentSize=1.21 MB, freeSize=1.57 GB, maxSize=1.57 GB, heapSize=1.21 MB,
minSize=1.50 GB, minFactor=0.95, multiSize=765.61 MB, multiFactor=0.5,
singleSize=382.80 MB, singleFactor=0.25}, cacheDataOnRead=true,
cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false,
cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false,
firstKey=Optional[00000000000000000001048576/info0:0/1530045423841/Put/seqid=0],
lastKey=Optional[00000000000000000003145727/info0:0/1530045423622/Put/seqid=0],
avgKeyLen=44, avgValueLen=1000, entries=477045, length=504687508,
cur=00000000000000000002937125/info0:0/1530045421477/Put/vlen=1000/seqid=12377]
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:216)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:120)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:639)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:385)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:325)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:65)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:126)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1403)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2111)
at
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:629)
at
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:671)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.channels.ClosedByInterruptException
at
java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:293)
at
org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.seek(RawLocalFileSystem.java:115)
at
org.apache.hadoop.fs.BufferedFSInputStream.seek(BufferedFSInputStream.java:96)
at
org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:62)
at
org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:62)
at
org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream.seek(ChecksumFileSystem.java:330)
at
org.apache.hadoop.hbase.io.hfile.HFileUtil.seekOnMultipleSources(HFileUtil.java:38)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readAtOffset(HFileBlock.java:1547)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1771)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1596)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.readBlock(HFileReaderImpl.java:1488)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.readNextDataBlock(HFileReaderImpl.java:926)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.isNextBlock(HFileReaderImpl.java:1059)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.positionForNextBlock(HFileReaderImpl.java:1053)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl._next(HFileReaderImpl.java:1071)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.next(HFileReaderImpl.java:1092)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:207)
... 13 more
2018-06-26 13:37:22,042 ERROR
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16022-longCompactions-1530045363453]
regionserver.CompactSplit: Compaction failed
region=TestTable,00000000000000000000299441,1529703743138.10fffe4a4c251917d4422e917a3f114d.,
storeName=info0, priority=2, startTime=1530045428634
java.io.IOException: Could not iterate StoreFileScanner[HFileScanner for reader
reader=file:/Users/mohit.goel/srcdir/distributed/data/data/default/TestTable/10fffe4a4c251917d4422e917a3f114d/info0/a9eefa45e359423caf5bd0542c55fa21,
compression=none, cacheConf=blockCache=LruBlockCache{blockCount=7,
currentSize=1.21 MB, freeSize=1.57 GB, maxSize=1.57 GB, heapSize=1.21 MB,
minSize=1.50 GB, minFactor=0.95, multiSize=765.61 MB, multiFactor=0.5,
singleSize=382.80 MB, singleFactor=0.25}, cacheDataOnRead=true,
cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false,
cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false,
firstKey=Optional[00000000000000000002574692/info0:0/1529710537577/Put/seqid=0],
lastKey=Optional[00000000000000000002933983/info0:0/1529710541075/Put/seqid=0],
avgKeyLen=44, avgValueLen=1000, entries=359292, length=380127020,
cur=00000000000000000002798971/info0:0/1529710539808/Put/vlen=1000/seqid=9557]
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:216)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:120)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:639)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:385)
at
org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:325)
at
org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:65)
at
org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:126)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1403)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2111)
at
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:629)
at
org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:671)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.channels.ClosedByInterruptException
at
java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:293)
at
org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileInputStream.seek(RawLocalFileSystem.java:115)
at
org.apache.hadoop.fs.BufferedFSInputStream.seek(BufferedFSInputStream.java:96)
at
org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:62)
at
org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:62)
at
org.apache.hadoop.fs.ChecksumFileSystem$FSDataBoundedInputStream.seek(ChecksumFileSystem.java:330)
at
org.apache.hadoop.hbase.io.hfile.HFileUtil.seekOnMultipleSources(HFileUtil.java:38)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readAtOffset(HFileBlock.java:1547)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1771)
at
org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1596)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl.readBlock(HFileReaderImpl.java:1488)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.readNextDataBlock(HFileReaderImpl.java:926)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.isNextBlock(HFileReaderImpl.java:1059)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.positionForNextBlock(HFileReaderImpl.java:1053)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl._next(HFileReaderImpl.java:1071)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.next(HFileReaderImpl.java:1092)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:207)
... 13 more
2018-06-26 13:37:23,009 WARN [regionserver/dhcp-10-16-2-111:16022]
compactions.CompactionProgress: totalCompactingKVs=2277787 less than
currentCompactedKVs=3889185
{code}
Logs from RS when compaction is switched on:
{code}
2018-06-26 13:40:17,499 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher:
Waited 90286ms on a compaction to clean up 'too many store files'; waited long
enough... proceeding with flush of
TestTable,00000000000000000000299441,1529703743138.10fffe4a4c251917d4422e917a3f114d.
2018-06-26 13:40:17,499 INFO [MemStoreFlusher.0] regionserver.HRegion:
Flushing 1/1 column families, dataSize=478.19 MB heapSize=512.73 MB
2018-06-26 13:40:18,941 INFO [MemStoreFlusher.0]
regionserver.DefaultStoreFlusher: Flushed memstore data size=478.19 MB at
sequenceid=13490 (bloomFilter=true),
to=file:/Users/mohit.goel/srcdir/distributed/data/data/default/TestTable/10fffe4a4c251917d4422e917a3f114d/.tmp/info0/6fcecdfc6dd64852b4595bedeedfb791
2018-06-26 13:40:18,948 INFO [MemStoreFlusher.0] regionserver.HStore: Added
file:/Users/mohit.goel/srcdir/distributed/data/data/default/TestTable/10fffe4a4c251917d4422e917a3f114d/info0/6fcecdfc6dd64852b4595bedeedfb791,
entries=476634, sequenceid=13490, filesize=480.9 M
2018-06-26 13:40:18,949 INFO [MemStoreFlusher.0] regionserver.HRegion:
Finished flush of dataSize ~478.19 MB/501418968, heapSize ~512.74 MB/537643368,
currentSize=0 B/0 for 10fffe4a4c251917d4422e917a3f114d in 1450ms,
sequenceid=13490, compaction requested=true
2018-06-26 13:40:20,352 WARN [regionserver/dhcp-10-16-2-111:16022]
compactions.CompactionProgress: totalCompactingKVs=2277787 less than
currentCompactedKVs=3889185
2018-06-26 13:40:23,360 WARN [regionserver/dhcp-10-16-2-111:16022]
compactions.CompactionProgress: totalCompactingKVs=2277787 less than
currentCompactedKVs=3889185
2018-06-26 13:40:26,366 WARN [regionserver/dhcp-10-16-2-111:16022]
compactions.CompactionProgress: totalCompactingKVs=2277787 less than
currentCompactedKVs=3889185
2018-06-26 13:40:26,838 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16022]
regionserver.CompactSplit: Re-Initializing compactions because user switched on
compactions
2018-06-26 13:40:29,373 WARN [regionserver/dhcp-10-16-2-111:16022]
compactions.CompactionProgress: totalCompactingKVs=2277787 less than
currentCompactedKVs=3889185
2018-06-26 13:40:32,377 WARN [regionserver/dhcp-10-16-2-111:16022]
compactions.CompactionProgress: totalCompactingKVs=2277787 less than
currentCompactedKVs=3889185
2018-06-26 13:40:35,381 WARN [regionserver/dhcp-10-16-2-111:16022]
compactions.CompactionProgress: totalCompactingKVs=2277787 less than
currentCompactedKVs=3889185
2018-06-26 13:40:38,259 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Rolled WAL
/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045548822
with entries=24, filesize=33.59 MB; new WAL
/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045638240
2018-06-26 13:40:38,259 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045436955
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045436955
2018-06-26 13:40:38,268 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045526237
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045526237
2018-06-26 13:40:38,271 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045526547
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045526547
2018-06-26 13:40:38,271 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045526992
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045526992
2018-06-26 13:40:38,272 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045527315
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045527315
2018-06-26 13:40:38,273 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045527605
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045527605
2018-06-26 13:40:38,273 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045527881
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045527881
2018-06-26 13:40:38,274 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045546663
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045546663
2018-06-26 13:40:38,275 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045546985
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045546985
2018-06-26 13:40:38,275 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045547281
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045547281
2018-06-26 13:40:38,275 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045547585
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045547585
2018-06-26 13:40:38,276 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045547899
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045547899
2018-06-26 13:40:38,276 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045548209
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045548209
2018-06-26 13:40:38,277 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Archiving
file:/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045548514
to
file:/Users/mohit.goel/srcdir/distributed/data/oldWALs/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045548514
2018-06-26 13:40:38,384 WARN [regionserver/dhcp-10-16-2-111:16022]
compactions.CompactionProgress: totalCompactingKVs=2277787 less than
currentCompactedKVs=3889185
2018-06-26 13:40:38,574 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Rolled WAL
/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045638240
with entries=21, filesize=32.06 MB; new WAL
/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045638567
2018-06-26 13:40:38,904 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Rolled WAL
/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045638567
with entries=21, filesize=32.06 MB; new WAL
/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045638896
2018-06-26 13:40:39,242 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Rolled WAL
/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045638896
with entries=21, filesize=32.06 MB; new WAL
/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045639235
2018-06-26 13:40:39,264 WARN [MemStoreFlusher.1] regionserver.MemStoreFlusher:
10fffe4a4c251917d4422e917a3f114d has too many store files(19); delaying flush
up to 90000 ms
2018-06-26 13:40:39,266 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16022-shortCompactions-1530045639264]
regionserver.HRegion: Starting compaction of info0 in
TestTable,00000000000000000000299441,1529703743138.10fffe4a4c251917d4422e917a3f114d.
2018-06-26 13:40:39,266 INFO
[RpcServer.default.FPBQ.Fifo.handler=29,queue=2,port=16022-shortCompactions-1530045639264]
regionserver.HStore: Starting compaction of
[file:/Users/mohit.goel/srcdir/distributed/data/data/default/TestTable/10fffe4a4c251917d4422e917a3f114d/info0/78ed22d67f1b42d88bbadf488a223a08,
file:/Users/mohit.goel/srcdir/distributed/data/data/default/TestTable/10fffe4a4c251917d4422e917a3f114d/info0/c44cedc3a76646648abab4479bd14bc5,
file:/Users/mohit.goel/srcdir/distributed/data/data/default/TestTable/10fffe4a4c251917d4422e917a3f114d/info0/1eef927ebb0d47d59196229e3ce566bd,
file:/Users/mohit.goel/srcdir/distributed/data/data/default/TestTable/10fffe4a4c251917d4422e917a3f114d/info0/163916598503459aa8feb20a8db1d35e,
file:/Users/mohit.goel/srcdir/distributed/data/data/default/TestTable/10fffe4a4c251917d4422e917a3f114d/info0/4aeeca0a5e464729a1c43943451ff600]
into
tmpdir=file:/Users/mohit.goel/srcdir/distributed/data/data/default/TestTable/10fffe4a4c251917d4422e917a3f114d/.tmp,
totalSize=1.3 G
2018-06-26 13:40:39,671 INFO [regionserver/dhcp-10-16-2-111:16022.logRoller]
wal.AbstractFSWAL: Rolled WAL
/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045639235
with entries=21, filesize=32.06 MB; new WAL
/Users/mohit.goel/srcdir/distributed/data/WALs/dhcp-10-16-2-111.pa.cloudera.com,16022,1530042902092/dhcp-10-16-2-111.pa.cloudera.com%2C16022%2C1530042902092.1530045639659
{code}
> Implement a cancel for in-progress compactions
> ----------------------------------------------
>
> Key: HBASE-6028
> URL: https://issues.apache.org/jira/browse/HBASE-6028
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Derek Wollenstein
> Assignee: Mohit Goel
> Priority: Minor
> Labels: beginner
>
> Depending on current server load, it can be extremely expensive to run
> periodic minor / major compactions. It would be helpful to have a feature
> where a user could use the shell or a client tool to explicitly cancel an
> in-progress compactions. This would allow a system to recover when too many
> regions became eligible for compactions at once
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
