[GitHub] [hbase] apurtell edited a comment on pull request #3730: HBASE-26316 Per-table or per-CF compression codec setting overrides
apurtell edited a comment on pull request #3730: URL: https://github.com/apache/hbase/pull/3730#issuecomment-938875365 The test failures are not related and appear environmental in nature. [ERROR] org.apache.hadoop.hbase.quotas.TestQuotaAdmin.null [ERROR] Run 1: TestQuotaAdmin.testSetModifyRemoveRPCQuota:503 » TestTimedOut test timed out a... [ERROR] Run 2: TestQuotaAdmin » Appears to be stuck in thread MiniHBaseClusterRegionServer-E... [ERROR] Run 3: TestQuotaAdmin.setUpBeforeClass:99 » IllegalState A mini-cluster is already ru... [ERROR] TestQuotaAdmin.clearQuotaTable:106 » TimeoutIO java.util.concurrent.TimeoutExc... [ERROR] org.apache.hadoop.hbase.quotas.TestQuotaAdmin.testSetModifyRemoveRPCQuota [ERROR] Run 1: TestQuotaAdmin.testSetModifyRemoveRPCQuota:503 » InterruptedIO [ERROR] Run 2: TestQuotaAdmin.clearQuotaTable:106 » TableNotEnabled org.apache.hadoop.hbase.T... [WARNING] org.apache.hadoop.hbase.TestZooKeeper.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry [ERROR] Run 1: TestZooKeeper.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry:199 [INFO] Run 2: PASS I ran the tests locally with this change applied and they passed. TestQuotaAdmin seems like a resource intensive unit (has a running time of 86 seconds on my macbook) and maybe we should break it up. [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Running org.apache.hadoop.hbase.quotas.TestQuotaAdmin [INFO] Running org.apache.hadoop.hbase.TestZooKeeper [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.494 s - in org.apache.hadoop.hbase.TestZooKeeper [INFO] Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 86.059 s - in org.apache.hadoop.hbase.quotas.TestQuotaAdmin [INFO] [INFO] Results: [INFO] [INFO] Tests run: 24, Failures: 0, Errors: 0, Skipped: 0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell edited a comment on pull request #3730: HBASE-26316 Per-table or per-CF compression codec setting overrides
apurtell edited a comment on pull request #3730: URL: https://github.com/apache/hbase/pull/3730#issuecomment-938875365 The test failures are not related and appear environmental in nature. [ERROR] org.apache.hadoop.hbase.quotas.TestQuotaAdmin.null [ERROR] Run 1: TestQuotaAdmin.testSetModifyRemoveRPCQuota:503 » TestTimedOut test timed out a... [ERROR] Run 2: TestQuotaAdmin » Appears to be stuck in thread MiniHBaseClusterRegionServer-E... [ERROR] Run 3: TestQuotaAdmin.setUpBeforeClass:99 » IllegalState A mini-cluster is already ru... [ERROR] TestQuotaAdmin.clearQuotaTable:106 » TimeoutIO java.util.concurrent.TimeoutExc... [ERROR] org.apache.hadoop.hbase.quotas.TestQuotaAdmin.testSetModifyRemoveRPCQuota [ERROR] Run 1: TestQuotaAdmin.testSetModifyRemoveRPCQuota:503 » InterruptedIO [ERROR] Run 2: TestQuotaAdmin.clearQuotaTable:106 » TableNotEnabled org.apache.hadoop.hbase.T... [WARNING] org.apache.hadoop.hbase.TestZooKeeper.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry [ERROR] Run 1: TestZooKeeper.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry:199 [INFO] Run 2: PASS I ran the tests locally with this change applied and they passed. TestQuotaAdmin seems like a resource intensive unit and maybe we should break it up. [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Running org.apache.hadoop.hbase.quotas.TestQuotaAdmin [INFO] Running org.apache.hadoop.hbase.TestZooKeeper [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.494 s - in org.apache.hadoop.hbase.TestZooKeeper [INFO] Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 86.059 s - in org.apache.hadoop.hbase.quotas.TestQuotaAdmin [INFO] [INFO] Results: [INFO] [INFO] Tests run: 24, Failures: 0, Errors: 0, Skipped: 0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell edited a comment on pull request #3730: HBASE-26316 Per-table or per-CF compression codec setting overrides
apurtell edited a comment on pull request #3730: URL: https://github.com/apache/hbase/pull/3730#issuecomment-938875365 The test failures are not related and appear environmental in nature. [ERROR] org.apache.hadoop.hbase.quotas.TestQuotaAdmin.null [ERROR] Run 1: TestQuotaAdmin.testSetModifyRemoveRPCQuota:503 » TestTimedOut test timed out a... [ERROR] Run 2: TestQuotaAdmin » Appears to be stuck in thread MiniHBaseClusterRegionServer-E... [ERROR] Run 3: TestQuotaAdmin.setUpBeforeClass:99 » IllegalState A mini-cluster is already ru... [ERROR] TestQuotaAdmin.clearQuotaTable:106 » TimeoutIO java.util.concurrent.TimeoutExc... [ERROR] org.apache.hadoop.hbase.quotas.TestQuotaAdmin.testSetModifyRemoveRPCQuota [ERROR] Run 1: TestQuotaAdmin.testSetModifyRemoveRPCQuota:503 » InterruptedIO [ERROR] Run 2: TestQuotaAdmin.clearQuotaTable:106 » TableNotEnabled org.apache.hadoop.hbase.T... [WARNING] org.apache.hadoop.hbase.TestZooKeeper.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry [ERROR] Run 1: TestZooKeeper.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry:199 [INFO] Run 2: PASS -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell edited a comment on pull request #3730: HBASE-26316 Per-table or per-CF compression codec setting overrides
apurtell edited a comment on pull request #3730: URL: https://github.com/apache/hbase/pull/3730#issuecomment-937381939 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell edited a comment on pull request #3730: HBASE-26316 Per-table or per-CF compression codec setting overrides
apurtell edited a comment on pull request #3730:
URL: https://github.com/apache/hbase/pull/3730#issuecomment-937381939
Confirmed functionality with a single host cluster:
hbase> create "IntegrationTestLoadCommonCrawl", \
{ NAME => 'c', VERSIONS => 1000, COMPRESSION => 'LZ4',
COMPRESSION_COMPACT => 'ZSTD', BLOCKSIZE => 131072 }, \
{ NAME => 'i', VERSIONS => 1000, COMPRESSION => 'LZ4',
COMPRESSION_COMPACT => 'ZSTD', BLOCKSIZE => 8192 }, \
CONFIGURATION => { 'hbase.io.compress.zstd.level' => '1' }
Loaded one WARC from common crawl.
Major compaction takes 11 seconds.
hbase> alter "IntegrationTestLoadCommonCrawl", \
CONFIGURATION => { 'hbase.io.compress.zstd.level' => '10' }
Major compaction now takes 42 seconds. Total size on disk reduced by 11.6%
vs level 1.
hbase> alter "IntegrationTestLoadCommonCrawl", \
CONFIGURATION => { 'hbase.io.compress.zstd.level' => '22' }
Major compaction now takes 17 minutes 15 seconds. Total size on disk reduced
by 13.3% vs level 1. (Sure, this level is crazy in practice.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell edited a comment on pull request #3730: HBASE-26316 Per-table or per-CF compression codec setting overrides
apurtell edited a comment on pull request #3730:
URL: https://github.com/apache/hbase/pull/3730#issuecomment-937381939
Confirmed functionality with a single host cluster:
hbase> create "IntegrationTestLoadCommonCrawl", \
{ NAME => 'c', VERSIONS => 1000, COMPRESSION => 'LZ4',
COMPRESSION_COMPACT => 'ZSTD', BLOCKSIZE => 131072 }, \
{ NAME => 'i', VERSIONS => 1000, COMPRESSION => 'LZ4',
COMPRESSION_COMPACT => 'ZSTD', BLOCKSIZE => 8192 }, \
CONFIGURATION => { 'hbase.io.compress.zstd.level' => '1' }
Loaded one WARC from common crawl.
Major compaction takes 11 seconds.
hbase> alter "IntegrationTestLoadCommonCrawl", \
CONFIGURATION => {'hbase.io.compress.zstd.level' => '10' }
Major compaction now takes 42 seconds. Total size on disk reduced by 11.6%.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] apurtell edited a comment on pull request #3730: HBASE-26316 Per-table or per-CF compression codec setting overrides
apurtell edited a comment on pull request #3730:
URL: https://github.com/apache/hbase/pull/3730#issuecomment-937381939
Confirmed functionality with a single host cluster:
hbase> create "IntegrationTestLoadCommonCrawl", \
{ NAME => 'c', VERSIONS => 1000, COMPRESSION => 'LZ4',
COMPRESSION_COMPACT => 'ZSTD', BLOCKSIZE => 131072 }, \
{ NAME => 'i', VERSIONS => 1000, COMPRESSION => 'LZ4',
COMPRESSION_COMPACT => 'ZSTD', BLOCKSIZE => 8192 }, \
CONFIGURATION => { 'hbase.io.compress.zstd.level' => '1' }
Loaded one WARC from common crawl.
Major compaction takes 11 seconds.
hbase> alter "IntegrationTestLoadCommonCrawl", \
CONFIGURATION => {'hbase.io.compress.zstd.level' => '10' }
Major compaction now takes 42 seconds. Total size on disk reduced by 11.6%.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
