[jira] [Updated] (HBASE-12712) skipLargeFiles in minor compact but not in major compact
[ https://issues.apache.org/jira/browse/HBASE-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12712: --- Status: Open (was: Patch Available) skipLargeFiles in minor compact but not in major compact Key: HBASE-12712 URL: https://issues.apache.org/jira/browse/HBASE-12712 Project: HBase Issue Type: New Feature Components: Compaction Affects Versions: 0.98.6 Reporter: Liu Junhong Labels: beginner Fix For: 0.98.6 Attachments: compact.diff Original Estimate: 72h Remaining Estimate: 72h Here is my case. After repeatedly minor compaction, the size of storefile is very large. Compaction with large storefile will waste much bandwidth, so i use the “hbase.hstore.compaction.max.size” to skip this case. But after use this config, i find that major compaction will be skipped forever when i read the source code and the deletes and muti-versions data my waste storage. So i had to modify the code. Now i'm try to submit my patch.But my patch is not perfect. I think there should be an other config to determine if the large size storefile should join major compaction in HColumnDescriptor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12712) skipLargeFiles in minor compact but not in major compact
[ https://issues.apache.org/jira/browse/HBASE-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-12712: Labels: beginner (was: newbie patch) skipLargeFiles in minor compact but not in major compact Key: HBASE-12712 URL: https://issues.apache.org/jira/browse/HBASE-12712 Project: HBase Issue Type: New Feature Components: Compaction Affects Versions: 0.98.6 Reporter: Liu Junhong Labels: beginner Fix For: 0.98.6 Attachments: compact.diff Original Estimate: 72h Remaining Estimate: 72h Here is my case. After repeatedly minor compaction, the size of storefile is very large. Compaction with large storefile will waste much bandwidth, so i use the “hbase.hstore.compaction.max.size” to skip this case. But after use this config, i find that major compaction will be skipped forever when i read the source code and the deletes and muti-versions data my waste storage. So i had to modify the code. Now i'm try to submit my patch.But my patch is not perfect. I think there should be an other config to determine if the large size storefile should join major compaction in HColumnDescriptor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12712) skipLargeFiles in minor compact but not in major compact
[ https://issues.apache.org/jira/browse/HBASE-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12712: --- Release Note: (was: It's my first time to submit a patch in jira. Sorry for mistake.) skipLargeFiles in minor compact but not in major compact Key: HBASE-12712 URL: https://issues.apache.org/jira/browse/HBASE-12712 Project: HBase Issue Type: New Feature Components: Compaction Affects Versions: 0.98.6 Reporter: Liu Junhong Labels: newbie, patch Fix For: 0.98.6 Attachments: compact.diff Original Estimate: 72h Remaining Estimate: 72h Here is my case. After repeatedly minor compaction, the size of storefile is very large. Compaction with large storefile will waste much bandwidth, so i use the “hbase.hstore.compaction.max.size” to skip this case. But after use this config, i find that major compaction will be skipped forever when i read the source code and the deletes and muti-versions data my waste storage. So i had to modify the code. Now i'm try to submit my patch.But my patch is not perfect. I think there should be an other config to determine if the large size storefile should join major compaction in HColumnDescriptor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12712) skipLargeFiles in minor compact but not in major compact
[ https://issues.apache.org/jira/browse/HBASE-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Junhong updated HBASE-12712: Status: Patch Available (was: Open) skipLargeFiles in minor compact but not in major compact Key: HBASE-12712 URL: https://issues.apache.org/jira/browse/HBASE-12712 Project: HBase Issue Type: New Feature Components: Compaction Affects Versions: 0.98.6 Reporter: Liu Junhong Original Estimate: 72h Remaining Estimate: 72h Here is my case. After repeatedly minor compaction, the size of storefile is very large. Compaction with large storefile will waste much bandwidth, so i use the “hbase.hstore.compaction.max.size” to skip this case. But after use this config, i find that major compaction will be skipped forever when i read the source code. So i had to modify the code. Now i'm try to submit my patch.But my patch is not perfect. I think there should be an other config to determine if the large size storefile should join major compaction in HColumnDescriptor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12712) skipLargeFiles in minor compact but not in major compact
[ https://issues.apache.org/jira/browse/HBASE-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Junhong updated HBASE-12712: Status: Open (was: Patch Available) skipLargeFiles in minor compact but not in major compact Key: HBASE-12712 URL: https://issues.apache.org/jira/browse/HBASE-12712 Project: HBase Issue Type: New Feature Components: Compaction Affects Versions: 0.98.6 Reporter: Liu Junhong Original Estimate: 72h Remaining Estimate: 72h Here is my case. After repeatedly minor compaction, the size of storefile is very large. Compaction with large storefile will waste much bandwidth, so i use the “hbase.hstore.compaction.max.size” to skip this case. But after use this config, i find that major compaction will be skipped forever when i read the source code. So i had to modify the code. Now i'm try to submit my patch.But my patch is not perfect. I think there should be an other config to determine if the large size storefile should join major compaction in HColumnDescriptor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12712) skipLargeFiles in minor compact but not in major compact
[
https://issues.apache.org/jira/browse/HBASE-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Junhong updated HBASE-12712:
Fix Version/s: 0.98.6
Labels: newbie patch (was: )
Release Note: It's my first time to submit a patch in jira. Sorry for
mistake.
Status: Patch Available (was: Open)
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java
old mode 100644
new mode 100755
index deca192..fdc9cba
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java
@@ -103,7 +103,6 @@ public class RatioBasedCompactionPolicy extends
CompactionPolicy {
return new CompactionRequest(expiredSelection);
}
}
- candidateSelection = skipLargeFiles(candidateSelection);
}
// Force a major compaction if this is a user-requested major compaction,
@@ -118,6 +117,9 @@ public class RatioBasedCompactionPolicy extends
CompactionPolicy {
);
if (!majorCompaction) {
+ if (!forceMajor) {
+candidateSelection = skipLargeFiles(candidateSelection);
+ }
// we're doing a minor compaction, let's see what files are applicable
candidateSelection = filterBulk(candidateSelection);
candidateSelection = applyCompactionPolicy(candidateSelection,
mayUseOffPeak, mayBeStuck);
skipLargeFiles in minor compact but not in major compact
Key: HBASE-12712
URL: https://issues.apache.org/jira/browse/HBASE-12712
Project: HBase
Issue Type: New Feature
Components: Compaction
Affects Versions: 0.98.6
Reporter: Liu Junhong
Labels: patch, newbie
Fix For: 0.98.6
Original Estimate: 72h
Remaining Estimate: 72h
Here is my case. After repeatedly minor compaction, the size of storefile is
very large. Compaction with large storefile will waste much bandwidth, so i
use the “hbase.hstore.compaction.max.size” to skip this case. But after use
this config, i find that major compaction will be skipped forever when i read
the source code. So i had to modify the code.
Now i'm try to submit my patch.But my patch is not perfect. I think there
should be an other config to determine if the large size storefile should
join major compaction in HColumnDescriptor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12712) skipLargeFiles in minor compact but not in major compact
[ https://issues.apache.org/jira/browse/HBASE-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Junhong updated HBASE-12712: Attachment: compact.diff skipLargeFiles in minor compact but not in major compact Key: HBASE-12712 URL: https://issues.apache.org/jira/browse/HBASE-12712 Project: HBase Issue Type: New Feature Components: Compaction Affects Versions: 0.98.6 Reporter: Liu Junhong Labels: newbie, patch Fix For: 0.98.6 Attachments: compact.diff Original Estimate: 72h Remaining Estimate: 72h Here is my case. After repeatedly minor compaction, the size of storefile is very large. Compaction with large storefile will waste much bandwidth, so i use the “hbase.hstore.compaction.max.size” to skip this case. But after use this config, i find that major compaction will be skipped forever when i read the source code. So i had to modify the code. Now i'm try to submit my patch.But my patch is not perfect. I think there should be an other config to determine if the large size storefile should join major compaction in HColumnDescriptor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12712) skipLargeFiles in minor compact but not in major compact
[ https://issues.apache.org/jira/browse/HBASE-12712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Junhong updated HBASE-12712: Description: Here is my case. After repeatedly minor compaction, the size of storefile is very large. Compaction with large storefile will waste much bandwidth, so i use the “hbase.hstore.compaction.max.size” to skip this case. But after use this config, i find that major compaction will be skipped forever when i read the source code and the deletes and muti-versions data my waste storage. So i had to modify the code. Now i'm try to submit my patch.But my patch is not perfect. I think there should be an other config to determine if the large size storefile should join major compaction in HColumnDescriptor. was: Here is my case. After repeatedly minor compaction, the size of storefile is very large. Compaction with large storefile will waste much bandwidth, so i use the “hbase.hstore.compaction.max.size” to skip this case. But after use this config, i find that major compaction will be skipped forever when i read the source code. So i had to modify the code. Now i'm try to submit my patch.But my patch is not perfect. I think there should be an other config to determine if the large size storefile should join major compaction in HColumnDescriptor. skipLargeFiles in minor compact but not in major compact Key: HBASE-12712 URL: https://issues.apache.org/jira/browse/HBASE-12712 Project: HBase Issue Type: New Feature Components: Compaction Affects Versions: 0.98.6 Reporter: Liu Junhong Labels: newbie, patch Fix For: 0.98.6 Attachments: compact.diff Original Estimate: 72h Remaining Estimate: 72h Here is my case. After repeatedly minor compaction, the size of storefile is very large. Compaction with large storefile will waste much bandwidth, so i use the “hbase.hstore.compaction.max.size” to skip this case. But after use this config, i find that major compaction will be skipped forever when i read the source code and the deletes and muti-versions data my waste storage. So i had to modify the code. Now i'm try to submit my patch.But my patch is not perfect. I think there should be an other config to determine if the large size storefile should join major compaction in HColumnDescriptor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
