[jira] [Updated] (HBASE-25104) The TABLE details of backup Master will always be empty, so I think we can consider dropping it
[ https://issues.apache.org/jira/browse/HBASE-25104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-25104: Priority: Minor (was: Trivial) > The TABLE details of backup Master will always be empty, so I think we can > consider dropping it > --- > > Key: HBASE-25104 > URL: https://issues.apache.org/jira/browse/HBASE-25104 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 2.3.2 >Reporter: shenshengli >Assignee: shenshengli >Priority: Minor > Attachments: 20200928133422.png, 20200928133446.png, HBASE-25104.patch > > > When the HBase Master is the Backup Master, the table details page on the UI > is always empty, so I think it can be deleted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-25104) The TABLE details of backup Master will always be empty, so I think we can consider dropping it
[ https://issues.apache.org/jira/browse/HBASE-25104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-25104 started by shenshengli. --- > The TABLE details of backup Master will always be empty, so I think we can > consider dropping it > --- > > Key: HBASE-25104 > URL: https://issues.apache.org/jira/browse/HBASE-25104 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 2.3.2 >Reporter: shenshengli >Assignee: shenshengli >Priority: Trivial > Attachments: 20200928133422.png, 20200928133446.png, HBASE-25104.patch > > > When the HBase Master is the Backup Master, the table details page on the UI > is always empty, so I think it can be deleted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25781) Add cacheBlocks option to RowCounter
[
https://issues.apache.org/jira/browse/HBASE-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25781:
Attachment: (was:
HBASE-25781_Add_isCacheBlocks_option_to_RowCounter1.patch)
> Add cacheBlocks option to RowCounter
>
>
> Key: HBASE-25781
> URL: https://issues.apache.org/jira/browse/HBASE-25781
> Project: HBase
> Issue Type: New Feature
> Components: mapreduce
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
>
> {{I want to preload the block cache quickly during performance testing,I find
> this:}}
> {{[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
> {{But it's too slow, adding cacheBlocks option to RowCounter maybe better
> for some case.}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25781) Add cacheBlocks option to RowCounter
[
https://issues.apache.org/jira/browse/HBASE-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25781:
Attachment: (was:
HBASE-25781_Add_isCacheBlocks_option_to_RowCounter.patch)
> Add cacheBlocks option to RowCounter
>
>
> Key: HBASE-25781
> URL: https://issues.apache.org/jira/browse/HBASE-25781
> Project: HBase
> Issue Type: New Feature
> Components: mapreduce
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
>
> {{I want to preload the block cache quickly during performance testing,I find
> this:}}
> {{[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
> {{But it's too slow, adding cacheBlocks option to RowCounter maybe better
> for some case.}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25781) Add cacheBlocks option to RowCounter
[
https://issues.apache.org/jira/browse/HBASE-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25781:
Description:
{{I want to preload the block cache quickly during performance testing,I find
this:}}
{{[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
{{But it's too slow, adding cacheBlocks option to RowCounter maybe better for
some case.}}
was:
{{I want to preload the block cache quickly during performance testing,I find
this:}}
{{[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
{{But it's too slow, adding isCacheBlocks option to RowCounter maybe better
for some case.}}
Summary: Add cacheBlocks option to RowCounter (was: Add isCacheBlocks
option to RowCounter)
> Add cacheBlocks option to RowCounter
>
>
> Key: HBASE-25781
> URL: https://issues.apache.org/jira/browse/HBASE-25781
> Project: HBase
> Issue Type: New Feature
> Components: mapreduce
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
> Attachments:
> HBASE-25781_Add_isCacheBlocks_option_to_RowCounter.patch,
> HBASE-25781_Add_isCacheBlocks_option_to_RowCounter1.patch
>
>
> {{I want to preload the block cache quickly during performance testing,I find
> this:}}
> {{[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
> {{But it's too slow, adding cacheBlocks option to RowCounter maybe better
> for some case.}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HBASE-25781) Add isCacheBlocks option to RowCounter
[
https://issues.apache.org/jira/browse/HBASE-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-25781 started by shenshengli.
---
> Add isCacheBlocks option to RowCounter
> --
>
> Key: HBASE-25781
> URL: https://issues.apache.org/jira/browse/HBASE-25781
> Project: HBase
> Issue Type: New Feature
> Components: mapreduce
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
> Attachments:
> HBASE-25781_Add_isCacheBlocks_option_to_RowCounter.patch,
> HBASE-25781_Add_isCacheBlocks_option_to_RowCounter1.patch
>
>
> {{I want to preload the block cache quickly during performance testing,I find
> this:}}
> {{[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
> {{But it's too slow, adding isCacheBlocks option to RowCounter maybe better
> for some case.}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25781) Add isCacheBlocks option to RowCounter
[
https://issues.apache.org/jira/browse/HBASE-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25781:
Component/s: mapreduce
> Add isCacheBlocks option to RowCounter
> --
>
> Key: HBASE-25781
> URL: https://issues.apache.org/jira/browse/HBASE-25781
> Project: HBase
> Issue Type: New Feature
> Components: mapreduce
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
> Attachments:
> HBASE-25781_Add_isCacheBlocks_option_to_RowCounter.patch,
> HBASE-25781_Add_isCacheBlocks_option_to_RowCounter1.patch
>
>
> {{I want to preload the block cache quickly during performance testing,I find
> this:}}
> {{[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
> {{But it's too slow, adding isCacheBlocks option to RowCounter maybe better
> for some case.}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25781) Add isCacheBlocks option to RowCounter
[
https://issues.apache.org/jira/browse/HBASE-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25781:
Attachment: HBASE-25781_Add_isCacheBlocks_option_to_RowCounter1.patch
> Add isCacheBlocks option to RowCounter
> --
>
> Key: HBASE-25781
> URL: https://issues.apache.org/jira/browse/HBASE-25781
> Project: HBase
> Issue Type: New Feature
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
> Attachments:
> HBASE-25781_Add_isCacheBlocks_option_to_RowCounter.patch,
> HBASE-25781_Add_isCacheBlocks_option_to_RowCounter1.patch
>
>
> {{I want to preload the block cache quickly during performance testing,I find
> this:}}
> {{[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
> {{But it's too slow, adding isCacheBlocks option to RowCounter maybe better
> for some case.}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25781) Add isCacheBlocks option to RowCounter
[
https://issues.apache.org/jira/browse/HBASE-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25781:
Attachment: HBASE-25781_Add_isCacheBlocks_option_to_RowCounter.patch
> Add isCacheBlocks option to RowCounter
> --
>
> Key: HBASE-25781
> URL: https://issues.apache.org/jira/browse/HBASE-25781
> Project: HBase
> Issue Type: New Feature
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
> Attachments: HBASE-25781_Add_isCacheBlocks_option_to_RowCounter.patch
>
>
> {{I want to preload the block cache quickly during performance testing,I find
> this:}}
> {{[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
> {{But it's too slow, adding isCacheBlocks option to RowCounter maybe better
> for some case.}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25781) Add isCacheBlocks option to RowCounter
[
https://issues.apache.org/jira/browse/HBASE-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25781:
Description:
{{I want to preload the block cache quickly during performance testing,I find
this:}}
{{[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
{{But it's too slow, adding isCacheBlocks option to RowCounter maybe better
for some case.}}
was:
{{I want to preload the block cache quickly during performance testing,I find
this:}}
{{--[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
{{But it's too slow, adding isCacheBlocks option to RowCounter maybe better
for some case.}}
> Add isCacheBlocks option to RowCounter
> --
>
> Key: HBASE-25781
> URL: https://issues.apache.org/jira/browse/HBASE-25781
> Project: HBase
> Issue Type: New Feature
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
>
> {{I want to preload the block cache quickly during performance testing,I find
> this:}}
> {{[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
> {{But it's too slow, adding isCacheBlocks option to RowCounter maybe better
> for some case.}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25781) Add isCacheBlocks option to RowCounter
[
https://issues.apache.org/jira/browse/HBASE-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25781:
Description:
{{I want to preload the block cache quickly during performance testing,I find
this:}}
{{--[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
{{But it's too slow, adding isCacheBlocks option to RowCounter maybe better
for some case.}}
> Add isCacheBlocks option to RowCounter
> --
>
> Key: HBASE-25781
> URL: https://issues.apache.org/jira/browse/HBASE-25781
> Project: HBase
> Issue Type: New Feature
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
>
> {{I want to preload the block cache quickly during performance testing,I find
> this:}}
> {{--[#HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb]}}
> {{But it's too slow, adding isCacheBlocks option to RowCounter maybe better
> for some case.}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25781) Add isCacheBlocks option to RowCounter
[
https://issues.apache.org/jira/browse/HBASE-25781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25781:
Environment: (was: {{I want to preload the block cache quickly during
performance testing,I find this:}}
{{HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb}}
{{But it's too slow, adding isCacheBlocks option to RowCounter maybe better for
some case.}})
> Add isCacheBlocks option to RowCounter
> --
>
> Key: HBASE-25781
> URL: https://issues.apache.org/jira/browse/HBASE-25781
> Project: HBase
> Issue Type: New Feature
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25781) Add isCacheBlocks option to RowCounter
shenshengli created HBASE-25781:
---
Summary: Add isCacheBlocks option to RowCounter
Key: HBASE-25781
URL: https://issues.apache.org/jira/browse/HBASE-25781
Project: HBase
Issue Type: New Feature
Environment: {{I want to preload the block cache quickly during
performance testing,I find this:}}
{{HBASE-25278 Add option to toggle CACHE_BLOCKS in count.rb}}
{{But it's too slow, adding isCacheBlocks option to RowCounter maybe better for
some case.}}
Reporter: shenshengli
Assignee: shenshengli
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25498) [Documentation] Add a comment when using Secure HTTP (HTTPS) for the Web UI
[
https://issues.apache.org/jira/browse/HBASE-25498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25498:
Component/s: documentation
> [Documentation] Add a comment when using Secure HTTP (HTTPS) for the Web UI
> ---
>
> Key: HBASE-25498
> URL: https://issues.apache.org/jira/browse/HBASE-25498
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 2.0.0
>Reporter: wenhao
>Assignee: wenhao
>Priority: Trivial
> Fix For: 3.0.0-alpha-1
>
>
> When I configured https for the HBase web UI according to the document(set
> {{hbase.ssl.enabled}} to {{true}} in _hbase-site.xml_), there are the
> following errors:
> {quote}2021-01-29 11:25:19,360 ERROR [main] master.HMasterCommandLine: Master
> exiting2021-01-29 11:25:19,360 ERROR [main] master.HMasterCommandLine: Master
> exitingjava.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMaster. at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2910) at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:236)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2921)Caused by:
> java.io.IOException: Problem starting http server at
> org.apache.hadoop.hbase.http.HttpServer.start(HttpServer.java:1006) at
> org.apache.hadoop.hbase.http.InfoServer.start(InfoServer.java:100) at
> org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:2015)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:627)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:472) at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2903) ...
> 5 moreCaused by: java.lang.IllegalStateException: no valid keystore
> {quote}
> According to the following link
> (https://community.cloudera.com/t5/Community-Articles/Enable-HTTPS-SSL-for-HBASE-Master-UI/ta-p/247031),
> I found that SSL certificate and SSL configuration are required. In the
> configuration document of hbase, we do not need to write a detailed
> configuration process, but we need to be reminded: only modifying the
> parameters according to the configuration steps will not take effect.
> I think we need to add hints to the document(Please prepare SSL certificate
> and ssl configuration file in advance)：
> {quote}
> h3. 59.1. Using Secure HTTP (HTTPS) for the Web UI
> A default HBase install uses insecure HTTP connections for Web UIs for the
> master and region servers. To enable secure HTTP (HTTPS) connections instead,
> set {{hbase.ssl.enabled}} to {{true}} in _hbase-site.xml(Please prepare SSL
> certificate and ssl configuration file in advance)_.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25710) During the recovery process, an error is thrown if there is an incremental backup of data that has not been updated
shenshengli created HBASE-25710: --- Summary: During the recovery process, an error is thrown if there is an incremental backup of data that has not been updated Key: HBASE-25710 URL: https://issues.apache.org/jira/browse/HBASE-25710 Project: HBase Issue Type: Bug Components: backuprestore Affects Versions: 2.0.0 Reporter: shenshengli Assignee: shenshengli The error is shown below: 19:49:24.213 [main] ERROR org.apache.hadoop.hbase.backup.RestoreDriver - Error while running restore backup java.io.IOException: Can not restore from backup directory (check Hadoop and HBase logs) at org.apache.hadoop.hbase.backup.mapreduce.MapReduceRestoreJob.run(MapReduceRestoreJob.java:110) ~[hbase-backup-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hbase.backup.util.RestoreTool.incrementalRestoreTable(RestoreTool.java:202) ~[hbase-backup-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hbase.backup.impl.RestoreTablesClient.restoreImages(RestoreTablesClient.java:178) ~[hbase-backup-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hbase.backup.impl.RestoreTablesClient.restore(RestoreTablesClient.java:221) ~[hbase-backup-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hbase.backup.impl.RestoreTablesClient.execute(RestoreTablesClient.java:258) ~[hbase-backup-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hbase.backup.impl.BackupAdminImpl.restore(BackupAdminImpl.java:520) ~[hbase-backup-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hbase.backup.RestoreDriver.parseAndRun(RestoreDriver.java:179) [hbase-backup-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hbase.backup.RestoreDriver.doWork(RestoreDriver.java:220) [hbase-backup-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.hbase.backup.RestoreDriver.run(RestoreDriver.java:256) [hbase-backup-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) [hadoop-common-3.1.1.3.0.1.0-187.jar:?] at org.apache.hadoop.hbase.backup.RestoreDriver.main(RestoreDriver.java:228) [hbase-backup-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] Caused by: java.io.IOException: No input paths specified in job -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25609) There is a problem with the SPLITS_FILE in the HBase shell statement
[
https://issues.apache.org/jira/browse/HBASE-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25609:
Component/s: shell
> There is a problem with the SPLITS_FILE in the HBase shell statement
>
>
> Key: HBASE-25609
> URL: https://issues.apache.org/jira/browse/HBASE-25609
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.1
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
> Fix For: 3.0.0-alpha-1, 2.2.7, 2.5.0, 2.3.5, 2.4.2
>
>
> The Ruby script here has a problem passing in a null value:
> hbase-shell/src/main/ruby/hbase/admin.rb
> {code:java}
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE]){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HBASE-25707) When restoring a table, create a namespace if it does not exist
[ https://issues.apache.org/jira/browse/HBASE-25707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli reassigned HBASE-25707: --- Assignee: shenshengli > When restoring a table, create a namespace if it does not exist > --- > > Key: HBASE-25707 > URL: https://issues.apache.org/jira/browse/HBASE-25707 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: shenshengli >Assignee: shenshengli >Priority: Minor > > It does not seem to have been taken into account that the namespace of the > table to be restored does not exist in the target environment, and if the > namespace does not exist, it will simply throw an error > (NamespaceNotFoundException ), which is unfriendly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25707) When restoring a table, create a namespace if it does not exist
[ https://issues.apache.org/jira/browse/HBASE-25707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-25707: Component/s: backup > When restoring a table, create a namespace if it does not exist > --- > > Key: HBASE-25707 > URL: https://issues.apache.org/jira/browse/HBASE-25707 > Project: HBase > Issue Type: Bug > Components: backuprestore >Affects Versions: 2.0.0 >Reporter: shenshengli >Assignee: shenshengli >Priority: Minor > > It does not seem to have been taken into account that the namespace of the > table to be restored does not exist in the target environment, and if the > namespace does not exist, it will simply throw an error > (NamespaceNotFoundException ), which is unfriendly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25707) When restoring a table, create a namespace if it does not exist
[ https://issues.apache.org/jira/browse/HBASE-25707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-25707: Affects Version/s: 2.0.0 > When restoring a table, create a namespace if it does not exist > --- > > Key: HBASE-25707 > URL: https://issues.apache.org/jira/browse/HBASE-25707 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: shenshengli >Priority: Minor > > It does not seem to have been taken into account that the namespace of the > table to be restored does not exist in the target environment, and if the > namespace does not exist, it will simply throw an error > (NamespaceNotFoundException ), which is unfriendly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25707) When restoring a table, create a namespace if it does not exist
shenshengli created HBASE-25707: --- Summary: When restoring a table, create a namespace if it does not exist Key: HBASE-25707 URL: https://issues.apache.org/jira/browse/HBASE-25707 Project: HBase Issue Type: Bug Reporter: shenshengli It does not seem to have been taken into account that the namespace of the table to be restored does not exist in the target environment, and if the namespace does not exist, it will simply throw an error (NamespaceNotFoundException ), which is unfriendly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25609) There is a problem with the SPLITS_FILE in the HBase shell statement
[
https://issues.apache.org/jira/browse/HBASE-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17291363#comment-17291363
]
shenshengli commented on HBASE-25609:
-
Before hbase-20819, this caused the Hbase shell to fail to create a table with
SPLITS_FILE.because the setValue method in HTableDescriptor is:
{code:java}
public HTableDescriptor setValue(String key, String value) {
getDelegateeForModification().setValue(key, value);
return this;
}{code}
After hbase-20819,The problem doesn't seem so serious because the setValue
method in TableDescriptorBuilder is:
{code:java}
public TableDescriptorBuilder setValue(final String key, final String value) {
desc.setValue(key, value);
return this;
}
public ModifyableTableDescriptor setValue(String key, String value) {
return setValue(toBytesOrNull(key, Bytes::toBytes),
toBytesOrNull(value, Bytes::toBytes));
}
{code}
> There is a problem with the SPLITS_FILE in the HBase shell statement
>
>
> Key: HBASE-25609
> URL: https://issues.apache.org/jira/browse/HBASE-25609
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.1
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
>
> The Ruby script here has a problem passing in a null value:
> hbase-shell/src/main/ruby/hbase/admin.rb
> {code:java}
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE]){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25609) There is a problem with the SPLITS_FILE in the HBase shell statement
[
https://issues.apache.org/jira/browse/HBASE-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25609:
Attachment: (was: HBASE-25609.patch)
> There is a problem with the SPLITS_FILE in the HBase shell statement
>
>
> Key: HBASE-25609
> URL: https://issues.apache.org/jira/browse/HBASE-25609
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.1
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
>
> The Ruby script here has a problem passing in a null value:
> hbase-shell/src/main/ruby/hbase/admin.rb
> {code:java}
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE]){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25609) There is a problem with the SPLITS_FILE in the HBase shell statement
[
https://issues.apache.org/jira/browse/HBASE-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25609:
External issue URL: (was: https://github.com/apache/hbase/pull/2992)
> There is a problem with the SPLITS_FILE in the HBase shell statement
>
>
> Key: HBASE-25609
> URL: https://issues.apache.org/jira/browse/HBASE-25609
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.1
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
> Attachments: HBASE-25609.patch
>
>
> The Ruby script here has a problem passing in a null value:
> hbase-shell/src/main/ruby/hbase/admin.rb
> {code:java}
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE]){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25609) There is a problem with the SPLITS_FILE in the HBase shell statement
[
https://issues.apache.org/jira/browse/HBASE-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25609:
External issue URL: https://github.com/apache/hbase/pull/2992
> There is a problem with the SPLITS_FILE in the HBase shell statement
>
>
> Key: HBASE-25609
> URL: https://issues.apache.org/jira/browse/HBASE-25609
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.1
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
> Attachments: HBASE-25609.patch
>
>
> The Ruby script here has a problem passing in a null value:
> hbase-shell/src/main/ruby/hbase/admin.rb
> {code:java}
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE]){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25609) There is a problem with the SPLITS_FILE in the HBase shell statement
[
https://issues.apache.org/jira/browse/HBASE-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25609:
Description:
The Ruby script here has a problem passing in a null value:
hbase-shell/src/main/ruby/hbase/admin.rb
{code:java}
if arg.key?(SPLITS_FILE)
splits_file = arg.delete(SPLITS_FILE)
unless File.exist?(splits_file)
raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
end
arg[SPLITS] = []
File.foreach(splits_file) do |line|
arg[SPLITS].push(line.chomp)
end
tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE]){code}
was:
The Ruby script here has a problem passing in a null value:
hbase-shell/src/main/ruby/hbase/admin.rb
if arg.key?(SPLITS_FILE)
splits_file = arg.delete(SPLITS_FILE)
unless File.exist?(splits_file)
raise(ArgumentError, "Splits file #\{splits_file} doesn't exist")
end
arg[SPLITS] = []
File.foreach(splits_file) do |line|
arg[SPLITS].push(line.chomp)
end
tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE])
> There is a problem with the SPLITS_FILE in the HBase shell statement
>
>
> Key: HBASE-25609
> URL: https://issues.apache.org/jira/browse/HBASE-25609
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.1
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
> Attachments: HBASE-25609.patch
>
>
> The Ruby script here has a problem passing in a null value:
> hbase-shell/src/main/ruby/hbase/admin.rb
> {code:java}
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE]){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25609) There is a problem with the SPLITS_FILE in the HBase shell statement
[
https://issues.apache.org/jira/browse/HBASE-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25609:
Affects Version/s: 2.0.1
> There is a problem with the SPLITS_FILE in the HBase shell statement
>
>
> Key: HBASE-25609
> URL: https://issues.apache.org/jira/browse/HBASE-25609
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.1
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
> Attachments: HBASE-25609.patch
>
>
> The Ruby script here has a problem passing in a null value:
> hbase-shell/src/main/ruby/hbase/admin.rb
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #\{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE])
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25609) There is a problem with the SPLITS_FILE in the HBase shell statement
[
https://issues.apache.org/jira/browse/HBASE-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25609:
Attachment: HBASE-25609.patch
Status: Patch Available (was: Open)
> There is a problem with the SPLITS_FILE in the HBase shell statement
>
>
> Key: HBASE-25609
> URL: https://issues.apache.org/jira/browse/HBASE-25609
> Project: HBase
> Issue Type: Bug
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
> Attachments: HBASE-25609.patch
>
>
> The Ruby script here has a problem passing in a null value:
> hbase-shell/src/main/ruby/hbase/admin.rb
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #\{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE])
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work stopped] (HBASE-25609) There is a problem with the SPLITS_FILE in the HBase shell statement
[
https://issues.apache.org/jira/browse/HBASE-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-25609 stopped by shenshengli.
---
> There is a problem with the SPLITS_FILE in the HBase shell statement
>
>
> Key: HBASE-25609
> URL: https://issues.apache.org/jira/browse/HBASE-25609
> Project: HBase
> Issue Type: Bug
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
>
> The Ruby script here has a problem passing in a null value:
> hbase-shell/src/main/ruby/hbase/admin.rb
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #\{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE])
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HBASE-25609) There is a problem with the SPLITS_FILE in the HBase shell statement
[
https://issues.apache.org/jira/browse/HBASE-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-25609 started by shenshengli.
---
> There is a problem with the SPLITS_FILE in the HBase shell statement
>
>
> Key: HBASE-25609
> URL: https://issues.apache.org/jira/browse/HBASE-25609
> Project: HBase
> Issue Type: Bug
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
>
> The Ruby script here has a problem passing in a null value:
> hbase-shell/src/main/ruby/hbase/admin.rb
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #\{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE])
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HBASE-25609) There is a problem with the SPLITS_FILE in the HBase shell statement
[
https://issues.apache.org/jira/browse/HBASE-25609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli reassigned HBASE-25609:
---
Assignee: shenshengli
> There is a problem with the SPLITS_FILE in the HBase shell statement
>
>
> Key: HBASE-25609
> URL: https://issues.apache.org/jira/browse/HBASE-25609
> Project: HBase
> Issue Type: Bug
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
>
> The Ruby script here has a problem passing in a null value:
> hbase-shell/src/main/ruby/hbase/admin.rb
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #\{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE])
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25609) There is a problem with the SPLITS_FILE in the HBase shell statement
shenshengli created HBASE-25609:
---
Summary: There is a problem with the SPLITS_FILE in the HBase
shell statement
Key: HBASE-25609
URL: https://issues.apache.org/jira/browse/HBASE-25609
Project: HBase
Issue Type: Bug
Reporter: shenshengli
The Ruby script here has a problem passing in a null value:
hbase-shell/src/main/ruby/hbase/admin.rb
if arg.key?(SPLITS_FILE)
splits_file = arg.delete(SPLITS_FILE)
unless File.exist?(splits_file)
raise(ArgumentError, "Splits file #\{splits_file} doesn't exist")
end
arg[SPLITS] = []
File.foreach(splits_file) do |line|
arg[SPLITS].push(line.chomp)
end
tdb.setValue(SPLITS_FILE, arg[SPLITS_FILE])
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-25449: Description: I think this parameter is not suitable for in hbase-default.xml, because in this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase rely on HDFS configuration, the parameters in hbase service still is false.Must be explicitly in hbase-site.xml is set to "dfs.client.read.shortcircuit=true" to take effect.The same goes for the parameter 'dfs.domain.socket.path', which is already covered in the specific change. (was: I think this parameter is not suitable for in hbase-default.xml, because in this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase rely on HDFS configuration, the parameters in hbase service still is false.Must be explicitly in hbase-site.xml is set to "dfs.client.read.shortcircuit=true" to take effect.The same goes for the parameter 'dfs.domain.socket.path', which is already covered in the specific change) > 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml > - > > Key: HBASE-25449 > URL: https://issues.apache.org/jira/browse/HBASE-25449 > Project: HBase > Issue Type: Improvement > Components: conf >Affects Versions: 2.0.1 >Reporter: shenshengli >Assignee: shenshengli >Priority: Major > Fix For: 3.0.0-alpha-1, 2.2.7, 2.5.0, 2.3.5, 2.4.2 > > > I think this parameter is not suitable for in hbase-default.xml, because in > this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase > rely on HDFS configuration, the parameters in hbase service still is > false.Must be explicitly in hbase-site.xml is set to > "dfs.client.read.shortcircuit=true" to take effect.The same goes for the > parameter 'dfs.domain.socket.path', which is already covered in the specific > change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-25449: Description: I think this parameter is not suitable for in hbase-default.xml, because in this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase rely on HDFS configuration, the parameters in hbase service still is false.Must be explicitly in hbase-site.xml is set to "dfs.client.read.shortcircuit=true" to take effect.The same goes for the parameter 'dfs.domain.socket.path', which is already covered in the specific change (was: I think this parameter is not suitable for in hbase-default.xml, because in this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase rely on HDFS configuration, the parameters in hbase service still is false.Must be explicitly in hbase-site.xml is set to "dfs.client.read.shortcircuit=true" to take effect.) > 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml > - > > Key: HBASE-25449 > URL: https://issues.apache.org/jira/browse/HBASE-25449 > Project: HBase > Issue Type: Improvement > Components: conf >Affects Versions: 2.0.1 >Reporter: shenshengli >Assignee: shenshengli >Priority: Major > Fix For: 3.0.0-alpha-1, 2.2.7, 2.5.0, 2.3.5, 2.4.2 > > > I think this parameter is not suitable for in hbase-default.xml, because in > this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase > rely on HDFS configuration, the parameters in hbase service still is > false.Must be explicitly in hbase-site.xml is set to > "dfs.client.read.shortcircuit=true" to take effect.The same goes for the > parameter 'dfs.domain.socket.path', which is already covered in the specific > change -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259491#comment-17259491
]
shenshengli commented on HBASE-25449:
-
{quote}I see. You want the hdfs-site.xml configuration to shine through if it
is NOT set in hbase? Is that so? If that is the case, the PR makes sense to me.
Is it possible to add a test? Thank you.
{quote}
I added a unit test to demonstrate my purpose : Make this configuration empty
in hbase-default.xml , get values in hdfs-default.xml or hdfs-site.xml.
> 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
> -
>
> Key: HBASE-25449
> URL: https://issues.apache.org/jira/browse/HBASE-25449
> Project: HBase
> Issue Type: Improvement
> Components: conf
>Affects Versions: 2.0.1
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Major
>
> I think this parameter is not suitable for in hbase-default.xml, because in
> this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase
> rely on HDFS configuration, the parameters in hbase service still is
> false.Must be explicitly in hbase-site.xml is set to
> "dfs.client.read.shortcircuit=true" to take effect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Issue Comment Deleted] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-25449: Comment: was deleted (was: I see. You want the hdfs-site.xml configuration to shine through if it is NOT set in hbase? Is that so? If that is the case, the PR makes sense to me. Is it possible to add a test? Thank you. I added a unit test to demonstrate my purpose : Make this configuration empty in hbase-default.xml , get values in hdfs-default.xml or hdfs-site.xml.) > 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml > - > > Key: HBASE-25449 > URL: https://issues.apache.org/jira/browse/HBASE-25449 > Project: HBase > Issue Type: Improvement > Components: conf >Affects Versions: 2.0.1 >Reporter: shenshengli >Assignee: shenshengli >Priority: Major > > I think this parameter is not suitable for in hbase-default.xml, because in > this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase > rely on HDFS configuration, the parameters in hbase service still is > false.Must be explicitly in hbase-site.xml is set to > "dfs.client.read.shortcircuit=true" to take effect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259490#comment-17259490 ] shenshengli commented on HBASE-25449: - I see. You want the hdfs-site.xml configuration to shine through if it is NOT set in hbase? Is that so? If that is the case, the PR makes sense to me. Is it possible to add a test? Thank you. I added a unit test to demonstrate my purpose : Make this configuration empty in hbase-default.xml , get values in hdfs-default.xml or hdfs-site.xml. > 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml > - > > Key: HBASE-25449 > URL: https://issues.apache.org/jira/browse/HBASE-25449 > Project: HBase > Issue Type: Improvement > Components: conf >Affects Versions: 2.0.1 >Reporter: shenshengli >Assignee: shenshengli >Priority: Major > > I think this parameter is not suitable for in hbase-default.xml, because in > this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase > rely on HDFS configuration, the parameters in hbase service still is > false.Must be explicitly in hbase-site.xml is set to > "dfs.client.read.shortcircuit=true" to take effect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257899#comment-17257899
]
shenshengli commented on HBASE-25449:
-
{quote}Do you mean, users could forget to set it in hbase and in hdfs?
{quote}
Yes.I just left the default value of this configuration empty so that I could
use the HDFS configuration directly.
{quote}I don't follow why this hbase-default configuration does not prevail
over the hdfs default? Is it that the hdfs jar with hdfs-default it in comes
before hbase-default in the classpath?
{quote}
This hbase-default configuration does prevail over the hdfs default,and prevail
over the hdfs-site.
{quote}Is it that you observe in operation that the default
dfs.client.read.shortcircuit=true in hbase-default is not happening?
{quote}
I find the default dfs.client.read.shortcircuit=false in hbase-default.So, even
if this value is true in my HDFs-site.xml, I still need to explicitly set it in
hbase-site.xml.
> 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
> -
>
> Key: HBASE-25449
> URL: https://issues.apache.org/jira/browse/HBASE-25449
> Project: HBase
> Issue Type: Improvement
> Components: conf
>Affects Versions: 2.0.1
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Major
>
> I think this parameter is not suitable for in hbase-default.xml, because in
> this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase
> rely on HDFS configuration, the parameters in hbase service still is
> false.Must be explicitly in hbase-site.xml is set to
> "dfs.client.read.shortcircuit=true" to take effect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25450) The parameter "hbase.bucketcache.size" is misdescribed
[
https://issues.apache.org/jira/browse/HBASE-25450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25450:
Affects Version/s: (was: 2.3.2)
2.0.0
> The parameter "hbase.bucketcache.size" is misdescribed
> --
>
> Key: HBASE-25450
> URL: https://issues.apache.org/jira/browse/HBASE-25450
> Project: HBase
> Issue Type: Improvement
> Components: conf
>Affects Versions: 2.0.0
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Major
> Attachments: image-2020-12-28-17-53-32-575.png
>
>
> You can see that the description of this configuration is outdated
> {quote}
> hbase.bucketcache.size
>
> A float that EITHER represents a percentage of total heap memory
> size to give to the cache (if < 1.0) OR, it is the total capacity in
> megabytes of BucketCache. Default: 0.0
>
> {quote}
> !image-2020-12-28-17-53-32-575.png!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25450) The parameter "hbase.bucketcache.size" is misdescribed
[
https://issues.apache.org/jira/browse/HBASE-25450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255830#comment-17255830
]
shenshengli commented on HBASE-25450:
-
The modification here is from Hbase-19187.Modify comments and documentation
here.
> The parameter "hbase.bucketcache.size" is misdescribed
> --
>
> Key: HBASE-25450
> URL: https://issues.apache.org/jira/browse/HBASE-25450
> Project: HBase
> Issue Type: Improvement
> Components: conf
>Affects Versions: 2.3.2
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Major
> Attachments: image-2020-12-28-17-53-32-575.png
>
>
> You can see that the description of this configuration is outdated
> {quote}
> hbase.bucketcache.size
>
> A float that EITHER represents a percentage of total heap memory
> size to give to the cache (if < 1.0) OR, it is the total capacity in
> megabytes of BucketCache. Default: 0.0
>
> {quote}
> !image-2020-12-28-17-53-32-575.png!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25450) The parameter "hbase.bucketcache.size" is misdescribed
[
https://issues.apache.org/jira/browse/HBASE-25450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenshengli updated HBASE-25450:
Description:
You can see that the description of this configuration is outdated
{quote}
hbase.bucketcache.size
A float that EITHER represents a percentage of total heap memory
size to give to the cache (if < 1.0) OR, it is the total capacity in
megabytes of BucketCache. Default: 0.0
{quote}
!image-2020-12-28-17-53-32-575.png!
was:
* You can see that the description of this configuration is outdated
{quote}
hbase.bucketcache.size
A float that EITHER represents a percentage of total heap memory
size to give to the cache (if 1.0) OR, it is the total capacity in
megabytes of BucketCache. Default: 0.0
{quote}
!image-2020-12-28-17-53-32-575.png!
> The parameter "hbase.bucketcache.size" is misdescribed
> --
>
> Key: HBASE-25450
> URL: https://issues.apache.org/jira/browse/HBASE-25450
> Project: HBase
> Issue Type: Improvement
> Components: conf
>Affects Versions: 2.3.2
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Major
> Attachments: image-2020-12-28-17-53-32-575.png
>
>
> You can see that the description of this configuration is outdated
> {quote}
> hbase.bucketcache.size
>
> A float that EITHER represents a percentage of total heap memory
> size to give to the cache (if < 1.0) OR, it is the total capacity in
> megabytes of BucketCache. Default: 0.0
>
> {quote}
> !image-2020-12-28-17-53-32-575.png!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-25449: Priority: Major (was: Minor) > 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml > - > > Key: HBASE-25449 > URL: https://issues.apache.org/jira/browse/HBASE-25449 > Project: HBase > Issue Type: Improvement > Components: conf >Affects Versions: 2.0.1 >Reporter: shenshengli >Assignee: shenshengli >Priority: Major > > I think this parameter is not suitable for in hbase-default.xml, because in > this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase > rely on HDFS configuration, the parameters in hbase service still is > false.Must be explicitly in hbase-site.xml is set to > "dfs.client.read.shortcircuit=true" to take effect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255764#comment-17255764
]
shenshengli commented on HBASE-25449:
-
{quote}Please explain more what you are thinking. The intention is that
short-circuit is enabled by default. Thank you.
{quote}
This change only happens when HDFS is already explicitly set to open
short-circuit reads, leaving HBase to open short-circuit reads by default.
> 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
> -
>
> Key: HBASE-25449
> URL: https://issues.apache.org/jira/browse/HBASE-25449
> Project: HBase
> Issue Type: Improvement
> Components: conf
>Affects Versions: 2.0.1
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Minor
>
> I think this parameter is not suitable for in hbase-default.xml, because in
> this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase
> rely on HDFS configuration, the parameters in hbase service still is
> false.Must be explicitly in hbase-site.xml is set to
> "dfs.client.read.shortcircuit=true" to take effect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255763#comment-17255763 ] shenshengli commented on HBASE-25449: - Because this is an HDFS client parameter, there are default values in HDFs-default.xml and explicit values in HDFs-site.xml, which HBase relies on.If THE HDFS is not explicitly set to on, use the HDFS default shutdown effect here as well.If HDFS is already explicitly set to on, and because of this default, hbase needs to be explicitly set to on, this can easily be ignored.Since open short circuit reading has been mentioned by many people as a way to improve performance, why have it set twice? > 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml > - > > Key: HBASE-25449 > URL: https://issues.apache.org/jira/browse/HBASE-25449 > Project: HBase > Issue Type: Improvement > Components: conf >Affects Versions: 2.0.1 >Reporter: shenshengli >Assignee: shenshengli >Priority: Minor > > I think this parameter is not suitable for in hbase-default.xml, because in > this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase > rely on HDFS configuration, the parameters in hbase service still is > false.Must be explicitly in hbase-site.xml is set to > "dfs.client.read.shortcircuit=true" to take effect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-25450) The parameter "hbase.bucketcache.size" is misdescribed
[
https://issues.apache.org/jira/browse/HBASE-25450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-25450 started by shenshengli.
---
> The parameter "hbase.bucketcache.size" is misdescribed
> --
>
> Key: HBASE-25450
> URL: https://issues.apache.org/jira/browse/HBASE-25450
> Project: HBase
> Issue Type: Improvement
> Components: conf
>Affects Versions: 2.3.2
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Major
> Attachments: image-2020-12-28-17-53-32-575.png
>
>
> * You can see that the description of this configuration is outdated
> {quote}
> hbase.bucketcache.size
>
> A float that EITHER represents a percentage of total heap memory
> size to give to the cache (if 1.0) OR, it is the total capacity in
> megabytes of BucketCache. Default: 0.0
>
> {quote}
> !image-2020-12-28-17-53-32-575.png!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25450) The parameter "hbase.bucketcache.size" is misdescribed
shenshengli created HBASE-25450:
---
Summary: The parameter "hbase.bucketcache.size" is misdescribed
Key: HBASE-25450
URL: https://issues.apache.org/jira/browse/HBASE-25450
Project: HBase
Issue Type: Improvement
Components: conf
Affects Versions: 2.3.2
Reporter: shenshengli
Assignee: shenshengli
Attachments: image-2020-12-28-17-53-32-575.png
* You can see that the description of this configuration is outdated
{quote}
hbase.bucketcache.size
A float that EITHER represents a percentage of total heap memory
size to give to the cache (if 1.0) OR, it is the total capacity in
megabytes of BucketCache. Default: 0.0
{quote}
!image-2020-12-28-17-53-32-575.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-25449: External issue URL: (was: https://github.com/apache/hbase/pull/2816) > 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml > - > > Key: HBASE-25449 > URL: https://issues.apache.org/jira/browse/HBASE-25449 > Project: HBase > Issue Type: Improvement > Components: conf >Affects Versions: 2.0.1 >Reporter: shenshengli >Assignee: shenshengli >Priority: Minor > > I think this parameter is not suitable for in hbase-default.xml, because in > this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase > rely on HDFS configuration, the parameters in hbase service still is > false.Must be explicitly in hbase-site.xml is set to > "dfs.client.read.shortcircuit=true" to take effect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-25449: External issue URL: https://github.com/apache/hbase/pull/2816 > 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml > - > > Key: HBASE-25449 > URL: https://issues.apache.org/jira/browse/HBASE-25449 > Project: HBase > Issue Type: Improvement > Components: conf >Affects Versions: 2.0.1 >Reporter: shenshengli >Assignee: shenshengli >Priority: Minor > > I think this parameter is not suitable for in hbase-default.xml, because in > this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase > rely on HDFS configuration, the parameters in hbase service still is > false.Must be explicitly in hbase-site.xml is set to > "dfs.client.read.shortcircuit=true" to take effect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-25449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-25449: Component/s: (was: Compaction) conf > 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml > - > > Key: HBASE-25449 > URL: https://issues.apache.org/jira/browse/HBASE-25449 > Project: HBase > Issue Type: Improvement > Components: conf >Affects Versions: 2.0.1 >Reporter: shenshengli >Assignee: shenshengli >Priority: Minor > > I think this parameter is not suitable for in hbase-default.xml, because in > this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase > rely on HDFS configuration, the parameters in hbase service still is > false.Must be explicitly in hbase-site.xml is set to > "dfs.client.read.shortcircuit=true" to take effect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25449) 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml
shenshengli created HBASE-25449: --- Summary: 'dfs.client.read.shortcircuit' should not be set in hbase-default.xml Key: HBASE-25449 URL: https://issues.apache.org/jira/browse/HBASE-25449 Project: HBase Issue Type: Improvement Components: Compaction Affects Versions: 2.0.1 Reporter: shenshengli Assignee: shenshengli I think this parameter is not suitable for in hbase-default.xml, because in this case, HDFS explicitly set to "dfs.client.read.shortcircuit=true", hbase rely on HDFS configuration, the parameters in hbase service still is false.Must be explicitly in hbase-site.xml is set to "dfs.client.read.shortcircuit=true" to take effect. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[
https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245645#comment-17245645
]
shenshengli commented on HBASE-21874:
-
I see this comment in the code:
{quote}// TODO this will support only one path over Pmem. To make use of
multiple Pmem devices mounted,
// we need to support multiple paths like files IOEngine. Support later.
{quote}
> Bucket cache on Persistent memory
> -
>
> Key: HBASE-21874
> URL: https://issues.apache.org/jira/browse/HBASE-21874
> Project: HBase
> Issue Type: New Feature
> Components: BucketCache
>Affects Versions: 3.0.0-alpha-1
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.0
>
> Attachments: HBASE-21874.branch-2.2.001.patch, HBASE-21874.patch,
> HBASE-21874.patch, HBASE-21874_V2.patch, HBASE-21874_V4.patch,
> HBASE-21874_V5.patch, HBASE-21874_V6.patch, Pmem_BC.png
>
>
> Non volatile persistent memory devices are byte addressable like DRAM (for
> eg. Intel DCPMM). Bucket cache implementation can take advantage of this new
> memory type and can make use of the existing offheap data structures to serve
> data directly from this memory area without having to bring the data to
> onheap.
> The patch is a new IOEngine implementation that works with the persistent
> memory.
> Note : Here we don't make use of the persistence nature of the device and
> just make use of the big memory it provides.
> Performance numbers to follow.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[ https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17244905#comment-17244905 ] shenshengli commented on HBASE-21874: - [~ram_krish] Only one directory can be configured here, but multiple hardware is usually used, such as 1T (8*128G) App Direct, in which case I want to configure 512G bucketCache, must poolSet be used? > Bucket cache on Persistent memory > - > > Key: HBASE-21874 > URL: https://issues.apache.org/jira/browse/HBASE-21874 > Project: HBase > Issue Type: New Feature > Components: BucketCache >Affects Versions: 3.0.0-alpha-1 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Major > Fix For: 3.0.0-alpha-1, 2.3.0 > > Attachments: HBASE-21874.branch-2.2.001.patch, HBASE-21874.patch, > HBASE-21874.patch, HBASE-21874_V2.patch, HBASE-21874_V4.patch, > HBASE-21874_V5.patch, HBASE-21874_V6.patch, Pmem_BC.png > > > Non volatile persistent memory devices are byte addressable like DRAM (for > eg. Intel DCPMM). Bucket cache implementation can take advantage of this new > memory type and can make use of the existing offheap data structures to serve > data directly from this memory area without having to bring the data to > onheap. > The patch is a new IOEngine implementation that works with the persistent > memory. > Note : Here we don't make use of the persistence nature of the device and > just make use of the big memory it provides. > Performance numbers to follow. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-24053) hbase/bin/region_status.rb Not available in hbase2
[ https://issues.apache.org/jira/browse/HBASE-24053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli reassigned HBASE-24053: --- Assignee: shenshengli > hbase/bin/region_status.rb Not available in hbase2 > -- > > Key: HBASE-24053 > URL: https://issues.apache.org/jira/browse/HBASE-24053 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Assignee: shenshengli >Priority: Minor > > A series of strange errors occurred while running this script in hbase2, > which was corrected as prompted, and note that in hbase2's MetaTableAccessor, > getHRegionInfo has been replaced by getRegionInfo -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25104) The TABLE details of backup Master will always be empty, so I think we can consider dropping it
[ https://issues.apache.org/jira/browse/HBASE-25104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-25104: Attachment: HBASE-25104.patch > The TABLE details of backup Master will always be empty, so I think we can > consider dropping it > --- > > Key: HBASE-25104 > URL: https://issues.apache.org/jira/browse/HBASE-25104 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 2.3.2 >Reporter: shenshengli >Assignee: shenshengli >Priority: Trivial > Attachments: 20200928133422.png, 20200928133446.png, HBASE-25104.patch > > > When the HBase Master is the Backup Master, the table details page on the UI > is always empty, so I think it can be deleted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-25104) The TABLE details of backup Master will always be empty, so I think we can consider dropping it
[ https://issues.apache.org/jira/browse/HBASE-25104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli reassigned HBASE-25104: --- Assignee: shenshengli > The TABLE details of backup Master will always be empty, so I think we can > consider dropping it > --- > > Key: HBASE-25104 > URL: https://issues.apache.org/jira/browse/HBASE-25104 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 2.3.2 >Reporter: shenshengli >Assignee: shenshengli >Priority: Trivial > Attachments: 20200928133422.png, 20200928133446.png > > > When the HBase Master is the Backup Master, the table details page on the UI > is always empty, so I think it can be deleted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25104) The TABLE details of backup Master will always be empty, so I think we can consider dropping it
[ https://issues.apache.org/jira/browse/HBASE-25104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-25104: Attachment: 20200928133422.png 20200928133446.png > The TABLE details of backup Master will always be empty, so I think we can > consider dropping it > --- > > Key: HBASE-25104 > URL: https://issues.apache.org/jira/browse/HBASE-25104 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 2.3.2 >Reporter: shenshengli >Priority: Trivial > Attachments: 20200928133422.png, 20200928133446.png > > > When the HBase Master is the Backup Master, the table details page on the UI > is always empty, so I think it can be deleted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25104) The TABLE details of backup Master will always be empty, so I think we can consider dropping it
shenshengli created HBASE-25104: --- Summary: The TABLE details of backup Master will always be empty, so I think we can consider dropping it Key: HBASE-25104 URL: https://issues.apache.org/jira/browse/HBASE-25104 Project: HBase Issue Type: Improvement Components: UI Affects Versions: 2.3.2 Reporter: shenshengli When the HBase Master is the Backup Master, the table details page on the UI is always empty, so I think it can be deleted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-20552) HBase RegionServer was shutdown due to UnexpectedStateException
[
https://issues.apache.org/jira/browse/HBASE-20552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147512#comment-17147512
]
shenshengli commented on HBASE-20552:
-
I reproduced the problem in my own environment, with more than 10,000 regions
on each RS.By adjusting the parameters of ’hbase.regionserver.msginterval‘, it
from 3 s and 30 s, greatly reduces the risk of the problem.Conversely, if you
go from 3s down to below 1s, this is almost certainly going to happen.
> HBase RegionServer was shutdown due to UnexpectedStateException
> ---
>
> Key: HBASE-20552
> URL: https://issues.apache.org/jira/browse/HBASE-20552
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Romil Choksi
>Assignee: Umesh Agashe
>Priority: Critical
> Attachments:
> 102143-master-ctr-e138-1518143905142-279227-01-03.hwx.site.log,
> 102143-master-ctr-e138-1518143905142-279227-01-05.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-02.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-07.hwx.site.log,
> 102143-regionserver-ctr-e138-1518143905142-279227-01-08.hwx.site.log
>
>
> This was observed during cluster testing (source code sync'ed with hbase-2.0,
> built May 2nd):
> {code}
> 2018-05-02 05:44:10,089 ERROR
> [RpcServer.default.FPBQ.Fifo.handler=28,queue=1,port=2]
> master.MasterRpcServices: Region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474 reported
> a fatal error:
> * ABORTING region server
> ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN, location=ctr-e138-
> 1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on server=ctr-e138-
> 1518143905142-279227-01-02.hwx.site,16020,1525239334474 but state has
> otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
> table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1037)
> ... 7 more
> *
> Cause:
> org.apache.hadoop.hbase.YouAreDeadException:
> org.apache.hadoop.hbase.YouAreDeadException: rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
>table=test_hbase_ha_load_test_tool_hbase,
> region=94f6ca283dbb4445b2bcdc321b734d28reported OPEN on
> server=ctr-e138-1518143905142-279227-01-02.hwx.site,16020,1525239334474
> but state has otherwise.
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.checkOnlineRegionsReport(AssignmentManager.java:1065)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.reportOnlineRegions(AssignmentManager.java:987)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.regionServerReport(MasterRpcServices.java:459)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:13118)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: org.apache.hadoop.hbase.exceptions.UnexpectedStateException:
> rit=OPEN,
> location=ctr-e138-1518143905142-279227-01-07.hwx.site,16020,1525239609353,
>
[jira] [Commented] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting
[
https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073477#comment-17073477
]
shenshengli commented on HBASE-24051:
-
{quote}Another easy way to avoid errors is to add "implements CanUnbuffer" to
the DFSStripedInputStream
Would this be a HDFS change?
{quote}
yes,org.apache.hadoop.hdfs.DFSStripedInputStream in
hadoop-hdfs-project/hadoop-hdfs-client/
{quote}{{I pushed your patch because it looked good but on one of the old PRs
there was a comment by a committer that we need a test that fails unless you
make your change. That possible? Thank you.}}{quote}
OK,I've tried again.
> When the data directory is set to EC,the amount of storefiles reaches a
> relatively high level,heap Occupancy Percent is above heap occupancy alarm
> watermark,resulting in frequent gc and eventual failure
> --
>
> Key: HBASE-24051
> URL: https://issues.apache.org/jira/browse/HBASE-24051
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.1
>Reporter: shenshengli
>Assignee: shenshengli
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.5
>
> Attachments: 1.png
>
>
> When the data directory is set to EC, during the continuous load process,
> regionserver is found down ,to view the log:
> regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below
> the heap occupancy alarm watermark (0.95)
> regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap
> occupancy alarm watermark (0.95)
> [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine
> (eg GC): pause of approximately 1254ms
> GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms
> GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms
> In particular, GC pauses occurred 283 times in half an hour.
> The memory analysis found that most of the memory footprint was generated by
> HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer
> See HDFS-14308，but after the fix, hbase did not return to normal.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073274#comment-17073274 ] shenshengli commented on HBASE-24051: - There was something wrong with that PR, so I closed it and opened a new one.Sorry，I'm a green hand at this。 > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Assignee: shenshengli >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.5 > > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > See HDFS-14308，but after the fix, hbase did not return to normal. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Priority: Critical (was: Major) > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Critical > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > See HDFS-14308，but after the fix, hbase did not return to normal. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Priority: Major (was: Critical) > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > See HDFS-14308，but after the fix, hbase did not return to normal. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17068235#comment-17068235 ] shenshengli commented on HBASE-24051: - Another easy way to avoid errors is to add "implements CanUnbuffer" to the DFSStripedInputStream > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > See HDFS-14308，but after the fix, hbase did not return to normal. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24053) hbase/bin/region_status.rb Not available in hbase2
[ https://issues.apache.org/jira/browse/HBASE-24053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17068231#comment-17068231 ] shenshengli commented on HBASE-24053: - Before modification, when./hbase-jruby region_status.rb is executed, an error is reported: NameError:uninitialized constant IOException Did you mean? Exception ... After I follow the instructions to modify,an error is reported: NoMethodError:undefined method 'getHRegionInfo' for Java::OrApacheHadoopHbase::MetaTableAccessor:class Did you mean? get_region_info So I revised it again,an error is reported: NameError: undefined local variable or method 'scanner' for main:Object When I last modified it, it returned correct > hbase/bin/region_status.rb Not available in hbase2 > -- > > Key: HBASE-24053 > URL: https://issues.apache.org/jira/browse/HBASE-24053 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Minor > > A series of strange errors occurred while running this script in hbase2, > which was corrected as prompted, and note that in hbase2's MetaTableAccessor, > getHRegionInfo has been replaced by getRegionInfo -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24053) hbase/bin/region_status.rb Not available in hbase2
[ https://issues.apache.org/jira/browse/HBASE-24053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24053: Description: A series of strange errors occurred while running this script in hbase2, which was corrected as prompted, and note that in hbase2's MetaTableAccessor, getHRegionInfo has been replaced by getRegionInfo (was: A series of strange errors occurred while running this script in hbase2, which was corrected as prompted, and it was important to note that MetaTableAccessor had been replaced with getHRegionInfo instead of getHRegionInfo) > hbase/bin/region_status.rb Not available in hbase2 > -- > > Key: HBASE-24053 > URL: https://issues.apache.org/jira/browse/HBASE-24053 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Minor > > A series of strange errors occurred while running this script in hbase2, > which was corrected as prompted, and note that in hbase2's MetaTableAccessor, > getHRegionInfo has been replaced by getRegionInfo -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17068225#comment-17068225 ] shenshengli commented on HBASE-24051: - I don't know why hbase keeps those inputstreams after load, just unbuffer.I'll keep watching. > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > See HDFS-14308，but after the fix, hbase did not return to normal. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17068218#comment-17068218 ] shenshengli commented on HBASE-24051: - I don't think you can pin your hopes on someone else thinking about perfection,So the hbase side as a client needs to be as thoughtful as possible. > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > See HDFS-14308，but after the fix, hbase did not return to normal. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-24053) hbase/bin/region_status.rb Not available in hbase2
[ https://issues.apache.org/jira/browse/HBASE-24053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17068215#comment-17068215 ] shenshengli commented on HBASE-24053: - Re-pull request > hbase/bin/region_status.rb Not available in hbase2 > -- > > Key: HBASE-24053 > URL: https://issues.apache.org/jira/browse/HBASE-24053 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Minor > > A series of strange errors occurred while running this script in hbase2, > which was corrected as prompted, and it was important to note that > MetaTableAccessor had been replaced with getHRegionInfo instead of > getHRegionInfo -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24054) The Jetty's version number leak occurred while using the thrift service
[ https://issues.apache.org/jira/browse/HBASE-24054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24054: Issue Type: Improvement (was: Bug) > The Jetty's version number leak occurred while using the thrift service > --- > > Key: HBASE-24054 > URL: https://issues.apache.org/jira/browse/HBASE-24054 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Minor > > When the port is checked with curl -I host:port, the version number of jetty > is displayed. > To be safe, jetty's version number should be blocked. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-24053) hbase/bin/region_status.rb Not available in hbase2
[ https://issues.apache.org/jira/browse/HBASE-24053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli reopened HBASE-24053: - > hbase/bin/region_status.rb Not available in hbase2 > -- > > Key: HBASE-24053 > URL: https://issues.apache.org/jira/browse/HBASE-24053 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Minor > > A series of strange errors occurred while running this script in hbase2, > which was corrected as prompted, and it was important to note that > MetaTableAccessor had been replaced with getHRegionInfo instead of > getHRegionInfo -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24054) The Jetty's version number leak occurred while using the thrift service
[ https://issues.apache.org/jira/browse/HBASE-24054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24054: Description: When the port is checked with curl -I host:port, the version number of jetty is displayed. To be safe, jetty's version number should be blocked. was:A series of strange errors occurred while running this script in hbase2, which was corrected as prompted, and it was important to note that MetaTableAccessor had been replaced with getHRegionInfo instead of getHRegionInfo > The Jetty's version number leak occurred while using the thrift service > --- > > Key: HBASE-24054 > URL: https://issues.apache.org/jira/browse/HBASE-24054 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Minor > > When the port is checked with curl -I host:port, the version number of jetty > is displayed. > To be safe, jetty's version number should be blocked. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24054) The Jetty's version number leak occurred while using the thrift service
[ https://issues.apache.org/jira/browse/HBASE-24054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24054: Summary: The Jetty's version number leak occurred while using the thrift service (was: hbase/bin/region_status.rb Not available in hbase2) > The Jetty's version number leak occurred while using the thrift service > --- > > Key: HBASE-24054 > URL: https://issues.apache.org/jira/browse/HBASE-24054 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Minor > > A series of strange errors occurred while running this script in hbase2, > which was corrected as prompted, and it was important to note that > MetaTableAccessor had been replaced with getHRegionInfo instead of > getHRegionInfo -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24054) hbase/bin/region_status.rb Not available in hbase2
shenshengli created HBASE-24054: --- Summary: hbase/bin/region_status.rb Not available in hbase2 Key: HBASE-24054 URL: https://issues.apache.org/jira/browse/HBASE-24054 Project: HBase Issue Type: Bug Affects Versions: 2.0.1 Reporter: shenshengli A series of strange errors occurred while running this script in hbase2, which was corrected as prompted, and it was important to note that MetaTableAccessor had been replaced with getHRegionInfo instead of getHRegionInfo -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24053) hbase/bin/region_status.rb Not available in hbase2
shenshengli created HBASE-24053: --- Summary: hbase/bin/region_status.rb Not available in hbase2 Key: HBASE-24053 URL: https://issues.apache.org/jira/browse/HBASE-24053 Project: HBase Issue Type: Bug Affects Versions: 2.0.1 Reporter: shenshengli A series of strange errors occurred while running this script in hbase2, which was corrected as prompted, and it was important to note that MetaTableAccessor had been replaced with getHRegionInfo instead of getHRegionInfo -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Description: When the data directory is set to EC, during the continuous load process, regionserver is found down ,to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer See HDFS-14308，but after the fix, hbase did not return to normal. was: When the data directory is set to EC, during the continuous load process, regionserver is found down ,to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer See HDFS-14308，but > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > See HDFS-14308，but after the fix, hbase did not return to normal. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Description: When the data directory is set to EC, during the continuous load process, regionserver is found down ,to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer See HDFS-14308，but was: When the data directory is set to EC, during the continuous load process, regionserver is found down ,to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > See HDFS-14308，but -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: External issue ID: (was: HDFS-14308) > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: External issue ID: HDFS-14308 > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Description: When the data directory is set to EC, during the continuous load process, regionserver is found down ,to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer was: When the data directory is set to EC, during the continuous load process, regionserver is found down to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down ,to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Description: When the data directory is set to EC, during the continuous load process, regionserver is found down to view the log: regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below the heap occupancy alarm watermark (0.95) regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap occupancy alarm watermark (0.95) [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 1254ms GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms In particular, GC pauses occurred 283 times in half an hour. The memory analysis found that most of the memory footprint was generated by HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > > When the data directory is set to EC, during the continuous load process, > regionserver is found down to view the log: > regionserver.HeapMemoryManager: heapOccupancyPercent 0.91630864 is now below > the heap occupancy alarm watermark (0.95) > regionserver.HeapMemoryManager: heapOccupancyPercent 0.969546 is above heap > occupancy alarm watermark (0.95) > [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM or host machine > (eg GC): pause of approximately 1254ms > GC pool 'G1 Young Generation' had collection(s): count=2 time=115ms > GC pool 'G1 Old Generation' had collection(s): count=1 time=1487ms > In particular, GC pauses occurred 283 times in half an hour. > The memory analysis found that most of the memory footprint was generated by > HeepByteBuffer,and DFSStripedInputStream produces the HeepByteBuffer > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Attachment: 1.png > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > Attachments: 1.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-24051) When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in
[ https://issues.apache.org/jira/browse/HBASE-24051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shenshengli updated HBASE-24051: Summary: When the data directory is set to EC,the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in frequent gc and eventual failure (was: When the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in frequent gc and eventual failure) > When the data directory is set to EC,the amount of storefiles reaches a > relatively high level,heap Occupancy Percent is above heap occupancy alarm > watermark,resulting in frequent gc and eventual failure > -- > > Key: HBASE-24051 > URL: https://issues.apache.org/jira/browse/HBASE-24051 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.1 >Reporter: shenshengli >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-24051) When the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in frequent gc and eventual failur
shenshengli created HBASE-24051: --- Summary: When the amount of storefiles reaches a relatively high level,heap Occupancy Percent is above heap occupancy alarm watermark,resulting in frequent gc and eventual failure Key: HBASE-24051 URL: https://issues.apache.org/jira/browse/HBASE-24051 Project: HBase Issue Type: Bug Affects Versions: 2.0.1 Reporter: shenshengli -- This message was sent by Atlassian Jira (v8.3.4#803005)
