[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[
https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15778964#comment-15778964
]
Matt Warhaftig commented on HBASE-16010:
{quote}
"Matt Warhaftig, would you like to work on the ACL task?"
{quote}
Would ACL just entail requiring that the user doing the draining functionality
has admin (table scope) access.
> Put draining function through Admin API
> ---
>
> Key: HBASE-16010
> URL: https://issues.apache.org/jira/browse/HBASE-16010
> Project: HBase
> Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Matt Warhaftig
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16010-v3.patch, hbase-16010-v1.patch,
> hbase-16010-v2.patch
>
>
> Currently, there is no Amdin API for draining function. Client has to
> interact directly with Zookeeper draining node to add and remove draining
> servers.
> For example, in draining_servers.rb:
> {code}
> zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config,
> "draining_servers", nil)
> parentZnode = zkw.drainingZNode
> begin
> for server in servers
> node = ZKUtil.joinZNode(parentZnode, server)
> ZKUtil.createAndFailSilent(zkw, node)
> end
> ensure
> zkw.close()
> end
> {code}
> This is not good in cases like secure clusters with protected Zookeeper nodes.
> Let's put draining function through Admin API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[
https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15767600#comment-15767600
]
Matt Warhaftig commented on HBASE-16010:
Thanks Jerry, Feel free to fix the first issue and commit.
As for the second issue, MasterRpcServices uses the fully qualified classname
of {{org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos.ServerName}}
because MasterRpcServices already has a {{ServerName}} import entry
({{org.apache.hadoop.hbase.ServerName}}) and they would conflict.
> Put draining function through Admin API
> ---
>
> Key: HBASE-16010
> URL: https://issues.apache.org/jira/browse/HBASE-16010
> Project: HBase
> Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch
>
>
> Currently, there is no Amdin API for draining function. Client has to
> interact directly with Zookeeper draining node to add and remove draining
> servers.
> For example, in draining_servers.rb:
> {code}
> zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config,
> "draining_servers", nil)
> parentZnode = zkw.drainingZNode
> begin
> for server in servers
> node = ZKUtil.joinZNode(parentZnode, server)
> ZKUtil.createAndFailSilent(zkw, node)
> end
> ensure
> zkw.close()
> end
> {code}
> This is not good in cases like secure clusters with protected Zookeeper nodes.
> Let's put draining function through Admin API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[
https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15765646#comment-15765646
]
Matt Warhaftig commented on HBASE-16010:
[~jinghe],
Can you or someone else please review the new patch soon to avoid the need for
another rebase later on.
Thanks,
Matt
> Put draining function through Admin API
> ---
>
> Key: HBASE-16010
> URL: https://issues.apache.org/jira/browse/HBASE-16010
> Project: HBase
> Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch
>
>
> Currently, there is no Amdin API for draining function. Client has to
> interact directly with Zookeeper draining node to add and remove draining
> servers.
> For example, in draining_servers.rb:
> {code}
> zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config,
> "draining_servers", nil)
> parentZnode = zkw.drainingZNode
> begin
> for server in servers
> node = ZKUtil.joinZNode(parentZnode, server)
> ZKUtil.createAndFailSilent(zkw, node)
> end
> ensure
> zkw.close()
> end
> {code}
> This is not good in cases like secure clusters with protected Zookeeper nodes.
> Let's put draining function through Admin API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16010) Put draining function through Admin API
[
https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-16010:
---
Attachment: hbase-16010-v2.patch
Attached 'hbase-16010-v2.patch' is rebase to current master.
> Put draining function through Admin API
> ---
>
> Key: HBASE-16010
> URL: https://issues.apache.org/jira/browse/HBASE-16010
> Project: HBase
> Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-16010-v1.patch, hbase-16010-v2.patch
>
>
> Currently, there is no Amdin API for draining function. Client has to
> interact directly with Zookeeper draining node to add and remove draining
> servers.
> For example, in draining_servers.rb:
> {code}
> zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config,
> "draining_servers", nil)
> parentZnode = zkw.drainingZNode
> begin
> for server in servers
> node = ZKUtil.joinZNode(parentZnode, server)
> ZKUtil.createAndFailSilent(zkw, node)
> end
> ensure
> zkw.close()
> end
> {code}
> This is not good in cases like secure clusters with protected Zookeeper nodes.
> Let's put draining function through Admin API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13838) Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
[
https://issues.apache.org/jira/browse/HBASE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15338279#comment-15338279
]
Matt Warhaftig commented on HBASE-13838:
{quote}
"Any progress here?"
{quote}
[~stack], Can you please take a look at the submitted patch. The associated
Hadoop QA results are at [this
comment|https://issues.apache.org/jira/browse/HBASE-13838?focusedCommentId=14629232&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14629232].
> Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
>
>
> Key: HBASE-13838
> URL: https://issues.apache.org/jira/browse/HBASE-13838
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 1.1.0
>Reporter: Lars George
>Assignee: Matt Warhaftig
> Labels: beginner
> Fix For: 2.0.0, 1.3.1
>
> Attachments: hbase-13838-v1.patch, hbase-13838_post.tiff,
> hbase-13838_post_put_command.txt, hbase-13838_pre.tiff
>
>
> There are a few issues with the shared TaskStatusTmpl:
> - "Client operations" tab is always empty
> For Master this is expected, but for RegionServers there is never anything
> listed either. Fix for RS status page (probably caused by params not
> containing Operation subclass anymore, but some PB generated classes?)
> - Hide “Client Operations” tab for master UI
> Since operations are RS only. Or we fix this and make other calls show here.
> - The "alert-error" for aborted tasks is not set in CSS at all
> When a task was aborted it should be amber or red, but the assigned style is
> not in any of the linked stylesheets (abort-error). Add.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15112) Allow coprocessors to extend 'software attributes' list
[
https://issues.apache.org/jira/browse/HBASE-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15337059#comment-15337059
]
Matt Warhaftig commented on HBASE-15112:
Would anyone be available to code review this patch? - It is fairly large so
would prefer to get feedback before I forgot the intricacies.
> Allow coprocessors to extend 'software attributes' list
> ---
>
> Key: HBASE-15112
> URL: https://issues.apache.org/jira/browse/HBASE-15112
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
> Attachments: hbase-15112-v1.patch, hbase-15112-v1.patch,
> hbase-15112.tiff
>
>
> Over on the {{/master-status}} and {{/rs-status}} pages we have a list of
> release properties, giving details about the cluster deployment. We should
> make this an extension point, allowing coprocessors to register information
> about themselves as well. For example, Phoenix, Trafodion, Tephra, &c might
> want to advertise installed version and build details as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16010) Put draining function through Admin API
[
https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-16010:
---
Attachment: hbase-16010-v1.patch
Attached patch 'hbase-16010-v1.patch' adds region server draining functionality
to the Admin API. API method signatures are slightly changed from my original
posting.
{quote}
"and we probably need to have a fallback to zk calls if the client is new but
the server is old to keep compat, at least in branch-1"
{quote}
This is good info Matteo, let me search through the HBase codebase to try and
find a previous example of this behavior for the branch-1 port.
> Put draining function through Admin API
> ---
>
> Key: HBASE-16010
> URL: https://issues.apache.org/jira/browse/HBASE-16010
> Project: HBase
> Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-16010-v1.patch
>
>
> Currently, there is no Amdin API for draining function. Client has to
> interact directly with Zookeeper draining node to add and remove draining
> servers.
> For example, in draining_servers.rb:
> {code}
> zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config,
> "draining_servers", nil)
> parentZnode = zkw.drainingZNode
> begin
> for server in servers
> node = ZKUtil.joinZNode(parentZnode, server)
> ZKUtil.createAndFailSilent(zkw, node)
> end
> ensure
> zkw.close()
> end
> {code}
> This is not good in cases like secure clusters with protected Zookeeper nodes.
> Let's put draining function through Admin API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16010) Put draining function through Admin API
[
https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-16010:
---
Status: Patch Available (was: Open)
> Put draining function through Admin API
> ---
>
> Key: HBASE-16010
> URL: https://issues.apache.org/jira/browse/HBASE-16010
> Project: HBase
> Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Matt Warhaftig
>Priority: Minor
>
> Currently, there is no Amdin API for draining function. Client has to
> interact directly with Zookeeper draining node to add and remove draining
> servers.
> For example, in draining_servers.rb:
> {code}
> zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config,
> "draining_servers", nil)
> parentZnode = zkw.drainingZNode
> begin
> for server in servers
> node = ZKUtil.joinZNode(parentZnode, server)
> ZKUtil.createAndFailSilent(zkw, node)
> end
> ensure
> zkw.close()
> end
> {code}
> This is not good in cases like secure clusters with protected Zookeeper nodes.
> Let's put draining function through Admin API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[
https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328275#comment-15328275
]
Matt Warhaftig commented on HBASE-16010:
{quote}
"or was the method supposed to be a List
getDrainingRegionServers()? which is what the
ServerManager.getDrainingServersList()?"
{quote}
Yes, you are on top of things Matteo - the method signature was changed from
that preliminary draft to return a List of the draining region
servers.
As of 6/13 the RPC and API methods are written and closely follow the ruby
script's logic. Am hoping to wrap up the development in the next 48 hours.
> Put draining function through Admin API
> ---
>
> Key: HBASE-16010
> URL: https://issues.apache.org/jira/browse/HBASE-16010
> Project: HBase
> Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Matt Warhaftig
>Priority: Minor
>
> Currently, there is no Amdin API for draining function. Client has to
> interact directly with Zookeeper draining node to add and remove draining
> servers.
> For example, in draining_servers.rb:
> {code}
> zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config,
> "draining_servers", nil)
> parentZnode = zkw.drainingZNode
> begin
> for server in servers
> node = ZKUtil.joinZNode(parentZnode, server)
> ZKUtil.createAndFailSilent(zkw, node)
> end
> ensure
> zkw.close()
> end
> {code}
> This is not good in cases like secure clusters with protected Zookeeper nodes.
> Let's put draining function through Admin API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16010) Put draining function through Admin API
[
https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15326049#comment-15326049
]
Matt Warhaftig commented on HBASE-16010:
Proposed Admin drain methods and signatures:
Drain RegionServer:
- void drainRegionServer(final ServerName sn);
List Draining Regions:
- List getDrainingRegions(final ServerName sn);
Clear Drain From RegionServer:
- void removeDrainFromRegionServer(final ServerName sn);
- void removeDrainFromAllRegionServers();
> Put draining function through Admin API
> ---
>
> Key: HBASE-16010
> URL: https://issues.apache.org/jira/browse/HBASE-16010
> Project: HBase
> Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Matt Warhaftig
>Priority: Minor
>
> Currently, there is no Amdin API for draining function. Client has to
> interact directly with Zookeeper draining node to add and remove draining
> servers.
> For example, in draining_servers.rb:
> {code}
> zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config,
> "draining_servers", nil)
> parentZnode = zkw.drainingZNode
> begin
> for server in servers
> node = ZKUtil.joinZNode(parentZnode, server)
> ZKUtil.createAndFailSilent(zkw, node)
> end
> ensure
> zkw.close()
> end
> {code}
> This is not good in cases like secure clusters with protected Zookeeper nodes.
> Let's put draining function through Admin API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-16010) Put draining function through Admin API
[
https://issues.apache.org/jira/browse/HBASE-16010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig reassigned HBASE-16010:
--
Assignee: Matt Warhaftig
> Put draining function through Admin API
> ---
>
> Key: HBASE-16010
> URL: https://issues.apache.org/jira/browse/HBASE-16010
> Project: HBase
> Issue Type: Improvement
>Reporter: Jerry He
>Assignee: Matt Warhaftig
>Priority: Minor
>
> Currently, there is no Amdin API for draining function. Client has to
> interact directly with Zookeeper draining node to add and remove draining
> servers.
> For example, in draining_servers.rb:
> {code}
> zkw = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(config,
> "draining_servers", nil)
> parentZnode = zkw.drainingZNode
> begin
> for server in servers
> node = ZKUtil.joinZNode(parentZnode, server)
> ZKUtil.createAndFailSilent(zkw, node)
> end
> ensure
> zkw.close()
> end
> {code}
> This is not good in cases like secure clusters with protected Zookeeper nodes.
> Let's put draining function through Admin API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15112) Allow coprocessors to extend 'software attributes' list
[
https://issues.apache.org/jira/browse/HBASE-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15112:
---
Attachment: hbase-15112-v1.patch
Don't see why JAVA_HOME error got thrown by Hadoop QA so reattaching to try
rerun.
> Allow coprocessors to extend 'software attributes' list
> ---
>
> Key: HBASE-15112
> URL: https://issues.apache.org/jira/browse/HBASE-15112
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
> Attachments: hbase-15112-v1.patch, hbase-15112-v1.patch,
> hbase-15112.tiff
>
>
> Over on the {{/master-status}} and {{/rs-status}} pages we have a list of
> release properties, giving details about the cluster deployment. We should
> make this an extension point, allowing coprocessors to register information
> about themselves as well. For example, Phoenix, Trafodion, Tephra, &c might
> want to advertise installed version and build details as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15112) Allow coprocessors to extend 'software attributes' list
[
https://issues.apache.org/jira/browse/HBASE-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15112:
---
Attachment: hbase-15112.tiff
> Allow coprocessors to extend 'software attributes' list
> ---
>
> Key: HBASE-15112
> URL: https://issues.apache.org/jira/browse/HBASE-15112
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
> Attachments: hbase-15112-v1.patch, hbase-15112.tiff
>
>
> Over on the {{/master-status}} and {{/rs-status}} pages we have a list of
> release properties, giving details about the cluster deployment. We should
> make this an extension point, allowing coprocessors to register information
> about themselves as well. For example, Phoenix, Trafodion, Tephra, &c might
> want to advertise installed version and build details as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15112) Allow coprocessors to extend 'software attributes' list
[
https://issues.apache.org/jira/browse/HBASE-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15112:
---
Attachment: hbase-15112-v1.patch
Attached patch 'hbase-15112-v1.patch' allows coprocessors to add attribute info
via SPI by implementing org.apache.hadoop.hbase.CoprocessorAttributes. The
coprocessors' attributes are published to GUI (screenshot attached) and
available via API.
> Allow coprocessors to extend 'software attributes' list
> ---
>
> Key: HBASE-15112
> URL: https://issues.apache.org/jira/browse/HBASE-15112
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
> Attachments: hbase-15112-v1.patch
>
>
> Over on the {{/master-status}} and {{/rs-status}} pages we have a list of
> release properties, giving details about the cluster deployment. We should
> make this an extension point, allowing coprocessors to register information
> about themselves as well. For example, Phoenix, Trafodion, Tephra, &c might
> want to advertise installed version and build details as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15112) Allow coprocessors to extend 'software attributes' list
[
https://issues.apache.org/jira/browse/HBASE-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15112:
---
Status: Patch Available (was: Open)
> Allow coprocessors to extend 'software attributes' list
> ---
>
> Key: HBASE-15112
> URL: https://issues.apache.org/jira/browse/HBASE-15112
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
>
> Over on the {{/master-status}} and {{/rs-status}} pages we have a list of
> release properties, giving details about the cluster deployment. We should
> make this an extension point, allowing coprocessors to register information
> about themselves as well. For example, Phoenix, Trafodion, Tephra, &c might
> want to advertise installed version and build details as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-15112) Allow coprocessors to extend 'software attributes' list
[
https://issues.apache.org/jira/browse/HBASE-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig reassigned HBASE-15112:
--
Assignee: Matt Warhaftig
> Allow coprocessors to extend 'software attributes' list
> ---
>
> Key: HBASE-15112
> URL: https://issues.apache.org/jira/browse/HBASE-15112
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
>
> Over on the {{/master-status}} and {{/rs-status}} pages we have a list of
> release properties, giving details about the cluster deployment. We should
> make this an extension point, allowing coprocessors to register information
> about themselves as well. For example, Phoenix, Trafodion, Tephra, &c might
> want to advertise installed version and build details as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15870) Specify columns in REST multi gets
[
https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313428#comment-15313428
]
Matt Warhaftig commented on HBASE-15870:
[~yuzhih...@gmail.com] would you be available to do a code review on this
patch? Sorry to point at you for a request but I like your work. :-)
As far as:
{quote}
"You might want to consider changing the URL structure a bit to be of the form
'columns=...&row=...&row...' "
{quote}
My opinion was that including the 'columns' property on the query string could
create confusion and add an edge case of multiple 'columns' tags. Also, having
columns as a path item matches existing REST urls (ex. GET on a single cell
value). Still, let us see the code reviewer's opinion.
> Specify columns in REST multi gets
> --
>
> Key: HBASE-15870
> URL: https://issues.apache.org/jira/browse/HBASE-15870
> Project: HBase
> Issue Type: New Feature
> Components: REST
>Reporter: Dean Gurvitz
>Assignee: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-15870-v1.patch
>
>
> The REST multi-gets feature currently does not allow specifying only certain
> columns or column families. Adding support for these should be quite simple
> and improve the usability of the multi-gets feature.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15112) Allow coprocessors to extend 'software attributes' list
[
https://issues.apache.org/jira/browse/HBASE-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302237#comment-15302237
]
Matt Warhaftig commented on HBASE-15112:
Eh, the following interfaces already extend
{{org.apache.hadoop.hbase.Coprocessor}} and would also need to have child
interfaces to handle software attributes:
* org.apache.hadoop.hbase.coprocessor.BulkLoadObserver
* org.apache.hadoop.hbase.coprocessor.EndpointObserver
* org.apache.hadoop.hbase.coprocessor.MasterObserver
* org.apache.hadoop.hbase.coprocessor.RegionObserver
* org.apache.hadoop.hbase.coprocessor.RegionServerObserver
* org.apache.hadoop.hbase.coprocessor.WALObserver
That makes my suggested approach even more inelegant. I would prefer waiting
until Java 8 to handle this through default method. If anyone has other
suggestions for patterns to approach this feature or wants to vote for the
current approach please comment.
> Allow coprocessors to extend 'software attributes' list
> ---
>
> Key: HBASE-15112
> URL: https://issues.apache.org/jira/browse/HBASE-15112
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Nick Dimiduk
>
> Over on the {{/master-status}} and {{/rs-status}} pages we have a list of
> release properties, giving details about the cluster deployment. We should
> make this an extension point, allowing coprocessors to register information
> about themselves as well. For example, Phoenix, Trafodion, Tephra, &c might
> want to advertise installed version and build details as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15112) Allow coprocessors to extend 'software attributes' list
[
https://issues.apache.org/jira/browse/HBASE-15112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297452#comment-15297452
]
Matt Warhaftig commented on HBASE-15112:
Hi [~ndimiduk], What do you think about approaching this by adding a new
interface that extends the existing {{org.apache.hadoop.hbase.Coprocessor}}
interface and only adds:
{noformat}
public Map coprocessorAttributes();
{noformat}
The implemented method's returned map of specified attribute key and value
would be published to the master or rs 'Software Attributes' list.
> Allow coprocessors to extend 'software attributes' list
> ---
>
> Key: HBASE-15112
> URL: https://issues.apache.org/jira/browse/HBASE-15112
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Reporter: Nick Dimiduk
>
> Over on the {{/master-status}} and {{/rs-status}} pages we have a list of
> release properties, giving details about the cluster deployment. We should
> make this an extension point, allowing coprocessors to register information
> about themselves as well. For example, Phoenix, Trafodion, Tephra, &c might
> want to advertise installed version and build details as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15295572#comment-15295572 ] Matt Warhaftig commented on HBASE-15870: It looks like the FindBug warning were for preexisting issues. > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: New Feature > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-15870-v1.patch > > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15870) Specify columns in REST multi gets
[
https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15870:
---
Attachment: hbase-15870-v1.patch
Attached patch 'hbase-15870-v1.patch' adds column support to multiget REST
calls. Enhanced format is:
{noformat}
http://example.com:8000//multiget/:[,:...]?row=[&row=...]
{noformat}
Where zero or more column:qualifier tuples can be specified (comma delimited)
and column can be specified without qualifier. Examples:
{noformat}
http://example.com:8000/t1/multiget/cf1:c1,cf2?row=r1
{noformat}
It doesn't look like [HBase Reference Guide|http://hbase.apache.org/book.html]
documents the multiget REST functionality. Should it get added?
> Specify columns in REST multi gets
> --
>
> Key: HBASE-15870
> URL: https://issues.apache.org/jira/browse/HBASE-15870
> Project: HBase
> Issue Type: New Feature
> Components: REST
>Reporter: Dean Gurvitz
>Assignee: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-15870-v1.patch
>
>
> The REST multi-gets feature currently does not allow specifying only certain
> columns or column families. Adding support for these should be quite simple
> and improve the usability of the multi-gets feature.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig updated HBASE-15870: --- Status: Patch Available (was: Open) > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: New Feature > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-15870) Specify columns in REST multi gets
[ https://issues.apache.org/jira/browse/HBASE-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig reassigned HBASE-15870: -- Assignee: Matt Warhaftig > Specify columns in REST multi gets > -- > > Key: HBASE-15870 > URL: https://issues.apache.org/jira/browse/HBASE-15870 > Project: HBase > Issue Type: New Feature > Components: REST >Reporter: Dean Gurvitz >Assignee: Matt Warhaftig >Priority: Minor > > The REST multi-gets feature currently does not allow specifying only certain > columns or column families. Adding support for these should be quite simple > and improve the usability of the multi-gets feature. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15802) ConnectionUtils should use ThreadLocalRandom instead of Random
[
https://issues.apache.org/jira/browse/HBASE-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15802:
---
Attachment: hbase-15802.patch
Attached patch 'hbase-15802.patch' switches to ThreadLocalRandom and adds a
simple test case.
> ConnectionUtils should use ThreadLocalRandom instead of Random
> ---
>
> Key: HBASE-15802
> URL: https://issues.apache.org/jira/browse/HBASE-15802
> Project: HBase
> Issue Type: Improvement
>Reporter: Hiroshi Ikeda
>Assignee: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-15802.patch
>
>
> {code}
> public final class ConnectionUtils {
> ...skip...
> private static final Random RANDOM = new Random();
> {code}
> In general, static fields are accessed by multi-threads. The class Random is
> thread-safe but ThreadLocalRandom is more preferable because of less
> contention.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-15802) ConnectionUtils should use ThreadLocalRandom instead of Random
[
https://issues.apache.org/jira/browse/HBASE-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig resolved HBASE-15802.
Resolution: Fixed
Assignee: Matt Warhaftig
> ConnectionUtils should use ThreadLocalRandom instead of Random
> ---
>
> Key: HBASE-15802
> URL: https://issues.apache.org/jira/browse/HBASE-15802
> Project: HBase
> Issue Type: Improvement
>Reporter: Hiroshi Ikeda
>Assignee: Matt Warhaftig
>Priority: Minor
>
> {code}
> public final class ConnectionUtils {
> ...skip...
> private static final Random RANDOM = new Random();
> {code}
> In general, static fields are accessed by multi-threads. The class Random is
> thread-safe but ThreadLocalRandom is more preferable because of less
> contention.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15769) Perform validation on cluster key for add_peer
[
https://issues.apache.org/jira/browse/HBASE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15286400#comment-15286400
]
Matt Warhaftig commented on HBASE-15769:
[~tedyu], thanks for the test fix.
> Perform validation on cluster key for add_peer
> --
>
> Key: HBASE-15769
> URL: https://issues.apache.org/jira/browse/HBASE-15769
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-15769-v1.patch, hbase-15769-v2.patch
>
>
> [~pateljaymin] reported the following.
> In hbase cluster, adding a peer for replication, incorrectly defined
> zookeeper znode does not fail new peer creation. However, replication breaks
> down.
> For example,
> below string incorrectly defines zookeeper znode as "hbase" instead of
> "/hbase"...
> {code}
> > add_peer '1', "host1.com,host2.com,host3.com:2181:hbase
> {code}
> This left replication broken, even after we dropped and added a new correct
> entry for the peer.
> It wasn't until we bounced the cluster that the issue was fixed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15769) Perform validation on cluster key for add_peer
[
https://issues.apache.org/jira/browse/HBASE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15769:
---
Attachment: hbase-15769-v1.patch
Attached patch 'hbase-15769-v1.patch' adds cluster key validation to the
cluster key parameter of the add_peer shell call. Also, the add_peer shell
command help message has been updated to have accurate samples - HBASE-12867 to
support custom replication endpoints removed the ability to use strings as
add_peer parameters.
> Perform validation on cluster key for add_peer
> --
>
> Key: HBASE-15769
> URL: https://issues.apache.org/jira/browse/HBASE-15769
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-15769-v1.patch
>
>
> [~pateljaymin] reported the following.
> In hbase cluster, adding a peer for replication, incorrectly defined
> zookeeper znode does not fail new peer creation. However, replication breaks
> down.
> For example,
> below string incorrectly defines zookeeper znode as "hbase" instead of
> "/hbase"...
> {code}
> > add_peer '1', "host1.com,host2.com,host3.com:2181:hbase
> {code}
> This left replication broken, even after we dropped and added a new correct
> entry for the peer.
> It wasn't until we bounced the cluster that the issue was fixed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15769) Perform validation on cluster key for add_peer
[
https://issues.apache.org/jira/browse/HBASE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15769:
---
Status: Patch Available (was: Open)
> Perform validation on cluster key for add_peer
> --
>
> Key: HBASE-15769
> URL: https://issues.apache.org/jira/browse/HBASE-15769
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Matt Warhaftig
>Priority: Minor
>
> [~pateljaymin] reported the following.
> In hbase cluster, adding a peer for replication, incorrectly defined
> zookeeper znode does not fail new peer creation. However, replication breaks
> down.
> For example,
> below string incorrectly defines zookeeper znode as "hbase" instead of
> "/hbase"...
> {code}
> > add_peer '1', "host1.com,host2.com,host3.com:2181:hbase
> {code}
> This left replication broken, even after we dropped and added a new correct
> entry for the peer.
> It wasn't until we bounced the cluster that the issue was fixed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-15769) Perform validation on cluster key for add_peer
[
https://issues.apache.org/jira/browse/HBASE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig reassigned HBASE-15769:
--
Assignee: Matt Warhaftig
> Perform validation on cluster key for add_peer
> --
>
> Key: HBASE-15769
> URL: https://issues.apache.org/jira/browse/HBASE-15769
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Matt Warhaftig
>Priority: Minor
>
> [~pateljaymin] reported the following.
> In hbase cluster, adding a peer for replication, incorrectly defined
> zookeeper znode does not fail new peer creation. However, replication breaks
> down.
> For example,
> below string incorrectly defines zookeeper znode as "hbase" instead of
> "/hbase"...
> {code}
> > add_peer '1', "host1.com,host2.com,host3.com:2181:hbase
> {code}
> This left replication broken, even after we dropped and added a new correct
> entry for the peer.
> It wasn't until we bounced the cluster that the issue was fixed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15799) Two Shell 'close_region' Example Syntaxes Don't Work
[
https://issues.apache.org/jira/browse/HBASE-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15799:
---
Attachment: hbase-15799-v1.patch
Attached patch 'hbase-15799-v1.patch' uses presence of period at end of
close_region command's region parameter to decide if region value is encoded
and choose proper HBaseAdmin close region call.
HBaseAdmin.closeRegionWithEncodedRegionName() requires servername so
close_region command's help message has been updated to not list incorrect
syntax of {{close_region 'ENCODED_REGIONNAME'}}.
> Two Shell 'close_region' Example Syntaxes Don't Work
>
>
> Key: HBASE-15799
> URL: https://issues.apache.org/jira/browse/HBASE-15799
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0
>Reporter: Matt Warhaftig
>Assignee: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-15799-v1.patch
>
>
> The close_region shell command's help message lists the following usage
> syntaxes:
> {noformat}
> hbase> close_region 'REGIONNAME'
> hbase> close_region 'REGIONNAME', 'SERVER_NAME'
> hbase> close_region 'ENCODED_REGIONNAME'
> hbase> close_region 'ENCODED_REGIONNAME', 'SERVER_NAME'
> {noformat}
> admin.rb's current code (with close_region method being the entry point) is:
> {code}
> def close_region(region_name, server)
> if (server == nil || !closeEncodedRegion?(region_name, server))
> @admin.closeRegion(region_name, server)
> end
> end
> def closeEncodedRegion?(region_name, server)
>@admin.closeRegionWithEncodedRegionName(region_name, server)
> end
> {code}
> The {{close_region 'ENCODED_REGIONNAME'}} syntax currently will not work
> because when server = nil the {{closeEncodedRegion}} method call is skipped.
> The {{close_region 'REGIONNAME', 'SERVER_NAME'}} syntax currently will not
> work because {{@admin.closeRegionWithEncodedRegionName}} throws an
> NotServingRegionException (for the non-encoded region_name) that is uncaught
> in and prevents execution from returning to {{close_region}} and the correct
> call of {{HBaseAdmin.closeRegion}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15799) Two Shell 'close_region' Example Syntaxes Don't Work
[
https://issues.apache.org/jira/browse/HBASE-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15799:
---
Status: Patch Available (was: Open)
> Two Shell 'close_region' Example Syntaxes Don't Work
>
>
> Key: HBASE-15799
> URL: https://issues.apache.org/jira/browse/HBASE-15799
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0
>Reporter: Matt Warhaftig
>Assignee: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-15799-v1.patch
>
>
> The close_region shell command's help message lists the following usage
> syntaxes:
> {noformat}
> hbase> close_region 'REGIONNAME'
> hbase> close_region 'REGIONNAME', 'SERVER_NAME'
> hbase> close_region 'ENCODED_REGIONNAME'
> hbase> close_region 'ENCODED_REGIONNAME', 'SERVER_NAME'
> {noformat}
> admin.rb's current code (with close_region method being the entry point) is:
> {code}
> def close_region(region_name, server)
> if (server == nil || !closeEncodedRegion?(region_name, server))
> @admin.closeRegion(region_name, server)
> end
> end
> def closeEncodedRegion?(region_name, server)
>@admin.closeRegionWithEncodedRegionName(region_name, server)
> end
> {code}
> The {{close_region 'ENCODED_REGIONNAME'}} syntax currently will not work
> because when server = nil the {{closeEncodedRegion}} method call is skipped.
> The {{close_region 'REGIONNAME', 'SERVER_NAME'}} syntax currently will not
> work because {{@admin.closeRegionWithEncodedRegionName}} throws an
> NotServingRegionException (for the non-encoded region_name) that is uncaught
> in and prevents execution from returning to {{close_region}} and the correct
> call of {{HBaseAdmin.closeRegion}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-15799) Two Shell 'close_region' Example Syntaxes Don't Work
Matt Warhaftig created HBASE-15799:
--
Summary: Two Shell 'close_region' Example Syntaxes Don't Work
Key: HBASE-15799
URL: https://issues.apache.org/jira/browse/HBASE-15799
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.0.0
Reporter: Matt Warhaftig
Assignee: Matt Warhaftig
Priority: Minor
The close_region shell command's help message lists the following usage
syntaxes:
{noformat}
hbase> close_region 'REGIONNAME'
hbase> close_region 'REGIONNAME', 'SERVER_NAME'
hbase> close_region 'ENCODED_REGIONNAME'
hbase> close_region 'ENCODED_REGIONNAME', 'SERVER_NAME'
{noformat}
admin.rb's current code (with close_region method being the entry point) is:
{code}
def close_region(region_name, server)
if (server == nil || !closeEncodedRegion?(region_name, server))
@admin.closeRegion(region_name, server)
end
end
def closeEncodedRegion?(region_name, server)
@admin.closeRegionWithEncodedRegionName(region_name, server)
end
{code}
The {{close_region 'ENCODED_REGIONNAME'}} syntax currently will not work
because when server = nil the {{closeEncodedRegion}} method call is skipped.
The {{close_region 'REGIONNAME', 'SERVER_NAME'}} syntax currently will not work
because {{@admin.closeRegionWithEncodedRegionName}} throws an
NotServingRegionException (for the non-encoded region_name) that is uncaught in
and prevents execution from returning to {{close_region}} and the correct call
of {{HBaseAdmin.closeRegion}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15746) RegionCoprocessor preClose() called 3 times
[ https://issues.apache.org/jira/browse/HBASE-15746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275600#comment-15275600 ] Matt Warhaftig commented on HBASE-15746: Hi [~mbertozzi], are you aware of HBASE-8075? - It looks to be an investigation of the same behavior. > RegionCoprocessor preClose() called 3 times > --- > > Key: HBASE-15746 > URL: https://issues.apache.org/jira/browse/HBASE-15746 > Project: HBase > Issue Type: Bug > Components: Coprocessors, regionserver >Affects Versions: 2.0.0, 1.3.0, 1.2.1, 1.1.4, 0.98.19 >Reporter: Matteo Bertozzi >Priority: Minor > > The preClose() region coprocessor call gets called 3 times via rpc. > The first one is when we receive the RPC > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java#L1329 > The second time is when ask the RS to close the region > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java#L2852 > The third time is when the doClose() on the region is executed. > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L1419 > I'm pretty sure the first one can be removed since, there is no code between > that and the second call. and they are a copy-paste. > The second one explicitly says that is to enforce ACLs before starting the > operation, which leads me to the fact that the 3rd one in the region gets > executed too late in the process. but the region.close() may be called by > someone other than the RS, so we should probably leave the preClose() in > there (e.g. OpenRegionHandler on failure cleanup). > any idea? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15563) 'counter' may overflow in BoundedGroupingStrategy
[
https://issues.apache.org/jira/browse/HBASE-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15563:
---
Attachment: hbase-15563-v2.patch
Expanded and reran the benchmark test. Counter resetting with _if_ beat out
reseting via mod and was more inline with AtomicLong this run:
|| Threads || AtomicLong Run Time || Reset With If Run Time || Reset With Mod
Run Time ||
| 1 | 66s | 52s | 76s |
| 2 | 194s | 203s | 218s |
| 4 | 299s | 302s | 315s |
Attached patch 'hbase-15563-v2.patch' uses if statements as this seems the best
route.
> 'counter' may overflow in BoundedGroupingStrategy
> -
>
> Key: HBASE-15563
> URL: https://issues.apache.org/jira/browse/HBASE-15563
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Matt Warhaftig
>Priority: Minor
> Labels: beginner
> Attachments: hbase-15563-v1.patch, hbase-15563-v2.patch
>
>
> {code}
> groupName = groupNames[counter.getAndIncrement() % groupNames.length];
> {code}
> Theoretically, counter can overflow and becomes negative then causes an
> ArrayIndexOutOfBoundsException.
> But in practice, we need 2 billions different identifiers to make this
> happen, and before the overflow we will run into OOM because of a huge
> groupNameCache...
> So not sure if it is worth to fix
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15563) 'counter' may overflow in BoundedGroupingStrategy
[
https://issues.apache.org/jira/browse/HBASE-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15268667#comment-15268667
]
Matt Warhaftig commented on HBASE-15563:
Just FYI counter reset logic was:
{code}
public static final int getAndIncrement() {
for (;;) {
int current = counterInt.get();
int next = (current + 1);
if (next == groupNames.length) {
next = 0;
}
if (counterInt.compareAndSet(current, next)) return current;
}
}
{code}
> 'counter' may overflow in BoundedGroupingStrategy
> -
>
> Key: HBASE-15563
> URL: https://issues.apache.org/jira/browse/HBASE-15563
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Matt Warhaftig
>Priority: Minor
> Labels: beginner
> Attachments: hbase-15563-v1.patch
>
>
> {code}
> groupName = groupNames[counter.getAndIncrement() % groupNames.length];
> {code}
> Theoretically, counter can overflow and becomes negative then causes an
> ArrayIndexOutOfBoundsException.
> But in practice, we need 2 billions different identifiers to make this
> happen, and before the overflow we will run into OOM because of a huge
> groupNameCache...
> So not sure if it is worth to fix
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15563) 'counter' may overflow in BoundedGroupingStrategy
[
https://issues.apache.org/jira/browse/HBASE-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15268622#comment-15268622
]
Matt Warhaftig commented on HBASE-15563:
Below are results for time to get groupNames array element for
Integer.Max_Value*2 records across multiple thread counts. Counter reset code
was closely adapted from AtomicInteger.getAndIncrement().
|| Threads || AtomicLong Run Time || Counter Reset Run Time || % Difference ||
| 1 | 65s | 66s | + 1.5% |
| 2 | 186s | 219s | +16% |
| 4 | 287s | 317s | +10% |
The Counter Reset method is a little slower but still I will put it into a
quick patch tonight.
> 'counter' may overflow in BoundedGroupingStrategy
> -
>
> Key: HBASE-15563
> URL: https://issues.apache.org/jira/browse/HBASE-15563
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Matt Warhaftig
>Priority: Minor
> Labels: beginner
> Attachments: hbase-15563-v1.patch
>
>
> {code}
> groupName = groupNames[counter.getAndIncrement() % groupNames.length];
> {code}
> Theoretically, counter can overflow and becomes negative then causes an
> ArrayIndexOutOfBoundsException.
> But in practice, we need 2 billions different identifiers to make this
> happen, and before the overflow we will run into OOM because of a huge
> groupNameCache...
> So not sure if it is worth to fix
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15563) 'counter' may overflow in BoundedGroupingStrategy
[
https://issues.apache.org/jira/browse/HBASE-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15563:
---
Attachment: hbase-15563-v1.patch
Attached patch 'hbase-15563-v1.patch' switches counter to AtomicLong. Existing
WAL test cases continue to pass but no new dedicated boundary tests are added.
{quote}
"before the overflow we will run into OOM because of a huge groupNameCache."
{quote}
Agreed, with 6GB of memory dedicated to just a ConcurrentHashMap and trying to
simulate the groupNameCache region entries I topped out at 28 million cache
entries. Counter's {{Integer.MAX_VALUE}} limit was a few orders of magnitude
larger at 2 billion. Still, since ConcurrentHashMap size is unbound, maybe
with server-sized memory allocations the limiting factor could shift to counter.
> 'counter' may overflow in BoundedGroupingStrategy
> -
>
> Key: HBASE-15563
> URL: https://issues.apache.org/jira/browse/HBASE-15563
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Matt Warhaftig
>Priority: Minor
> Labels: beginner
> Attachments: hbase-15563-v1.patch
>
>
> {code}
> groupName = groupNames[counter.getAndIncrement() % groupNames.length];
> {code}
> Theoretically, counter can overflow and becomes negative then causes an
> ArrayIndexOutOfBoundsException.
> But in practice, we need 2 billions different identifiers to make this
> happen, and before the overflow we will run into OOM because of a huge
> groupNameCache...
> So not sure if it is worth to fix
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15563) 'counter' may overflow in BoundedGroupingStrategy
[
https://issues.apache.org/jira/browse/HBASE-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15563:
---
Assignee: Matt Warhaftig
Status: Patch Available (was: Open)
> 'counter' may overflow in BoundedGroupingStrategy
> -
>
> Key: HBASE-15563
> URL: https://issues.apache.org/jira/browse/HBASE-15563
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Duo Zhang
>Assignee: Matt Warhaftig
>Priority: Minor
> Labels: beginner
>
> {code}
> groupName = groupNames[counter.getAndIncrement() % groupNames.length];
> {code}
> Theoretically, counter can overflow and becomes negative then causes an
> ArrayIndexOutOfBoundsException.
> But in practice, we need 2 billions different identifiers to make this
> happen, and before the overflow we will run into OOM because of a huge
> groupNameCache...
> So not sure if it is worth to fix
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15676:
---
Attachment: hbase-15287-0.98-v1.patch
Attached 'hbase-15287-0.98-v1.patch' for port to 0.98. Compilation on this
branch required two object type changes in
TestFuzzyRowFilterEndToEnd.testAllFixedBits().
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 0.98.13, 1.0.2, 1.2.0, 1.1.1
>Reporter: Rohit Sinha
>Assignee: Matt Warhaftig
> Fix For: 2.0.0, 1.3.0, 1.4.0, 1.1.5, 1.2.2
>
> Attachments: hbase-15287-0.98-v1.patch, hbase-15676-v1.patch,
> hbase-15676-v2.patch, hbase-15676-v3.patch, hbase-15676-v4.patch
>
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15676:
---
Attachment: hbase-15676-v4.patch
{quote}
"Please fix checkstyle and findbugs"
{quote}
Had overlooked the checkstyle error, fixed now, 'hbase-15676-v4.patch'.
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 0.98.13, 1.0.2, 1.2.0, 1.1.1
>Reporter: Rohit Sinha
> Attachments: hbase-15676-v1.patch, hbase-15676-v2.patch,
> hbase-15676-v3.patch, hbase-15676-v4.patch
>
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15676:
---
Attachment: hbase-15676-v3.patch
Attached patch 'hbase-15676-v3.patch' corrects the smell FindBugs found and
updates the code comment.
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 0.98.13, 1.0.2, 1.2.0, 1.1.1
>Reporter: Rohit Sinha
> Attachments: hbase-15676-v1.patch, hbase-15676-v2.patch,
> hbase-15676-v3.patch
>
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15260014#comment-15260014
]
Matt Warhaftig commented on HBASE-15676:
Actually, [~chenheng] you can probably do the code review too. - Please let me
know what you think.
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 0.98.13, 1.0.2, 1.2.0, 1.1.1
>Reporter: Rohit Sinha
> Attachments: hbase-15676-v1.patch, hbase-15676-v2.patch
>
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15260013#comment-15260013
]
Matt Warhaftig commented on HBASE-15676:
Hi [~chenheng], the first code snippet you list shifts the new mask encoding
values of -1 & 2 back to -1 & 0 respectively which FuzzyRowFilter uses for its
bitwise scan operations.
As for your second code snippet, yes, good catch that comment should be
updated. I'll update the line once I get feedback on the patch from [~tedyu].
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 0.98.13, 1.0.2, 1.2.0, 1.1.1
>Reporter: Rohit Sinha
> Attachments: hbase-15676-v1.patch, hbase-15676-v2.patch
>
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15676:
---
Attachment: hbase-15676-v2.patch
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 0.98.13, 1.0.2, 1.2.0, 1.1.1
>Reporter: Rohit Sinha
> Attachments: hbase-15676-v1.patch, hbase-15676-v2.patch
>
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259418#comment-15259418
]
Matt Warhaftig commented on HBASE-15676:
{noformat}
"Have you thought about changing the encoding ( 0 -> -1 (0xff), 1 -> 0) so that
we can tell whether the mask has been preprocessed without introducing marker ?"
{noformat}
[~tedyu], thanks for the suggestion, that approach seems to work and is much
more elegant. Attached patch 'hbase-15676-v2.patch' uses new mask encoding of
0 -> -1 & 1 -> 2 in the FuzzyRowFilter constructor and then switches the mask
to needed values of -1 & 0 at filterKeyValue().
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 0.98.13, 1.0.2, 1.2.0, 1.1.1
>Reporter: Rohit Sinha
> Attachments: hbase-15676-v1.patch
>
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15676:
---
Attachment: hbase-15676-v1.patch
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 0.98.13, 1.0.2, 1.2.0, 1.1.1
>Reporter: Rohit Sinha
> Attachments: hbase-15676-v1.patch
>
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15676:
---
Status: Patch Available (was: Open)
The FuzzyRowFilter constructor as a side effect modifies passed mask values
(0->-1 & 1->0). When a FuzzyRowFilter reuses a previously used mask, the
FuzzyRowFilter constructor gets a mask that is already updated to -1s & 0s.
FuzzyRowFilter logic has isPreprocessedMask() to check mask state and only
update new mask. However, as [~talktorohi...@gmail.com] found, that check
fails for a mask of all 0s since that value is ambiguous (both new and
previously updated masks could contain all 0s).
Attached patch 'hbase-15676-v1.patch' adds an isPreprocessed flag byte to the
mask to track if previously updated. Adding a flag byte is a bit inelegant but
FuzzyRowFilter expects masks to be reusable while retaining state and being
modified by FuzzyRowFilter's constructor.
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 1.1.1, 1.2.0, 1.0.2, 0.98.13, 2.0.0
>Reporter: Rohit Sinha
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15676) FuzzyRowFilter fails and matches all the rows in the table if the mask consists of all 0s
[
https://issues.apache.org/jira/browse/HBASE-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15255246#comment-15255246
]
Matt Warhaftig commented on HBASE-15676:
[~chenheng]], you ok if I take this ticket? If so can you assign it to me -
the JIRA lockdown is preventing me from assigning.
[~talktorohi...@gmail.com] thanks for the detailed info and investigation -
your findings look right to me.
> FuzzyRowFilter fails and matches all the rows in the table if the mask
> consists of all 0s
> -
>
> Key: HBASE-15676
> URL: https://issues.apache.org/jira/browse/HBASE-15676
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 2.0.0, 0.98.13, 1.0.2, 1.2.0, 1.1.1
>Reporter: Rohit Sinha
>
> While using FuzzyRowFilter we noticed that if the mask array consists of all
> 0s (fixed) the FuzzyRowFilter matches all the rows in the table. We noticed
> this on HBase 1.1, 1.2 and higher.
> After some digging we suspect that this is because of isPreprocessedMask()
> check which is used in preprocessMask() which was added here:
> https://issues.apache.org/jira/browse/HBASE-13761
> If the mask consists of all 0s then the isPreprocessedMask() returns true and
> the preprocessing which responsible for changing 0s to -1 doesn't happen and
> hence all rows are matched in scan.
> This scenario can be tested in TestFuzzyRowFilterEndToEnd#testHBASE14782() If
> we change the
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x00\\x00\\x044");
> byte[] mask = new byte[] {1,0,0,0};
> to
> byte[] fuzzyKey = Bytes.toBytesBinary("\\x9B\\x00\\x044e");
> byte[] mask = new byte[] {0,0,0,0,0};
> We expect one match but this will match all the rows in the table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14363) Print more details on the row behind an Empty REGIONINFO_QUALIFIER warning
[ https://issues.apache.org/jira/browse/HBASE-14363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15254915#comment-15254915 ] Matt Warhaftig commented on HBASE-14363: FYI, should be safe for an admin to close this ticket. > Print more details on the row behind an Empty REGIONINFO_QUALIFIER warning > -- > > Key: HBASE-14363 > URL: https://issues.apache.org/jira/browse/HBASE-14363 > Project: HBase > Issue Type: Improvement > Components: hbck, Operability >Affects Versions: 1.0.0 >Reporter: Harsh J >Assignee: Matt Warhaftig > > Currently HBCK just prints a vague "Empty REGIONINFO_QUALIFIER found" > warning, and does not print the row it found that on. While fixing this is > easy thanks to HBCK, some more detail (say the row/region ID) would be good > to print, to avoid people manually scanning meta to obtain the very same info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15641) Shell "alter" should do a single modifyTable operation
[
https://issues.apache.org/jira/browse/HBASE-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15641:
---
Attachment: hbase-15641-branch-1-v1.patch
Attached patch 'hbase-15641-branch-1-v1.patch' is the port to branch-1.
> Shell "alter" should do a single modifyTable operation
> --
>
> Key: HBASE-15641
> URL: https://issues.apache.org/jira/browse/HBASE-15641
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Gary Helmling
>Assignee: Matt Warhaftig
> Fix For: 2.0.0
>
> Attachments: hbase-15641-branch-1-v1.patch, hbase-15641-v1.patch
>
>
> When performing an "alter" on multiple column families in a table, then shell
> will perform a separate {{Admin.modifyColumn()}} call for each column family
> being modified, with all of the table regions being bulk-reopened each time.
> It would be much better to simply apply all the changes to the table
> descriptor, then do a single call to {{Admin.modifyTable()}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15641) Shell "alter" should do a single modifyTable operation
[
https://issues.apache.org/jira/browse/HBASE-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15251784#comment-15251784
]
Matt Warhaftig commented on HBASE-15641:
Just tested, looked good on a fully distributed cluster - alters affected all
nodes as expected.
Also, to be clear, by batching the table updates the alter command has become
atomic. Prior to this patch the below command would still apply the 'cf'
version change even if the table attribute unset failed:
{noformat}
alter 'tableCluster', { NAME => 'cf', VERSIONS => 5 }, { METHOD =>
'table_att_unset', NAME => 'REGION_REPLICATION'}
{noformat}
With this patch the command fails wholesale, which IMHO makes more sense.
> Shell "alter" should do a single modifyTable operation
> --
>
> Key: HBASE-15641
> URL: https://issues.apache.org/jira/browse/HBASE-15641
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Gary Helmling
>Assignee: Matt Warhaftig
> Attachments: hbase-15641-v1.patch
>
>
> When performing an "alter" on multiple column families in a table, then shell
> will perform a separate {{Admin.modifyColumn()}} call for each column family
> being modified, with all of the table regions being bulk-reopened each time.
> It would be much better to simply apply all the changes to the table
> descriptor, then do a single call to {{Admin.modifyTable()}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15641) Shell "alter" should do a single modifyTable operation
[
https://issues.apache.org/jira/browse/HBASE-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-15641:
---
Attachment: hbase-15641-v1.patch
Attached 'hbase-15641-v1.patch' updates the shell alter to batch update all
column and table modifying commands.
> Shell "alter" should do a single modifyTable operation
> --
>
> Key: HBASE-15641
> URL: https://issues.apache.org/jira/browse/HBASE-15641
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Gary Helmling
>Assignee: Matt Warhaftig
> Attachments: hbase-15641-v1.patch
>
>
> When performing an "alter" on multiple column families in a table, then shell
> will perform a separate {{Admin.modifyColumn()}} call for each column family
> being modified, with all of the table regions being bulk-reopened each time.
> It would be much better to simply apply all the changes to the table
> descriptor, then do a single call to {{Admin.modifyTable()}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-15641) Shell "alter" should do a single modifyTable operation
[
https://issues.apache.org/jira/browse/HBASE-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig reassigned HBASE-15641:
--
Assignee: Matt Warhaftig
> Shell "alter" should do a single modifyTable operation
> --
>
> Key: HBASE-15641
> URL: https://issues.apache.org/jira/browse/HBASE-15641
> Project: HBase
> Issue Type: Improvement
> Components: shell
>Reporter: Gary Helmling
>Assignee: Matt Warhaftig
>
> When performing an "alter" on multiple column families in a table, then shell
> will perform a separate {{Admin.modifyColumn()}} call for each column family
> being modified, with all of the table regions being bulk-reopened each time.
> It would be much better to simply apply all the changes to the table
> descriptor, then do a single call to {{Admin.modifyTable()}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15287) mapreduce.RowCounter returns incorrect result with binary row key inputs
[ https://issues.apache.org/jira/browse/HBASE-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig updated HBASE-15287: --- Attachment: hbase-15287-branch-1-v1.patch Attached 'hbase-15287-branch-1-v1.patch' ports this patch to branch-1. Note that TestRowCounter was not ported because the --range argument does not work on this branch (see [HBASE-14770|https://issues.apache.org/jira/browse/HBASE-14770?focusedCommentId=15135623&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15135623] for details). > mapreduce.RowCounter returns incorrect result with binary row key inputs > > > Key: HBASE-15287 > URL: https://issues.apache.org/jira/browse/HBASE-15287 > Project: HBase > Issue Type: Bug > Components: mapreduce, util >Affects Versions: 1.1.1 >Reporter: Randy Hu >Assignee: Matt Warhaftig > Fix For: 2.0.0, 1.4.0 > > Attachments: 15287-v2.patch, hbase-15287-branch-1-v1.patch, > hbase-15287-v1.patch, hbase-15287-v2.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > org.apache.hadoop.hbase.mapreduce.RowCounter takes optional start/end key as > inputs (-range option). It would work only when the string representation of > value is identical to the string. When row key is binary, the string > representation of the value would look like this: "\x00\x01", which would be > incorrect interpreted as 8 char string in the current implementation: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/RowCounter.java > To fix that, we need change how the value is converted from command line > inputs: > Change > scan.setStartRow(Bytes.toBytes(startKey)); > to > scan.setStartRow(Bytes.toBytesBinary(startKey)); > Do the same conversion to end key as well. > The issue was discovered when the utility was used to calcualte row > distribution on regions from table with binary row keys. The hbase:meta > contains the start key of each region in format of above example. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15287) org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with binary row key inputs
[ https://issues.apache.org/jira/browse/HBASE-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig updated HBASE-15287: --- Attachment: hbase-15287-v2.patch > org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with > binary row key inputs > > > Key: HBASE-15287 > URL: https://issues.apache.org/jira/browse/HBASE-15287 > Project: HBase > Issue Type: Bug > Components: mapreduce, util >Affects Versions: 1.1.1 >Reporter: Randy Hu >Assignee: Matt Warhaftig > Attachments: hbase-15287-v1.patch, hbase-15287-v2.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > org.apache.hadoop.hbase.mapreduce.RowCounter takes optional start/end key as > inputs (-range option). It would work only when the string representation of > value is identical to the string. When row key is binary, the string > representation of the value would look like this: "\x00\x01", which would be > incorrect interpreted as 8 char string in the current implementation: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/RowCounter.java > To fix that, we need change how the value is converted from command line > inputs: > Change > scan.setStartRow(Bytes.toBytes(startKey)); > to > scan.setStartRow(Bytes.toBytesBinary(startKey)); > Do the same conversion to end key as well. > The issue was discovered when the utility was used to calcualte row > distribution on regions from table with binary row keys. The hbase:meta > contains the start key of each region in format of above example. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15287) org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with binary row key inputs
[ https://issues.apache.org/jira/browse/HBASE-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15244179#comment-15244179 ] Matt Warhaftig commented on HBASE-15287: Ok, attached 'hbase-15287-v2.patch' adds binary row test cases for TestImportExport, TestCopyTable, and TestCellCounter. > org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with > binary row key inputs > > > Key: HBASE-15287 > URL: https://issues.apache.org/jira/browse/HBASE-15287 > Project: HBase > Issue Type: Bug > Components: mapreduce, util >Affects Versions: 1.1.1 >Reporter: Randy Hu >Assignee: Matt Warhaftig > Attachments: hbase-15287-v1.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > org.apache.hadoop.hbase.mapreduce.RowCounter takes optional start/end key as > inputs (-range option). It would work only when the string representation of > value is identical to the string. When row key is binary, the string > representation of the value would look like this: "\x00\x01", which would be > incorrect interpreted as 8 char string in the current implementation: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/RowCounter.java > To fix that, we need change how the value is converted from command line > inputs: > Change > scan.setStartRow(Bytes.toBytes(startKey)); > to > scan.setStartRow(Bytes.toBytesBinary(startKey)); > Do the same conversion to end key as well. > The issue was discovered when the utility was used to calcualte row > distribution on regions from table with binary row keys. The hbase:meta > contains the start key of each region in format of above example. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15238378#comment-15238378
]
Matt Warhaftig commented on HBASE-15615:
[~zghaobac], yes, you and I are completely on the same page.
For
{quote}
"RETRY_BACKOFF\[0\] will never be used"
{quote}
I had noticed the same thing and also thought that after the first RPC attempt
RETRY_BACKOFF[0] should be the backoff factor, not RETRY_BACKOFF[1].
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-15615.patch
>
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-15615) Wrong sleep time when RegionServerCallable need retry
[
https://issues.apache.org/jira/browse/HBASE-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15236282#comment-15236282
]
Matt Warhaftig commented on HBASE-15615:
Hi [~zghaobac], Thanks for finding this bug and submitting a patch. I had
actually glanced at this ticket before you grabbed it and noticed that some of
the other classes implementing RetryingCallable (ex MasterCallable) do not do
tries +1 and so was thinking about if all implementation should have tries +1.
Anyway, this is just a stream of consciousness question from my head, so please
disregard if you know better. :-).
Cheers,
Matt
> Wrong sleep time when RegionServerCallable need retry
> -
>
> Key: HBASE-15615
> URL: https://issues.apache.org/jira/browse/HBASE-15615
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 2.0.0
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
> Attachments: HBASE-15615.patch
>
>
> In RpcRetryingCallerImpl, it get pause time by expectedSleep =
> callable.sleep(pause, tries + 1); And in RegionServerCallable, it get pasue
> time by sleep = ConnectionUtils.getPauseTime(pause, tries + 1). So tries will
> be bumped up twice. And the pasue time is 3 * hbase.client.pause when tries
> is 0.
> RETRY_BACKOFF = {1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-15287) org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with binary row key inputs
[ https://issues.apache.org/jira/browse/HBASE-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig updated HBASE-15287: --- Attachment: hbase-15287-v1.patch Attached patch 'hbase-15287-v1.patch' switches row value for HBase's mapred/mapreduce jobs from Bytes.toByte() to Bytes.toBytesBinary(). Family and qualifier were not switched to binary bytes as I assume they are unlikely to contain binary values. > org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with > binary row key inputs > > > Key: HBASE-15287 > URL: https://issues.apache.org/jira/browse/HBASE-15287 > Project: HBase > Issue Type: Bug > Components: mapreduce, util >Affects Versions: 1.1.1 >Reporter: Randy Hu >Assignee: Matt Warhaftig > Attachments: hbase-15287-v1.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > org.apache.hadoop.hbase.mapreduce.RowCounter takes optional start/end key as > inputs (-range option). It would work only when the string representation of > value is identical to the string. When row key is binary, the string > representation of the value would look like this: "\x00\x01", which would be > incorrect interpreted as 8 char string in the current implementation: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/RowCounter.java > To fix that, we need change how the value is converted from command line > inputs: > Change > scan.setStartRow(Bytes.toBytes(startKey)); > to > scan.setStartRow(Bytes.toBytesBinary(startKey)); > Do the same conversion to end key as well. > The issue was discovered when the utility was used to calcualte row > distribution on regions from table with binary row keys. The hbase:meta > contains the start key of each region in format of above example. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15287) org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with binary row key inputs
[ https://issues.apache.org/jira/browse/HBASE-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15221601#comment-15221601 ] Matt Warhaftig commented on HBASE-15287: Apologies, I've been busy with other things. Hopefully should have a patch ready this weekend. > org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with > binary row key inputs > > > Key: HBASE-15287 > URL: https://issues.apache.org/jira/browse/HBASE-15287 > Project: HBase > Issue Type: Bug > Components: mapreduce, util >Affects Versions: 1.1.1 >Reporter: Randy Hu >Assignee: Matt Warhaftig > Original Estimate: 24h > Remaining Estimate: 24h > > org.apache.hadoop.hbase.mapreduce.RowCounter takes optional start/end key as > inputs (-range option). It would work only when the string representation of > value is identical to the string. When row key is binary, the string > representation of the value would look like this: "\x00\x01", which would be > incorrect interpreted as 8 char string in the current implementation: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/RowCounter.java > To fix that, we need change how the value is converted from command line > inputs: > Change > scan.setStartRow(Bytes.toBytes(startKey)); > to > scan.setStartRow(Bytes.toBytesBinary(startKey)); > Do the same conversion to end key as well. > The issue was discovered when the utility was used to calcualte row > distribution on regions from table with binary row keys. The hbase:meta > contains the start key of each region in format of above example. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14147) REST Support for Namespaces
[ https://issues.apache.org/jira/browse/HBASE-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207710#comment-15207710 ] Matt Warhaftig commented on HBASE-14147: It doesn't look like the HTTP protocol specifically requires data when content-type is in the header. I would guess this 400 Bad Request quirk is getting introduced by Java Framework or HBase's REST implementation (speculating). > REST Support for Namespaces > --- > > Key: HBASE-14147 > URL: https://issues.apache.org/jira/browse/HBASE-14147 > Project: HBase > Issue Type: Sub-task > Components: REST >Affects Versions: 1.1.1 >Reporter: Rick Kellogg >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.15 > > Attachments: SuccessfulLocalBuild.txt, hbase-14147-v1.patch, > hbase-14147-v1.patch, hbase-14147-v1.patch > > > Expand REST services to include addition features: > * Create namespace > * Alter namespace > * Describe namespace > * Drop namespace > * List tables in a specific namespace > * List all namespaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15287) org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with binary row key inputs
[ https://issues.apache.org/jira/browse/HBASE-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205663#comment-15205663 ] Matt Warhaftig commented on HBASE-15287: Hi [~enis], Randy's description of the issue (and fix) are correct and would be a quick patch. I had been investigating the larger question of whether other HBase MapReduce functions should get the same change. I grabbed the ticket figuring on a 3/30 ETA for patch submission but am ok giving it up to someone else. > org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with > binary row key inputs > > > Key: HBASE-15287 > URL: https://issues.apache.org/jira/browse/HBASE-15287 > Project: HBase > Issue Type: Bug > Components: mapreduce, util >Affects Versions: 1.1.1 >Reporter: Randy Hu >Assignee: Matt Warhaftig > Original Estimate: 24h > Remaining Estimate: 24h > > org.apache.hadoop.hbase.mapreduce.RowCounter takes optional start/end key as > inputs (-range option). It would work only when the string representation of > value is identical to the string. When row key is binary, the string > representation of the value would look like this: "\x00\x01", which would be > incorrect interpreted as 8 char string in the current implementation: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/RowCounter.java > To fix that, we need change how the value is converted from command line > inputs: > Change > scan.setStartRow(Bytes.toBytes(startKey)); > to > scan.setStartRow(Bytes.toBytesBinary(startKey)); > Do the same conversion to end key as well. > The issue was discovered when the utility was used to calcualte row > distribution on regions from table with binary row keys. The hbase:meta > contains the start key of each region in format of above example. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-15287) org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with binary row key inputs
[ https://issues.apache.org/jira/browse/HBASE-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig reassigned HBASE-15287: -- Assignee: Matt Warhaftig > org.apache.hadoop.hbase.mapreduce.RowCounter returns incorrect result with > binary row key inputs > > > Key: HBASE-15287 > URL: https://issues.apache.org/jira/browse/HBASE-15287 > Project: HBase > Issue Type: Bug > Components: mapreduce, util >Affects Versions: 1.1.1 >Reporter: Randy Hu >Assignee: Matt Warhaftig > Original Estimate: 24h > Remaining Estimate: 24h > > org.apache.hadoop.hbase.mapreduce.RowCounter takes optional start/end key as > inputs (-range option). It would work only when the string representation of > value is identical to the string. When row key is binary, the string > representation of the value would look like this: "\x00\x01", which would be > incorrect interpreted as 8 char string in the current implementation: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/RowCounter.java > To fix that, we need change how the value is converted from command line > inputs: > Change > scan.setStartRow(Bytes.toBytes(startKey)); > to > scan.setStartRow(Bytes.toBytesBinary(startKey)); > Do the same conversion to end key as well. > The issue was discovered when the utility was used to calcualte row > distribution on regions from table with binary row keys. The hbase:meta > contains the start key of each region in format of above example. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14147) REST Support for Namespaces
[
https://issues.apache.org/jira/browse/HBASE-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205636#comment-15205636
]
Matt Warhaftig commented on HBASE-14147:
Hi [~misty], It looks like specifying {{-H "Content-Type: text/xml"}} results
in a 400 Bad Request if the curl statement does not also include the -data
argument.
To create a namespace with no namespace properties specified you can do:
{noformat}curl -vi -X POST -H "Accept: text/xml"
"http://localhost:20550/namespaces/namespaceName"{noformat}
Or to create a namespace with property key1=value1 specified via xml you can do:
{noformat}curl -vi -X POST -H "Accept: text/xml" -H "Content-Type: text/xml" -d
"key1value1"
"http://localhost:20550/namespaces/namespaceName"{noformat}
> REST Support for Namespaces
> ---
>
> Key: HBASE-14147
> URL: https://issues.apache.org/jira/browse/HBASE-14147
> Project: HBase
> Issue Type: Sub-task
> Components: REST
>Affects Versions: 1.1.1
>Reporter: Rick Kellogg
>Assignee: Matt Warhaftig
>Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.15
>
> Attachments: SuccessfulLocalBuild.txt, hbase-14147-v1.patch,
> hbase-14147-v1.patch, hbase-14147-v1.patch
>
>
> Expand REST services to include addition features:
> * Create namespace
> * Alter namespace
> * Describe namespace
> * Drop namespace
> * List tables in a specific namespace
> * List all namespaces.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14363) Print more details on the row behind an Empty REGIONINFO_QUALIFIER warning
[
https://issues.apache.org/jira/browse/HBASE-14363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182168#comment-15182168
]
Matt Warhaftig commented on HBASE-14363:
{quote}"Is that appearing in 1.x already, or was that after your local
changes?"{quote}
The output listed earlier is HBase source without any local changes.
{quote}"I didn't see a key being logged in 0.98.x I was originally
troubleshooting when I'd filed this."{quote}
Empty REGIONINFO_QUALIFIER details was added 3/2012 (f2d637) and should be
available in 0.98.x. Please respond if the HBCK details flag doesn't yield the
needed results for your and I will investigate further.
> Print more details on the row behind an Empty REGIONINFO_QUALIFIER warning
> --
>
> Key: HBASE-14363
> URL: https://issues.apache.org/jira/browse/HBASE-14363
> Project: HBase
> Issue Type: Improvement
> Components: hbck, Operability
>Affects Versions: 1.0.0
>Reporter: Harsh J
>Assignee: Matt Warhaftig
>
> Currently HBCK just prints a vague "Empty REGIONINFO_QUALIFIER found"
> warning, and does not print the row it found that on. While fixing this is
> easy thanks to HBCK, some more detail (say the row/region ID) would be good
> to print, to avoid people manually scanning meta to obtain the very same info.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14363) Print more details on the row behind an Empty REGIONINFO_QUALIFIER warning
[
https://issues.apache.org/jira/browse/HBASE-14363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954219#comment-14954219
]
Matt Warhaftig commented on HBASE-14363:
Hi [~qwertymaniac], The HBCK -details flag provides keys (but not values) for
'hbase:meta' row with empty REGIONINFO_QUALIFIER's.
For example here is HBCK -details output when the 'hbase:meta' table is missing
a 'info:region info' column for a 't1' table with one region:
{noformat}
ERROR: Empty REGIONINFO_QUALIFIER found in hbase:meta
Number of empty REGIONINFO_QUALIFIER rows in hbase:meta: 1
keyvalues={t1,,1444650621481.3da6b22dcf0498f89fe3a3cb8b8afe39./info:seqnumDuringOpen/1444692021935/Put/vlen=8/seqid=0,
t1,,1444650621481.3da6b22dcf0498f89fe3a3cb8b8afe39./info:server/1444692021935/Put/vlen=15/seqid=0,
t1,,1444650621481.3da6b22dcf0498f89fe3a3cb8b8afe39./info:serverstartcode/1444692021935/Put/vlen=8/seqid=0,
t1,,1444650621481.3da6b22dcf0498f89fe3a3cb8b8afe39./info:sn/1444692021907/Put/vlen=29/seqid=0,
t1,,1444650621481.3da6b22dcf0498f89fe3a3cb8b8afe39./info:state/1444692021935/Put/vlen=4/seqid=0}
{noformat}
It doesn't have regionserver info (as that is a value) but is the currently
displayed info enough to avoid manually scanning 'hbase:meta'? I can log more
if needed.
> Print more details on the row behind an Empty REGIONINFO_QUALIFIER warning
> --
>
> Key: HBASE-14363
> URL: https://issues.apache.org/jira/browse/HBASE-14363
> Project: HBase
> Issue Type: Improvement
> Components: hbck, Operability
>Affects Versions: 1.0.0
>Reporter: Harsh J
>Assignee: Matt Warhaftig
>
> Currently HBCK just prints a vague "Empty REGIONINFO_QUALIFIER found"
> warning, and does not print the row it found that on. While fixing this is
> easy thanks to HBCK, some more detail (say the row/region ID) would be good
> to print, to avoid people manually scanning meta to obtain the very same info.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14954195#comment-14954195
]
Matt Warhaftig commented on HBASE-14584:
TestBaseModel#testFromXML (which runs as a test case for
TestNamespacesInstanceModel via inheritance) requires AS_XML.
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
> Attachments: 14584.00.patch, disable.txt, hbase-14584-v1.patch
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-14363) Print more details on the row behind an Empty REGIONINFO_QUALIFIER warning
[ https://issues.apache.org/jira/browse/HBASE-14363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig reassigned HBASE-14363: -- Assignee: Matt Warhaftig > Print more details on the row behind an Empty REGIONINFO_QUALIFIER warning > -- > > Key: HBASE-14363 > URL: https://issues.apache.org/jira/browse/HBASE-14363 > Project: HBase > Issue Type: Improvement > Components: hbck, Operability >Affects Versions: 1.0.0 >Reporter: Harsh J >Assignee: Matt Warhaftig > > Currently HBCK just prints a vague "Empty REGIONINFO_QUALIFIER found" > warning, and does not print the row it found that on. While fixing this is > easy thanks to HBCK, some more detail (say the row/region ID) would be good > to print, to avoid people manually scanning meta to obtain the very same info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14560) TestNamespacesInstanceModel#testToXML fails when JDK 1.8 is used
[
https://issues.apache.org/jira/browse/HBASE-14560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14952501#comment-14952501
]
Matt Warhaftig commented on HBASE-14560:
Thanks Ted for finding and reporting this issue.
A later ticket, HBASE-14584, got more love and a potential patch even though
your ticket is earlier.
> TestNamespacesInstanceModel#testToXML fails when JDK 1.8 is used
>
>
> Key: HBASE-14560
> URL: https://issues.apache.org/jira/browse/HBASE-14560
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Matt Warhaftig
>Priority: Minor
>
> From
> https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/237/consoleFull
> :
> {code}
> org.apache.hadoop.hbase.rest.model.TestNamespacesInstanceModel
> testToXML(org.apache.hadoop.hbase.rest.model.TestNamespacesInstanceModel)
> Time elapsed: 0.017 sec <<< FAILURE!
> junit.framework.ComparisonFailure:
> expected:<...perties>[NAMEtestNamespaceKEY_2VALUE_2KEY_1VALUE_1]
> but
> was:<...perties>[KEY_1VALUE_1KEY_2VALUE_2NAMEtestNamespace]
> at junit.framework.Assert.assertEquals(Assert.java:100)
> at junit.framework.Assert.assertEquals(Assert.java:107)
> at junit.framework.TestCase.assertEquals(TestCase.java:269)
> at
> org.apache.hadoop.hbase.rest.model.TestModelBase.testToXML(TestModelBase.java:115)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The above test failure can be reproduced locally.
> It was likely caused by the different behavior w.r.t. JAXBContext between JDK
> 1.7 and 1.8
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-14560) TestNamespacesInstanceModel#testToXML fails when JDK 1.8 is used
[
https://issues.apache.org/jira/browse/HBASE-14560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig reassigned HBASE-14560:
--
Assignee: Matt Warhaftig
> TestNamespacesInstanceModel#testToXML fails when JDK 1.8 is used
>
>
> Key: HBASE-14560
> URL: https://issues.apache.org/jira/browse/HBASE-14560
> Project: HBase
> Issue Type: Test
>Reporter: Ted Yu
>Assignee: Matt Warhaftig
>Priority: Minor
>
> From
> https://builds.apache.org/job/HBase-1.3/jdk=latest1.8,label=Hadoop/237/consoleFull
> :
> {code}
> org.apache.hadoop.hbase.rest.model.TestNamespacesInstanceModel
> testToXML(org.apache.hadoop.hbase.rest.model.TestNamespacesInstanceModel)
> Time elapsed: 0.017 sec <<< FAILURE!
> junit.framework.ComparisonFailure:
> expected:<...perties>[NAMEtestNamespaceKEY_2VALUE_2KEY_1VALUE_1]
> but
> was:<...perties>[KEY_1VALUE_1KEY_2VALUE_2NAMEtestNamespace]
> at junit.framework.Assert.assertEquals(Assert.java:100)
> at junit.framework.Assert.assertEquals(Assert.java:107)
> at junit.framework.TestCase.assertEquals(TestCase.java:269)
> at
> org.apache.hadoop.hbase.rest.model.TestModelBase.testToXML(TestModelBase.java:115)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> The above test failure can be reproduced locally.
> It was likely caused by the different behavior w.r.t. JAXBContext between JDK
> 1.7 and 1.8
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-14584:
---
Attachment: hbase-14584-v1.patch
Attached patch 'hbase-14584-v1.patch' does the following:
- Remove alphabetic XML accessor ordering from NamespacesInstanceModel because
it will have no effect with only one element name used for all properties.
- Modify TestModelBase#testToXML() to reconvert to the model for verification
since XML element ordering is not guaranteed.
- Reenable TestNamespacesInstanceModel#testFromXML() as it wouldn't be affected
by XML element ordering and never seemed to have failed.
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
> Attachments: 14584.00.patch, disable.txt, hbase-14584-v1.patch
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14951815#comment-14951815
]
Matt Warhaftig commented on HBASE-14584:
Oh, I see. The commit was just for disabling tests. I'll take a look at fixing.
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig reassigned HBASE-14584:
--
Assignee: Matt Warhaftig
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
>Assignee: Matt Warhaftig
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14584) TestNamespacesInstanceModel fails on jdk8
[
https://issues.apache.org/jira/browse/HBASE-14584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14951811#comment-14951811
]
Matt Warhaftig commented on HBASE-14584:
Thanks [~ndimiduk] and [~stack].
> TestNamespacesInstanceModel fails on jdk8
> -
>
> Key: HBASE-14584
> URL: https://issues.apache.org/jira/browse/HBASE-14584
> Project: HBase
> Issue Type: Test
> Components: REST, test
>Reporter: Nick Dimiduk
> Attachments: 14584.00.patch, disable.txt
>
>
> Noticed this in the [build
> output|https://builds.apache.org/job/HBase-1.2/jdk=latest1.8,label=Hadoop/235/consoleFull]
> of HBASE-12911. Seems this test has been failing for a long time, got all
> the way back to {{6534583}}, {{master~44}} and it's still failing there. I
> guess tests usually fail in {{hbase-server}}, so we don't often get to
> {{hbase-rest}} module.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14147) REST Support for Namespaces
[ https://issues.apache.org/jira/browse/HBASE-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig updated HBASE-14147: --- Attachment: SuccessfulLocalBuild.txt [~stack] I believe the compilation errors are due to a bad patch merge into source. If you compare my patch attached to this ticket versus what was committed in fa730f all of the new files are missing (e.g. NamespacesInstanceResource.java). Please try committing again. > REST Support for Namespaces > --- > > Key: HBASE-14147 > URL: https://issues.apache.org/jira/browse/HBASE-14147 > Project: HBase > Issue Type: Sub-task > Components: REST >Affects Versions: 1.1.1 >Reporter: Rick Kellogg >Assignee: Matt Warhaftig >Priority: Minor > Fix For: 2.0.0, 1.3.0 > > Attachments: SuccessfulLocalBuild.txt, hbase-14147-v1.patch, > hbase-14147-v1.patch, hbase-14147-v1.patch > > > Expand REST services to include addition features: > * Create namespace > * Alter namespace > * Describe namespace > * Drop namespace > * List tables in a specific namespace > * List all namespaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14291) NPE On StochasticLoadBalancer Balance Involving RS With No Regions
[
https://issues.apache.org/jira/browse/HBASE-14291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708017#comment-14708017
]
Matt Warhaftig commented on HBASE-14291:
Also, thanks for the quick response. :-)
> NPE On StochasticLoadBalancer Balance Involving RS With No Regions
> --
>
> Key: HBASE-14291
> URL: https://issues.apache.org/jira/browse/HBASE-14291
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 2.0.0
> Environment: Pseudo-distributed (2 local RegionServers), Hadoop
> 2.5.1, Java 1.7.0_71
>Reporter: Matt Warhaftig
>Assignee: Ted Yu
>Priority: Minor
> Attachments: 14291-v1.txt, hbase-mwarhaftig-master-Matts-MBP.log
>
>
> When StochasticLoadBalancer attempts to balance a local RS with multiple
> regions with another local RS that had no regions the HBase shell call of
> 'balancer' gets the following NPE:
> {noformat}
> ERROR: java.io.IOException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2175)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.getLeastLoadedTopServerForRegion(BaseLoadBalancer.java:863)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer$LocalityBasedCandidateGenerator.generate(StochasticLoadBalancer.java:724)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:325)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:263)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1264)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.balance(MasterRpcServices.java:413)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:52450)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2133)
> ... 4 more
> {noformat}
> Issue only occurs when one of the RSs has no regions before balancing. Also,
> unsure if distributed RSs would also have same issue. Attached
> 'hbase-mwarhaftig-master-Matts-MBP.log' is master's log of the error
> occurring.
> SimpleLoadBalancer rebalances correctly when used in the same situation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14291) NPE On StochasticLoadBalancer Balance Involving RS With No Regions
[
https://issues.apache.org/jira/browse/HBASE-14291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708015#comment-14708015
]
Matt Warhaftig commented on HBASE-14291:
Patch looks good [~ted_yu] - described scenario no longer NPEs and regions are
now balanced.
> NPE On StochasticLoadBalancer Balance Involving RS With No Regions
> --
>
> Key: HBASE-14291
> URL: https://issues.apache.org/jira/browse/HBASE-14291
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 2.0.0
> Environment: Pseudo-distributed (2 local RegionServers), Hadoop
> 2.5.1, Java 1.7.0_71
>Reporter: Matt Warhaftig
>Assignee: Ted Yu
>Priority: Minor
> Attachments: 14291-v1.txt, hbase-mwarhaftig-master-Matts-MBP.log
>
>
> When StochasticLoadBalancer attempts to balance a local RS with multiple
> regions with another local RS that had no regions the HBase shell call of
> 'balancer' gets the following NPE:
> {noformat}
> ERROR: java.io.IOException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2175)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.getLeastLoadedTopServerForRegion(BaseLoadBalancer.java:863)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer$LocalityBasedCandidateGenerator.generate(StochasticLoadBalancer.java:724)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:325)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:263)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1264)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.balance(MasterRpcServices.java:413)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:52450)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2133)
> ... 4 more
> {noformat}
> Issue only occurs when one of the RSs has no regions before balancing. Also,
> unsure if distributed RSs would also have same issue. Attached
> 'hbase-mwarhaftig-master-Matts-MBP.log' is master's log of the error
> occurring.
> SimpleLoadBalancer rebalances correctly when used in the same situation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14291) NPE On StochasticLoadBalancer Balance Involving RS With No Regions
[
https://issues.apache.org/jira/browse/HBASE-14291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-14291:
---
Attachment: hbase-mwarhaftig-master-Matts-MBP.log
> NPE On StochasticLoadBalancer Balance Involving RS With No Regions
> --
>
> Key: HBASE-14291
> URL: https://issues.apache.org/jira/browse/HBASE-14291
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Affects Versions: 2.0.0
> Environment: Pseudo-distributed (2 local RegionServers), Hadoop
> 2.5.1, Java 1.7.0_71
>Reporter: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-mwarhaftig-master-Matts-MBP.log
>
>
> When StochasticLoadBalancer attempts to balance a local RS with multiple
> regions with another local RS that had no regions the HBase shell call of
> 'balancer' gets the following NPE:
> {noformat}
> ERROR: java.io.IOException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2175)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.getLeastLoadedTopServerForRegion(BaseLoadBalancer.java:863)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer$LocalityBasedCandidateGenerator.generate(StochasticLoadBalancer.java:724)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:325)
> at
> org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:263)
> at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1264)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.balance(MasterRpcServices.java:413)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:52450)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2133)
> ... 4 more
> {noformat}
> Issue only occurs when one of the RSs has no regions before balancing. Also,
> unsure if distributed RSs would also have same issue. Attached
> 'hbase-mwarhaftig-master-Matts-MBP.log' is master's log of the error
> occurring.
> SimpleLoadBalancer rebalances correctly when used in the same situation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14291) NPE On StochasticLoadBalancer Balance Involving RS With No Regions
Matt Warhaftig created HBASE-14291:
--
Summary: NPE On StochasticLoadBalancer Balance Involving RS With
No Regions
Key: HBASE-14291
URL: https://issues.apache.org/jira/browse/HBASE-14291
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 2.0.0
Environment: Pseudo-distributed (2 local RegionServers), Hadoop 2.5.1,
Java 1.7.0_71
Reporter: Matt Warhaftig
Priority: Minor
When StochasticLoadBalancer attempts to balance a local RS with multiple
regions with another local RS that had no regions the HBase shell call of
'balancer' gets the following NPE:
{noformat}
ERROR: java.io.IOException
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2175)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.getLeastLoadedTopServerForRegion(BaseLoadBalancer.java:863)
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer$LocalityBasedCandidateGenerator.generate(StochasticLoadBalancer.java:724)
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:325)
at
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:263)
at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1264)
at
org.apache.hadoop.hbase.master.MasterRpcServices.balance(MasterRpcServices.java:413)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:52450)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2133)
... 4 more
{noformat}
Issue only occurs when one of the RSs has no regions before balancing. Also,
unsure if distributed RSs would also have same issue. Attached
'hbase-mwarhaftig-master-Matts-MBP.log' is master's log of the error occurring.
SimpleLoadBalancer rebalances correctly when used in the same situation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14078) improve error message when HMaster can't bind to port
[
https://issues.apache.org/jira/browse/HBASE-14078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-14078:
---
Attachment: hbase-14708-v3.patch
Ok, attached 'hbase-14708-v3.patch' tightens the catch logic to only add config
message to BindExceptions and adds spaces near '{' and '}'. I'll individually
add spaces to my other pending patches after their reviews.
> improve error message when HMaster can't bind to port
> -
>
> Key: HBASE-14078
> URL: https://issues.apache.org/jira/browse/HBASE-14078
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Matt Warhaftig
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: hbase-14078_post_stack.txt, hbase-14708-v1.patch,
> hbase-14708-v2.patch, hbase-14708-v3.patch
>
>
> When the master fails to start becahse hbase.master.port is already taken,
> the log messages could make it easier to tell.
> {quote}
> 2015-07-14 13:10:02,667 INFO [main] regionserver.RSRpcServices:
> master/master01.example.com/10.20.188.121:16000 server-side HConnection
> retries=350
> 2015-07-14 13:10:02,879 INFO [main] ipc.SimpleRpcScheduler: Using deadline
> as user call queue, count=3
> 2015-07-14 13:10:02,895 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMaster
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:234)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2272)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2513)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener.(RpcServer.java:599)
> at org.apache.hadoop.hbase.ipc.RpcServer.(RpcServer.java:2000)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:919)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.(MasterRpcServices.java:211)
> at
> org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:509)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:535)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:351)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2253)
> ... 5 more
> {quote}
> I recognize that the "RSRpcServices" log message shows port 16000, but I
> don't know why a new operator would. Additionally, it'd be nice to tell them
> that the port is controlled by {{hbase.master.port}}. Maybe give a hint on
> how to see what's using the port. Could be too os-dist specific?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14078) improve error message when HMaster can't bind to port
[
https://issues.apache.org/jira/browse/HBASE-14078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-14078:
---
Attachment: hbase-14078_post_stack.txt
Thanks for the feedback [~stack]. Below are responses inline.
{quote}
Formatting is a little off. Please check.
{quote}
Can you point me towards the misformatted code? - I didn't see it when looking
at the diff.
{quote}
Do you have example of what the emission looks like now?
{quote}
See attached 'hbase-14078_post_stack.txt'.
{quote}
Whatever the IOE exception that comes up out of setting up the rpc server, we
will always have this suffix about how to config port. Will it always be a port
issue? Perhaps work on BindException only? And only if 'Address in use'?
{quote}
You are correct with your questioning. I will tighten port issue error logic
when making the earlier mentioned formatting change.
{quote}
The changes in HMaster.java do not seem to make for the same emission. Is that
intentional? For example, before patch, if an Exception, we used to
e.getCause().getMessage() if non-null but now if I read it right, we do
e.toString
{quote}
Yes, the change was intentional because existing HMaster thrown errors include
useful messages that were previously ignored when only e.getCause() was
displayed. The e.getCause() message is still displayed after this change, just
one level down the error stack now.
> improve error message when HMaster can't bind to port
> -
>
> Key: HBASE-14078
> URL: https://issues.apache.org/jira/browse/HBASE-14078
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Matt Warhaftig
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: hbase-14078_post_stack.txt, hbase-14708-v1.patch,
> hbase-14708-v2.patch
>
>
> When the master fails to start becahse hbase.master.port is already taken,
> the log messages could make it easier to tell.
> {quote}
> 2015-07-14 13:10:02,667 INFO [main] regionserver.RSRpcServices:
> master/master01.example.com/10.20.188.121:16000 server-side HConnection
> retries=350
> 2015-07-14 13:10:02,879 INFO [main] ipc.SimpleRpcScheduler: Using deadline
> as user call queue, count=3
> 2015-07-14 13:10:02,895 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMaster
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:234)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2272)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2513)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener.(RpcServer.java:599)
> at org.apache.hadoop.hbase.ipc.RpcServer.(RpcServer.java:2000)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:919)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.(MasterRpcServices.java:211)
> at
> org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:509)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:535)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:351)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2253)
> ... 5 more
> {quote}
> I recognize that the "RSRpcServices" log message shows port 16000, but I
> don't know why a new operator would. Additionally, it'd be nice to tell them
> that the port is controlled by {{hbase.master.port}}. Maybe give a hint on
> how to s
[jira] [Updated] (HBASE-13838) Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
[ https://issues.apache.org/jira/browse/HBASE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig updated HBASE-13838: --- Attachment: hbase-13838_post_put_command.txt hbase-13838_post.tiff hbase-13838_pre.tiff Attached 'hbase-13838_pre.tiff' and 'hbase-13838_post.tiff' show that master UI page no longer has Operations tab and that alerts are now properly colored via Bootstrap. Attached 'hbase-13838_post_put_command.txt' shows that the Operations JSON lists actual data for a PUT (and any RPC operation). This displayed data was the open question about security concerns. > Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.) > > > Key: HBASE-13838 > URL: https://issues.apache.org/jira/browse/HBASE-13838 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Matt Warhaftig > Labels: beginner > Fix For: 2.0.0, 1.3.0 > > Attachments: hbase-13838-v1.patch, hbase-13838_post.tiff, > hbase-13838_post_put_command.txt, hbase-13838_pre.tiff > > > There are a few issues with the shared TaskStatusTmpl: > - "Client operations" tab is always empty > For Master this is expected, but for RegionServers there is never anything > listed either. Fix for RS status page (probably caused by params not > containing Operation subclass anymore, but some PB generated classes?) > - Hide “Client Operations” tab for master UI > Since operations are RS only. Or we fix this and make other calls show here. > - The "alert-error" for aborted tasks is not set in CSS at all > When a task was aborted it should be amber or red, but the assigned style is > not in any of the linked stylesheets (abort-error). Add. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13839) Fix AssgnmentManagerTmpl.jamon issues (coloring, content etc.)
[
https://issues.apache.org/jira/browse/HBASE-13839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-13839:
---
Attachment: HBASE-13838_post.tiff
HBASE-13838_pre.tiff
Attached 'HBASE-13838_pre.tiff' and 'HBASE-13838_post.tiff' to show the 'Region
In Transition' UI improvements (hopefully got all the enhancements in that
single screenshot :-) ).
If you want to recreate, to have long lasting RITs add the following line and
split 'TestTable':
{noformat}
diff --git
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java
b/hbase-server/src/main/java/org/a
pache/hadoop/hbase/master/RegionStates.java
index 5528b07..3e37ca5 100644
---
a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java
+++
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java
@@ -410,6 +435,7 @@ public class RegionStates {
updateRegionState(hri, State.OPEN, serverName, openSeqNum);
synchronized (this) {
+ if(hri.getTable().getNameAsString().equals("TestTable")){ return; }
regionsInTransition.remove(encodedName);
ServerName oldServerName = regionAssignments.put(hri, serverName);
if (!serverName.equals(oldServerName)) {
{noformat}
> Fix AssgnmentManagerTmpl.jamon issues (coloring, content etc.)
> --
>
> Key: HBASE-13839
> URL: https://issues.apache.org/jira/browse/HBASE-13839
> Project: HBase
> Issue Type: Bug
> Components: master, UI
>Affects Versions: 1.1.0
>Reporter: Lars George
>Assignee: Matt Warhaftig
> Labels: beginner
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-13838_post.tiff, HBASE-13838_pre.tiff,
> hbase-13839-v1.patch
>
>
> The template for the RIT in the Master status page,
> AssignmentManagerTmpl.jamon) has a few issues:
> - The oldest RIT should not be _red_, looks like a failed entry
> The RIT entries should be for example yellow/amber when over the threshold
> time, and red if 2x the threshold - or red for the oldest once over the
> threshold.
> - Region count over RIT threshold should only be colored if > 0
> The summary line (first of two) should not be colored unless there is a value
> > 0 in it.
> - Color is overriden by table-stripped CSS style!
> The Bootstrap stylesheet cancels out the hardcoded coloring! The
> table-stripped resets the conditional coloring and should be fixed. Best is
> to use "alert-warning" etc. that come from the Bootstrap theme stylesheet.
> That should maybe already work in combination with the "table-stripped" from
> the same.
> - Should sort descending by time
> Currently the list of regions is sorted by encoded region name. Better is to
> have the table sorted by RIT time descending.
> We should also think about a pagination option for the currently hardcoded
> 100 entries max. Maybe a separate issue?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13708) PE - Add option to specify the range
[
https://issues.apache.org/jira/browse/HBASE-13708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697913#comment-14697913
]
Matt Warhaftig commented on HBASE-13708:
The rowRangeSize is in the logging at the start with:
{noformat}
2015-08-14 07:24:25,523 INFO [main] hbase.PerformanceEvaluation:
RandomReadTest test run options={..."rowRangeSize":100...
{noformat}
And then at PE completion:
{noformat}
2015-08-14 07:24:27,148 INFO [TestClient-0] hbase.PerformanceEvaluation:
Finished class org.apache.hadoop.hbase.PerformanceEvaluation$RandomReadTest in
92ms at offset 0 for 10 rows from a range of 100 rows (0.11 MB/s)
{noformat}
The line you highlighted
{noformat}
2015-08-14 07:24:27,013 INFO [TestClient-0] hbase.PerformanceEvaluation:
Sampling 1 every 1 out of 10 total rows.
{noformat}
is logging for sample sizing versus number of reads so 10 total rows is apt to
display.
If you want rowRangeSize to be more prominent in the logging I can append it to
that line.
> PE - Add option to specify the range
>
>
> Key: HBASE-13708
> URL: https://issues.apache.org/jira/browse/HBASE-13708
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.0.1
>Reporter: Jean-Marc Spaggiari
>Assignee: Matt Warhaftig
>Priority: Minor
> Labels: beginner
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-13708-master_v1.patch, PE_mapred_output.txt,
> PE_nomapred_output.txt, hbase-13708-v2.patch, hbase-13708-v2.patch
>
>
> When specifying --rows=YYY for randomReads in PE, it will read YYY rows but
> only betweem 0 and YYY.
> We might want to read YYY rows randomly between 0 and ZZZ.
> YYY should only be the number of rows we want to ready. Not the high range.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13966) Limit column width in table.jsp
[
https://issues.apache.org/jira/browse/HBASE-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697272#comment-14697272
]
Matt Warhaftig commented on HBASE-13966:
{quote}
Reviewing the patch, adding in the little styling seems good. Any chance of
same change to start and end key columns? If long keys in the region, they'll
likely blow up start and/or end columns?
{quote}
Since the table-layout is fixed if the start and end key columns get too long
for their allotted space they will automatically wrap as the name column did.
> Limit column width in table.jsp
> ---
>
> Key: HBASE-13966
> URL: https://issues.apache.org/jira/browse/HBASE-13966
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 1.1.0.1
>Reporter: Jean-Marc Spaggiari
>Assignee: Matt Warhaftig
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-13966_post.tiff, HBASE-13966_pre.tiff,
> hbase-13966-v1.patch
>
>
> In table.jsp, for tables with very wide keys like URLs, the page can be very
> wide. On my own cluster, this page is 8 screens wide, almost un-usable.
> Might be good to have a way to resize the columns, or wrap the long keys or
> truncate them, or anything else. When we want to look at the last colums,
> this is a big difficult.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13708) PE - Add option to specify the range
[ https://issues.apache.org/jira/browse/HBASE-13708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig updated HBASE-13708: --- Attachment: PE_mapred_output.txt PE_nomapred_output.txt Attached 'PE_nomapred_output.txt' and 'PE_mapred_output.txt' are output files of Performance Evaluation random read runs for map reduce and non-map reduce with --rowRangeSize=100 & --rows=10. Added extra logging to show that for random read's gets, even though rows < 10, it will randomly read rows between 0 and 100. Prior to patch would only read rows between 0 and 10. > PE - Add option to specify the range > > > Key: HBASE-13708 > URL: https://issues.apache.org/jira/browse/HBASE-13708 > Project: HBase > Issue Type: Bug >Affects Versions: 1.0.1 >Reporter: Jean-Marc Spaggiari >Assignee: Matt Warhaftig >Priority: Minor > Labels: beginner > Fix For: 2.0.0, 1.3.0 > > Attachments: HBASE-13708-master_v1.patch, PE_mapred_output.txt, > PE_nomapred_output.txt, hbase-13708-v2.patch, hbase-13708-v2.patch > > > When specifying --rows=YYY for randomReads in PE, it will read YYY rows but > only betweem 0 and YYY. > We might want to read YYY rows randomly between 0 and ZZZ. > YYY should only be the number of rows we want to ready. Not the high range. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13966) Limit column width in table.jsp
[ https://issues.apache.org/jira/browse/HBASE-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig updated HBASE-13966: --- Attachment: HBASE-13966_post.tiff HBASE-13966_pre.tiff [~stack], Attached 'HBASE-13966_pre.tiff' and 'HBASE-13966_post.tiff' show the formatting improvements. Note that table of tables no longer displays data off the page. > Limit column width in table.jsp > --- > > Key: HBASE-13966 > URL: https://issues.apache.org/jira/browse/HBASE-13966 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 1.1.0.1 >Reporter: Jean-Marc Spaggiari >Assignee: Matt Warhaftig >Priority: Minor > Labels: beginner > Attachments: HBASE-13966_post.tiff, HBASE-13966_pre.tiff, > hbase-13966-v1.patch > > > In table.jsp, for tables with very wide keys like URLs, the page can be very > wide. On my own cluster, this page is 8 screens wide, almost un-usable. > Might be good to have a way to resize the columns, or wrap the long keys or > truncate them, or anything else. When we want to look at the last colums, > this is a big difficult. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-14078) improve error message when HMaster can't bind to port
[
https://issues.apache.org/jira/browse/HBASE-14078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig reassigned HBASE-14078:
--
Assignee: Matt Warhaftig
> improve error message when HMaster can't bind to port
> -
>
> Key: HBASE-14078
> URL: https://issues.apache.org/jira/browse/HBASE-14078
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
>Assignee: Matt Warhaftig
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: hbase-14708-v1.patch, hbase-14708-v2.patch
>
>
> When the master fails to start becahse hbase.master.port is already taken,
> the log messages could make it easier to tell.
> {quote}
> 2015-07-14 13:10:02,667 INFO [main] regionserver.RSRpcServices:
> master/master01.example.com/10.20.188.121:16000 server-side HConnection
> retries=350
> 2015-07-14 13:10:02,879 INFO [main] ipc.SimpleRpcScheduler: Using deadline
> as user call queue, count=3
> 2015-07-14 13:10:02,895 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMaster
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:234)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2272)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2513)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener.(RpcServer.java:599)
> at org.apache.hadoop.hbase.ipc.RpcServer.(RpcServer.java:2000)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:919)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.(MasterRpcServices.java:211)
> at
> org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:509)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:535)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:351)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2253)
> ... 5 more
> {quote}
> I recognize that the "RSRpcServices" log message shows port 16000, but I
> don't know why a new operator would. Additionally, it'd be nice to tell them
> that the port is controlled by {{hbase.master.port}}. Maybe give a hint on
> how to see what's using the port. Could be too os-dist specific?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14147) REST Support for Namespaces
[
https://issues.apache.org/jira/browse/HBASE-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-14147:
---
Attachment: hbase-14147-v1.patch
Attached patch 'hbase-14147-v1.patch' adds namespace CRUD functionality to
rest. A brief overview of the functionality:
|| REST Endpoint || REST Verb || Operation ||
| /namespaces | GET | list all namespaces|
| /namespaces/\{namespace\} | GET |describe namespace|
| /namespaces/\{namespace\}/tables | GET | list namespace tables|
| /namespaces/\{namespace\} | POST |create namespace|
| /namespaces/\{namespace\} | PUT | alter namespace|
| /namespaces/\{namespace\} | DELETE | delete namespace|
Please closely peer review this functionality as it is a large update.
Also note that the HBase REST documentation on the following wiki will need to
be updated to include the new fields:
https://wiki.apache.org/hadoop/Hbase/Stargate
> REST Support for Namespaces
> ---
>
> Key: HBASE-14147
> URL: https://issues.apache.org/jira/browse/HBASE-14147
> Project: HBase
> Issue Type: Sub-task
> Components: REST
>Affects Versions: 1.1.1
>Reporter: Rick Kellogg
>Assignee: Matt Warhaftig
>Priority: Minor
> Attachments: hbase-14147-v1.patch
>
>
> Expand REST services to include addition features:
> * Create namespace
> * Alter namespace
> * Describe namespace
> * Drop namespace
> * List tables in a specific namespace
> * List all namespaces.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14147) REST Support for Namespaces
[ https://issues.apache.org/jira/browse/HBASE-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig updated HBASE-14147: --- Status: Patch Available (was: In Progress) > REST Support for Namespaces > --- > > Key: HBASE-14147 > URL: https://issues.apache.org/jira/browse/HBASE-14147 > Project: HBase > Issue Type: Sub-task > Components: REST >Affects Versions: 1.1.1 >Reporter: Rick Kellogg >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-14147-v1.patch > > > Expand REST services to include addition features: > * Create namespace > * Alter namespace > * Describe namespace > * Drop namespace > * List tables in a specific namespace > * List all namespaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HBASE-14147) REST Support for Namespaces
[ https://issues.apache.org/jira/browse/HBASE-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-14147 started by Matt Warhaftig. -- > REST Support for Namespaces > --- > > Key: HBASE-14147 > URL: https://issues.apache.org/jira/browse/HBASE-14147 > Project: HBase > Issue Type: Sub-task > Components: REST >Affects Versions: 1.1.1 >Reporter: Rick Kellogg >Assignee: Matt Warhaftig >Priority: Minor > Attachments: hbase-14147-v1.patch > > > Expand REST services to include addition features: > * Create namespace > * Alter namespace > * Describe namespace > * Drop namespace > * List tables in a specific namespace > * List all namespaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-14147) REST Support for Namespaces
[ https://issues.apache.org/jira/browse/HBASE-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig reassigned HBASE-14147: -- Assignee: Matt Warhaftig > REST Support for Namespaces > --- > > Key: HBASE-14147 > URL: https://issues.apache.org/jira/browse/HBASE-14147 > Project: HBase > Issue Type: Sub-task > Components: REST >Affects Versions: 1.1.1 >Reporter: Rick Kellogg >Assignee: Matt Warhaftig >Priority: Minor > > Expand REST services to include addition features: > * Create namespace > * Alter namespace > * Describe namespace > * Drop namespace > * List tables in a specific namespace > * List all namespaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14147) REST Support for Namespaces
[ https://issues.apache.org/jira/browse/HBASE-14147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647109#comment-14647109 ] Matt Warhaftig commented on HBASE-14147: I am going to grab this ticket unless there are objectors. Look relatively straight forward - ETA of 7/9. > REST Support for Namespaces > --- > > Key: HBASE-14147 > URL: https://issues.apache.org/jira/browse/HBASE-14147 > Project: HBase > Issue Type: Sub-task > Components: REST >Affects Versions: 1.1.1 >Reporter: Rick Kellogg >Priority: Minor > > Expand REST services to include addition features: > * Create namespace > * Alter namespace > * Describe namespace > * Drop namespace > * List tables in a specific namespace > * List all namespaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-14079) improve error message when Master fails to connect to Hadoop-auth
[
https://issues.apache.org/jira/browse/HBASE-14079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig reassigned HBASE-14079:
--
Assignee: Matt Warhaftig
> improve error message when Master fails to connect to Hadoop-auth
> -
>
> Key: HBASE-14079
> URL: https://issues.apache.org/jira/browse/HBASE-14079
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.94.0, 0.98.0, 1.0.0, 1.1.0
>Reporter: Sean Busbey
>Assignee: Matt Warhaftig
> Fix For: 2.0.0, 0.94.28, 1.1.2, 1.3.0, 0.98.15, 1.2.1, 1.0.3
>
>
> Current error message at INFO level doesn't give any hint about what keytab
> and principle are in use
> {quote}
> 2015-07-14 13:32:48,514 INFO [main] impl.MetricsConfig: loaded properties
> from hadoop-metrics2-hbase.properties
> 2015-07-14 13:32:48,640 INFO [main] impl.MetricsSystemImpl: Scheduled
> snapshot period at 10 second(s).
> 2015-07-14 13:32:48,640 INFO [main] impl.MetricsSystemImpl: HBase metrics
> system started
> 2015-07-14 13:32:48,776 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMaster
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:234)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2272)
> Caused by: javax.security.auth.login.LoginException: Unable to obtain
> password from user
> at
> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:856)
> at
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:719)
> at
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:584)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
> at
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:912)
> at
> org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:242)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.login(User.java:385)
> at org.apache.hadoop.hbase.security.User.login(User.java:252)
> at
> org.apache.hadoop.hbase.security.UserProvider.login(UserProvider.java:115)
> at org.apache.hadoop.hbase.master.HMaster.login(HMaster.java:464)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:553)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:351)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2253)
> ... 5 more
> {quote}
> increasing to DEBUG also doesn't help.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14078) improve error message when HMaster can't bind to port
[
https://issues.apache.org/jira/browse/HBASE-14078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-14078:
---
Attachment: hbase-14708-v2.patch
Attached patch 'hbase-14708-v2.patch' rewrites the catch logic in
HMaster#constructMaster to log all exceptions (including chained) which also
resolves HBASE-14709. Port config info message is now added in the
RSRpcServices constructor to have messages for both master and rs bind
exceptions.
> improve error message when HMaster can't bind to port
> -
>
> Key: HBASE-14078
> URL: https://issues.apache.org/jira/browse/HBASE-14078
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: hbase-14708-v1.patch, hbase-14708-v2.patch
>
>
> When the master fails to start becahse hbase.master.port is already taken,
> the log messages could make it easier to tell.
> {quote}
> 2015-07-14 13:10:02,667 INFO [main] regionserver.RSRpcServices:
> master/master01.example.com/10.20.188.121:16000 server-side HConnection
> retries=350
> 2015-07-14 13:10:02,879 INFO [main] ipc.SimpleRpcScheduler: Using deadline
> as user call queue, count=3
> 2015-07-14 13:10:02,895 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMaster
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:234)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2272)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2513)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener.(RpcServer.java:599)
> at org.apache.hadoop.hbase.ipc.RpcServer.(RpcServer.java:2000)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:919)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.(MasterRpcServices.java:211)
> at
> org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:509)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:535)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:351)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2253)
> ... 5 more
> {quote}
> I recognize that the "RSRpcServices" log message shows port 16000, but I
> don't know why a new operator would. Additionally, it'd be nice to tell them
> that the port is controlled by {{hbase.master.port}}. Maybe give a hint on
> how to see what's using the port. Could be too os-dist specific?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14079) improve error message when Master fails to connect to Hadoop-auth
[
https://issues.apache.org/jira/browse/HBASE-14079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642126#comment-14642126
]
Matt Warhaftig commented on HBASE-14079:
In HBASE-14078 the catch logic in HMaster#constructMaster is improved to log
all exceptions (including chained). Linked the two tickets together as the
HBASE-14078 patch should resolve this ticket too.
> improve error message when Master fails to connect to Hadoop-auth
> -
>
> Key: HBASE-14079
> URL: https://issues.apache.org/jira/browse/HBASE-14079
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 0.94.0, 0.98.0, 1.0.0, 1.1.0
>Reporter: Sean Busbey
> Fix For: 2.0.0, 0.94.28, 1.1.2, 1.3.0, 0.98.15, 1.2.1, 1.0.3
>
>
> Current error message at INFO level doesn't give any hint about what keytab
> and principle are in use
> {quote}
> 2015-07-14 13:32:48,514 INFO [main] impl.MetricsConfig: loaded properties
> from hadoop-metrics2-hbase.properties
> 2015-07-14 13:32:48,640 INFO [main] impl.MetricsSystemImpl: Scheduled
> snapshot period at 10 second(s).
> 2015-07-14 13:32:48,640 INFO [main] impl.MetricsSystemImpl: HBase metrics
> system started
> 2015-07-14 13:32:48,776 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMaster
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:234)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2272)
> Caused by: javax.security.auth.login.LoginException: Unable to obtain
> password from user
> at
> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:856)
> at
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:719)
> at
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:584)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
> at
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:595)
> at
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:912)
> at
> org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:242)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.login(User.java:385)
> at org.apache.hadoop.hbase.security.User.login(User.java:252)
> at
> org.apache.hadoop.hbase.security.UserProvider.login(UserProvider.java:115)
> at org.apache.hadoop.hbase.master.HMaster.login(HMaster.java:464)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:553)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:351)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2253)
> ... 5 more
> {quote}
> increasing to DEBUG also doesn't help.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14078) improve error message when HMaster can't bind to port
[
https://issues.apache.org/jira/browse/HBASE-14078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-14078:
---
Attachment: hbase-14708-v1.patch
> improve error message when HMaster can't bind to port
> -
>
> Key: HBASE-14078
> URL: https://issues.apache.org/jira/browse/HBASE-14078
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: hbase-14708-v1.patch
>
>
> When the master fails to start becahse hbase.master.port is already taken,
> the log messages could make it easier to tell.
> {quote}
> 2015-07-14 13:10:02,667 INFO [main] regionserver.RSRpcServices:
> master/master01.example.com/10.20.188.121:16000 server-side HConnection
> retries=350
> 2015-07-14 13:10:02,879 INFO [main] ipc.SimpleRpcScheduler: Using deadline
> as user call queue, count=3
> 2015-07-14 13:10:02,895 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMaster
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:234)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2272)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2513)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener.(RpcServer.java:599)
> at org.apache.hadoop.hbase.ipc.RpcServer.(RpcServer.java:2000)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:919)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.(MasterRpcServices.java:211)
> at
> org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:509)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:535)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:351)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2253)
> ... 5 more
> {quote}
> I recognize that the "RSRpcServices" log message shows port 16000, but I
> don't know why a new operator would. Additionally, it'd be nice to tell them
> that the port is controlled by {{hbase.master.port}}. Maybe give a hint on
> how to see what's using the port. Could be too os-dist specific?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14078) improve error message when HMaster can't bind to port
[
https://issues.apache.org/jira/browse/HBASE-14078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Warhaftig updated HBASE-14078:
---
Status: Patch Available (was: Open)
Attached patch 'hbase-14708-v1.patch' adds current master hostname and config
info to the BindException via the HMaster class. The config info is added at
HMaster#constructMaster() to keep the socket code generic.
> improve error message when HMaster can't bind to port
> -
>
> Key: HBASE-14078
> URL: https://issues.apache.org/jira/browse/HBASE-14078
> Project: HBase
> Issue Type: Improvement
> Components: master
>Affects Versions: 2.0.0
>Reporter: Sean Busbey
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: hbase-14708-v1.patch
>
>
> When the master fails to start becahse hbase.master.port is already taken,
> the log messages could make it easier to tell.
> {quote}
> 2015-07-14 13:10:02,667 INFO [main] regionserver.RSRpcServices:
> master/master01.example.com/10.20.188.121:16000 server-side HConnection
> retries=350
> 2015-07-14 13:10:02,879 INFO [main] ipc.SimpleRpcScheduler: Using deadline
> as user call queue, count=3
> 2015-07-14 13:10:02,895 ERROR [main] master.HMasterCommandLine: Master exiting
> java.lang.RuntimeException: Failed construction of Master: class
> org.apache.hadoop.hbase.master.HMaster
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2258)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:234)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2272)
> Caused by: java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method)
> at sun.nio.ch.Net.bind(Net.java:444)
> at sun.nio.ch.Net.bind(Net.java:436)
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2513)
> at
> org.apache.hadoop.hbase.ipc.RpcServer$Listener.(RpcServer.java:599)
> at org.apache.hadoop.hbase.ipc.RpcServer.(RpcServer.java:2000)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.(RSRpcServices.java:919)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.(MasterRpcServices.java:211)
> at
> org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:509)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:535)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:351)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2253)
> ... 5 more
> {quote}
> I recognize that the "RSRpcServices" log message shows port 16000, but I
> don't know why a new operator would. Additionally, it'd be nice to tell them
> that the port is controlled by {{hbase.master.port}}. Maybe give a hint on
> how to see what's using the port. Could be too os-dist specific?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13838) Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
[
https://issues.apache.org/jira/browse/HBASE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14631243#comment-14631243
]
Matt Warhaftig commented on HBASE-13838:
Note that after this change the message content of 'Client Operation' tasks
will be shown in the JSON (because the {{ClientProtos}} toString method is
used). If this is a security concern let me know and will generate a sanitized
JSON body.
> Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
>
>
> Key: HBASE-13838
> URL: https://issues.apache.org/jira/browse/HBASE-13838
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 1.1.0
>Reporter: Lars George
>Assignee: Matt Warhaftig
> Labels: beginner
> Fix For: 2.0.0, 1.3.0
>
> Attachments: hbase-13838-v1.patch
>
>
> There are a few issues with the shared TaskStatusTmpl:
> - "Client operations" tab is always empty
> For Master this is expected, but for RegionServers there is never anything
> listed either. Fix for RS status page (probably caused by params not
> containing Operation subclass anymore, but some PB generated classes?)
> - Hide “Client Operations” tab for master UI
> Since operations are RS only. Or we fix this and make other calls show here.
> - The "alert-error" for aborted tasks is not set in CSS at all
> When a task was aborted it should be amber or red, but the assigned style is
> not in any of the linked stylesheets (abort-error). Add.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13838) Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.)
[ https://issues.apache.org/jira/browse/HBASE-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Warhaftig updated HBASE-13838: --- Attachment: hbase-13838-v1.patch > Fix shared TaskStatusTmpl.jamon issues (coloring, content, etc.) > > > Key: HBASE-13838 > URL: https://issues.apache.org/jira/browse/HBASE-13838 > Project: HBase > Issue Type: Bug > Components: UI >Affects Versions: 1.1.0 >Reporter: Lars George >Assignee: Matt Warhaftig > Labels: beginner > Fix For: 2.0.0, 1.3.0 > > Attachments: hbase-13838-v1.patch > > > There are a few issues with the shared TaskStatusTmpl: > - "Client operations" tab is always empty > For Master this is expected, but for RegionServers there is never anything > listed either. Fix for RS status page (probably caused by params not > containing Operation subclass anymore, but some PB generated classes?) > - Hide “Client Operations” tab for master UI > Since operations are RS only. Or we fix this and make other calls show here. > - The "alert-error" for aborted tasks is not set in CSS at all > When a task was aborted it should be amber or red, but the assigned style is > not in any of the linked stylesheets (abort-error). Add. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
