Re: Reworking the use of log levels
Changing logging level at runtime is good, but we should also consider non-reproducible cases. After a system becomes complicated enough and gets widely used, I believe it would get such cases. Agree that stack trace should be logged only when it is really needed. It looks to me hbase is doing good here. E.g. In hbase UT log, I don't see useless stacktrace at least in 98.5(It is not the case for Zookeeper, sometimes I see quite many stacktraces logged with INFO or WARN...) , things have changed in master? Thanks. 在 2014-10-26，3:42，Sean Busbey bus...@cloudera.com 写道： That sounds great! Either a shell or hbase command would be my suggestion, preferably with some logic for mapping from the short logger name that ends up in the log to the actual logger to adjust. The ref guide would need a section on how to use the proposed log changing tool along based on seeing a log message. Would being able to make the logger with the failure change at runtime to show the stack trace address your concerns Andrew and Qiang? To be clear, I'm not saying we should *never* have stack traces above DEBUG, just that they should be rare and reserved for when we can't point an operator at something to do. -- Sean On Oct 25, 2014 1:50 PM, Nick Dimiduk ndimi...@gmail.com wrote: We have the ability to alter log levels at runtime. This would allow an operator to temporarily increase log level for afflicted components, even in production. Doing this on a server-by-server basis should have minimal impact on overall cluster performance. Maybe this needs to be better documented? Maybe we need a script that makes this easier, or could be managed via a new shell command? On Saturday, October 25, 2014, Andrew Purtell apurt...@apache.org wrote: On Sat, Oct 25, 2014 at 6:34 AM, Sean Busbey bus...@cloudera.com javascript:; wrote: Even if debug is disabled in production, it could be enabled on a non-production system for reproducing the problem, no? In my experience, often enough, no. I do hear the complaint that Hadoop ecosystem projects are quite operator unfriendly because error messages most often come in the form of a stacktrace. It's a totally valid point. I think we could certainly improve the exception message printed ahead of the stacktrace in a large number of cases. On Sat, Oct 25, 2014 at 6:34 AM, Sean Busbey bus...@cloudera.com javascript:; wrote: Even if debug is disabled in production, it could be enabled on a non-production system for reproducing the problem, no? -- Sean On Oct 25, 2014 7:11 AM, Qiang Tian tian...@gmail.com javascript:; wrote: perhaps case by case is better. stacktrace is one of most important problem determination methods. debug is mostly disabled in production, we may lose important clues. On Sat, Oct 25, 2014 at 1:14 PM, Sean Busbey bus...@cloudera.com javascript:; wrote: Hi! Right now we have many failure paths where we send stack traces to log files at ERROR / WARN. In an effort to make things easier to operate, I'd like to propose we move towards: * INFO/WARN/ERROR : description of failure and if possible an action an operator could take to fix/diagnose * DEBUG : information needed to handle failures that require developer action, i.e. stack traces I figure this can go as one or more subtasks off of HBASE-12341, but wanted to float things here before I get started. -- Sean -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
[jira] [Created] (HBASE-12346) Scan's default auths behavior under Visibility labels
Jerry He created HBASE-12346:
Summary: Scan's default auths behavior under Visibility labels
Key: HBASE-12346
URL: https://issues.apache.org/jira/browse/HBASE-12346
Project: HBase
Issue Type: Bug
Components: API, security
Affects Versions: 0.99.1, 0.98.7
Reporter: Jerry He
In Visibility Labels security, a set of labels (auths) are administered and
associated with a user.
A user can normally only see cell data during scan that are part of the user's
label set (auths).
Scan uses setAuthorizations to indicates its wants to use the auths to access
the cells.
Similarly in the shell:
{code}
scan 'table1', AUTHORIZATIONS = ['private']
{code}
But it is a surprise to find that setAuthorizations seems to be 'mandatory' in
the default visibility label security setting. Every scan needs to
setAuthorizations before the scan can get any cells even the cells are under
the labels the request user is part of.
The following steps will illustrate the issue:
Run as superuser.
{code}
1. create a visibility label called 'private'
2. create 'table1'
3. put into 'table1' data and label the data as 'private'
4. set_auths 'user1', 'private'
5. grant 'user1', 'RW', 'table1'
{code}
Run as 'user1':
{code}
1. scan 'table1'
This show no cells.
2. scan 'table1', scan 'table1', AUTHORIZATIONS = ['private']
This will show all the data.
{code}
I am not sure if this is expected by design or a bug.
But a more reasonable, more client application backward compatible, and less
surprising default behavior should probably look like this:
A scan's default auths, if its Authorizations attributes is not set explicitly,
should be all the auths the request user is administered and allowed on the
server.
If scan.setAuthorizations is used, then the server further filter the auths
during scan: use the input auths minus what is not in user's label set on the
server.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Jenkins log
I put the config -Dmaven.test.redirectTestOutputToFile=true back on the branch-1 builds and set log level back to WARN. It look like the -D param had no effect. Something to do w/ log amount. See HBASE-12285. St.Ack On Sun, Oct 26, 2014 at 4:21 PM, Stack st...@duboce.net wrote: I reenabled logging at DEBUG level on branch-1 builds up on jenkins (HBASE-12285). St.Ack On Sat, Oct 25, 2014 at 7:58 PM, Stack st...@duboce.net wrote: Lets keep a running notice of changes made up on jenkins here on this thread. I just changed branch-1 removing: -Dmaven.test.redirectTestOutputToFile=true Internal tests have been passing. They didn't have the above in place. branch-1 has been passing since we changed logging to WARN level only. I'll let the above config sit awhile and then put back branch-1 to DEBUG level. St.Ack
[jira] [Created] (HBASE-12347) Fix the edge case where Hadoop QA's parsing of attached patches breaks the JIRA status checker in dev-support/rebase_all_git_branches.sh
Misty Stanley-Jones created HBASE-12347:
---
Summary: Fix the edge case where Hadoop QA's parsing of attached
patches breaks the JIRA status checker in dev-support/rebase_all_git_branches.sh
Key: HBASE-12347
URL: https://issues.apache.org/jira/browse/HBASE-12347
Project: HBase
Issue Type: Sub-task
Components: scripts
Reporter: Misty Stanley-Jones
Priority: Minor
The rebase_all_hbase_branches.sh script is unable to detect that HBASE-12207 is
closed, because for that one JIRA, the curl command that detects the status is
returning the status, but also the text from Hadoop QA for each patch it has
evaluated on the JIRA:
{code}
$ curl -s https://issues.apache.org/jira/browse/HBASE-12207 | grep
resolution-val
span id=resolution-val class=value resolved
+ jira_status=$(curl -s $jira_url/$jira|grep resolution-val|sed
-e s/.*class=\value\ //|cut -d'' -f 1)br/
+ jira_status=$(curl -s $jira_url/$jira|grep resolution-val|sed
-e s/.*class=\value\ //|cut -d'' -f 1)
+ jira_status=$(curl -s $jira_url/$jira|grep resolution-val|sed
-e s/.*class=\value\ //|cut -d'' -f 1)br/
+ jira_status=$(curl -s $jira_url/$jira|grep resolution-val|sed
-e s/.*class=\value\ //|cut -d'' -f 1)
+ jira_status=$(curl -s $jira_url/$jira|grep resolution-val|sed
-e s/.*class=\value\ //|cut -d'' -f 1)br/
+ jira_status=$(curl -s $jira_url/$jira|grep resolution-val|sed
-e s/.*class=\value\ //|cut -d'' -f 1)
+ jira_status=$(curl -s $jira_url/$jira|grep resolution-val|sed
-e s/.*class=\value\ //|cut -d'' -f 1)br/
+ jira_status=$(curl -s $jira_url/$jira|grep resolution-val|sed
-e s/.*class=\value\ //|cut -d'' -f 1)
{code}
All but the top line of output are from parsing comments from Hadoop QA. I
think this is an edge case that will only to patches against that section of
dev-support/rebase_all_git_branches.sh. However, I'm not sure the cleanest way
to fix it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
