Normalizer in 1.2+?

2016-01-13 Thread Lars George 
Hi,

Just looking through the new properties and seeing this

  
hbase.normalizer.enabled
false
If set to true, Master will try to keep region size
  within each table approximately the same.
  

Searching both branches 1.2 and master reveals it is only used in a test to
enable it, but never anywhere else to check it is disabled. What am I
missing? Is the normalizer always on no matter what you set (with the
default "false" not working then)?

Lars

Re: Normalizer in 1.2+?

2016-01-13 Thread Ted Yu 
There is RegionNormalizerTracker which tracks region normalizer state up in
ZK.

When user toggles normalizer switch through shell, the following method
in MasterRpcServices is called:

  public boolean normalizerSwitch(boolean on) {

boolean oldValue = master.getRegionNormalizerTracker().isNormalizerOn();

...

master.getRegionNormalizerTracker().setNormalizerOn(newValue);

In RegionNormalizerTracker :

  public boolean isNormalizerOn() {

byte [] upData = super.getData(false);

try {

  // if data in ZK is null, use default of on.

  return upData == null || parseFrom(upData).getNormalizerOn();

So I guess the config parameter hbase.normalizer.enabled can be dropped.


Cheers

On Wed, Jan 13, 2016 at 3:01 AM, Lars George  wrote:

> Hi,
>
> Just looking through the new properties and seeing this
>
>   
> hbase.normalizer.enabled
> false
> If set to true, Master will try to keep region size
>   within each table approximately the same.
>   
>
> Searching both branches 1.2 and master reveals it is only used in a test to
> enable it, but never anywhere else to check it is disabled. What am I
> missing? Is the normalizer always on no matter what you set (with the
> default "false" not working then)?
>
> Lars
>

[jira] [Created] (HBASE-15097) When the scan operation covered two regions,sometimes the final results have duplicated rows.

2016-01-13 Thread chenrongwei (JIRA) 
chenrongwei created HBASE-15097:
---

 Summary: When the scan operation covered two regions,sometimes the 
final results have duplicated rows.
 Key: HBASE-15097
 URL: https://issues.apache.org/jira/browse/HBASE-15097
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 1.1.2
 Environment: centos 6.5
hbase 1.1.2 
Reporter: chenrongwei
Assignee: chenrongwei
 Fix For: 1.1.2


When the scan operation‘s start key and end key covered two regions,the first 
region returned the rows which were beyond of its' end key.So,this finally 
leads to duplicated rows in the results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15098) Normalizer switch in configuration is not used

2016-01-13 Thread Lars George (JIRA) 
Lars George created HBASE-15098:
---

 Summary: Normalizer switch in configuration is not used
 Key: HBASE-15098
 URL: https://issues.apache.org/jira/browse/HBASE-15098
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 1.2.0
Reporter: Lars George
 Fix For: 2.0.0, 1.2.0, 1.2.1


The newly added global switch to enable the new normalizer functionality is 
never used apparently, meaning it is always on. The {{hbase-default.xml}} has 
this:

{noformat}
  
hbase.normalizer.enabled
false
If set to true, Master will try to keep region size
  within each table approximately the same.
  
{noformat}

But only a test class uses it to set the switch to "true". We should implement 
a proper {{if}} statement that checks this value and properly disables the 
feature cluster wide if not wanted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Normalizer in 1.2+?

2016-01-13 Thread Lars George 
Hi Ted,

Wait a minute... this means it is always on by default. Is this wanted?
This is a new feature and should be tested in the field first?

But you agree this is not right, so I will create a JIRA to track this.

Cheers,
Lars

On Wed, Jan 13, 2016 at 11:13 AM, Ted Yu  wrote:

> There is RegionNormalizerTracker which tracks region normalizer state up in
> ZK.
>
> When user toggles normalizer switch through shell, the following method
> in MasterRpcServices is called:
>
>   public boolean normalizerSwitch(boolean on) {
>
> boolean oldValue =
> master.getRegionNormalizerTracker().isNormalizerOn();
>
> ...
>
> master.getRegionNormalizerTracker().setNormalizerOn(newValue);
>
> In RegionNormalizerTracker :
>
>   public boolean isNormalizerOn() {
>
> byte [] upData = super.getData(false);
>
> try {
>
>   // if data in ZK is null, use default of on.
>
>   return upData == null || parseFrom(upData).getNormalizerOn();
>
> So I guess the config parameter hbase.normalizer.enabled can be dropped.
>
>
> Cheers
>
> On Wed, Jan 13, 2016 at 3:01 AM, Lars George 
> wrote:
>
> > Hi,
> >
> > Just looking through the new properties and seeing this
> >
> >   
> > hbase.normalizer.enabled
> > false
> > If set to true, Master will try to keep region size
> >   within each table approximately the same.
> >   
> >
> > Searching both branches 1.2 and master reveals it is only used in a test
> to
> > enable it, but never anywhere else to check it is disabled. What am I
> > missing? Is the normalizer always on no matter what you set (with the
> > default "false" not working then)?
> >
> > Lars
> >
>

[jira] [Created] (HBASE-15100) Master WALProcs still never clean up

2016-01-13 Thread Elliott Clark (JIRA) 
Elliott Clark created HBASE-15100:
-

 Summary: Master WALProcs still never clean up
 Key: HBASE-15100
 URL: https://issues.apache.org/jira/browse/HBASE-15100
 Project: HBase
  Issue Type: Bug
  Components: master
Affects Versions: 1.2.0
Reporter: Elliott Clark


{code}
bin/hdfs dfs -ls /hbase/MasterProcWALs | wc -l
218631
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: branch-1.1.0 showed back up

2016-01-13 Thread Nick Dimiduk 
INFRA-10736 was closed and I have successfully deleted this errant branch.

On Sun, Nov 8, 2015 at 5:27 PM, Nick Dimiduk  wrote:

> Rgr that: https://issues.apache.org/jira/browse/INFRA-10736
>
> On Sun, Nov 8, 2015 at 5:05 PM, Andrew Purtell 
> wrote:
>
>> Nope. Infra has disallowed force pushing and deletes. Please open an
>> INFRA ticket to clean up after it.
>>
>> Everyone, please be extra careful when you push.
>>
>> > On Nov 8, 2015, at 5:00 PM, Nick Dimiduk  wrote:
>> >
>> > Heya devs,
>> >
>> > Looks like someone pushed branch-1.1.0 back to our repo. It has only a
>> > single commit, which is already on branch-1.1. I tried to delete it, but
>> > the operation failed as "prohibited". Someone with the correct karma
>> mind
>> > deleting this one?
>> >
>> > Thanks,
>> > Nick
>> >
>> > $ git push apache-rw :branch-1.1.0
>> > remote: error: denying ref deletion for refs/heads/branch-1.1.0
>> > To https://git-wip-us.apache.org/repos/asf/hbase.git
>> > ! [remote rejected] branch-1.1.0 (deletion prohibited)
>> > error: failed to push some refs to '
>> > https://git-wip-us.apache.org/repos/asf/hbase.git'
>>
>
>

[jira] [Created] (HBASE-15103) hadoopcheck test should provide diff file showing what's new

2016-01-13 Thread Ted Yu (JIRA) 
Ted Yu created HBASE-15103:
--

 Summary: hadoopcheck test should provide diff file showing what's 
new
 Key: HBASE-15103
 URL: https://issues.apache.org/jira/browse/HBASE-15103
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Priority: Minor


Currently developer has to go to 'build artifacts' folder to read output from 
hadoopcheck test.
e.g.
https://builds.apache.org/job/PreCommit-HBASE-Build/98/artifact/patchprocess/patch-javac-2.6.1.txt

hadoopcheck test should provide diff file showing what exactly is new

Thanks to Sean for offline discussion



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15108) TestReplicationAdmin failed on branch-1.0

2016-01-13 Thread Heng Chen (JIRA) 
Heng Chen created HBASE-15108:
-

 Summary: TestReplicationAdmin failed on branch-1.0 
 Key: HBASE-15108
 URL: https://issues.apache.org/jira/browse/HBASE-15108
 Project: HBase
  Issue Type: Bug
Reporter: Heng Chen


I notice it on HBASE-15095.

See
https://builds.apache.org/job/PreCommit-HBASE-Build/95/artifact/patchprocess/patch-unit-hbase-server-jdk1.8.0.txt



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15101) Leak References to StoreFile.Reader after HBASE-13082

2016-01-13 Thread deepankar (JIRA) 
deepankar created HBASE-15101:
-

 Summary: Leak References to StoreFile.Reader after HBASE-13082
 Key: HBASE-15101
 URL: https://issues.apache.org/jira/browse/HBASE-15101
 Project: HBase
  Issue Type: Bug
  Components: HFile, io
Affects Versions: 2.0.0
Reporter: deepankar


We observed this production that after a region server dies there are huge 
number of hfiles in that region for the region server running the version with 
HBASE-13082, In the doc it is given that it is expected to happen, but we found 
a one place where scanners are not being closed. If the scanners are not closed 
their references are not decremented and that is leading to the issue of huge 
number of store files not being finalized

All I was able to find is in the selectScannersFrom, where we discard some of 
the scanners and we are not closing them. I am attaching a patch for that.

Also to avoid these issues should the files that are done be logged and 
finalized (moved to archive) as a part of region close operation. This will 
solve any leaks that can happen and does not cause any dire consequences?











--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15102) HeapMemoryTuner can "overtune" memstore size and suddenly drop it into blocking zone

2016-01-13 Thread Ashu Pachauri (JIRA) 
Ashu Pachauri created HBASE-15102:
-

 Summary: HeapMemoryTuner can "overtune" memstore size and suddenly 
drop it into blocking zone
 Key: HBASE-15102
 URL: https://issues.apache.org/jira/browse/HBASE-15102
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Affects Versions: 2.0.0, 1.2.0, 1.3.0, 1.2.1
Reporter: Ashu Pachauri
Assignee: Ashu Pachauri
Priority: Critical


DefaultHeapMemoryTuner often resets the maximum step size for tuning to 8% of 
total heap size. Often, when the size of memstore is to be decreased while 
tuning, the 8% tuning can suddenly drop the memstore size below the low water 
mark of the previous memstore size (which could potentially be  the used size 
of the memstore)
This is problematic because suddenly it blocks all the updates by suddenly 
causing a situation where memstore used size is above high water mark. This has 
a very bad performance impact on an otherwise fine HBase cluster. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15106) Procedure V2 - Procedure Queue pass Procedure for better debuggability

2016-01-13 Thread Matteo Bertozzi (JIRA) 
Matteo Bertozzi created HBASE-15106:
---

 Summary: Procedure V2 - Procedure Queue pass Procedure for better 
debuggability
 Key: HBASE-15106
 URL: https://issues.apache.org/jira/browse/HBASE-15106
 Project: HBase
  Issue Type: Sub-task
  Components: proc-v2
Affects Versions: 2.0.0, 1.3.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 2.0.0, 1.3.0


Changes the various acquire/release methods to take the Procedure as argument.
That allows better debuggability. 
(The patch it is just a refactor, it does not introduce any new thing)

https://reviews.apache.org/r/42271/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-14944) Minimize or eliminate source incompatible changes due to HBASE-14605, HBASE-14631, and HBASE-14655

2016-01-13 Thread Nick Dimiduk (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-14944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nick Dimiduk resolved HBASE-14944.
--
   Resolution: Invalid
Fix Version/s: (was: 1.0.4)
   (was: 1.1.3)

Absence of response taken as no further opinion. Resolving as 'Invalid'.

> Minimize or eliminate source incompatible changes due to HBASE-14605, 
> HBASE-14631, and HBASE-14655
> --
>
> Key: HBASE-14944
> URL: https://issues.apache.org/jira/browse/HBASE-14944
> Project: HBase
>  Issue Type: Sub-task
>Affects Versions: 1.1.3, 1.0.4
>Reporter: Andrew Purtell
>Priority: Blocker
>
> Minimize or eliminate source incompatible changes due to HBASE-14605, 
> HBASE-14631, and HBASE-14655. The changes are due to abstract method 
> additions to carry the correct (not current) {{User}} through to where 
> authoritative decisions or audit is performed.
> HBASE-14605 introduces source incompatible changes to the SplitTransaction 
> interface:
> - Adds abstract method execute(Server, RegionServerServices, User)
> - Adds abstract method rollback(Server, RegionServerServices, User)
> HBASE-14631 introduces source incompatible changes to the 
> RegionMergeTransaction interface:
> - Adds abstract method execute(Server, RegionServerServices, User)
> - Adds abstract method rollback(Server, RegionServerServices, User)
> HBASE-14655 introduces source incompatible changes to the Store interface:
> - Adds abstract method compact(CompactionContext, 
> CompactionThroughputController, User)
> - Adds abstract method requestCompaction( int, CompactionRequest, User)
> Default implementations are provided for binary compatibility but 
> implementors of these interface won't recompile until implementations of the 
> new methods are added.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15107) Procedure V2 - Procedure Queue with Regions

2016-01-13 Thread Matteo Bertozzi (JIRA) 
Matteo Bertozzi created HBASE-15107:
---

 Summary: Procedure V2 - Procedure Queue with Regions
 Key: HBASE-15107
 URL: https://issues.apache.org/jira/browse/HBASE-15107
 Project: HBase
  Issue Type: Sub-task
  Components: proc-v2
Affects Versions: 2.0.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 2.0.0


Adds the "region locking" that will be used to perform assign/unassign, 
split/merge operations.
An operation take the xlock on the regions is working on, all the other 
procedures will be suspended (removed from the runnable queue) and resumed (put 
back in the runnable queue) when the operation that has the lock on the region 
is completed.

https://reviews.apache.org/r/42213/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15104) IntegrationTestAcitGuarantees occasionally fails when trying to cleanup

2016-01-13 Thread huaxiang sun (JIRA) 
huaxiang sun created HBASE-15104:


 Summary: IntegrationTestAcitGuarantees occasionally fails when 
trying to cleanup
 Key: HBASE-15104
 URL: https://issues.apache.org/jira/browse/HBASE-15104
 Project: HBase
  Issue Type: Bug
  Components: integration tests
Affects Versions: 2.0.0, 1.2.0
Reporter: huaxiang sun


IntegrationTestAcidGuarantees fails when trying to cleanup with 
NotServerRegionExceptions giving up (after 36 attempts) .

5/11/09 09:19:24 INFO client.AsyncProcess: #33, waiting for some tasks to 
finish. Expected max=0, tasksInProgress=9
15/11/09 09:19:33 INFO client.AsyncProcess: #45, table=TestAcidGuarantees, 
attempt=10/35 failed=1ops, last exception: 
org.apache.hadoop.hbase.NotServingRegionException: 
org.apache.hadoop.hbase.NotServingRegionException: Region 
TestAcidGuarantees,test_row_1,1447089367019.032439ef4f3353cb894d20337ba043bc. 
is not online on node-4.internal,22101,1447089152259
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2786)
at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.getRegion(RSRpcServices.java:922)
at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:1893)
at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at 
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:745)
...
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed 
after attempts=36, exceptions:
Mon Nov 09 09:19:53 PST 2015, null, java.net.SocketTimeoutException: 
callTimeout=6, callDuration=68104: row 'test_row_1'

Looked at the RS log, the following exception is found:
2015-11-10 10:07:49,091 ERROR 
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open of 
region=TestAcidGuarantees,,1447177733243.f1be6b850fe3958c5c9b5e330b5dfb00., 
starting to roll back the global memstore size.
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.RuntimeException: 
java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzoCodec
at 
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:102)
at 
org.apache.hadoop.hbase.regionserver.HRegion.checkCompressionCodecs(HRegion.java:6011)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5995)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5967)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5938)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5894)
at 
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:5845)
at 
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:356)
at 
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:126)
at 
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15105) Procedure V2 - Procedure Queue with Namespaces

2016-01-13 Thread Matteo Bertozzi (JIRA) 
Matteo Bertozzi created HBASE-15105:
---

 Summary: Procedure V2 - Procedure Queue with Namespaces
 Key: HBASE-15105
 URL: https://issues.apache.org/jira/browse/HBASE-15105
 Project: HBase
  Issue Type: Sub-task
  Components: proc-v2
Affects Versions: 2.0.0, 1.3.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
 Fix For: 2.0.0, 1.3.0


Replace the rwlock in TableNamespaceManager with the locking/scheduling from 
the Master procedure queue (see HBASE-14837)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-15059) Allow 0.94 to compile against Hadoop 2.7.x

2016-01-13 Thread Sean Busbey (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-15059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey reopened HBASE-15059:
-

reopening, because I don't think this has all the docs needed.

When attempting to build I get a failure from the enforcer plugin because 
hadoop 2.7 is jdk7+ only:

{{mvn clean site install assembly:single -Prelease -Dhadoop.profile=2.7 
-Dlicense.debug.print.included=true}} (last bit is to avoid license failure 
while fixing HBASE-14213)
{code}
[INFO] --- maven-enforcer-plugin:1.0.1:enforce (enforce) @ hbase ---
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-yarn-server-common:jar:2.7.1:compile contains 
org/apache/hadoop/yarn/lib/package-info.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-yarn-api:jar:2.7.1:compile contains 
org/apache/hadoop/yarn/api/ApplicationBaseProtocol.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-annotations:jar:2.7.1:compile contains 
org/apache/hadoop/classification/InterfaceAudience$LimitedPrivate.class 
targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-mapreduce-client-app:jar:2.7.1:compile contains 
org/apache/hadoop/CustomOutputCommitter.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-yarn-server-applicationhistoryservice:jar:2.7.1:compile
 contains 
org/apache/hadoop/yarn/proto/YarnServerTimelineServerRecoveryProtos$1.class 
targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-mapreduce-client-core:jar:2.7.1:compile contains 
org/apache/hadoop/filecache/DistributedCache.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet jdk.tools:jdk.tools:jar:1.7:system contains 
sun/tools/asm/ArrayData.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-common:test-jar:tests:2.7.1:compile contains 
org/apache/hadoop/cli/CLITestHelper$TestConfigFileParser.class targeted to JDK 
1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-hdfs:jar:2.7.1:compile contains 
org/apache/hadoop/fs/BlockStorageLocation.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-yarn-server-nodemanager:jar:2.7.1:compile contains 
org/apache/hadoop/yarn/proto/LocalizationProtocol$1.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-mapreduce-client-common:jar:2.7.1:compile contains 
org/apache/hadoop/mapred/LocalClientProtocolProvider.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.1:compile contains 
org/apache/hadoop/yarn/proto/YarnSecurityTestClientAMTokenProtos$1.class 
targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:2.7.1:compile contains 
org/apache/hadoop/mapred/ClientCache$1.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-mapreduce-client-shuffle:jar:2.7.1:compile contains 
org/apache/hadoop/mapred/FadvisedChunkedFile.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-hdfs:test-jar:tests:2.7.1:compile contains 
org/apache/hadoop/cli/CLITestCmdDFS.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-mapreduce-client-jobclient:test-jar:tests:2.7.1:compile
 contains org/apache/hadoop/cli/CLITestCmdMR.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:2.7.1:compile contains 
org/apache/hadoop/yarn/server/webproxy/amfilter/AmFilterInitializer.class 
targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-common:jar:2.7.1:compile contains 
org/apache/hadoop/conf/Configurable.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-mapreduce-client-hs:jar:2.7.1:compile contains 
org/apache/hadoop/mapreduce/v2/hs/CachedHistoryStorage$1.class targeted to JDK 
1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-auth:jar:2.7.1:compile contains 
org/apache/hadoop/security/authentication/client/AuthenticatedURL$Token.class 
targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-yarn-server-tests:test-jar:tests:2.7.1:compile 
contains org/apache/hadoop/yarn/server/ContainerTokenIdentifierForTest.class 
targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-yarn-client:jar:2.7.1:compile contains 
org/apache/hadoop/yarn/client/api/AHSClient.class targeted to JDK 1.7
[INFO] Restricted to JDK 1.6 yet 
org.apache.hadoop:hadoop-yarn-common:jar:2.7.1:compile contains 
org/apache/hadoop/yarn/api/ApplicationClientProtocolPB.class targeted to JDK 1.7
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.EnforceBytecodeVersion 
failed with message:
HBase has unsupported dependencies.
  HBase requires that all dependencies be compiled with

Successful: HBase Generate Website

2016-01-13 Thread Apache Jenkins Server 
Build status: Successful

If successful, the website and docs have been generated. If failed, skip to the 
bottom of this email.

Use the following commands to download the patch and apply it to a clean branch 
based on origin/asf-site. If you prefer to keep the hbase-site repo around 
permanently, you can skip the clone step.

  git clone https://git-wip-us.apache.org/repos/asf/hbase-site.git
  cd hbase-site
  wget -O- 
https://builds.apache.org/job/hbase_generate_website/99/artifact/website.patch.zip
 | funzip > 5e89ebcc2f37366fd1efa7f2317a60552cf9e2b8.patch
  git fetch
  git checkout -b asf-site-5e89ebcc2f37366fd1efa7f2317a60552cf9e2b8 
origin/asf-site
  git am 5e89ebcc2f37366fd1efa7f2317a60552cf9e2b8.patch

At this point, you can preview the changes by opening index.html or any of the 
other HTML pages in your local 
asf-site-5e89ebcc2f37366fd1efa7f2317a60552cf9e2b8 branch, and you can review 
the differences by running:

  git diff origin/asf-site

There are lots of spurious changes, such as timestamps and CSS styles in 
tables. To see a list of files that have been added, deleted, renamed, changed 
type, or are otherwise interesting, use the following command:

  git diff --name-status --diff-filter=ADCRTXUB origin/asf-site

To see only files that had 10 or more lines changed:

  git diff --stat origin/asf-site | grep -Ev "\|\s+\ [1-9]\ [\+-]+$"

When you are satisfied, publish your changes to origin/asf-site using this 
command:

  git push origin asf-site-5e89ebcc2f37366fd1efa7f2317a60552cf9e2b8:asf-site

Changes take a couple of minutes to be propagated. You can then remove your 
asf-site-5e89ebcc2f37366fd1efa7f2317a60552cf9e2b8 branch:

  git checkout asf-site && git branch -d 
asf-site-5e89ebcc2f37366fd1efa7f2317a60552cf9e2b8



If failed, see https://builds.apache.org/job/hbase_generate_website/99/console

Re: ASF git repository policy update

2016-01-13 Thread Sean Busbey 
no, definitely not. the "rel" space is reserved for PMC-approved releases.
There's no foundation-backed need to preserve candidates that didn't get
approved.

On Wed, Jan 13, 2016 at 9:32 AM, Nick Dimiduk  wrote:

> Should we be pushing RC tags into the rel space as well as release tags?
>
> On Tuesday, January 12, 2016,  wrote:
>
> > +1
> >
> >
> >   From: Enis Söztutar >
> >  To: "dev@hbase.apache.org "  > >
> >  Sent: Monday, January 11, 2016 11:17 AM
> >  Subject: Re: ASF git repository policy update
> >
> > +1 on re-tagging releases under rel/ and using it going forward.
> >
> > Enis
> >
> > On Mon, Jan 11, 2016 at 11:05 AM, Sean Busbey  > > wrote:
> >
> > > just to confirm, 'master' is no longer protected?
> > >
> > > On Mon, Jan 11, 2016 at 12:56 PM, Andrew Purtell  > >
> > > wrote:
> > >
> > > > Infa recently announced on infrastructure@ that, following direction
> > > from
> > > > the Board, ASF git is now allowing force pushes and branch/tag
> deletion
> > > > again. In accordance with the guidance given by the Board there are
> > some
> > > > policy changes you should be aware of:
> > > >
> > > >- First, if a forced commit is pushed, the subsequent commit email
> > to
> > > >commits@ will contain '[Forced Update!]' in the subject line. The
> > > hope
> > > >here is that it draws extra attention to the situation for a
> project
> > > >community tobe aware, and take appropriate action if needed.
> > > >
> > > >- Second, the 'protected' Git ref-space is now refs/tags/rel/* .
> Any
> > > >tags under rel/, can be assured to carry the complete immutable
> > commit
> > > >history which lead up to the commit. This provides the provenance
> > that
> > > > the
> > > >ASF needs for releases. Otherwise projects can mold their
> repository
> > > ref
> > > >structure any way they see fit. When a release vote is successful
> > part
> > > > of
> > > >the release process should include tagging the voted upon commit
> SHA
> > > > under
> > > >rel/ to make it indelible. ('# git tag rel/v15.4.2 ' or something
> > > > similar.)
> > > >
> > > > Let's incorporate the second point into our release process
> > immediately.
> > > We
> > > > can also go back and add corresponding tags in refs/tags/rel/ for all
> > of
> > > > our past releases. Unless there are any objections I will take care
> of
> > > this
> > > > housekeeping later this week.
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > >- Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> > > >
> > >
> > >
> > >
> > > --
> > > Sean
> > >
> >
> >
> >
>



-- 
Sean

[jira] [Reopened] (HBASE-14213) Ensure ASF policy compliant headers and correct LICENSE and NOTICE files in artifacts for 0.94

2016-01-13 Thread Sean Busbey (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HBASE-14213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey reopened HBASE-14213:
-

> Ensure ASF policy compliant headers and correct LICENSE and NOTICE files in 
> artifacts for 0.94
> --
>
> Key: HBASE-14213
> URL: https://issues.apache.org/jira/browse/HBASE-14213
> Project: HBase
>  Issue Type: Task
>  Components: build
>Reporter: Nick Dimiduk
>Assignee: Sean Busbey
>Priority: Blocker
> Fix For: 0.94.28
>
> Attachments: 14213-LICENSE.txt, 14213-addendum.txt, 
> 14213-addendum2.txt, 14213-combined.txt, 14213-part1.txt, 14213-part2.txt, 
> 14213-part3.sh, 14213-part4.sh, 14213-part5.sh, HBASE-14213.1.0.94.patch
>
>
> From tail of thread on HBASE-14085, opening a backport ticket for 0.94. Took 
> the liberty of assigning to [~busbey].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: ASF git repository policy update

2016-01-13 Thread Nick Dimiduk 
Should we be pushing RC tags into the rel space as well as release tags?

On Tuesday, January 12, 2016,  wrote:

> +1
>
>
>   From: Enis Söztutar >
>  To: "dev@hbase.apache.org "  >
>  Sent: Monday, January 11, 2016 11:17 AM
>  Subject: Re: ASF git repository policy update
>
> +1 on re-tagging releases under rel/ and using it going forward.
>
> Enis
>
> On Mon, Jan 11, 2016 at 11:05 AM, Sean Busbey  > wrote:
>
> > just to confirm, 'master' is no longer protected?
> >
> > On Mon, Jan 11, 2016 at 12:56 PM, Andrew Purtell  >
> > wrote:
> >
> > > Infa recently announced on infrastructure@ that, following direction
> > from
> > > the Board, ASF git is now allowing force pushes and branch/tag deletion
> > > again. In accordance with the guidance given by the Board there are
> some
> > > policy changes you should be aware of:
> > >
> > >- First, if a forced commit is pushed, the subsequent commit email
> to
> > >commits@ will contain '[Forced Update!]' in the subject line. The
> > hope
> > >here is that it draws extra attention to the situation for a project
> > >community tobe aware, and take appropriate action if needed.
> > >
> > >- Second, the 'protected' Git ref-space is now refs/tags/rel/* . Any
> > >tags under rel/, can be assured to carry the complete immutable
> commit
> > >history which lead up to the commit. This provides the provenance
> that
> > > the
> > >ASF needs for releases. Otherwise projects can mold their repository
> > ref
> > >structure any way they see fit. When a release vote is successful
> part
> > > of
> > >the release process should include tagging the voted upon commit SHA
> > > under
> > >rel/ to make it indelible. ('# git tag rel/v15.4.2 ' or something
> > > similar.)
> > >
> > > Let's incorporate the second point into our release process
> immediately.
> > We
> > > can also go back and add corresponding tags in refs/tags/rel/ for all
> of
> > > our past releases. Unless there are any objections I will take care of
> > this
> > > housekeeping later this week.
> > >
> > > --
> > > Best regards,
> > >
> > >- Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
> >
> >
> > --
> > Sean
> >
>
>
>

Re: ASF git repository policy update

2016-01-13 Thread Nick Dimiduk 
ACK

On Wednesday, January 13, 2016, Sean Busbey  wrote:

> no, definitely not. the "rel" space is reserved for PMC-approved releases.
> There's no foundation-backed need to preserve candidates that didn't get
> approved.
>
> On Wed, Jan 13, 2016 at 9:32 AM, Nick Dimiduk  > wrote:
>
> > Should we be pushing RC tags into the rel space as well as release tags?
> >
> > On Tuesday, January 12, 2016, > wrote:
> >
> > > +1
> > >
> > >
> > >   From: Enis Söztutar 
> >
> > >  To: "dev@hbase.apache.org  " <
> dev@hbase.apache.org 
> > > >
> > >  Sent: Monday, January 11, 2016 11:17 AM
> > >  Subject: Re: ASF git repository policy update
> > >
> > > +1 on re-tagging releases under rel/ and using it going forward.
> > >
> > > Enis
> > >
> > > On Mon, Jan 11, 2016 at 11:05 AM, Sean Busbey  
> > > > wrote:
> > >
> > > > just to confirm, 'master' is no longer protected?
> > > >
> > > > On Mon, Jan 11, 2016 at 12:56 PM, Andrew Purtell <
> apurt...@apache.org 
> > > >
> > > > wrote:
> > > >
> > > > > Infa recently announced on infrastructure@ that, following
> direction
> > > > from
> > > > > the Board, ASF git is now allowing force pushes and branch/tag
> > deletion
> > > > > again. In accordance with the guidance given by the Board there are
> > > some
> > > > > policy changes you should be aware of:
> > > > >
> > > > >- First, if a forced commit is pushed, the subsequent commit
> email
> > > to
> > > > >commits@ will contain '[Forced Update!]' in the subject line.
> The
> > > > hope
> > > > >here is that it draws extra attention to the situation for a
> > project
> > > > >community tobe aware, and take appropriate action if needed.
> > > > >
> > > > >- Second, the 'protected' Git ref-space is now refs/tags/rel/* .
> > Any
> > > > >tags under rel/, can be assured to carry the complete immutable
> > > commit
> > > > >history which lead up to the commit. This provides the
> provenance
> > > that
> > > > > the
> > > > >ASF needs for releases. Otherwise projects can mold their
> > repository
> > > > ref
> > > > >structure any way they see fit. When a release vote is
> successful
> > > part
> > > > > of
> > > > >the release process should include tagging the voted upon commit
> > SHA
> > > > > under
> > > > >rel/ to make it indelible. ('# git tag rel/v15.4.2 ' or
> something
> > > > > similar.)
> > > > >
> > > > > Let's incorporate the second point into our release process
> > > immediately.
> > > > We
> > > > > can also go back and add corresponding tags in refs/tags/rel/ for
> all
> > > of
> > > > > our past releases. Unless there are any objections I will take care
> > of
> > > > this
> > > > > housekeeping later this week.
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > >- Andy
> > > > >
> > > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein
> > > > > (via Tom White)
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Sean
> > > >
> > >
> > >
> > >
> >
>
>
>
> --
> Sean
>