Re: [VOTE] Release Apache Hadoop 2.2.0

[Suresh Srinivas]

+1 (binding)


On Mon, Oct 7, 2013 at 12:00 AM, Arun C Murthy  wrote:

> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.2.0 that I would like
> to get released - this release fixes a small number of bugs and some
> protocol/api issues which should ensure they are now stable and will not
> change in hadoop-2.x.
>
> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.2.0-rc0
> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> P.S.: Thanks to Colin, Andrew, Daryn, Chris and others for helping nail
> down the symlinks-related issues. I'll release note the fact that we have
> disabled it in 2.2. Also, thanks to Vinod for some heavy-lifting on the
> YARN side in the last couple of weeks.
>
>
>
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>



-- 
http://hortonworks.com/download/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Hbase to hive

[Shekhar Sharma]

You can very well port the the data to from Hbase to hive using
hive-hbase handler..But if you want to do it on specific criteria then
you might have to write the code to pull the data from hbase and put
in hive..

Its pretty straightforward to reflect the data in HBase into Hive
table..I have really not tried writing an intermediatory code to
filter the data and put in hive..but since we have filters in hbase i
think it might be easy to do it..


Regards,
Som Shekhar Sharma
+91-8197243810


On Tue, Oct 15, 2013 at 3:11 AM, abhisekgun...@yahoo.com
 wrote:
> Hi All,
>
> Can somebody suggest a solution on my problem below.
>
> 1. The data is maintained and constantly updated in hbase.
>
> 2. The user wants to query the data from hive.
>
> The requirement is to write a programs that can move data from hbase to hive 
> based on certain criteria.
>
> Please advice.
>
> Thanks,
>
> Abhishek
>
> Sent from my HTC
>

Re: streaming documentation in Hadoop 2?

[Sandy Ryza]

Thanks Eli, that explains it.


On Mon, Oct 14, 2013 at 3:30 PM, Eli Collins  wrote:

> This is MAPREDUCE-4282
>
> On Mon, Oct 14, 2013 at 3:28 PM, Sandy Ryza 
> wrote:
> > Doc existed in MR1 http://hadoop.apache.org/docs/stable/streaming.html,
> but
> > it looks like it and a bunch of other stuff (e.g. Rumen and the MapReduce
> > Tutorial) weren't ported over.
> >
> >
> > On Mon, Oct 14, 2013 at 3:20 PM, Eli Collins  wrote:
> >
> >> It probably just needs doc, I'd go ahead and file a jira for it. The
> >> wiki content here could be a good starting point.
> >>
> >> On Mon, Oct 14, 2013 at 2:56 PM, Sandy Ryza 
> >> wrote:
> >> > Hi All,
> >> >
> >> > I noticed that the hadoop streaming documentation does not exist in
> the
> >> > Hadoop 2 source tree, and also cannot be found on the internet.   Is
> this
> >> > on purpose?  I found this wiki page
> >> > http://wiki.apache.org/hadoop/HadoopStreaming - is that where doc is
> >> > supposed to go?  As this page isn't tied to a specific version, how
> does
> >> it
> >> > work if new options are added?
> >> >
> >> > thanks,
> >> > -Sandy
> >>
>

Re: streaming documentation in Hadoop 2?

[Eli Collins]

This is MAPREDUCE-4282

On Mon, Oct 14, 2013 at 3:28 PM, Sandy Ryza  wrote:
> Doc existed in MR1 http://hadoop.apache.org/docs/stable/streaming.html, but
> it looks like it and a bunch of other stuff (e.g. Rumen and the MapReduce
> Tutorial) weren't ported over.
>
>
> On Mon, Oct 14, 2013 at 3:20 PM, Eli Collins  wrote:
>
>> It probably just needs doc, I'd go ahead and file a jira for it. The
>> wiki content here could be a good starting point.
>>
>> On Mon, Oct 14, 2013 at 2:56 PM, Sandy Ryza 
>> wrote:
>> > Hi All,
>> >
>> > I noticed that the hadoop streaming documentation does not exist in the
>> > Hadoop 2 source tree, and also cannot be found on the internet.   Is this
>> > on purpose?  I found this wiki page
>> > http://wiki.apache.org/hadoop/HadoopStreaming - is that where doc is
>> > supposed to go?  As this page isn't tied to a specific version, how does
>> it
>> > work if new options are added?
>> >
>> > thanks,
>> > -Sandy
>>

Re: streaming documentation in Hadoop 2?

[Sandy Ryza]

Doc existed in MR1 http://hadoop.apache.org/docs/stable/streaming.html, but
it looks like it and a bunch of other stuff (e.g. Rumen and the MapReduce
Tutorial) weren't ported over.


On Mon, Oct 14, 2013 at 3:20 PM, Eli Collins  wrote:

> It probably just needs doc, I'd go ahead and file a jira for it. The
> wiki content here could be a good starting point.
>
> On Mon, Oct 14, 2013 at 2:56 PM, Sandy Ryza 
> wrote:
> > Hi All,
> >
> > I noticed that the hadoop streaming documentation does not exist in the
> > Hadoop 2 source tree, and also cannot be found on the internet.   Is this
> > on purpose?  I found this wiki page
> > http://wiki.apache.org/hadoop/HadoopStreaming - is that where doc is
> > supposed to go?  As this page isn't tied to a specific version, how does
> it
> > work if new options are added?
> >
> > thanks,
> > -Sandy
>

Re: streaming documentation in Hadoop 2?

[Eli Collins]

It probably just needs doc, I'd go ahead and file a jira for it. The
wiki content here could be a good starting point.

On Mon, Oct 14, 2013 at 2:56 PM, Sandy Ryza  wrote:
> Hi All,
>
> I noticed that the hadoop streaming documentation does not exist in the
> Hadoop 2 source tree, and also cannot be found on the internet.   Is this
> on purpose?  I found this wiki page
> http://wiki.apache.org/hadoop/HadoopStreaming - is that where doc is
> supposed to go?  As this page isn't tied to a specific version, how does it
> work if new options are added?
>
> thanks,
> -Sandy

streaming documentation in Hadoop 2?

[Sandy Ryza]

Hi All,

I noticed that the hadoop streaming documentation does not exist in the
Hadoop 2 source tree, and also cannot be found on the internet.   Is this
on purpose?  I found this wiki page
http://wiki.apache.org/hadoop/HadoopStreaming - is that where doc is
supposed to go?  As this page isn't tied to a specific version, how does it
work if new options are added?

thanks,
-Sandy

Hbase to hive

[abhisekgun...@yahoo.com]

Hi All,

Can somebody suggest a solution on my problem below.

1. The data is maintained and constantly updated in hbase.

2. The user wants to query the data from hive.

The requirement is to write a programs that can move data from hbase to hive 
based on certain criteria.

Please advice.

Thanks,

Abhishek

Sent from my HTC

Re: [VOTE] Release Apache Hadoop 2.2.0

[Arpit Agarwal]

+1 (non-binding)

- Verified md5/SHA checksums
- Installed binary distribution on Centos 6.4
- Ran a few MapReduce jobs on a single-node cluster
- Copied files to/from the cluster using HDFS commands

No issues encountered.


On Mon, Oct 7, 2013 at 12:00 AM, Arun C Murthy  wrote:

> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.2.0 that I would like
> to get released - this release fixes a small number of bugs and some
> protocol/api issues which should ensure they are now stable and will not
> change in hadoop-2.x.
>
> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.2.0-rc0
> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> P.S.: Thanks to Colin, Andrew, Daryn, Chris and others for helping nail
> down the symlinks-related issues. I'll release note the fact that we have
> disabled it in 2.2. Also, thanks to Vinod for some heavy-lifting on the
> YARN side in the last couple of weeks.
>
>
>
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

[jira] [Created] (HADOOP-10046) Print a log message when SSL is enabled

[David S. Wang (JIRA)]

David S. Wang created HADOOP-10046:
--

 Summary: Print a log message when SSL is enabled
 Key: HADOOP-10046
 URL: https://issues.apache.org/jira/browse/HADOOP-10046
 Project: Hadoop Common
  Issue Type: Improvement
Affects Versions: 3.0.0
Reporter: David S. Wang
Priority: Trivial


It would be nice to have a log message that indicates that SSL is enabled in 
org.apache.hadoop.http.HttpServer.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Created] (HADOOP-10045) QueueMetrics 'maps_killed' and 'reduces_killed' are never incremented in branch-1

[Akira AJISAKA (JIRA)]

Akira AJISAKA created HADOOP-10045:
--

 Summary: QueueMetrics 'maps_killed' and 'reduces_killed' are never 
incremented in branch-1
 Key: HADOOP-10045
 URL: https://issues.apache.org/jira/browse/HADOOP-10045
 Project: Hadoop Common
  Issue Type: Bug
  Components: metrics
Affects Versions: 1.2.1
 Environment: branch-1
Reporter: Akira AJISAKA
Priority: Minor


In branch-1, maps/reduces_failed metric will be incremented when map/reduce 
task is failed or killed, otherwise maps/reduces_killed metric won't be 
incremented then.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Re: [VOTE] Release Apache Hadoop 2.2.0

[Jonathan Eagles]

+1. Compiled native on ubuntu and found that sample jobs on single node
cluster are working.


On Mon, Oct 7, 2013 at 2:00 AM, Arun C Murthy  wrote:

> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.2.0 that I would like
> to get released - this release fixes a small number of bugs and some
> protocol/api issues which should ensure they are now stable and will not
> change in hadoop-2.x.
>
> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.2.0-rc0
> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> P.S.: Thanks to Colin, Andrew, Daryn, Chris and others for helping nail
> down the symlinks-related issues. I'll release note the fact that we have
> disabled it in 2.2. Also, thanks to Vinod for some heavy-lifting on the
> YARN side in the last couple of weeks.
>
>
>
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

[jira] [Resolved] (HADOOP-10040) hadoop.cmd in UNIX format and would not run by default on Windows

[Chris Nauroth (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-10040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Nauroth resolved HADOOP-10040.


Resolution: Fixed

I have committed the changes described above.  I waited for the git sync and 
refreshed by git repos.  I don't see any problems.  I think this worked.

Luke, thanks again for help with this.

> hadoop.cmd in UNIX format and would not run by default on Windows
> -
>
> Key: HADOOP-10040
> URL: https://issues.apache.org/jira/browse/HADOOP-10040
> Project: Hadoop Common
>  Issue Type: Bug
>Reporter: Yingda Chen
>Assignee: Chris Nauroth
> Fix For: 3.0.0, 2.2.1
>
>
> The hadoop.cmd currently checked in into hadoop-common is in UNIX format, 
> same as most of other src files. However, the hadoop.cmd is meant to be used 
> on Windows only, the fact that it is in UNIX format makes it unrunnable as is 
> on Window platform.
> An exception shall be made on hadoop.cmd (and other cmd files for what 
> matters) to make sure they are in DOS format, for them to be runnable as is 
> when checked out from source repository.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Build failed in Jenkins: Hadoop-Common-trunk #921

[Apache Jenkins Server]

See 

Changes:

[sandy] YARN-305. Fair scheduler logs too many "Node offered to app" messages. 
(Lohit Vijayarenu via Sandy Ryza)

[sseth] MAPREDUCE-5329. Allow MR applications to use additional AuxServices, 
which are compatible with the default MapReduce shuffle. Contributed by Avner 
BenHanoch.

--
[...truncated 57618 lines...]
Adding reference: maven.local.repository
[DEBUG] Initialize Maven Ant Tasks
parsing buildfile 
jar:file:/home/jenkins/.m2/repository/org/apache/maven/plugins/maven-antrun-plugin/1.6/maven-antrun-plugin-1.6.jar!/org/apache/maven/ant/tasks/antlib.xml
 with URI = 
jar:file:/home/jenkins/.m2/repository/org/apache/maven/plugins/maven-antrun-plugin/1.6/maven-antrun-plugin-1.6.jar!/org/apache/maven/ant/tasks/antlib.xml
 from a zip file
parsing buildfile 
jar:file:/home/jenkins/.m2/repository/org/apache/ant/ant/1.8.1/ant-1.8.1.jar!/org/apache/tools/ant/antlib.xml
 with URI = 
jar:file:/home/jenkins/.m2/repository/org/apache/ant/ant/1.8.1/ant-1.8.1.jar!/org/apache/tools/ant/antlib.xml
 from a zip file
Class org.apache.maven.ant.tasks.AttachArtifactTask loaded from parent loader 
(parentFirst)
 +Datatype attachartifact org.apache.maven.ant.tasks.AttachArtifactTask
Class org.apache.maven.ant.tasks.DependencyFilesetsTask loaded from parent 
loader (parentFirst)
 +Datatype dependencyfilesets org.apache.maven.ant.tasks.DependencyFilesetsTask
Setting project property: test.build.dir -> 

Setting project property: test.exclude.pattern -> _
Setting project property: hadoop.assemblies.version -> 3.0.0-SNAPSHOT
Setting project property: test.exclude -> _
Setting project property: distMgmtSnapshotsId -> apache.snapshots.https
Setting project property: project.build.sourceEncoding -> UTF-8
Setting project property: java.security.egd -> file:///dev/urandom
Setting project property: distMgmtSnapshotsUrl -> 
https://repository.apache.org/content/repositories/snapshots
Setting project property: distMgmtStagingUrl -> 
https://repository.apache.org/service/local/staging/deploy/maven2
Setting project property: avro.version -> 1.7.4
Setting project property: test.build.data -> 

Setting project property: commons-daemon.version -> 1.0.13
Setting project property: hadoop.common.build.dir -> 

Setting project property: testsThreadCount -> 4
Setting project property: maven.test.redirectTestOutputToFile -> true
Setting project property: jdiff.version -> 1.0.9
Setting project property: distMgmtStagingName -> Apache Release Distribution 
Repository
Setting project property: project.reporting.outputEncoding -> UTF-8
Setting project property: build.platform -> Linux-i386-32
Setting project property: protobuf.version -> 2.5.0
Setting project property: failIfNoTests -> false
Setting project property: protoc.path -> ${env.HADOOP_PROTOC_PATH}
Setting project property: jersey.version -> 1.9
Setting project property: distMgmtStagingId -> apache.staging.https
Setting project property: distMgmtSnapshotsName -> Apache Development Snapshot 
Repository
Setting project property: ant.file -> 

[DEBUG] Setting properties with prefix: 
Setting project property: project.groupId -> org.apache.hadoop
Setting project property: project.artifactId -> hadoop-common-project
Setting project property: project.name -> Apache Hadoop Common Project
Setting project property: project.description -> Apache Hadoop Common Project
Setting project property: project.version -> 3.0.0-SNAPSHOT
Setting project property: project.packaging -> pom
Setting project property: project.build.directory -> 

Setting project property: project.build.outputDirectory -> 

Setting project property: project.build.testOutputDirectory -> 

Setting project property: project.build.sourceDirectory -> 

Setting project property: project.build.testSourceDirectory -> 

Setting project property: localRepository ->id: local
  url: file:///home/jenkins/.m2/repository/
   layout: none
Setting project property: settings.localRepository -> 
/home/jenkins/.m2/repository
Setting project propert