Re: hadoop-2 on windows How To ?
HDP 2.0 for Windows isnt out yet. HDInsight based upon 2.0 isnt out yet either. On Wed, Nov 20, 2013 at 8:56 AM, Joy George wrote: > May be my bad googling skills. I searched a lot but didn’t get a good > point which explains how to install Hadoop 2 in windows > > > > There is HDInsight for Windows 7 but it uses Hadoop 1.# versions. > > > > *Joy George K* > > *Jr.Architect* > > *Orion India Systems Pvt. Ltd.* > > *103, Tejomaya, L&T Tech Park, Infopark, Kusumagiri-PO, Kakkanad, Kochi - > 682030* > > Desk: *+91 484 6616384 <%2B91%20484%206616384> *| Cell: *+91 95444 00706 > <%2B91%2095444%2000706>* | Ext *384* | Email : > j...@orioninc.com<%7cem...@orioninc.com> > > www.orioninc.com | *Delivering trusted solutions to our clients for over > 2 decades* > > *From:* Nitin Pawar [mailto:nitinpawar...@gmail.com] > *Sent:* Tuesday, November 12, 2013 5:16 PM > > *To:* user@hadoop.apache.org > *Subject:* Re: hadoop-2 on windows How To ? > > > > it does support > > Please google it :) > > > > I can not promote a vendor on general mailing list. You can reach out > their forums or google it or tweet to their engineers :) > > > > > > > > On Tue, Nov 12, 2013 at 2:27 PM, Joy George wrote: > > Hartonworks HDP is currently using Hadoop 1.2 version. > > http://hortonworks.com/products/hdp-windows/ > > > > Are there any links which explain how to install Hadoop 2 in windows > machines? > > > > *From:* Nitin Pawar [mailto:nitinpawar...@gmail.com] > *Sent:* Monday, November 11, 2013 4:54 PM > *To:* user@hadoop.apache.org > *Subject:* Re: hadoop-2 on windows How To ? > > > > you do not ssh to run windows either its windows or linux. > > > > ssh is used to for only making it easy to install. hadoop does not use > ssh. > > > > All network communication in hadoop happen over tcp. > > > > I personally never installed hadoop on windows but there is this blog with > pictorial representation > > http://hortonworks.com/blog/installing-hadoop-on-windows/ > > > > On Mon, Nov 11, 2013 at 3:53 PM, sagar nikam > wrote: > > I read some latest features of hadoop-2.0 version, that it also runs > on windows system. > > I found some .cmd files along side with .sh files. > > I want to run them for windows. I have JDk-7 installed. > > I don't want to install openssh component in my windows (as this requires > cygwin to be installed, that have many other issues with regular command > prompts) > > Does without openssh can i run hadoop on windows ? > > any tutorials/blog showing latest hadoop installation on windows ? > > > > > > > > -- > Nitin Pawar > > > > Confidentiality Notice: This e-mail message, including any attachments, is > for the sole use of the intended recipient(s) and may contain confidential > and privileged information. Any unauthorized review, use, disclosure or > distribution is prohibited. If you are not the intended recipient, please > contact the sender by reply e-mail, delete and then destroy all copies of > the original message. > > > > > > -- > Nitin Pawar > > > Confidentiality Notice: This e-mail message, including any attachments, is > for the sole use of the intended recipient(s) and may contain confidential > and privileged information. Any unauthorized review, use, disclosure or > distribution is prohibited. If you are not the intended recipient, please > contact the sender by reply e-mail, delete and then destroy all copies of > the original message. > -- * Adam Diaz * Solution Engineer - Big Data -- Phone:919 609 4842 Email: ad...@hortonworks.com Website: http://www.hortonworks.com/ * Follow Us: * <http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> <http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> <http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> [image: photo] Latest From Our Blog: Using Hive to interact with HBase, Part 2 <http://hortonworks.com/blog/using-hive-to-interact-with-hbase-part-2/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: HDP 2.0 Install fails on repo unavailability
The servers have been very busy since the release. You probably just need
to try again.
On Oct 24, 2013 6:37 PM, "Jain, Prem" wrote:
> Folks,
>
> ** **
>
> Trying to install the newly release Hadoop 2.0 using Ambari. I am able to
> install Ambari, but when I try to install Hadoop 2.0 on rest of the
> cluster, the installation fails erroring on repo mirror unavailability.
> Not sure where I messed up.
>
> ** **
>
> Here are the error messages
>
> ** **
>
> *Output log from AMBARI during Installation*
>
> ** **
>
> notice:
> /Stage[1]/Hdp::Snappy::Package/Hdp::Snappy::Package::Ln[32]/Hdp::Exec[hdp::snappy::package::ln
> 32]/Exec[hdp::snappy::package::ln 32]/returns: executed successfully
>
> err:
> /Stage[main]/Hdp-hadoop/Hdp-hadoop::Package[hadoop]/Hdp::Package[hadoop
> 64]/Hdp::Package::Process_pkg[hadoop 64]/Package[hadoop-libhdfs]/ensure:
> change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0
> -y install hadoop-libhdfs' returned 1:
>
> ** **
>
> Error Downloading Packages:
>
> hadoop-2.2.0.2.0.6.0-76.el6.x86_64: failure:
> hadoop/hadoop-2.2.0.2.0.6.0-76.el6.x86_64.rpm from HDP-2.0.6: [Errno 256]
> No more mirrors to try.
>
> zookeeper-3.4.5.2.0.6.0-76.el6.noarch: failure:
> zookeeper/zookeeper-3.4.5.2.0.6.0-76.el6.noarch.rpm from HDP-2.0.6: [Errno
> 256] No more mirrors to try.
>
> hadoop-hdfs-2.2.0.2.0.6.0-76.el6.x86_64: failure:
> hadoop/hadoop-hdfs-2.2.0.2.0.6.0-76.el6.x86_64.rpm from HDP-2.0.6: [Errno
> 256] No more mirrors to try.
>
> ** **
>
> ** **
>
> err:
> /Stage[main]/Hdp-hadoop/Hdp-hadoop::Package[hadoop]/Hdp::Package[hadoop
> 64]/Hdp::Package::Process_pkg[hadoop 64]/Package[hadoop-lzo]/ensure: change
> from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y
> install hadoop-lzo' returned 1:
>
> ** **
>
> Error Downloading Packages:
>
> hadoop-2.2.0.2.0.6.0-76.el6.x86_64: failure:
> hadoop/hadoop-2.2.0.2.0.6.0-76.el6.x86_64.rpm from HDP-2.0.6: [Errno 256]
> No more mirrors to try.
>
> zookeeper-3.4.5.2.0.6.0-76.el6.noarch: failure:
> zookeeper/zookeeper-3.4.5.2.0.6.0-76.el6.noarch.rpm from HDP-2.0.6: [Errno
> 256] No more mirrors to try.
>
> ** **
>
> ** **
>
> notice:
> /Stage[2]/Hdp-hadoop::Initialize/Configgenerator::Configfile[hdfs-site]/File[/etc/hadoop/conf/hdfs-site.xml]/content:
> content changed '{md5}117224b1cf67c151f8a3d7ac0a157fa5' to
> '{md5}ba383a94bdde1a0b2eb5c59b1f5b61e7'
>
> notice:
> /Stage[2]/Hdp-hadoop::Initialize/Configgenerator::Configfile[capacity-scheduler]/File[/etc/hadoop/conf/capacity-scheduler.xml]/content:
> content changed '{md5}08d7e952b3e2d4fd5a2a880dfcd3a2df' to
> '{md5}dd3922fc27f72cd78cf2b47f57351b08'
>
> notice:
> /Stage[2]/Hdp-hadoop::Initialize/Configgenerator::Configfile[core-site]/File[/etc/hadoop/conf/core-site.xml]/content:
> content changed '{md5}76d06ebce1310be7e65ae0c7e8c3068a' to
> '{md5}1b626aa016a6f916271f67f3aa22cbbb'
>
> err:
> /Stage[main]/Hdp-hadoop/Hdp-hadoop::Package[hadoop]/Hdp::Package[hadoop
> 64]/Hdp::Package::Process_pkg[hadoop 64]/Package[hadoop]/ensure: change
> from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y
> install hadoop' returned 1:
>
> ** **
>
> Error Downloading Packages:
>
> hadoop-2.2.0.2.0.6.0-76.el6.x86_64: failure:
> hadoop/hadoop-2.2.0.2.0.6.0-76.el6.x86_64.rpm from HDP-2.0.6: [Errno 256]
> No more mirrors to try.
>
> zookeeper-3.4.5.2.0.6.0-76.el6.noarch: failure:
> zookeeper/zookeeper-3.4.5.2.0.6.0-76.el6.noarch.rpm from HDP-2.0.6: [Errno
> 256] No more mirrors to try.
>
> ** **
>
> ** **
>
> notice:
> /Stage[main]/Hdp-hadoop/Hdp-hadoop::Package[hadoop]/Hdp::Package[hadoop
> 64]/Hdp::Package::Process_pkg[hadoop 64]/Anchor[hdp::package::hadoop
> 64::end]: Dependency Package[hadoop-libhdfs] has failures: true
>
> notice:
> /Stage[main]/Hdp-hadoop/Hdp-hadoop::Package[hadoop]/Hdp::Package[hadoop
> 64]/Hdp::Package::Process_pkg[hadoop 64]/Anchor[hdp::package::hadoop
> 64::end]: Dependency Package[hadoop-lzo] has failures: true
>
> notice:
> /Stage[main]/Hdp-hadoop/Hdp-hadoop::Package[hadoop]/Hdp::Package[hadoop
> 64]/Hdp::Package::Process_pkg[hadoop 64]/Anchor[hdp::package::hadoop
> 64::end]: Dependency Package[hadoop] has failures: true
>
> notice:
> /Stage[main]/Hdp-hadoop/Hdp-hadoop::Package[hadoop]/Anchor[hdp-hadoop::package::helper::end]:
> Dependency Package[hadoop-libhdfs] has failures: true
>
> notice:
> /Stage[main]/Hdp-hadoop/Hdp-hadoop::Package[hadoop]/Anchor[hdp-hadoop::package::helper::end]:
> Dependency Package[hadoop-lzo] has failures: true
>
> ** **
>
> *Repo :*
>
> ** **
>
> http://public-repo-1.hortonworks.com/ambari/centos6/1.x/updates/1.4.1.25**
> **
>
> NoSuchKeyThe specified key does not
> exist.ambari/centos6/1.x/updates/1.4.1.25
> 4693487CE703DB53B87iAHvcpH7im27HOuEKBJ0F+qPFf+7aXuTe+O7OhLb9WscyxTbV/2yUPXO+KPOJ
>
> ** **
>
> ** **
>
> *Manual install:*
>
> ** **
>
> [root@dn5 ~]# /usr/bin/yum -d 0 -e 0 -y install ha
Re: Documentation on installing Hadoop 2.2.0 on Microsoft Windows
No Cygwin is involved...We have the installation via MSI clearly documented on our site. Python, Visual C++, JDK and .Net framework. This work was done in conjunction with Microsoft who is not in the business of supporting Cygwin. Hadoop is Java and Java runs on windows. On Thu, Oct 24, 2013 at 9:20 AM, DSuiter RDX wrote: > It was my understanding that HortonWorks depended on CygWin (UNIX > emulation on Windows) for most of the "Bigtop" family of tools - Hadoop > core, MapReduce, etc. - so, you will probably make all your configuration > files in Windows, since XML is agnostic, and can develop in Windows, since > JARs and other Java constructs are agnostic by design, but when things are > actually happening on your cluster, CygWin is in the middle. For a > framework like Hadoop, I question the wisdom of deciding to use a host > environment that uses more memory to begin with, then adding an emulation > layer that complicates things and takes still more memory (since most > Hadoop constraints are memory-based) simply for the convenience of being > able to use my mouse. > > If you have a competent *NIX admin, you may consider the benefits of using > Hadoop in Linux/UNIX, and leveraging the many web-based management tools > (usually will come with a Hadoop distribution) and 3rd-party development > tools from Informatica or Pentaho (let you build MapReduce, Pig, etc jobs > in GUI) rather than going the Windows route - not only are you using > hardware resources better, you also don't need to worry about licensing, > and you won't need to reboot your cluster every Patch Tuesday. :-) > > Thanks! > > *Devin Suiter* > Jr. Data Solutions Software Engineer > 100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212 > Google Voice: 412-256-8556 | www.rdx.com > > > On Thu, Oct 24, 2013 at 8:49 AM, Nitin Pawar wrote: > >> Disclosure: I do not work for Hortonworks, I just use their product. >> please do not bash me up >> Angelo, >> thats not entirely correct now. >> >> Hortonworks has done tremendous amount of work to port hadoop to windows >> os as well. >> Here is there press release: >> http://hortonworks.com/about-us/news/hadoop-on-windows/ >> >> Chris, >> I do remember seeing hadoop 2.x for windows ( >> http://hortonworks.com/blog/announcing-beta-release-of-apache-hadoop-2/) >> I never tried it myself it on windows but you can reach out to their >> support forum and I am sure someone will be happy to help . >> >> >> >> >> On Thu, Oct 24, 2013 at 6:08 PM, Angelo Matarazzo < >> matarazzoang...@gmail.com> wrote: >> >>> Hi, >>> I think that it is not useful trying to install hadoop on Windows >>> because hadoop is very integrated in Linux and there is no support for >>> Windows >>> >>> >>> 2013/10/23 chris bidwell >>> >>>> Is there any documentation or instructions on installing Hadoop 2.2.0 >>>> on Microsoft Windows? >>>> Thank you. >>>> >>>> -Chris >>>> >>>> >>> >> >> >> -- >> Nitin Pawar >> > > -- * Adam Diaz * * * * * Solution Engineer - Big Data -- Phone:919 609 4842 Email: ad...@hortonworks.com Website: http://www.hortonworks.com/ * Follow Us: * <http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> <http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> <http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> [image: photo] Latest From Our Blog: Strong Ecosystem Support for HDP 2.0, Enabling the Modern Data Architecture <http://hortonworks.com/blog/strong-ecosystem-support-for-hdp-2-0/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature> -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
