Re: yarn jobhistory server not displaying all jobs

I found the root cause. Sharing in case someone else runs into this issue. I'm running Yarn, Hadoop 2.3. The reason the jobs weren’t showing up in JobHistoryServer had to do with how we submit jobs. If the same job is submitted via “hadoop jar …” everything works fine. But if the job is submitted

Re: Stopping ntpd signals SIGTERM, then causes namenode exit

Absolutely a critical error to lose the configured ntpd time source in Hadoop. The replication and many other services require absolutely millisecond time sync between the nodes. Interesting that your SRE design called for ntpd running on each node. Curious. What is the problem you are trying to

Re: Stopping ntpd signals SIGTERM, then causes namenode exit

I would spot on Jan 7 14:52:48 host1 ntpd[44765]: no servers reachable looks for me like an network / DNS issue. You can check per dmesg whats going on, too. BR - Alexander On 09 Feb 2015, at 17:57, daemeon reiydelle daeme...@gmail.com wrote: Absolutely a critical error to lose the

Re: Max Connect retries

Are your nodes actually stuck or are you in e.g. a reduce step that is reading so much data across the network that the node SEEMS unreachable? Since you mention gets stuck for a while at 25%, that suggests that eventually the node finishes up its work ... *...* *“Life should not be

Re: Max Connect retries

It did finish, but it took hours, and in one case it didnt finish at all. The same thing happened running the pi estimator On Mon Feb 09 2015 at 15:24:11 daemeon reiydelle daeme...@gmail.com wrote: Are your nodes actually stuck or are you in e.g. a reduce step that is reading so much data

Re: Adding datanodes to Hadoop cluster - Will data redistribute?

Thank you all for answering, the hdfs balancer worked. Now the datanodes capacity is more or less equally balanced. Regards, Manoj From: Arpit Agarwal aagar...@hortonworks.commailto:aagar...@hortonworks.com Reply-To: user@hadoop.apache.orgmailto:user@hadoop.apache.org

Re: Kill one node on map start

In unit tests MiniMRYarnCluster is used to do this kind of stuff. On Friday, February 6, 2015 3:51 AM, Telles Nobrega tellesnobr...@gmail.com wrote: Hi, I'm working on a experiment and I need to do something like, start a hadoop job (wordcount, terasort, pi) and let the application

Re: Max Connect retries

Thanks On Mon Feb 09 2015 at 01:43:24 Xuan Gong xg...@hortonworks.com wrote: That is for client connect retry in ipc level. You can decrease the max.retries by configuring ipc.client.connect.max.retries.on.timeouts in core-site.xml Thanks Xuan Gong From: Telles Nobrega

Re: Home for Apache Big Data Solutions?

I believe Apache Bigtop is what you're looking for. Artem Ervits On Feb 9, 2015 8:15 AM, Jean-Baptiste Onofré j...@nanthrax.net wrote: Hi Amir, thanks for the update. Please, let me know if you need some help on the proposal and to qualify your ideas. Regards JB On 02/09/2015 02:05

Re: Home for Apache Big Data Solutions?

Bigtop.. Yup! Mr Asanjar : why don't you post an email about what your doing on the Apache bigtop list, we'd love to hear from you. There could possibly be some overlap and our goal is to plumb the hadoop ecosystem as well On Feb 9, 2015, at 4:41 PM, Artem Ervits artemerv...@gmail.com

Re: Home for Apache Big Data Solutions?

Hi Chris, thanks for the information, will get on it ... Hi JB Glad that you are familiar with Juju, however my personal goal is not to promote any tool but to take the next step, which is to build a community for apache big data solutions. do you already have a kind of proposal/description of

Re: Home for Apache Big Data Solutions?

Hi Amir, thanks for the update. Please, let me know if you need some help on the proposal and to qualify your ideas. Regards JB On 02/09/2015 02:05 PM, MrAsanjar . wrote: Hi Chris, thanks for the information, will get on it ... Hi JB Glad that you are familiar with Juju, however my