Anseh,
Let's assume that your job is fully scalable, then it should take: 100 000
000 / 600 000 times the amount of time of the first job, which is 1000 / 6
= 167 times longer. This is an ideal, probably it will be something like
200 times. Also try using units in your questions + scientific
Boris, what does hbck say?
We have had this issue a couple times before. To fix it I had to stop the
cluster, run offline meta repair tool,
delete zk-store on each zk quorum node
Offline Meta repair tool will not work if there are inconsistencies in HBase
- you better try hbase hbck
-fixAll
Hi, Boris
Did you check RS logs ? There should be exception regarding why assignment
failed. Can you past that exception ?
Cheers :)
On Mon, Oct 21, 2013 at 9:53 AM, Boris Emelyanov emelya...@post.km.ruwrote:
Boris, what does hbck say?
We have had this issue a couple times before. To fix
On 21.10.2013 12:17, Samir Ahmic wrote:
Hi, Boris
Did you check RS logs ? There should be exception regarding why
assignment failed. Can you past that exception ?
Cheers :)
On Mon, Oct 21, 2013 at 9:53 AM, Boris Emelyanov emelya...@post.km.ru
mailto:emelya...@post.km.ru wrote:
Hi all,
I am currently pushing the limits of my hadoop cluster. Unfortunately I am
a bit confused about the memory requirements during the copy phase and the
sort phases. I have made an effort to fully explain my setup and problems
at the following link:
Dear All,
Please let me know how to get the CDR Sample Data files. And how to deploy and
analysis CDR (call detail records) files on Hadoop Cluster.
Thanks Regards,
Aijas Mohammed
Ext:- 1148
DISCLAIMER:
This email may contain confidential information and is
Are you sure you wanted to send this mail to common hadoop users and dev?
If you want to put files on hadoop cluster, there are ways like hadoop cli,
java client, webhdfs etc.
What analysis you want to do is in your brain, not really sure what help
you need on that.
How do download CDR data, you
I can't see anything wrong in your logs, but fact that you trigger this
issue by running balancer makes me think that some of your RS may have some
problem. Here is what would i do in this situation:
1. Make sure that system time, OS configuration, hadoop/HBase configuration
is synced on all
Dear ALL,
I want to ANALYZE Call Detail Records Data. For Example
CDR Analysis System:
* Capture for extended periods of time from hours to months
* Once calls are captured, a search for calls of interest can be
performed while live capturing continues
* Drill-down to problem calls
Hi,
I am seeing the following call to start() on AMRMClientAsync taking from
0.9 to 1 second. Why does it take that long? Is there a way to reduce it, I
mean does it depend on any of the interval parameters or so in
configuration files? I have tried reducing the value of the first argument
below
Thanks again. This gives me a lot of options; we will see what works.
Do you know if there are any permissions issues if we directly access the
folders of LOCAL_DIR_ENV?
Regarding LocalDirAllocator, I see its constructor: LocalDirAllocator(String
contextCfgItemName) and a note mentioning that
Hi Krishna,
Those 900ms seems consistent with the numbers we found while doing some
benchmarks in the context of Llama:
http://cloudera.github.io/llama/
We found that the first application master created from a client process
takes around 900 ms to be ready to submit resource requests.
The dirs in that env-var are app-specific and are for the app's user
to utilize. You shouldn't have any permission issues working within
them.
The LocalDirAllocator is still somewhat MR-bound but you can still be
able to make it work by giving it a config with the values it needs.
On Mon, Oct
Hi,
I installed Hadoop 2.1 from this site:
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.5.0/bk_installing_manually_book/content/rpm-chap13.html
And have followed the installation guide.
I am at the section for formatting and starting HDFS. However I am getting
this error saying $JAVA
This post might help a bit.
http://hortonworks.com/blog/management-of-application-dependencies-in-yarn/
Thanks,
Jian
On Mon, Oct 21, 2013 at 11:11 AM, Harsh J ha...@cloudera.com wrote:
The dirs in that env-var are app-specific and are for the app's user
to utilize. You shouldn't have any
Hi,
Due to some security concerns I can't share the real time CDR logs but as
an alternative you can create your own script that will generate dummy CDR
records for your analysis.
Below link might be helpful.
http://www.gedis-studio.com/online-call-detail-records-cdr-generator.html
Regards
Thanks, sounds like LOCAL_DIR_ENV is the way to go.
john
-Original Message-
From: Harsh J [mailto:ha...@cloudera.com]
Sent: Monday, October 21, 2013 12:11 PM
To: user@hadoop.apache.org
Subject: Re: temporary file locations for YARN applications
The dirs in that env-var are app-specific
Hello,
We are noticing the RM running out of memory in the webapp code. It happens
in
org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlock.renderBlock(Block
html).
The StringBuilder object appsTableData grows too large in this case while
appending AppInfo. Ignoring the heap size (this
Right, that's very useful for ensuring that copies of read-only data are
available to all nodes. We do use LocalResources for the transport of our
executable environment to the nodes.
Cheers,
John
From: Jian He [mailto:j...@hortonworks.com]
Sent: Monday, October 21, 2013 12:22 PM
To:
Dear all
I'm was using hadoop-1.2 for some project, and be very enthousiast. Now,
i want to switch to the new version 2.2, with Yarn. But reading the
Getting Started document, i'm facing a egg and chicken problem: in the
Setting up a Single Node Cluster, it is asuming that we get HDFS
I recently setup a 2.2.0 test cluster. For some reason, all of my MR jobs
are failing. The maps and reduces all run to completion, without any
errors. Yet the app is marked failed and there is no final output. Any
ideas?
Application Type: MAPREDUCE
State: FINISHED
FinalStatus: FAILED
If you follow the links on the web-ui to the logs of the map/reduce tasks, what
do you see there?
Arun
On Oct 21, 2013, at 9:55 PM, Robert Dyer psyb...@gmail.com wrote:
I recently setup a 2.2.0 test cluster. For some reason, all of my MR jobs
are failing. The maps and reduces all run to
22 matches
Mail list logo
