Hello
I need to upload large file using WEBHDFS (from local disk into HDFS,
WEBHDFS is my only option, don't have direct access). Because in my case
network connection is bottleneck I decided to compress file with snappy
before sending. I am using Java application, compiled with
"org.apache.hadoop
Hi, all:
I configured a hadoop cluster with 9 hosts, each with 2 VCPU and 4G Ram.
I noticed when I run the example pi program, only when I configure it with
at least 8*9=72 mappers will all hosts be busy.
Which means there is a default 8 mappers per host?
How is this value decided? And where can
Maybe the user 'test' has no privilege of write operation.
You can refer the ERROR log like:
org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
as:test (auth:SIMPLE)
2014-07-15 2:07 GMT+08:00 Bogdan Raducanu :
> I'm getting this error while writing many files.
> org.apac
Right.
hbase is different from Cassandra in this regard.
On Mon, Jul 14, 2014 at 2:57 PM, Adaryl "Bob" Wakefield, MBA <
adaryl.wakefi...@hotmail.com> wrote:
> Now this is different from Cassandra which does NOT use HDFS correct?
> (Sorry. Don’t know why that needed two emails.)
>
> B.
>
> *Fr
Now this is different from Cassandra which does NOT use HDFS correct? (Sorry.
Don’t know why that needed two emails.)
B.
From: Ted Yu
Sent: Monday, July 14, 2014 4:53 PM
To: mailto:user@hadoop.apache.org
Subject: Re: clarification on HBASE functionality
Yes.
See http://hbase.apache.org/book.
Yes.
See http://hbase.apache.org/book.html#arch.hdfs
On Mon, Jul 14, 2014 at 2:52 PM, Adaryl "Bob" Wakefield, MBA <
adaryl.wakefi...@hotmail.com> wrote:
> HBASE uses HDFS to store it's data correct?
>
> B.
>
HBASE uses HDFS to store it's data correct?
B.
There shouldn't be any - it basically streams over the existing local
fsimage file.
On Tue, Jul 15, 2014 at 12:21 AM, Ashish Dobhal
wrote:
> Sir I tried it it works. Are there any issues in downloading the gsimage
> using wget.
>
>
> On Tue, Jul 15, 2014 at 12:17 AM, Harsh J wrote:
>>
>> Sure, y
For what it's worth, mapreduce.jobtracker.split.metainfo.maxsize is related
to the size of the file containing the information describing the input
splits. It is not related directly to the volume of data but to the number
of splits which might explode when using too many (small) files. It's
basica
Sir I tried it it works. Are there any issues in downloading the gsimage
using wget.
On Tue, Jul 15, 2014 at 12:17 AM, Harsh J wrote:
> Sure, you could try that. I've not tested that mix though, and OIV
> relies on some known formats support, but should hopefully work.
>
> On Mon, Jul 14, 2014
Sure, you could try that. I've not tested that mix though, and OIV
relies on some known formats support, but should hopefully work.
On Mon, Jul 14, 2014 at 11:56 PM, Ashish Dobhal
wrote:
> Could I download the fsimage of a hadoop 1.0 using wget and then interpret
> it in offline mode using the to
Could I download the fsimage of a hadoop 1.0 using wget and then interpret
it in offline mode using the tool in the hadoop 1.2 or higher
distributions.I guess the structure of fsimage would be same for both the
distributions.
On Mon, Jul 14, 2014 at 11:53 PM, Ashish Dobhal
wrote:
> Harsh thanks
Harsh thanks
On Mon, Jul 14, 2014 at 11:39 PM, Harsh J wrote:
> The OIV for 1.x series is available in release 1.2.0 and higher. You
> can use it from the 'hadoop oiv' command.
>
> It is not available in 1.0.x.
>
> On Mon, Jul 14, 2014 at 9:49 PM, Ashish Dobhal
> wrote:
> > Hey everyone ;
> >
The OIV for 1.x series is available in release 1.2.0 and higher. You
can use it from the 'hadoop oiv' command.
It is not available in 1.0.x.
On Mon, Jul 14, 2014 at 9:49 PM, Ashish Dobhal
wrote:
> Hey everyone ;
> Could anyone tell me how to use the OIV tool in hadoop 1.0 as there is no
> hdfs.s
I'm getting this error while writing many files.
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy: Not
able to place enough replicas, still in need of 4 to reach 4
I've set logging to DEBUG but still there is no reason printed. There
should've been a reason after this line but in
It sounds like JobTracker setting, so the restart looks to be required.
You verify it in pseudo-distributed mode by setting it to a very low value,
restarting JT and seeing if you get the exception that prints this new value.
Sent from my iPhone
> On 14 jul 2014, at 16:03, Jan Warchoł wrote:
>
Hey everyone ;
Could anyone tell me how to use the OIV tool in hadoop 1.0 as there is no
hdfs.sh file there.
Thanks.
Hello,
I recently got "Split metadata size exceeded 1000" error when running
Cascading jobs with very big joins. I found that I should change
mapreduce.jobtracker.split.metainfo.maxsize property in hadoop
configuration by adding this to the mapred-site.xml file:
mapreduce.jobtrac
HI ，
I didn't try Hadoop rebalancer 。Because I remember rebalancer only considers
disk load, and won't consider that data blocks which rack 。
I can try 。Thank you for your reply 。
-- 原始邮件 --
发件人: "Yehia Elshater";;
发送时间: 2014年7月14日(星期一) 下午4:52
收件人: "user";
Hi,
Did you try Hadoop rebalancer ?
http://hadoop.apache.org/docs/r1.0.4/hdfs_user_guide.html#Rebalancer
On 14 July 2014 04:10, 风雨无阻 <232341...@qq.com> wrote:
> HI all:
>
> After the cluster configuration rack awareness,run " hadoop fsck / "
> A lot of the following error occurred:
> Replica
HI all:
After the cluster configuration rack awareness,run " hadoop fsck / "
A lot of the following error occurred:
Replica placement policy is violated for blk_-1267324897180563985_11130670.
Block should be additionally replicated on 1 more rack(s).
Online said "The reason is that three cop
Hadoop 2.4.1 doesn't support symbolic link.
(2014/07/14 11:34), cho ju il wrote:
My hadoop version is 2.4.1.
Hdfs(version 2.4.1) is a symbolic link support?
How do I create symbolic links?
22 matches
Mail list logo
