Yes, you can solve the bottleneck by starting a webdav server on each
client. But this would include the burden to manage the servers etc. and
it may not be the intended use case for webdav. But we can further
discuss the architecture in the relevant issue.
Alban Chevignard wrote:
Thanks for
Hello,
When launching a map-reduce job, I am interested in copying a certain file
to the datanodes, but not HDFS - the local file system, so I can access that
file from my job on the datanode. (The file is around 500KB, so I don't
think there will be much overhead). Is there a way to tell hadoop
Hi Jerrro,
take a look at
http://hadoop.apache.org/core/docs/r0.15.3/mapred_tutorial.html#DistributedCache
The DistributedCache looks like what you are searching for. I think the
interesting part is the example
http://hadoop.apache.org/core/docs/r0.15.3/mapred_tutorial.html#Example%3A+WordCoun
You can either pack the files with your job jar or use the distributed
cache if the file size is big. See http://wiki.apache.org/hadoop/FAQ#8.
Hairong
-Original Message-
From: jerrro [mailto:[EMAIL PROTECTED]
Sent: Thursday, January 24, 2008 8:06 AM
To: [EMAIL PROTECTED]
Subject: hado
On Tue, 22 Jan 2008 22:03:03 +0100, Jeff Hammerbacher
<[EMAIL PROTECTED]> wrote:
we use FUSE: who wants a gui when you could have a shell?
http://issues.apache.org/jira/browse/HADOOP-4
Does this work with newer versions of Hadoop?
[...]
--
Using Opera's revolutionary e-mail client: http:/
With very minor changes it works with 0.15.2, read only.
Vetle Roeim wrote:
On Tue, 22 Jan 2008 22:03:03 +0100, Jeff Hammerbacher
<[EMAIL PROTECTED]> wrote:
we use FUSE: who wants a gui when you could have a shell?
http://issues.apache.org/jira/browse/HADOOP-4
Does this work with newer ve
Right now its tested with 0.14.4. It also includes rmdir, rm, mkdir, mv.
I¹ve implemented write, but it has to wait for appends to work in Hadoop
because of the Fuse protocol.
Our strategy thus far has been to use FUSE on a single box and then NFS
export it to other machines. We don¹t do heavy, h
Another note is we implemented the trash feature in fuse. This could be
turned on and off with ioctl. We also don¹t allow removal of certain
directories which again can be configured with ioctl. (but isn¹t yet)
-- pete
On 1/24/08 10:39 AM, "Vetle Roeim" <[EMAIL PROTECTED]> wrote:
> On Tue, 22
Great! Where can I get it? :)
On Thu, 24 Jan 2008 19:48:57 +0100, Pete Wyckoff <[EMAIL PROTECTED]>
wrote:
Right now its tested with 0.14.4. It also includes rmdir, rm, mkdir, mv.
I¹ve implemented write, but it has to wait for appends to work in Hadoop
because of the Fuse protocol.
Our stra
The only change needed for 0.15.2 was to change the references to
info[0].mCreationTime into references to info[0].mLastMod
Pete Wyckoff wrote:
Right now its tested with 0.14.4. It also includes rmdir, rm, mkdir, mv.
I¹ve implemented write, but it has to wait for appends to work in Hadoop
becau
I can post it again, but it doesn¹t include ioctl commands so the trash
feature cannot be configured. I can still create a flag and default it to
false. And also the directory protection isn¹t configurable so I can set a
flag to false. The main directory we protect here is /user/facebook for data
Yes, please post it again. :) Lack of trash and directory protection
shouldn't be an issue for my needs.
On Thu, 24 Jan 2008 20:11:26 +0100, Pete Wyckoff <[EMAIL PROTECTED]>
wrote:
I can post it again, but it doesn¹t include ioctl commands so the trash
feature cannot be configured. I can
I attached the newest version to:
https://issues.apache.org/jira/browse/HADOOP-4
Still a work in progress and any help appreciated. Not much by way of
instructions but here are some:
1. download and install fuse and do a modprobe fuse
2. modify fuse_dfs.c¹s Makefile to have the right paths for f
Thanks!
On Thu, 24 Jan 2008 20:29:20 +0100, Pete Wyckoff <[EMAIL PROTECTED]>
wrote:
I attached the newest version to:
https://issues.apache.org/jira/browse/HADOOP-4
Still a work in progress and any help appreciated. Not much by way of
instructions but here are some:
1. download and instal
Is anyone else having the same problems as me with regard to frequently
seeing "NotServingRegionException" and "IllegalStateException: region
offline" exceptions when trying to load data into an hbase instance?
My setup uses
- hadoop (2008-01-14 snapshot)
- a single server hbase cluster, as descri
Does an hbase client java application just need a correctly configured
hbase-site.xml or does it need a hadoop-site.xml as well? By client
application I mean not a map-reduce job but something similar to the
sample application on the hbase FAQ page
http://wiki.apache.org/hadoop/Hbase/FAQ#1
Thanks,
When there are splits going on, NSREs are expected. I would say that
it is fairly unexpected for them to bubble all the way up to the
client application, though.
Is there anything else in your master or regionserver logs? Are you
running at DEBUG log level for HBase? I'd like to try and fig
I've seen the ISE's myself (HADOOP-2692). As Bryan says, the NSREs are
part of 'normal' operation; they only show if running at DEBUG level
unless we run out of retries and then the NSRE is thrown as an error.
FYI, 5M rows single-threaded will take forever to load. I'd suggest you
set up a M
Yes, unless you copy all of your hadoop-site.xml to hbase-site.xml.
Hbase on startup -- server or client -- will add $HBASE_CONF_DIR and
$HADOOP_CONF_DIR to its CLASSPATH. Any hadoop-*xml and hbase-*.xml
configuration files found therein will be loaded. Hbase then uses such
as the hadoop conf
Hi,
I have some questions on the network settings.
What is the different between the following two entries?
The first one is obvious, but what does the second one mean? How is it
related to DNS?
dfs.datanode.bindAddress
0.0.0.0
the address where the datanode will listen to.
Hi,
I am very new to Hadoop, and I have a project where I need to use Lucene to
index some input given either as a a huge collection of Java objects or one
huge java object.
I read about Hadoop's MapReduce utilities and I want to leverage that feature
in my case described above.
Can som
I'm actually going to be doing something similar, with Nutch. I just
started learning about Hadoop this week, so I'm interested in what
everyone has to say :)
On Jan 24, 2008 5:00 PM, roger dimitri <[EMAIL PROTECTED]> wrote:
> Hi,
>I am very new to Hadoop, and I have a project where I need to
Hi all;
When I start my hbase,the error print as follows: localhost:
regionserver running as process 6893. Stop it first.
Can you tell me how to solve this problem ?Why after I stop my
hbase the regionserver still run?
Best Wishes
Its safe to 'kill' it if it won't go down. See logs to see if you can
figure why it didn't go down when master went down.
St.Ack
ma qiang wrote:
Hi all;
When I start my hbase,the error print as follows: localhost:
regionserver running as process 6893. Stop it first.
Can you tell me how t
On Jan 25, 2008 6:30 AM, roger dimitri <[EMAIL PROTECTED]> wrote:
> Hi,
> I am very new to Hadoop, and I have a project where I need to use Lucene
> to index some input given either as a a huge collection of Java objects or
> one huge java object.
> I read about Hadoop's MapReduce utilities and
25 matches
Mail list logo
