This is for the edification of the group.
The clone solution worked. Here's how I handled it.
Second Reducer (redux) :
protected void reduce(Text key, Iterable visitors, Context ctx)
throws IOException, InterruptedException {
List list = new ArrayList();
for (Text visitor : visitors) {
Geoffry,
The problem here is that the Reducer in Hadoop reuses the same
container object to pass on all values and keys. Thus, what you're
really holding in your second reducer's code are "References" to this
object -> Which upon writing will all be a mess of duplicates and what
not cause they are
All,
I have come across a situation that I don't understand.
*First Reducer:
*Behold the first of two reducers. A fragment of it's output follows.
Simple no? It doesn't do anything. I've highlighted two records from the
output. Keep them in mind. Now lets look at the second reducer.
*
*prot
Nothing can see of slaves command. Am I missing something ?
Background: When my first time installation of hadoop on last month, i
followed the instructions of mapreduce wordcount example, and it works.
And this is the second time i use, the computer is
restarted, and call bin/s
On 06/17/2011 09:51 AM, Lemon Cheng wrote:
Hi,
Thanks for your reply.
I am not sure that. How can I prove that?
Which is your dfs.tmp.dir and dfs.data.dir values?
You can check the DataNodes´s health with bin/slaves.sh jps | grep
Datanode | sort
Which is the output of bin/hadoop dfsadmin -r
[appuser@localhost bin]$ ./hadoop dfs -ls
Found 2 items
drwxr-xr-x - appuser supergroup 0 2011-04-26 09:41
/user/appuser/input
drwxr-xr-x - appuser supergroup 0 2011-04-26 09:42
/user/appuser/output
[appuser@localhost bin]$ ./hadoop dfs -ls /usr/lemon/wordcount/input
Found 2 i
Can you send us the output of "hadoop dfs -ls"?
On Jun 17, 2011, at 10:21, Lemon Cheng wrote:
> Hi,
>
> Thanks for your reply.
> I am not sure that. How can I prove that?
>
> I checked the localhost:50070, it shows 1 live node and 0 dead node.
> And the log "hadoop-appuser-datanode-localhos
On Fri, Jun 17, 2011 at 10:29 AM, Marcos Ortiz wrote:
> On 06/17/2011 08:50 AM, Shaun Martinec wrote:
>
> I have a MapReduce job that is failing occasionally at the reduce
> phase. I believe it's during the shuffle phase, but am not positive. I
> have copied the end of the job log below. As you ca
Hi,
Thanks for your reply.
I am not sure that. How can I prove that?
I checked the localhost:50070, it shows 1 live node and 0 dead node.
And the log "hadoop-appuser-datanode-localhost.localdomain.log" shows:
/
2011-06-17 19:59:38,658
On 06/17/2011 08:50 AM, Shaun Martinec wrote:
I have a MapReduce job that is failing occasionally at the reduce
phase. I believe it's during the shuffle phase, but am not positive. I
have copied the end of the job log below. As you can see, I have a
very large number of maps (2910) and only 1 red
I have a MapReduce job that is failing occasionally at the reduce
phase. I believe it's during the shuffle phase, but am not positive. I
have copied the end of the job log below. As you can see, I have a
very large number of maps (2910) and only 1 reducer that is used to
cat the results together (/
On 06/17/2011 07:41 AM, Lemon Cheng wrote:
Hi,
I am using the hadoop-0.20.2. After calling ./start-all.sh, i can type
"hadoop dfs -ls".
However, when i type "hadoop dfs -cat
/usr/lemon/wordcount/input/file01", the error is shown as follow.
I have searched the related problem in the web, but i
Hi,
I am using the hadoop-0.20.2. After calling ./start-all.sh, i can type
"hadoop dfs -ls".
However, when i type "hadoop dfs -cat /usr/lemon/wordcount/input/file01",
the error is shown as follow.
I have searched the related problem in the web, but i can't find a solution
for helping me to solve t
13 matches
Mail list logo
