Reporter: Runping Qi
I have a program that reads a block compressed sequence file, does some
processing on the records and writes the
processed records into another block compressed sequence file.
During execution of the program, I got the following exception
[
https://issues.apache.org/jira/browse/HADOOP-2663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560637#action_12560637
]
Runping Qi commented on HADOOP-2663:
On the flip side, the Buffer should provi
Project: Hadoop
Issue Type: Improvement
Components: record
Reporter: Runping Qi
Assignee: Milind Bhandarkar
Currently, if one call toString("UTF-8"), a String object is created using
Java's converion code.
That does not work properl
[
https://issues.apache.org/jira/browse/HADOOP-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12559815#action_12559815
]
Runping Qi commented on HADOOP-2608:
Forgot one point: when scanning sequence f
[
https://issues.apache.org/jira/browse/HADOOP-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12559812#action_12559812
]
Runping Qi commented on HADOOP-2608:
I profiled the program of reading sequ
-2608
Project: Hadoop
Issue Type: Improvement
Components: io
Reporter: Runping Qi
I did some tests on the throughput of scanning block-compressed sequence files.
The sustained throughput was bounded at 5MB/sec per process, with the cpu of
each process
[
https://issues.apache.org/jira/browse/HADOOP-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558749#action_12558749
]
Runping Qi commented on HADOOP-2491:
A great analysis.
+2
Especially like
[
https://issues.apache.org/jira/browse/HADOOP-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558147#action_12558147
]
Runping Qi commented on HADOOP-2581:
Cool.
That will address the first 2 i
: Improvement
Components: mapred
Reporter: Runping Qi
The following stats are useful and available to JT but not logged job history
log:
1. The counters of each job
2. The counters of each mapper/reducer attempt
3. The info about the input splits (filename, split size, on
[
https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558029#action_12558029
]
Runping Qi commented on HADOOP-2178:
Even with hod JT, we still need to address
[
https://issues.apache.org/jira/browse/HADOOP-2570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12557720#action_12557720
]
Runping Qi commented on HADOOP-2570:
Lohit's suggestion should work.
&g
Combining multiple input blocks into one mapper
---
Key: HADOOP-2560
URL: https://issues.apache.org/jira/browse/HADOOP-2560
Project: Hadoop
Issue Type: Bug
Reporter: Runping Qi
: Runping Qi
Currently, when writing out a block, dfs will place one copy to a local data
node, one copy to a rack local node
and another one to a remote node. This leads to a number of undesired
properties:
1. The block will be rack-local to two tacks instead of three, reducing the
advantage of
[
https://issues.apache.org/jira/browse/HADOOP-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12557292#action_12557292
]
Runping Qi commented on HADOOP-1876:
I am fine with the approach of this patch i
[
https://issues.apache.org/jira/browse/HADOOP-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi reassigned HADOOP-2094:
--
Assignee: dhruba borthakur
> DFS should not use round robin policy in determing on wh
[
https://issues.apache.org/jira/browse/HADOOP-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi reassigned HADOOP-2014:
--
Assignee: Devaraj Das
> Job Tracker should not clobber the data locality of ta
[
https://issues.apache.org/jira/browse/HADOOP-2144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi reassigned HADOOP-2144:
--
Assignee: dhruba borthakur
> Data node process consumes 180%
[
https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556914#action_12556914
]
Runping Qi commented on HADOOP-2178:
+1.
This is much clearer.
Need coordinatio
[
https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556630#action_12556630
]
Runping Qi commented on HADOOP-2178:
BTW, Is this issue the same as H-1876
(h
[
https://issues.apache.org/jira/browse/HADOOP-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556584#action_12556584
]
Runping Qi commented on HADOOP-1876:
The point is that you don't need
[
https://issues.apache.org/jira/browse/HADOOP-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556498#action_12556498
]
Runping Qi commented on HADOOP-1876:
You can add counter information to JobHis
[
https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12555439#action_12555439
]
Runping Qi commented on HADOOP-2178:
The output data may be deleted anytime whe
[
https://issues.apache.org/jira/browse/HADOOP-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12555372#action_12555372
]
Runping Qi commented on HADOOP-2501:
bin/hadoop seq -head assumes the key/v
[
https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12553958
]
Runping Qi commented on HADOOP-1298:
Since the file sizes to read/write are very small, the NNBench should not
[
https://issues.apache.org/jira/browse/HADOOP-2437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12552244
]
Runping Qi commented on HADOOP-2437:
Similar problem due to round robin placement policy happens in DFS data
[
https://issues.apache.org/jira/browse/HADOOP-1336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12552074
]
Runping Qi commented on HADOOP-1336:
Speculative execution does not work well without the patch for hadoop
[
https://issues.apache.org/jira/browse/HADOOP-2433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12552033
]
Runping Qi commented on HADOOP-2433:
There is already a proper input format class: KeyValueTextInputFormat for
[
https://issues.apache.org/jira/browse/HADOOP-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12551923
]
Runping Qi commented on HADOOP-2429:
+1
To enforce that, the API should use concrete classes (such as Buffer
[
https://issues.apache.org/jira/browse/HADOOP-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12550763
]
Runping Qi commented on HADOOP-2369:
+1
> Representative mix of jobs for large cluster throughput benchmark
Project: Hadoop
Issue Type: Bug
Components: mapred
Reporter: Runping Qi
When some tasks failed, the job tracker writes an line to the history file with
error message.
However, the error message may mess up with the history file format, choking
the
[
https://issues.apache.org/jira/browse/HADOOP-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12542896
]
Runping Qi commented on HADOOP-2141:
I don't think this Jira is that urgent and we have to have a quick
[
https://issues.apache.org/jira/browse/HADOOP-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12542861
]
Runping Qi commented on HADOOP-2141:
The above proposal sounds reasonable.
Here are some points to consider
[
https://issues.apache.org/jira/browse/HADOOP-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541828
]
Runping Qi commented on HADOOP-1984:
It will be really helpful if we know the overall job progress status
[
https://issues.apache.org/jira/browse/HADOOP-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541817
]
Runping Qi commented on HADOOP-1984:
Ten minutes waiting interval seems too much.
When the interval reach a
[
https://issues.apache.org/jira/browse/HADOOP-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541388
]
Runping Qi commented on HADOOP-2141:
+1
Speculative execution should start of the original execution is a lot
[
https://issues.apache.org/jira/browse/HADOOP-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541386
]
Runping Qi commented on HADOOP-2175:
The criteria for starting a speculative execution should also include
Components: mapred
Reporter: Runping Qi
I noticed that sometimes, a tasktracker started 2 or three speculative mapper
tasks.
That seems counter productive. You want to speculative execution complete as
soon as possible.
Thus, it is better to spread speculative execution
[
https://issues.apache.org/jira/browse/HADOOP-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541350
]
Runping Qi commented on HADOOP-2175:
then we need to try the patch for hadoop-1984.
Currently, a job may stall
[
https://issues.apache.org/jira/browse/HADOOP-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541240
]
Runping Qi commented on HADOOP-2175:
I was using hadoop 0.15.0.
is the patch for hadoop-1158 in 0.15.0?
if so
Reporter: Runping Qi
One mapper of my job stuck when it reached 87.7%.
Speculative execution was set to true.
But no speculative execution was fired for that task.
The whole job was stalled.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the
: mapred
Reporter: Runping Qi
After a node fails 4 mappers (tasks), it is added to blacklist thus it will no
longer accept tasks.
But, it will continue serve the map outputs of any mappers that ran
successfully there.
However, the node may not be able serve the map outputs either.
This
[
https://issues.apache.org/jira/browse/HADOOP-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540774
]
Runping Qi commented on HADOOP-2164:
My input data is in SequenceFile.
Only one map outputt segment file had
[
https://issues.apache.org/jira/browse/HADOOP-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540772
]
Runping Qi commented on HADOOP-2164:
All attempts failed. The job failed eventually.
> Reducer sort fai
: Runping Qi
One of my job's reducers failed due to the following exception:
java.io.IOException: wrong key class: class org.apache.hadoop.io.LongWritable
is not class org.apache.hadoop.io.Text
at
org.apache.hadoop.io.SequenceFile$Sorter$SegmentDescriptor.nextRawKey(SequenceFile
mapper failed due to exceptions
---
Key: HADOOP-2163
URL: https://issues.apache.org/jira/browse/HADOOP-2163
Project: Hadoop
Issue Type: Bug
Components: mapred
Reporter: Runping Qi
>F
[
https://issues.apache.org/jira/browse/HADOOP-2144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12539635
]
Runping Qi commented on HADOOP-2144:
Overall cpu usage is 90+%.
It is easy to reproduce.
> Data n
Data node process consumes 180% cpu
Key: HADOOP-2144
URL: https://issues.apache.org/jira/browse/HADOOP-2144
Project: Hadoop
Issue Type: Improvement
Reporter: Runping Qi
I did a test on
[
https://issues.apache.org/jira/browse/HADOOP-2144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2144:
---
Component/s: dfs
Description:
I did a test on DFS read throughput and found that the data node
[
https://issues.apache.org/jira/browse/HADOOP-2060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi resolved HADOOP-2060.
Resolution: Invalid
The suggested feature is already in.
> DFSClient should choose a bl
[
https://issues.apache.org/jira/browse/HADOOP-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12539581
]
Runping Qi commented on HADOOP-2086:
+1
Looks good.
> ability to add dependencies to a job af
[
https://issues.apache.org/jira/browse/HADOOP-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12538678
]
Runping Qi commented on HADOOP-2119:
Yes, the #running-mappers kept on going up and went beyond the actual
Issue Type: Bug
Components: mapred
Reporter: Runping Qi
Fix For: 0.15.0
I ran a job with 0 reducer on a cluster with 390 nodes.
The mappers ran very fast.
The jobtracker lacks behind on committing completed mapper tasks.
The number of running mappers displayed
[
https://issues.apache.org/jira/browse/HADOOP-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12538534
]
Runping Qi commented on HADOOP-2086:
There is no real harm to make getState synchronized,
although either way
[
https://issues.apache.org/jira/browse/HADOOP-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12538051
]
Runping Qi commented on HADOOP-2086:
+1.
> ability to add dependencies to a job after construct
[
https://issues.apache.org/jira/browse/HADOOP-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12538017
]
Runping Qi commented on HADOOP-2086:
Right. WAITING is the only valid state we can add a dependending job
[
https://issues.apache.org/jira/browse/HADOOP-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537706
]
runping edited comment on HADOOP-2086 at 10/25/07 1:54 PM:
--
I like 1).
Yes, there is a
[
https://issues.apache.org/jira/browse/HADOOP-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537706
]
Runping Qi commented on HADOOP-2086:
I like 1).
Yes, there is a possibility of race condition when the
[
https://issues.apache.org/jira/browse/HADOOP-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537653
]
Runping Qi commented on HADOOP-2086:
+0 the code looks good.
However, the semantics of the new api
[
https://issues.apache.org/jira/browse/HADOOP-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537492
]
Runping Qi commented on HADOOP-2095:
Yes.
> Reducer failed due to Out ofMem
[
https://issues.apache.org/jira/browse/HADOOP-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537162
]
Runping Qi commented on HADOOP-2095:
The problem was gone after I set the compressMapOutput attribute to false
[
https://issues.apache.org/jira/browse/HADOOP-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2095:
---
Component/s: mapred
Fix Version/s: 0.15.0
Description:
One of the reducers of my job
Reducer failed due to Out ofMemory
--
Key: HADOOP-2095
URL: https://issues.apache.org/jira/browse/HADOOP-2095
Project: Hadoop
Issue Type: Bug
Reporter: Runping Qi
One of the reducers of my job
[
https://issues.apache.org/jira/browse/HADOOP-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi reassigned HADOOP-2094:
--
Assignee: (was: Runping Qi)
> DFS should not use round robin policy in determing on wh
URL: https://issues.apache.org/jira/browse/HADOOP-2094
Project: Hadoop
Issue Type: Improvement
Components: dfs
Reporter: Runping Qi
Assignee: Runping Qi
When multiple file system partitions are configured for the data storage of a
data
[
https://issues.apache.org/jira/browse/HADOOP-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2093:
---
Component/s: mapred
dfs
Description:
The summary is a bit of long. But the
--
Key: HADOOP-2093
URL: https://issues.apache.org/jira/browse/HADOOP-2093
Project: Hadoop
Issue Type: New Feature
Reporter: Runping Qi
The summary is a bit of long. But the
[
https://issues.apache.org/jira/browse/HADOOP-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12535529
]
Runping Qi commented on HADOOP-1965:
It seems clear that threaded spill performed much better than sequence
[
https://issues.apache.org/jira/browse/HADOOP-2060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534962
]
Runping Qi commented on HADOOP-2060:
OK, that is the part I didn't get.
I'll examine the locati
Project: Hadoop
Issue Type: Bug
Components: dfs
Reporter: Runping Qi
When I chase down the DFSClient code to see how the data locality impact the
dfs read throughput,
I realized that DFSClient does not use data locality info (at least not obvious
to me)
when it
[
https://issues.apache.org/jira/browse/HADOOP-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534716
]
Runping Qi commented on HADOOP-2050:
It turned out to be a problem in CopyFile class.
After a mapper got killed
[
https://issues.apache.org/jira/browse/HADOOP-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2050:
---
Component/s: (was: dfs)
mapred
> distcp failed due to problem in creat
[
https://issues.apache.org/jira/browse/HADOOP-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534570
]
Runping Qi commented on HADOOP-2050:
This problem does not happen if the dfs write load is low.
When a few
[
https://issues.apache.org/jira/browse/HADOOP-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2050:
---
Description:
When I run a distcp program to copy files from one dfs to another, my job
failed with
[
https://issues.apache.org/jira/browse/HADOOP-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2052:
---
Description:
When the mappers of distcp finish, the status page in the web gui reports the
data
[
https://issues.apache.org/jira/browse/HADOOP-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2052:
---
Component/s: mapred
Description:
When the mappers of distcp finish, the status page in the web
distcp mapper's status report misleading
Key: HADOOP-2052
URL: https://issues.apache.org/jira/browse/HADOOP-2052
Project: Hadoop
Issue Type: Bug
Reporter: Runping Qi
When the ma
[
https://issues.apache.org/jira/browse/HADOOP-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534462
]
Runping Qi commented on HADOOP-2050:
Some mappers failed with the following exception
[
https://issues.apache.org/jira/browse/HADOOP-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2050:
---
Component/s: dfs
I suspect this is a problem in dfs.
> distcp failed due to problem in creat
distcp failed due to problem in creating files
--
Key: HADOOP-2050
URL: https://issues.apache.org/jira/browse/HADOOP-2050
Project: Hadoop
Issue Type: Bug
Reporter: Runping Qi
When I
[
https://issues.apache.org/jira/browse/HADOOP-2050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2050:
---
Description:
When I run a distcp program to copy files from one dfs to another, my job
DISTCP mapper should report progress more often
---
Key: HADOOP-2048
URL: https://issues.apache.org/jira/browse/HADOOP-2048
Project: Hadoop
Issue Type: Bug
Reporter: Runping Qi
When
[
https://issues.apache.org/jira/browse/HADOOP-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534401
]
Runping Qi commented on HADOOP-2042:
The current trunk. I guess it is 0.15
> distcp job fai
[
https://issues.apache.org/jira/browse/HADOOP-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2042:
---
Component/s: dfs
Description:
I was running distcp to copy data from one dfs to another.
The
distcp job failed
-
Key: HADOOP-2042
URL: https://issues.apache.org/jira/browse/HADOOP-2042
Project: Hadoop
Issue Type: Bug
Affects Versions: 0.15.0
Reporter: Runping Qi
I was running distcp to copy data
Reporter: Runping Qi
With the current implementation, distcp will always assign multiple files to
one mapper to copy, no matter how large
are the files. This is because the CopyFiles class uses a sequencefile to store
the list of files to be copied,
one record per file. CopyFile class
[
https://issues.apache.org/jira/browse/HADOOP-2028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533904
]
Runping Qi commented on HADOOP-2028:
the job fails too if the specified log dir is not empty
> distcp fa
[
https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2015:
---
Status: Patch Available (was: Open)
> Job tracker should report the number of splits that
[
https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2015:
---
Attachment: (was: hadoop-2015.txt)
> Job tracker should report the number of splits that
[
https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2015:
---
Status: Open (was: Patch Available)
a more optimized patch is available
> Job tracker sho
[
https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2015:
---
Attachment: hadoop-2015.txt
a better one
> Job tracker should report the number of splits t
[
https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2015:
---
Attachment: hadoop-2015.txt
Fix a bug in the patch
> Job tracker should report the number
[
https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2015:
---
Status: Patch Available (was: Open)
> Job tracker should report the number of splits that
[
https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2015:
---
Attachment: (was: hadoop-2015.txt)
> Job tracker should report the number of splits that
[
https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2015:
---
Status: Open (was: Patch Available)
Need some more change
> Job tracker should report the num
[
https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2015:
---
Status: Patch Available (was: Open)
> Job tracker should report the number of splits that
[
https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2015:
---
Attachment: hadoop-2015.txt
a patch that adds a counter for the total number of tasks
that may be
[
https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi reassigned HADOOP-2015:
--
Assignee: Runping Qi
> Job tracker should report the number of splits that are local to s
: Hadoop
Issue Type: Improvement
Components: mapred
Reporter: Runping Qi
Right now, jon tracker keeps track the number of launched mappers with local
data.
However, it is not clear how many mappers that are potentially be launched with
data locality.
This
[
https://issues.apache.org/jira/browse/HADOOP-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Runping Qi updated HADOOP-2014:
---
Component/s: mapred
Description:
Currently, when the Job Tracker assigns a mapper task to a
: Runping Qi
Currently, when the Job Tracker assigns a mapper task to a task tracker and
there is no local split to the task tracker, the
job tracker will find the first runable task in the mast task list and assign
the task to the task tracker.
The split for the task is not local to the task
1 - 100 of 456 matches
Mail list logo
