[ http://issues.apache.org/jira/browse/HADOOP-423?page=all ]
Wendy Chien updated HADOOP-423:
---
Status: Patch Available (was: Open)
> file paths are not normalized
> -
>
> Key: HADOOP-423
> URL: ht
[ http://issues.apache.org/jira/browse/HADOOP-423?page=all ]
Wendy Chien updated HADOOP-423:
---
Attachment: (was: pathnorm2.patch)
> file paths are not normalized
> -
>
> Key: HADOOP-423
> URL:
[ http://issues.apache.org/jira/browse/HADOOP-423?page=all ]
Wendy Chien updated HADOOP-423:
---
Attachment: pathnorm3.patch
> file paths are not normalized
> -
>
> Key: HADOOP-423
> URL: http://issu
[
http://issues.apache.org/jira/browse/HADOOP-423?page=comments#action_12438586 ]
Doug Cutting commented on HADOOP-423:
-
> I don't understand where you don't want to call Path.toString(), in DFSClient?
You're right. I mistakenly assumed tha
[
http://issues.apache.org/jira/browse/HADOOP-423?page=comments#action_12438585 ]
Wendy Chien commented on HADOOP-423:
You're right, I've changed the logic in the namenode check.
I don't understand where you don't want to call Path.toString(
[
http://issues.apache.org/jira/browse/HADOOP-489?page=comments#action_12438584 ]
Doug Cutting commented on HADOOP-489:
-
+1 These sound like good changes. They will permit folks to get 'tail -f'-like
log output while jobs are running, to co
[
http://issues.apache.org/jira/browse/HADOOP-489?page=comments#action_12438579 ]
Owen O'Malley commented on HADOOP-489:
--
In terms of managing the user logs in the task tracker, there would be 2
settings:
1. mapred.task.log.size.max
[ http://issues.apache.org/jira/browse/HADOOP-519?page=all ]
Milind Bhandarkar updated HADOOP-519:
-
Attachment: pread.patch
This is the new patch that fixes the problems in earlier patch mentioned by
Doug.
> HDFS File API should be extended to inc
[ http://issues.apache.org/jira/browse/HADOOP-519?page=all ]
Milind Bhandarkar updated HADOOP-519:
-
Status: Patch Available (was: Open)
Patch submitted.
> HDFS File API should be extended to include positional read
> ---
[ http://issues.apache.org/jira/browse/HADOOP-519?page=all ]
Milind Bhandarkar updated HADOOP-519:
-
Attachment: (was: pread.patch)
> HDFS File API should be extended to include positional read
> ---
[
http://issues.apache.org/jira/browse/HADOOP-489?page=comments#action_12438557 ]
Owen O'Malley commented on HADOOP-489:
--
I'd like to have:
1. A jsp on the task trackers that let's me fetch stdout/stderr from the
Tasks with urls like:
htt
[ http://issues.apache.org/jira/browse/HADOOP-549?page=all ]
Owen O'Malley reassigned HADOOP-549:
Assignee: Owen O'Malley
> NullPointerException in TaskReport's serialization code
> ---
>
>
[ http://issues.apache.org/jira/browse/HADOOP-557?page=all ]
Sameer Paranjpye updated HADOOP-557:
Component/s: mapred
Description:
I ran a large job on our cluster over the weekend. At some point, some map
tasks were re-run successfully. However
[ http://issues.apache.org/jira/browse/HADOOP-466?page=all ]
Sameer Paranjpye updated HADOOP-466:
Component/s: conf
> Startup scripts will not start instances of Hadoop daemons w/different
> configs w/o setting separate PID directories
> ---
[ http://issues.apache.org/jira/browse/HADOOP-543?page=all ]
Sameer Paranjpye updated HADOOP-543:
Component/s: mapred
Description:
I was running a faily large job on Hadoop release 0.6.2.
The job failed because a lot of map tasks failed with foll
[ http://issues.apache.org/jira/browse/HADOOP-439?page=all ]
Sameer Paranjpye updated HADOOP-439:
Component/s: contrib/streaming
> Streaming does not work for text data if the records don't fit in a short
> UTF8 [2^16/3 characters]
> ---
[ http://issues.apache.org/jira/browse/HADOOP-230?page=all ]
Sameer Paranjpye updated HADOOP-230:
Component/s: dfs
> improve syntax of the hadoop dfs command
>
>
> Key: HADOOP-230
>
[
http://issues.apache.org/jira/browse/HADOOP-550?page=comments#action_12438547 ]
Doug Cutting commented on HADOOP-550:
-
Two minor nits:
1. Instead of ignoring the CharacterCodingException that should never be
thrown, it would be better to
[ http://issues.apache.org/jira/browse/HADOOP-555?page=all ]
Sameer Paranjpye updated HADOOP-555:
Component/s: mapred
> Tasks should inherit some of the server's environment
> -
>
>
[ http://issues.apache.org/jira/browse/HADOOP-513?page=all ]
Sameer Paranjpye updated HADOOP-513:
Component/s: mapred
> IllegalStateException is thrown by TaskTracker
> --
>
> Key: HADOOP-51
[ http://issues.apache.org/jira/browse/HADOOP-564?page=all ]
Sameer Paranjpye updated HADOOP-564:
Component/s: dfs
> we should use hdfs:// in all API URIs
> -
>
> Key: HADOOP-564
> U
[ http://issues.apache.org/jira/browse/HADOOP-563?page=all ]
Sameer Paranjpye updated HADOOP-563:
Component/s: dfs
Description:
In the current DFS client implementation, there is one thread responsible for
renewing leases. If for whatever reason
[ http://issues.apache.org/jira/browse/HADOOP-535?page=all ]
Sameer Paranjpye updated HADOOP-535:
Component/s: io
> back to back testing of codecs
> --
>
> Key: HADOOP-535
> URL: http://issu
[ http://issues.apache.org/jira/browse/HADOOP-485?page=all ]
Sameer Paranjpye updated HADOOP-485:
Component/s: mapred
> allow a different comparator for grouping keys in calls to reduce
> --
[ http://issues.apache.org/jira/browse/HADOOP-489?page=all ]
Owen O'Malley reassigned HADOOP-489:
Assignee: Owen O'Malley (was: Mahadev konar)
> Seperating user logs from system logs in map reduce
> --
[ http://issues.apache.org/jira/browse/HADOOP-429?page=all ]
Sameer Paranjpye updated HADOOP-429:
Component/s: dfs
> Periodically move blocks from full nodes to those with space
> -
>
>
[ http://issues.apache.org/jira/browse/HADOOP-552?page=all ]
Sameer Paranjpye updated HADOOP-552:
Component/s: mapred
> getMapOutput doesn't reliably detect errors and throw to the caller
>
[ http://issues.apache.org/jira/browse/HADOOP-544?page=all ]
Sameer Paranjpye updated HADOOP-544:
Component/s: mapred
> Replace the job, tip and task ids with objects.
> ---
>
> Key: HADOOP-
[ http://issues.apache.org/jira/browse/HADOOP-500?page=all ]
Sameer Paranjpye updated HADOOP-500:
Component/s: dfs
> Datanode should scan blocks continuously to detect bad blocks / CRC errors
>
[ http://issues.apache.org/jira/browse/HADOOP-511?page=all ]
Sameer Paranjpye updated HADOOP-511:
Component/s: mapred
> mapred.reduce.tasks not used
>
>
> Key: HADOOP-511
> URL: http://issu
[ http://issues.apache.org/jira/browse/HADOOP-475?page=all ]
Sameer Paranjpye updated HADOOP-475:
Component/s: mapred
Description:
In the current framework, when the user implements the reduce method of Reducer
class,
the user can only iterate
[ http://issues.apache.org/jira/browse/HADOOP-448?page=all ]
Sameer Paranjpye updated HADOOP-448:
Component/s: dfs
> DistributedFileSystem uses the wrong user.name to set the working directory.
> --
[ http://issues.apache.org/jira/browse/HADOOP-445?page=all ]
Sameer Paranjpye updated HADOOP-445:
Component/s: dfs
> Parallel data/socket writing for DFSOutputStream
>
>
> Key: HADOOP-4
[ http://issues.apache.org/jira/browse/HADOOP-379?page=all ]
Sameer Paranjpye updated HADOOP-379:
Component/s: mapred
> provide progress feedback while the reducer is sorting
> --
>
>
[ http://issues.apache.org/jira/browse/HADOOP-308?page=all ]
Sameer Paranjpye updated HADOOP-308:
Component/s: mapred
Description:
In case that the local dir is not writable on a node, the tasks on the node
will fail as expected, with an except
[ http://issues.apache.org/jira/browse/HADOOP-357?page=all ]
Sameer Paranjpye updated HADOOP-357:
Component/s: mapred
> hadoop doesn't handle 0 reduces
> ---
>
> Key: HADOOP-357
> URL: http:
[ http://issues.apache.org/jira/browse/HADOOP-338?page=all ]
Sameer Paranjpye updated HADOOP-338:
Component/s: mapred
> the number of maps in the JobConf does not match reality
>
>
>
[ http://issues.apache.org/jira/browse/HADOOP-333?page=all ]
Sameer Paranjpye updated HADOOP-333:
Component/s: mapred
> we should have some checks that the sort benchmark generates correct outputs
> ---
[ http://issues.apache.org/jira/browse/HADOOP-249?page=all ]
Sameer Paranjpye updated HADOOP-249:
Component/s: mapred
> Improving Map -> Reduce performance and Task JVM reuse
> --
>
>
[ http://issues.apache.org/jira/browse/HADOOP-550?page=all ]
Hairong Kuang updated HADOOP-550:
-
Status: Patch Available (was: Open)
Fix Version/s: 0.7.0
Affects Version/s: 0.6.2
> Text constructure can throw exception
> --
[ http://issues.apache.org/jira/browse/HADOOP-550?page=all ]
Hairong Kuang updated HADOOP-550:
-
Attachment: text.patch
> Text constructure can throw exception
> -
>
> Key: HADOOP-550
> U
[
http://issues.apache.org/jira/browse/HADOOP-550?page=comments#action_12438541 ]
Hairong Kuang commented on HADOOP-550:
--
Thanks for your comments, Addison. Currently Text is the default clas for
map/reduce text input files, in which record
[ http://issues.apache.org/jira/browse/HADOOP-560?page=all ]
Doug Cutting updated HADOOP-560:
Status: Resolved (was: Patch Available)
Resolution: Fixed
I just committed this. Thanks, Owen!
> tasks should have a "killed" state
> ---
I'm largely at fault for the "user code running in the JobTracker" that
exists.
I support this change - but, I might reformulate it. Why not make this a
sort of special Job? It can even be formulated roughly like this:
input -> map(Job,FilePath) ->
reduce(Job,FileSplits) -> SchedulableJob
It mi
[
http://issues.apache.org/jira/browse/HADOOP-239?page=comments#action_12438516 ]
Doug Cutting commented on HADOOP-239:
-
> does this sound reasonable?
Yes, it sounds great to me! Thanks!
> job tracker WI drops jobs after 24 hours
> ---
Benjamin Reed wrote:
One of the things that bothers me about the JobTracker is that it is
running user code when it creates the FileSplits. In the long term this
puts the JobTracker JVM at risk due to errors in the user code.
JVM's are supposed to be able to do this kind of stuff securely. Sti
[
http://issues.apache.org/jira/browse/HADOOP-538?page=comments#action_12438503 ]
Arun C Murthy commented on HADOOP-538:
--
Sounds like shipping libhadoop-linux-i386.so along should be a sweet-spot
between performance and maintainence/release
[ http://issues.apache.org/jira/browse/HADOOP-423?page=all ]
Doug Cutting updated HADOOP-423:
Status: Open (was: Patch Available)
Fix Version/s: 0.7.0
I think the check in the NameNode isn't quite right. We should never permit
"." or ".." a
[
http://issues.apache.org/jira/browse/HADOOP-538?page=comments#action_12438492 ]
Owen O'Malley commented on HADOOP-538:
--
I was mostly putting in DF since it had been mentioned and I didn't think the
NativeCode class should be buried up in
[
http://issues.apache.org/jira/browse/HADOOP-239?page=comments#action_12438487 ]
Sanjay Dahiya commented on HADOOP-239:
--
Here are some new changes ( will submit patch in a while )
Moved code to Java 5
Replaced all data attributes in JobI
[
http://issues.apache.org/jira/browse/HADOOP-538?page=comments#action_12438483 ]
Doug Cutting commented on HADOOP-538:
-
Let me refine that a bit: if we ship with any pre-built binaries, I think it
should be libhadoop-linux-i386.so only. Al
[
http://issues.apache.org/jira/browse/HADOOP-538?page=comments#action_12438481 ]
Doug Cutting commented on HADOOP-538:
-
> we need a class that is responsible for trying to load libhadoop.so and
> having fall back replacements
+1
I questio
[ http://issues.apache.org/jira/browse/HADOOP-560?page=all ]
Owen O'Malley updated HADOOP-560:
-
Status: Patch Available (was: Open)
> tasks should have a "killed" state
> --
>
> Key: HADOOP-560
>
[ http://issues.apache.org/jira/browse/HADOOP-560?page=all ]
Owen O'Malley updated HADOOP-560:
-
Attachment: kill-state-2.patch
I fixed another couple of things:
1. my previous patch tickled a previous bug that was causing jobs to not
complete when the
[
http://issues.apache.org/jira/browse/HADOOP-538?page=comments#action_12438478 ]
Owen O'Malley commented on HADOOP-538:
--
Ok, I think we need a class that is responsible for trying to load libhadoop.so
and having fall back replacements.
pa
Should this be true (main() calls and reports exceptions) for all
system components? Especially for user-facing ones like hadoop client?
On Sep 20, 2006, at 3:31 PM, Konstantin Shvachko (JIRA) wrote:
DataNode and NameNode main() should catch and report exceptions.
[
http://issues.apache.org/jira/browse/HADOOP-538?page=comments#action_12438395 ]
Andrzej Bialecki commented on HADOOP-538:
--
If we are to permit use of a native libhadoop there is at least one more thing
that could be optimized - the a
[
http://issues.apache.org/jira/browse/HADOOP-538?page=comments#action_12438384 ]
Arun C Murthy commented on HADOOP-538:
--
Good points Doug; thanks!
Some thoughts...
I like the 'fallback' mechanism, it should ease hadoop adoption for people
One of the things that bothers me about the JobTracker is that it is
running user code when it creates the FileSplits. In the long term this
puts the JobTracker JVM at risk due to errors in the user code.
The JobTracker uses the InputFormat to create a set of tasks that it
then schedules. The task
59 matches
Mail list logo
