[jira] [Created] (HADOOP-16839) SparkLauncher does not read SPARK_CONF_DIR/spark-defaults.conf

Koji Noguchi created HADOOP-16839: - Summary: SparkLauncher does not read SPARK_CONF_DIR/spark-defaults.conf Key: HADOOP-16839 URL: https://issues.apache.org/jira/browse/HADOOP-16839 Project: Hadoop C

[jira] [Updated] (HADOOP-13340) Compress Hadoop Archive output

[ https://issues.apache.org/jira/browse/HADOOP-13340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated HADOOP-13340: -- Attachment: HADOOP-13340-example-v02.patch bq. Hmm, updated unit test is failing for me.  Ple

[jira] [Commented] (HADOOP-13340) Compress Hadoop Archive output

[ https://issues.apache.org/jira/browse/HADOOP-13340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547046#comment-16547046 ] Koji Noguchi commented on HADOOP-13340: --- Hmm, updated unit test is failing for me

[jira] [Commented] (HADOOP-13340) Compress Hadoop Archive output

[ https://issues.apache.org/jira/browse/HADOOP-13340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547033#comment-16547033 ] Koji Noguchi commented on HADOOP-13340: --- Just to clarify on my previous comment,

[jira] [Updated] (HADOOP-13340) Compress Hadoop Archive output

[ https://issues.apache.org/jira/browse/HADOOP-13340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated HADOOP-13340: -- Attachment: HADOOP-13340-example-v01.patch > Compress Hadoop Archive output > ---

[jira] [Commented] (HADOOP-13340) Compress Hadoop Archive output

[ https://issues.apache.org/jira/browse/HADOOP-13340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534993#comment-16534993 ] Koji Noguchi commented on HADOOP-13340: --- I do sometimes want this compression fea

[jira] [Commented] (HADOOP-13114) DistCp should have option to compress data on write

[ https://issues.apache.org/jira/browse/HADOOP-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15819365#comment-15819365 ] Koji Noguchi commented on HADOOP-13114: --- bq. Could you please elucidate your conce

[jira] [Commented] (HADOOP-13114) DistCp should have option to compress data on write

[ https://issues.apache.org/jira/browse/HADOOP-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15815124#comment-15815124 ] Koji Noguchi commented on HADOOP-13114: --- bq. I guess it'd be useful for any files

[jira] [Commented] (HADOOP-13114) DistCp should have option to compress data on write

[ https://issues.apache.org/jira/browse/HADOOP-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15684893#comment-15684893 ] Koji Noguchi commented on HADOOP-13114: --- Sorry for joining late on this jira but t

[jira] [Commented] (HADOOP-10876) The constructor of Path should not take an empty URL as a parameter

[ https://issues.apache.org/jira/browse/HADOOP-10876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14090245#comment-14090245 ] Koji Noguchi commented on HADOOP-10876: --- Thanks Andrew, Zhihai. (Sorry I was too l

[jira] [Commented] (HADOOP-10876) The constructor of Path should not take an empty URL as a parameter

[ https://issues.apache.org/jira/browse/HADOOP-10876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070605#comment-14070605 ] Koji Noguchi commented on HADOOP-10876: --- My comment and suggestion can be found at

[jira] [Commented] (HADOOP-10820) Empty entry in libjars results in working directory being recursively localized

[ https://issues.apache.org/jira/browse/HADOOP-10820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070441#comment-14070441 ] Koji Noguchi commented on HADOOP-10820: --- Given careless ",," started uploading the

[jira] [Commented] (HADOOP-10820) Empty entry in libjars results in working directory being recursively localized

[ https://issues.apache.org/jira/browse/HADOOP-10820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068776#comment-14068776 ] Koji Noguchi commented on HADOOP-10820: --- This reminds me of a very old bug HADOOP-

[jira] [Commented] (HADOOP-9639) truly shared cache for jars (jobjar/libjar)

[ https://issues.apache.org/jira/browse/HADOOP-9639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751400#comment-13751400 ] Koji Noguchi commented on HADOOP-9639: -- I'm not comfortable with the proposed design

[jira] [Commented] (HADOOP-9757) Har metadata cache can grow without limit

[ https://issues.apache.org/jira/browse/HADOOP-9757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13725577#comment-13725577 ] Koji Noguchi commented on HADOOP-9757: -- Hi Daryn, bq. The har AFS delegates to the

[jira] [Commented] (HADOOP-8690) Shell may remove a file without going to trash even if skipTrash is not enabled

[ https://issues.apache.org/jira/browse/HADOOP-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434240#comment-13434240 ] Koji Noguchi commented on HADOOP-8690: -- bq. Seems like in this case we should move i

[jira] [Commented] (HADOOP-8230) Enable sync by default and disable append

[ https://issues.apache.org/jira/browse/HADOOP-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269678#comment-13269678 ] Koji Noguchi commented on HADOOP-8230: -- bq. , but what is that use case? I'm probab

[jira] [Updated] (HADOOP-7519) hadoop fs commands should support tar/gzip or an equivalent

[ https://issues.apache.org/jira/browse/HADOOP-7519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated HADOOP-7519: - Attachment: hadoop-7519-0.20.2XX-1.patch Some of my users had this need in the past. Wrote a sho

[jira] [Commented] (HADOOP-7262) Update CS docs with better example configs

[ https://issues.apache.org/jira/browse/HADOOP-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030912#comment-13030912 ] Koji Noguchi commented on HADOOP-7262: -- bq. Capacity Guarantees bq. Isn't 'guarantee

[jira] [Commented] (HADOOP-7262) Update CS docs with better example configs

[ https://issues.apache.org/jira/browse/HADOOP-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030901#comment-13030901 ] Koji Noguchi commented on HADOOP-7262: -- Arun, "mapred.capacity-scheduler.queue." is

[jira] [Commented] (HADOOP-7207) fs member of FSShell is not really needed

[ https://issues.apache.org/jira/browse/HADOOP-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13010787#comment-13010787 ] Koji Noguchi commented on HADOOP-7207: -- Duplicate of HADOOP-5749 ? > fs member of F

[jira] Commented: (HADOOP-7115) Add a cache for getpwuid_r and getpwgid_r calls

[ https://issues.apache.org/jira/browse/HADOOP-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12985048#action_12985048 ] Koji Noguchi commented on HADOOP-7115: -- Devaraj, can we make this cache timeout perio

[jira] Commented: (HADOOP-7104) Remove unnecessary DNS reverse lookups from RPC layer

[ https://issues.apache.org/jira/browse/HADOOP-7104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983339#action_12983339 ] Koji Noguchi commented on HADOOP-7104: -- Kan, Deveraj, isn't this a regression bug ins

[jira] Commented: (HADOOP-6985) Suggest that HADOOP_OPTS be preserved in hadoop-env.sh.template

[ https://issues.apache.org/jira/browse/HADOOP-6985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924763#action_12924763 ] Koji Noguchi commented on HADOOP-6985: -- Curious. bq. else FOO+=" -server"; fi Where

[jira] Reopened: (HADOOP-6857) FsShell should report raw disk usage including replication factor

[ https://issues.apache.org/jira/browse/HADOOP-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi reopened HADOOP-6857: -- I think this number(raw usage) would be helpful. Not sure whether this should be in -du or -coun

[jira] Commented: (HADOOP-6857) FsShell should report raw disk usage including replication factor

[ https://issues.apache.org/jira/browse/HADOOP-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12909502#action_12909502 ] Koji Noguchi commented on HADOOP-6857: -- A little confused. I thought "fs -count" sho

[jira] Commented: (HADOOP-6760) WebServer shouldn't increase port number in case of negative port setting caused by Jetty's race

[ https://issues.apache.org/jira/browse/HADOOP-6760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12869298#action_12869298 ] Koji Noguchi commented on HADOOP-6760: -- bq. Thus a daemon's webserver becomes pretty

[jira] Commented: (HADOOP-6766) Spill can fail with bad call to Random

[ https://issues.apache.org/jira/browse/HADOOP-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12867950#action_12867950 ] Koji Noguchi commented on HADOOP-6766: -- Peter, if you're using yahoo clusters it coul

[jira] Commented: (HADOOP-6702) Incorrect exit codes for "dfs -chown", "dfs -chgrp" when input is given in wildcard format.

[ https://issues.apache.org/jira/browse/HADOOP-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12865958#action_12865958 ] Koji Noguchi commented on HADOOP-6702: -- bq. HADOOP-6701 Fixes this issue. Ravi, HA

[jira] Commented: (HADOOP-6713) The RPC server Listener thread is a scalability bottleneck

[ https://issues.apache.org/jira/browse/HADOOP-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12859905#action_12859905 ] Koji Noguchi commented on HADOOP-6713: -- Nice. Any performance numbers ? > The RPC se

[jira] Commented: (HADOOP-6702) Incorrect exit codes for "dfs -chown", "dfs -chgrp" when input is given in wildcard format.

[ https://issues.apache.org/jira/browse/HADOOP-6702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12857862#action_12857862 ] Koji Noguchi commented on HADOOP-6702: -- Incorrect return code for wildcard in hadoop

[jira] Commented: (HADOOP-6669) zlib.compress.level ignored for DefaultCodec initialization

[ https://issues.apache.org/jira/browse/HADOOP-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852428#action_12852428 ] Koji Noguchi commented on HADOOP-6669: -- Hm. Didn't know that native library is not us

[jira] Updated: (HADOOP-6669) zlib.compress.level ignored for DefaultCodec initialization

[ https://issues.apache.org/jira/browse/HADOOP-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated HADOOP-6669: - Status: Patch Available (was: Open) > zlib.compress.level ignored for DefaultCodec initializat

[jira] Updated: (HADOOP-6669) zlib.compress.level ignored for DefaultCodec initialization

[ https://issues.apache.org/jira/browse/HADOOP-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated HADOOP-6669: - Attachment: HADOOP-6669-0.patch Attaching a patch that 1) Pass the conf when compressor is creat

[jira] Created: (HADOOP-6669) zlib.compress.level ignored for DefaultCodec initialization

zlib.compress.level ignored for DefaultCodec initialization - Key: HADOOP-6669 URL: https://issues.apache.org/jira/browse/HADOOP-6669 Project: Hadoop Common Issue Type: Bug

[jira] Commented: (HADOOP-6284) Any hadoop commands crashing jvm (SIGBUS) when /tmp (tmpfs) is full

[ https://issues.apache.org/jira/browse/HADOOP-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785423#action_12785423 ] Koji Noguchi commented on HADOOP-6284: -- FYI, we deployed the fix with '-XX:-UsePerfDa

[jira] Commented: (HADOOP-6335) Support reading of concatenated gzip files

[ https://issues.apache.org/jira/browse/HADOOP-6335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772183#action_12772183 ] Koji Noguchi commented on HADOOP-6335: -- Is this a duplicate of MAPREDUCE-469 ? Also,

[jira] Commented: (HADOOP-4933) ConcurrentModificationException in JobHistory.java

[ https://issues.apache.org/jira/browse/HADOOP-4933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771969#action_12771969 ] Koji Noguchi commented on HADOOP-4933: -- FYI, when hitting this bug, 1) Job history lo

[jira] Updated: (HADOOP-6284) Any hadoop commands crashing jvm (SIGBUS) when /tmp (tmpfs) is full

[ https://issues.apache.org/jira/browse/HADOOP-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated HADOOP-6284: - Attachment: HADOOP-6284-y0.20.1.patch Patch for 0.20. (not meant for commit) > Any hadoop comma

[jira] Commented: (HADOOP-6284) Any hadoop commands crashing jvm (SIGBUS) when /tmp (tmpfs) is full

[ https://issues.apache.org/jira/browse/HADOOP-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759693#action_12759693 ] Koji Noguchi commented on HADOOP-6284: -- bq. I think a bug is needed to be files again

[jira] Updated: (HADOOP-6284) Any hadoop commands crashing jvm (SIGBUS) when /tmp (tmpfs) is full

[ https://issues.apache.org/jira/browse/HADOOP-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated HADOOP-6284: - Attachment: hadoop-6284-patch-v1.txt It's a silly patch but introduces a new env HADOOP_JAVA_PLA

[jira] Commented: (HADOOP-6097) Multiple bugs w/ Hadoop archives

[ https://issues.apache.org/jira/browse/HADOOP-6097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759316#action_12759316 ] Koji Noguchi commented on HADOOP-6097: -- bq. since the core problem was fixed in HADOO

[jira] Commented: (HADOOP-6097) Multiple bugs w/ Hadoop archives

[ https://issues.apache.org/jira/browse/HADOOP-6097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12759193#action_12759193 ] Koji Noguchi commented on HADOOP-6097: -- Do we want a test case of handling two har fi

[jira] Commented: (HADOOP-6284) Any hadoop commands crashing jvm (SIGBUS) when /tmp (tmpfs) is full

[ https://issues.apache.org/jira/browse/HADOOP-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12758913#action_12758913 ] Koji Noguchi commented on HADOOP-6284: -- bq. How about we add a HADOOP_JVM_OPTS? I on

[jira] Commented: (HADOOP-6284) Any hadoop commands crashing jvm (SIGBUS) when /tmp (tmpfs) is full

[ https://issues.apache.org/jira/browse/HADOOP-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12758896#action_12758896 ] Koji Noguchi commented on HADOOP-6284: -- Reproducing this error, it is crashing when t

[jira] Created: (HADOOP-6284) Any hadoop commands crashing jvm (SIGBUS) when /tmp (tmpfs) is full

Any hadoop commands crashing jvm (SIGBUS) when /tmp (tmpfs) is full Key: HADOOP-6284 URL: https://issues.apache.org/jira/browse/HADOOP-6284 Project: Hadoop Common Issue Ty

[jira] Commented: (HADOOP-6280) Uniform way of setting default param for ToolRunner (like passing -Ddfs.umask)

[ https://issues.apache.org/jira/browse/HADOOP-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12758779#action_12758779 ] Koji Noguchi commented on HADOOP-6280: -- My initial thought was to simply insert + e

[jira] Created: (HADOOP-6280) Uniform way of setting default param for ToolRunner (like passing -Ddfs.umask)

Uniform way of setting default param for ToolRunner (like passing -Ddfs.umask) -- Key: HADOOP-6280 URL: https://issues.apache.org/jira/browse/HADOOP-6280 Project: Hadoop Commo

[jira] Created: (HADOOP-6239) Command-line for append

Command-line for append --- Key: HADOOP-6239 URL: https://issues.apache.org/jira/browse/HADOOP-6239 Project: Hadoop Common Issue Type: New Feature Components: fs Reporter: Koji Noguchi Once append

[jira] Updated: (HADOOP-6239) Command-line for append

[ https://issues.apache.org/jira/browse/HADOOP-6239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated HADOOP-6239: - Description: Once append is implemented in hdfs, it would be nice if users can append files fro

[jira] Commented: (HADOOP-6097) Multiple bugs w/ Hadoop archives

[ https://issues.apache.org/jira/browse/HADOOP-6097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745477#action_12745477 ] Koji Noguchi commented on HADOOP-6097: -- Ben, sorry about that. I was wrong. I got co

[jira] Commented: (HADOOP-6097) Multiple bugs w/ Hadoop archives

[ https://issues.apache.org/jira/browse/HADOOP-6097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745096#action_12745096 ] Koji Noguchi commented on HADOOP-6097: -- Ben, for the caching part, HarFileSystem.jav

[jira] Resolved: (HADOOP-6202) harchive: Har doesn't work on files having '%' character

[ https://issues.apache.org/jira/browse/HADOOP-6202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved HADOOP-6202. -- Resolution: Duplicate This is reported in HADOOP-6097. > harchive: Har doesn't work on file

[jira] Commented: (HADOOP-6202) harchive: Har doesn't work on files having '%' character

[ https://issues.apache.org/jira/browse/HADOOP-6202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744687#action_12744687 ] Koji Noguchi commented on HADOOP-6202: -- Probably this is happening because HarFileSys

[jira] Updated: (HADOOP-6202) harchive: Har doesn't work on files having '%' character

[ https://issues.apache.org/jira/browse/HADOOP-6202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated HADOOP-6202: - Description: If I have a harchive file test.har that contain /a/b /a/b/abc%cde /a/b/fgh {n

[jira] Created: (HADOOP-6202) harchive: Har doesn't work on files having '%' character

harchive: Har doesn't work on files having '%' character -- Key: HADOOP-6202 URL: https://issues.apache.org/jira/browse/HADOOP-6202 Project: Hadoop Common Issue Type: Bug Co

[jira] Commented: (HADOOP-6167) bin/hadoop script doesn't allow for different memory settings for each daemon type

[ https://issues.apache.org/jira/browse/HADOOP-6167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12735846#action_12735846 ] Koji Noguchi commented on HADOOP-6167: -- bq. how does this not work? I agree with rya