Jenkins build is back to normal : Hadoop-common-trunk-Java8 #278
See https://builds.apache.org/job/Hadoop-common-trunk-Java8/278/changes
Re: Planning Hadoop 2.6.1 release
Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
+YARN-3011: without which, NM is likely to crash, and cannot come back with recovery enabled. Thanks, Zhijie From: Wangda Tan wheele...@gmail.com Sent: Wednesday, August 05, 2015 10:42 AM To: yarn-...@hadoop.apache.org Cc: mapreduce-...@hadoop.apache.org; common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
+1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
Another +1 to add those fixes. YARN-3487 bug can grind a large cluster to a halt repeatedly -Allan Allan Wilson | Sr. Software Engineer | Pandora m 919.841.2449 | awils...@pandora.com On 8/5/15, 1:52 PM, Rich Haase rha...@pandora.com wrote: +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
[jira] [Created] (HADOOP-12303) test-patch pylint plugin fails silently and votes +1 incorrectly
Kengo Seki created HADOOP-12303:
---
Summary: test-patch pylint plugin fails silently and votes +1
incorrectly
Key: HADOOP-12303
URL: https://issues.apache.org/jira/browse/HADOOP-12303
Project: Hadoop Common
Issue Type: Sub-task
Components: yetus
Affects Versions: HADOOP-12111
Reporter: Kengo Seki
This patch
{code}
[sekikn@mobile hadoop]$ cat /tmp/test.patch
diff --git a/dev-support/releasedocmaker.py b/dev-support/releasedocmaker.py
index 37bd58a..7cd6dd3 100755
--- a/dev-support/releasedocmaker.py
+++ b/dev-support/releasedocmaker.py
@@ -580,4 +580,4 @@ def main():
sys.exit(1)
if __name__ == __main__:
-main()
+main( )
{code}
is supposed to cause the following pylint errors.
{code}
C:583, 0: No space allowed after bracket
main( )
^ (bad-whitespace)
C:583, 0: No space allowed before bracket
main( )
^ (bad-whitespace)
{code}
But the system locale is set as follows, pylint check is passed, and there is
no pylint output.
{code}
[sekikn@mobile hadoop]$ locale
LANG=
LC_COLLATE=C
LC_CTYPE=UTF-8
LC_MESSAGES=C
LC_MONETARY=C
LC_NUMERIC=C
LC_TIME=C
LC_ALL=
[sekikn@mobile hadoop]$ dev-support/test-patch.sh
--basedir=/Users/sekikn/dev/hadoop --project=hadoop /tmp/test.patch
(snip)
| Vote | Subsystem | Runtime | Comment
| +1 | @author | 0m 00s| The patch does not contain any @author
| ||| tags.
| +1 |asflicense | 0m 21s| Patch does not generate ASF License
| ||| warnings.
| +1 |pylint | 0m 01s| There were no new pylint issues.
| +1 |whitespace | 0m 00s| Patch has no whitespace issues.
| || 0m 24s|
(snip)
[sekikn@mobile hadoop]$ cat
/private/tmp/test-patch-hadoop/8656/branch-pylint-result.txt
[sekikn@mobile hadoop]$ cat
/private/tmp/test-patch-hadoop/8656/patch-pylint-result.txt
[sekikn@mobile hadoop]$ cat
/private/tmp/test-patch-hadoop/8656/diff-patch-pylint.txt
{code}
Removing '2/dev/null' from pylint.sh reveals the root cause. Setting LC_ALL or
LC_CTYPE such like 'en_US.UTF-8' solves this problem.
{code}
pylint plugin: prepatch
Running pylint against modified python scripts.
No config file found, using default configuration
Traceback (most recent call last):
File /usr/local/bin/pylint, line 11, in module
sys.exit(run_pylint())
File /Library/Python/2.7/site-packages/pylint/__init__.py, line 23, in
run_pylint
Run(sys.argv[1:])
File /Library/Python/2.7/site-packages/pylint/lint.py, line 1332, in
__init__
linter.check(args)
File /Library/Python/2.7/site-packages/pylint/lint.py, line 747, in check
self._do_check(files_or_modules)
File /Library/Python/2.7/site-packages/pylint/lint.py, line 869, in
_do_check
self.check_astroid_module(ast_node, walker, rawcheckers, tokencheckers)
File /Library/Python/2.7/site-packages/pylint/lint.py, line 944, in
check_astroid_module
checker.process_tokens(tokens)
File /Library/Python/2.7/site-packages/pylint/checkers/format.py, line 743,
in process_tokens
self.check_indent_level(token, indents[-1]+1, line_num)
File /Library/Python/2.7/site-packages/pylint/checkers/format.py, line 963,
in check_indent_level
expected * unit_size))
File /Library/Python/2.7/site-packages/pylint/checkers/__init__.py, line
101, in add_message
self.linter.add_message(msg_id, line, node, args, confidence)
File /Library/Python/2.7/site-packages/pylint/utils.py, line 410, in
add_message
(abspath, path, module, obj, line or 1, col_offset or 0), msg, confidence))
File /Library/Python/2.7/site-packages/pylint/reporters/text.py, line 61,
in handle_message
self.write_message(msg)
File /Library/Python/2.7/site-packages/pylint/reporters/text.py, line 51,
in write_message
self.writeln(msg.format(self._template))
File /Library/Python/2.7/site-packages/pylint/reporters/__init__.py, line
94, in writeln
print(self.encode(string), file=self.out)
File /Library/Python/2.7/site-packages/pylint/reporters/__init__.py, line
84, in encode
locale.getdefaultlocale()[1] or
File
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py,
line 511, in getdefaultlocale
return _parse_localename(localename)
File
[jira] [Created] (HADOOP-12304) Applications using FileContext fail with the default file system configured to be wasb/s3/etc.
Chris Nauroth created HADOOP-12304:
--
Summary: Applications using FileContext fail with the default file
system configured to be wasb/s3/etc.
Key: HADOOP-12304
URL: https://issues.apache.org/jira/browse/HADOOP-12304
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.7.1, 2.7.0
Reporter: Chris Nauroth
Assignee: Chris Nauroth
Priority: Blocker
HADOOP-11618 fixed a bug with {{DelegateToFileSystem}} using the wrong default
port. As a side effect of this patch, file path URLs that previously had no
port now insert :0 for the port, as per the default implementation of
{{FileSystem#getDefaultPort}}. At runtime, this can cause an application to
erroneously try contacting port 0 for a remote blob store service. The
connection fails. Ultimately, this renders wasb, s3, and probably custom file
system implementations outside the Hadoop source tree completely unusable as
the default file system.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Planning Hadoop 2.6.1 release
I would like to nominate YARN-3832 as 2.6.1 candidate which is critical and I also saw it happened recently on a 2.6.0 cluster. Hope this is not too late. Thanks, Junping From: Rich Haase rha...@pandora.com Sent: Thursday, August 06, 2015 1:52 AM To: mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
Re: Planning Hadoop 2.6.1 release
If we haven't frozen yet, HDFS-8850 is a straight forward fix that is currently only in 2.8+ and would benefit 2.6 and 2.7. On Wed, Aug 5, 2015 at 2:56 PM, Junping Du j...@hortonworks.com wrote: I would like to nominate YARN-3832 as 2.6.1 candidate which is critical and I also saw it happened recently on a 2.6.0 cluster. Hope this is not too late. Thanks, Junping From: Rich Haase rha...@pandora.com Sent: Thursday, August 06, 2015 1:52 AM To: mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: common-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin -- Sean
RE: Planning Hadoop 2.6.1 release
Can we add following fixes to 2.6.1? YARN-3733 YARN-2865 YARN-3990 YARN-2894 Thanks Regards Rohith Sharma K S -Original Message- From: Allan Wilson [mailto:awils...@pandora.com] Sent: 05 August 2015 23:25 To: common-dev@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Cc: hdfs-...@hadoop.apache.org Subject: Re: Planning Hadoop 2.6.1 release Another +1 to add those fixes. YARN-3487 bug can grind a large cluster to a halt repeatedly -Allan Allan Wilson | Sr. Software Engineer | Pandora m 919.841.2449 | awils...@pandora.com On 8/5/15, 1:52 PM, Rich Haase rha...@pandora.com wrote: +1 to add those fixes. Rich Haase | Sr. Software Engineer | Pandora m 303.887.1146 | rha...@pandora.com On 8/5/15, 11:42 AM, Wangda Tan wheele...@gmail.com wrote: Can we add following two fixes to 2.6.1? https://issues.apache.org/jira/browse/YARN-2922 and https://issues.apache.org/jira/browse/YARN-3487. They're not fatal issue, but they can cause lots of issue in a large cluster. Thanks, Wangda On Mon, Aug 3, 2015 at 1:21 PM, Sangjin Lee sj...@apache.org wrote: See my later update in the thread. HDFS-7704 is in the list. Thanks, Sangjin On Mon, Aug 3, 2015 at 1:19 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Makes sense, it was caused by HDFS-7704 which got into 2.7.0 only and is not part of the candidate list. Removed HDFS-7916 from the list. Thanks +Vinod On Jul 24, 2015, at 6:32 PM, Sangjin Lee sj...@apache.org wrote: Out of the JIRAs we proposed, please remove HDFS-7916. I don't think it applies to 2.6. Thanks, Sangjin
