[jira] [Created] (HADOOP-9465) TestJobImpl->testJobNoTasks fails ..
Amir Sanjar created HADOOP-9465:
---
Summary: TestJobImpl->testJobNoTasks fails ..
Key: HADOOP-9465
URL: https://issues.apache.org/jira/browse/HADOOP-9465
Project: Hadoop Common
Issue Type: Bug
Components: conf
Affects Versions: 2.0.3-alpha
Environment: Linux + IBM JAVA 6
Reporter: Amir Sanjar
I am not sure if this is a testcase or a design issue. During execution of
TestJobImpl->testJobNoTasks() there is an assertion made based on the order of
key/value pairs stored in adjacency list. However adjacency list was created by
Configuration->getValByRegex() as a HashMap (order is not guaranteed):
Testcase:
JobSubmittedEventHandler jseHandler = new JobSubmittedEventHandler("testId",
"testName", "testNodeName", "\"key2\"=\"value2\" \"key1\"=\"value1\" ");
try {
Assert.assertTrue(jseHandler.getAssertValue()); <===
Configuration->getValByRegex():
public Map getValByRegex(String regex) {
Pattern p = Pattern.compile(regex);
Map result = new HashMap(); <===
as we all know, HashMap makes absolutely no guarantees about the iteration
order. It can (and will) even change completely when new elements are added.
Changing HashMap to LinkedHashMap fixes the ordering inconsistency, however
with a small performance side effect.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9464) In test*Conf.xml regexp replication factor set to 1
Anatoli Fomenko created HADOOP-9464:
---
Summary: In test*Conf.xml regexp replication factor set to 1
Key: HADOOP-9464
URL: https://issues.apache.org/jira/browse/HADOOP-9464
Project: Hadoop Common
Issue Type: Bug
Components: test
Affects Versions: 2.0.4-alpha
Reporter: Anatoli Fomenko
Priority: Critical
Fix For: 2.0.4-alpha
In some Hadoop smoke tests (testHDFSConf.xml), in expected output for
RegexpComparator, a replication factor is hard coded to 1,
{noformat}
ls: file using absolute path
-fs NAMENODE -touchz /file1
-fs NAMENODE -ls /file1
-fs NAMENODE -rm /file1
TokenComparator
Found 1 items
RegexpComparator
^-rw-r--r--( )*1( )*[a-z]*( )*supergroup( )*0(
)*[0-9]{4,}-[0-9]{2,}-[0-9]{2,} [0-9]{2,}:[0-9]{2,}( )*/file1
{noformat}
such as the first "1" in
{noformat}
^-rw-r--r--( )*1( )*[a-z]*( )*supergroup( )*0(
)*[0-9]{4,}-[0-9]{2,}-[0-9]{2,} [0-9]{2,}:[0-9]{2,}( )*/file1
{noformat}.
We found in Bigtop testing on a standalone cluster that such tests fail with a
default replication factor.
Please update the regexp in test*Conf.xml files to add a flexibility for a
replication factor that would allow to execute these tests with a variety of
clusters, or inside the Bigtop.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9463) branch-1-win fails to build with OpenJDK7
Ivan Mitic created HADOOP-9463: -- Summary: branch-1-win fails to build with OpenJDK7 Key: HADOOP-9463 URL: https://issues.apache.org/jira/browse/HADOOP-9463 Project: Hadoop Common Issue Type: Bug Reporter: Ivan Mitic Assignee: Ivan Mitic Fix For: 1-win Build fails with the following error: I:\svn\trunk_rebase\hadoop-common [branch-1-win]> ant clean winpkg Buildfile: I:\svn\trunk_rebase\hadoop-common\build.xml BUILD FAILED I:\svn\trunk_rebase\hadoop-common\build.xml:87: Unable to create javax script engine for javascript -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-9462) ulimit output is displayed on stdout each time I start a daemon.
[
https://issues.apache.org/jira/browse/HADOOP-9462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruno Mahé resolved HADOOP-9462.
Resolution: Duplicate
Duplicate of HADOOP-9379.
> ulimit output is displayed on stdout each time I start a daemon.
>
>
> Key: HADOOP-9462
> URL: https://issues.apache.org/jira/browse/HADOOP-9462
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Bruno Mahé
>
> {noformat}
> [root@master ~]# /etc/init.d/hadoop-hdfs-namenode start
> Starting Hadoop namenode: [ OK ]
> starting namenode, logging to
> /var/log/hadoop-hdfs/hadoop-hdfs-namenode-master.out
> ulimit -a for user hdfs
> core file size (blocks, -c) 0
> data seg size (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size (blocks, -f) unlimited
> pending signals (-i) 30731
> max locked memory (kbytes, -l) 64
> max memory size (kbytes, -m) unlimited
> open files (-n) 32768
> pipe size(512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority (-r) 0
> stack size (kbytes, -s) 10240
> cpu time (seconds, -t) unlimited
> max user processes (-u) 65536
> virtual memory (kbytes, -v) unlimited
> file locks (-x) unlimited
> {noformat}
> This should be displayed on daemon startup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9462) ulimit output is displayed on stdout each time I start a daemon.
Bruno Mahé created HADOOP-9462:
--
Summary: ulimit output is displayed on stdout each time I start a
daemon.
Key: HADOOP-9462
URL: https://issues.apache.org/jira/browse/HADOOP-9462
Project: Hadoop Common
Issue Type: Bug
Reporter: Bruno Mahé
{noformat}
[root@master ~]# /etc/init.d/hadoop-hdfs-namenode start
Starting Hadoop namenode: [ OK ]
starting namenode, logging to
/var/log/hadoop-hdfs/hadoop-hdfs-namenode-master.out
ulimit -a for user hdfs
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 30731
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 32768
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 65536
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
{noformat}
This should be displayed on daemon startup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9461) JobTracker and NameNode both grant delegation tokens to non-secure clients
Harsh J created HADOOP-9461: --- Summary: JobTracker and NameNode both grant delegation tokens to non-secure clients Key: HADOOP-9461 URL: https://issues.apache.org/jira/browse/HADOOP-9461 Project: Hadoop Common Issue Type: Bug Components: security Reporter: Harsh J Assignee: Harsh J Priority: Minor If one looks at the MAPREDUCE-1516 added logic in JobTracker.java's isAllowedDelegationTokenOp() method, and apply non-secure states of UGI.isSecurityEnabled == false and authMethod == SIMPLE, the return result is true when the intention is false (due to the shorted conditionals). This is allowing non-secure JobClients to easily request and use DelegationTokens and cause unwanted errors to be printed in the JobTracker when the renewer attempts to run. Ideally such clients ought to get an error if they request a DT in non-secure mode. HDFS in trunk and branch-1 both too have the same problem. Trunk MR (HistoryServer) and YARN are however, unaffected due to a simpler, inlined logic instead of reuse of this faulty method. Note that fixing this will break Oozie today, due to the merged logic of OOZIE-734. Oozie will require a fix as well if this is to be fixed in branch-1. As a result, I'm going to mark this as an Incompatible Change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
