[jira] [Updated] (HADOOP-17011) Tolerate leading and trailing spaces in fs.defaultFS
[ https://issues.apache.org/jira/browse/HADOOP-17011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated HADOOP-17011: Target Version/s: 3.4.0 Affects Version/s: 3.4.0 > Tolerate leading and trailing spaces in fs.defaultFS > > > Key: HADOOP-17011 > URL: https://issues.apache.org/jira/browse/HADOOP-17011 > Project: Hadoop Common > Issue Type: Bug > Components: common >Affects Versions: 3.4.0 >Reporter: Ctest >Assignee: Ctest >Priority: Major > Fix For: 3.4.0 > > Attachments: HADOOP-17011-001.patch, HADOOP-17011-002.patch, > HADOOP-17011-003.patch, HADOOP-17011-004.patch, HADOOP-17011-005.patch > > > *Problem:* > Currently, `getDefaultUri` is using `conf.get` to get the value of > `fs.defaultFS`, which means that the trailing whitespace after a valid URI > won’t be removed and could stop namenode and datanode from starting up. > > *How to reproduce (Hadoop-2.8.5):* > Set the configuration > {code:java} > > fs.defaultFS > hdfs://localhost:9000 > {code} > In core-site.xml (there is a whitespace after 9000) and start HDFS. > Namenode and datanode won’t start and the log message is: > {code:java} > 2020-04-23 11:09:48,198 ERROR > org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. > java.lang.IllegalArgumentException: Illegal character in authority at index > 7: hdfs://localhost:9000 > at java.net.URI.create(URI.java:852) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.setClientNamenodeAddress(NameNode.java:440) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:897) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:885) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1626) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1694) > Caused by: java.net.URISyntaxException: Illegal character in authority at > index 7: hdfs://localhost:9000 > at java.net.URI$Parser.fail(URI.java:2848) > at java.net.URI$Parser.parseAuthority(URI.java:3186) > at java.net.URI$Parser.parseHierarchical(URI.java:3097) > at java.net.URI$Parser.parse(URI.java:3053) > at java.net.URI.(URI.java:588) > at java.net.URI.create(URI.java:850) > ... 5 more > {code} > > *Solution:* > Use `getTrimmed` instead of `get` for `fs.defaultFS`: > {code:java} > public static URI getDefaultUri(Configuration conf) { > URI uri = > URI.create(fixName(conf.getTrimmed(FS_DEFAULT_NAME_KEY, DEFAULT_FS))); > if (uri.getScheme() == null) { > throw new IllegalArgumentException("No scheme in default FS: " + uri); > } > return uri; > } > {code} > I have submitted a patch for trunk about this. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17011) Tolerate leading and trailing spaces in fs.defaultFS
[ https://issues.apache.org/jira/browse/HADOOP-17011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-17011: --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) +1 # The failing test {{TestSnappyCompressorDecompressor}} an d{{TestCompressorDecompressor}} are tracked by HADOOP-16768 # The failing test {{TestBalancer}} is tracked by HDFS-13975 # Findbug etc are not related to this patch. Committed to {{trunk}} branch. Thanks for your contribution [~ctest.team]. Thanks for reviewing [~ayushtkn] > Tolerate leading and trailing spaces in fs.defaultFS > > > Key: HADOOP-17011 > URL: https://issues.apache.org/jira/browse/HADOOP-17011 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: Ctest >Assignee: Ctest >Priority: Major > Fix For: 3.4.0 > > Attachments: HADOOP-17011-001.patch, HADOOP-17011-002.patch, > HADOOP-17011-003.patch, HADOOP-17011-004.patch, HADOOP-17011-005.patch > > > *Problem:* > Currently, `getDefaultUri` is using `conf.get` to get the value of > `fs.defaultFS`, which means that the trailing whitespace after a valid URI > won’t be removed and could stop namenode and datanode from starting up. > > *How to reproduce (Hadoop-2.8.5):* > Set the configuration > {code:java} > > fs.defaultFS > hdfs://localhost:9000 > {code} > In core-site.xml (there is a whitespace after 9000) and start HDFS. > Namenode and datanode won’t start and the log message is: > {code:java} > 2020-04-23 11:09:48,198 ERROR > org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. > java.lang.IllegalArgumentException: Illegal character in authority at index > 7: hdfs://localhost:9000 > at java.net.URI.create(URI.java:852) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.setClientNamenodeAddress(NameNode.java:440) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:897) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:885) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1626) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1694) > Caused by: java.net.URISyntaxException: Illegal character in authority at > index 7: hdfs://localhost:9000 > at java.net.URI$Parser.fail(URI.java:2848) > at java.net.URI$Parser.parseAuthority(URI.java:3186) > at java.net.URI$Parser.parseHierarchical(URI.java:3097) > at java.net.URI$Parser.parse(URI.java:3053) > at java.net.URI.(URI.java:588) > at java.net.URI.create(URI.java:850) > ... 5 more > {code} > > *Solution:* > Use `getTrimmed` instead of `get` for `fs.defaultFS`: > {code:java} > public static URI getDefaultUri(Configuration conf) { > URI uri = > URI.create(fixName(conf.getTrimmed(FS_DEFAULT_NAME_KEY, DEFAULT_FS))); > if (uri.getScheme() == null) { > throw new IllegalArgumentException("No scheme in default FS: " + uri); > } > return uri; > } > {code} > I have submitted a patch for trunk about this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17011) Tolerate leading and trailing spaces in fs.defaultFS
[ https://issues.apache.org/jira/browse/HADOOP-17011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ctest updated HADOOP-17011: --- Attachment: HADOOP-17011-005.patch > Tolerate leading and trailing spaces in fs.defaultFS > > > Key: HADOOP-17011 > URL: https://issues.apache.org/jira/browse/HADOOP-17011 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: Ctest >Assignee: Ctest >Priority: Major > Attachments: HADOOP-17011-001.patch, HADOOP-17011-002.patch, > HADOOP-17011-003.patch, HADOOP-17011-004.patch, HADOOP-17011-005.patch > > > *Problem:* > Currently, `getDefaultUri` is using `conf.get` to get the value of > `fs.defaultFS`, which means that the trailing whitespace after a valid URI > won’t be removed and could stop namenode and datanode from starting up. > > *How to reproduce (Hadoop-2.8.5):* > Set the configuration > {code:java} > > fs.defaultFS > hdfs://localhost:9000 > {code} > In core-site.xml (there is a whitespace after 9000) and start HDFS. > Namenode and datanode won’t start and the log message is: > {code:java} > 2020-04-23 11:09:48,198 ERROR > org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. > java.lang.IllegalArgumentException: Illegal character in authority at index > 7: hdfs://localhost:9000 > at java.net.URI.create(URI.java:852) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.setClientNamenodeAddress(NameNode.java:440) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:897) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:885) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1626) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1694) > Caused by: java.net.URISyntaxException: Illegal character in authority at > index 7: hdfs://localhost:9000 > at java.net.URI$Parser.fail(URI.java:2848) > at java.net.URI$Parser.parseAuthority(URI.java:3186) > at java.net.URI$Parser.parseHierarchical(URI.java:3097) > at java.net.URI$Parser.parse(URI.java:3053) > at java.net.URI.(URI.java:588) > at java.net.URI.create(URI.java:850) > ... 5 more > {code} > > *Solution:* > Use `getTrimmed` instead of `get` for `fs.defaultFS`: > {code:java} > public static URI getDefaultUri(Configuration conf) { > URI uri = > URI.create(fixName(conf.getTrimmed(FS_DEFAULT_NAME_KEY, DEFAULT_FS))); > if (uri.getScheme() == null) { > throw new IllegalArgumentException("No scheme in default FS: " + uri); > } > return uri; > } > {code} > I have submitted a patch for trunk about this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17011) Tolerate leading and trailing spaces in fs.defaultFS
[ https://issues.apache.org/jira/browse/HADOOP-17011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ctest updated HADOOP-17011: --- Attachment: HADOOP-17011-004.patch > Tolerate leading and trailing spaces in fs.defaultFS > > > Key: HADOOP-17011 > URL: https://issues.apache.org/jira/browse/HADOOP-17011 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: Ctest >Assignee: Ctest >Priority: Major > Attachments: HADOOP-17011-001.patch, HADOOP-17011-002.patch, > HADOOP-17011-003.patch, HADOOP-17011-004.patch > > > *Problem:* > Currently, `getDefaultUri` is using `conf.get` to get the value of > `fs.defaultFS`, which means that the trailing whitespace after a valid URI > won’t be removed and could stop namenode and datanode from starting up. > > *How to reproduce (Hadoop-2.8.5):* > Set the configuration > {code:java} > > fs.defaultFS > hdfs://localhost:9000 > {code} > In core-site.xml (there is a whitespace after 9000) and start HDFS. > Namenode and datanode won’t start and the log message is: > {code:java} > 2020-04-23 11:09:48,198 ERROR > org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. > java.lang.IllegalArgumentException: Illegal character in authority at index > 7: hdfs://localhost:9000 > at java.net.URI.create(URI.java:852) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.setClientNamenodeAddress(NameNode.java:440) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:897) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:885) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1626) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1694) > Caused by: java.net.URISyntaxException: Illegal character in authority at > index 7: hdfs://localhost:9000 > at java.net.URI$Parser.fail(URI.java:2848) > at java.net.URI$Parser.parseAuthority(URI.java:3186) > at java.net.URI$Parser.parseHierarchical(URI.java:3097) > at java.net.URI$Parser.parse(URI.java:3053) > at java.net.URI.(URI.java:588) > at java.net.URI.create(URI.java:850) > ... 5 more > {code} > > *Solution:* > Use `getTrimmed` instead of `get` for `fs.defaultFS`: > {code:java} > public static URI getDefaultUri(Configuration conf) { > URI uri = > URI.create(fixName(conf.getTrimmed(FS_DEFAULT_NAME_KEY, DEFAULT_FS))); > if (uri.getScheme() == null) { > throw new IllegalArgumentException("No scheme in default FS: " + uri); > } > return uri; > } > {code} > I have submitted a patch for trunk about this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17011) Tolerate leading and trailing spaces in fs.defaultFS
[ https://issues.apache.org/jira/browse/HADOOP-17011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ctest updated HADOOP-17011: --- Attachment: HADOOP-17011-003.patch > Tolerate leading and trailing spaces in fs.defaultFS > > > Key: HADOOP-17011 > URL: https://issues.apache.org/jira/browse/HADOOP-17011 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: Ctest >Assignee: Ctest >Priority: Major > Attachments: HADOOP-17011-001.patch, HADOOP-17011-002.patch, > HADOOP-17011-003.patch > > > *Problem:* > Currently, `getDefaultUri` is using `conf.get` to get the value of > `fs.defaultFS`, which means that the trailing whitespace after a valid URI > won’t be removed and could stop namenode and datanode from starting up. > > *How to reproduce (Hadoop-2.8.5):* > Set the configuration > {code:java} > > fs.defaultFS > hdfs://localhost:9000 > {code} > In core-site.xml (there is a whitespace after 9000) and start HDFS. > Namenode and datanode won’t start and the log message is: > {code:java} > 2020-04-23 11:09:48,198 ERROR > org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. > java.lang.IllegalArgumentException: Illegal character in authority at index > 7: hdfs://localhost:9000 > at java.net.URI.create(URI.java:852) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.setClientNamenodeAddress(NameNode.java:440) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:897) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:885) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1626) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1694) > Caused by: java.net.URISyntaxException: Illegal character in authority at > index 7: hdfs://localhost:9000 > at java.net.URI$Parser.fail(URI.java:2848) > at java.net.URI$Parser.parseAuthority(URI.java:3186) > at java.net.URI$Parser.parseHierarchical(URI.java:3097) > at java.net.URI$Parser.parse(URI.java:3053) > at java.net.URI.(URI.java:588) > at java.net.URI.create(URI.java:850) > ... 5 more > {code} > > *Solution:* > Use `getTrimmed` instead of `get` for `fs.defaultFS`: > {code:java} > public static URI getDefaultUri(Configuration conf) { > URI uri = > URI.create(fixName(conf.getTrimmed(FS_DEFAULT_NAME_KEY, DEFAULT_FS))); > if (uri.getScheme() == null) { > throw new IllegalArgumentException("No scheme in default FS: " + uri); > } > return uri; > } > {code} > I have submitted a patch for trunk about this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17011) Tolerate leading and trailing spaces in fs.defaultFS
[ https://issues.apache.org/jira/browse/HADOOP-17011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ctest updated HADOOP-17011: --- Attachment: HADOOP-17011-002.patch > Tolerate leading and trailing spaces in fs.defaultFS > > > Key: HADOOP-17011 > URL: https://issues.apache.org/jira/browse/HADOOP-17011 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: Ctest >Assignee: Ctest >Priority: Major > Attachments: HADOOP-17011-001.patch, HADOOP-17011-002.patch > > > *Problem:* > Currently, `getDefaultUri` is using `conf.get` to get the value of > `fs.defaultFS`, which means that the trailing whitespace after a valid URI > won’t be removed and could stop namenode and datanode from starting up. > > *How to reproduce (Hadoop-2.8.5):* > Set the configuration > {code:java} > > fs.defaultFS > hdfs://localhost:9000 > {code} > In core-site.xml (there is a whitespace after 9000) and start HDFS. > Namenode and datanode won’t start and the log message is: > {code:java} > 2020-04-23 11:09:48,198 ERROR > org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. > java.lang.IllegalArgumentException: Illegal character in authority at index > 7: hdfs://localhost:9000 > at java.net.URI.create(URI.java:852) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.setClientNamenodeAddress(NameNode.java:440) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:897) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:885) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1626) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1694) > Caused by: java.net.URISyntaxException: Illegal character in authority at > index 7: hdfs://localhost:9000 > at java.net.URI$Parser.fail(URI.java:2848) > at java.net.URI$Parser.parseAuthority(URI.java:3186) > at java.net.URI$Parser.parseHierarchical(URI.java:3097) > at java.net.URI$Parser.parse(URI.java:3053) > at java.net.URI.(URI.java:588) > at java.net.URI.create(URI.java:850) > ... 5 more > {code} > > *Solution:* > Use `getTrimmed` instead of `get` for `fs.defaultFS`: > {code:java} > public static URI getDefaultUri(Configuration conf) { > URI uri = > URI.create(fixName(conf.getTrimmed(FS_DEFAULT_NAME_KEY, DEFAULT_FS))); > if (uri.getScheme() == null) { > throw new IllegalArgumentException("No scheme in default FS: " + uri); > } > return uri; > } > {code} > I have submitted a patch for trunk about this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17011) Tolerate leading and trailing spaces in fs.defaultFS
[ https://issues.apache.org/jira/browse/HADOOP-17011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ctest updated HADOOP-17011: --- Summary: Tolerate leading and trailing spaces in fs.defaultFS (was: Trailing whitespace in fs.defaultFS will crash namenode and datanode) > Tolerate leading and trailing spaces in fs.defaultFS > > > Key: HADOOP-17011 > URL: https://issues.apache.org/jira/browse/HADOOP-17011 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: Ctest >Assignee: Ctest >Priority: Major > Attachments: HADOOP-17011-001.patch > > > *Problem:* > Currently, `getDefaultUri` is using `conf.get` to get the value of > `fs.defaultFS`, which means that the trailing whitespace after a valid URI > won’t be removed and could stop namenode and datanode from starting up. > > *How to reproduce (Hadoop-2.8.5):* > Set the configuration > {code:java} > > fs.defaultFS > hdfs://localhost:9000 > {code} > In core-site.xml (there is a whitespace after 9000) and start HDFS. > Namenode and datanode won’t start and the log message is: > {code:java} > 2020-04-23 11:09:48,198 ERROR > org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. > java.lang.IllegalArgumentException: Illegal character in authority at index > 7: hdfs://localhost:9000 > at java.net.URI.create(URI.java:852) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.setClientNamenodeAddress(NameNode.java:440) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:897) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:885) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1626) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1694) > Caused by: java.net.URISyntaxException: Illegal character in authority at > index 7: hdfs://localhost:9000 > at java.net.URI$Parser.fail(URI.java:2848) > at java.net.URI$Parser.parseAuthority(URI.java:3186) > at java.net.URI$Parser.parseHierarchical(URI.java:3097) > at java.net.URI$Parser.parse(URI.java:3053) > at java.net.URI.(URI.java:588) > at java.net.URI.create(URI.java:850) > ... 5 more > {code} > > *Solution:* > Use `getTrimmed` instead of `get` for `fs.defaultFS`: > {code:java} > public static URI getDefaultUri(Configuration conf) { > URI uri = > URI.create(fixName(conf.getTrimmed(FS_DEFAULT_NAME_KEY, DEFAULT_FS))); > if (uri.getScheme() == null) { > throw new IllegalArgumentException("No scheme in default FS: " + uri); > } > return uri; > } > {code} > I have submitted a patch for trunk about this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org