[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17717711#comment-17717711
]
Steve Loughran commented on HADOOP-18724:
-
usual homework: version, module etc
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ayush Saxena
>Priority: Major
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718087#comment-17718087
]
Steve Loughran commented on HADOOP-18724:
-
Ok, I see the problem, adding opt(str, double) alongside opt(string, long) is
causing the compiler to (how?) choose the more relaxed type over the one of the
type passed in. This must be some quirk of numeric type inference which is
possibly different from the rest of java overloading.
This is bad. I think it'll be from HADOOP-16202, and only going to hit code
compiled against 3.3.5, not 3.3.3-.3.3.4.
Immediate Workaround for apps: use opt(string, Long.toString(l)) to convert the
string.p
I have an idea for a fix for this which is
* add explicit optLong() optDouble() calls
* have all the existing opt(string, long), double and float calls *forward to
`optLong()`
this means existing code can't set doubles, but we can be confident that
whatever overloaded method is linked to, a long is always set.
what breaks?
* nothing can set opt(string, double) but AFAIK nothing does this, it was added
due to some perceived need for it, not an actual one at present
* I've been doing some passthrough code which collects this for some testing on
hbase-fs; not sure of its state
* my WiP fs api should go for the new setters only.
https://github.com/steveloughran/fs-api-shim/blob/main/fs-api-shim-library/src/main/java/org/apache/hadoop/fs/shim/api/FSBuilder.java
Shall I do this?
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Major
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718299#comment-17718299
]
Ayush Saxena commented on HADOOP-18724:
---
Sounds good to me
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718342#comment-17718342
]
ASF GitHub Bot commented on HADOOP-18724:
-
steveloughran opened a new pull request, #5611:
URL: https://github.com/apache/hadoop/pull/5611
+s3a openfile parser downgrades on all parser failures.
No tests for this yet, this is up to get the API reviewed
### How was this patch tested?
### For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718386#comment-17718386
]
ASF GitHub Bot commented on HADOOP-18724:
-
hadoop-yetus commented on PR #5611:
URL: https://github.com/apache/hadoop/pull/5611#issuecomment-1530685675
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 17m 21s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 0s | | trunk passed |
| +1 :green_heart: | compile | 23m 11s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 20m 31s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 46s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 54s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 4s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 35s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 33s | | the patch passed |
| +1 :green_heart: | compile | 22m 20s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| -1 :x: | javac | 22m 20s |
[/results-compile-javac-root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/1/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt)
| root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 generated 3 new + 2806 unchanged
- 0 fixed = 2809 total (was 2806) |
| +1 :green_heart: | compile | 20m 32s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| -1 :x: | javac | 20m 32s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/1/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt)
| root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 generated 3 new + 2604
unchanged - 0 fixed = 2607 total (was 2604) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 35s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 47s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 1m 24s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 4m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 34s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 39s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 11s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 4s | | The patch does not
generate ASF License warnings. |
| | | 231m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5611 |
| Optional Tests | dupname asflicens
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719481#comment-17719481
]
ASF GitHub Bot commented on HADOOP-18724:
-
steveloughran commented on PR #5611:
URL: https://github.com/apache/hadoop/pull/5611#issuecomment-1535233676
testing
* azure cardiff: -Dparallel-tests=abfs -DtestsThreadCount=6
* s3 london: -Dparallel-tests -DtestsThreadCount=8
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719530#comment-17719530
]
ASF GitHub Bot commented on HADOOP-18724:
-
hadoop-yetus commented on PR #5611:
URL: https://github.com/apache/hadoop/pull/5611#issuecomment-1535415371
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 14s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 19m 35s | | trunk passed |
| +1 :green_heart: | compile | 15m 44s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 14m 20s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 45s | | trunk passed |
| +1 :green_heart: | mvnsite | 5m 8s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 22s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 2s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 8m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 12s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 43s | | the patch passed |
| +1 :green_heart: | compile | 14m 56s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 14m 56s | | the patch passed |
| +1 :green_heart: | compile | 14m 21s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 14m 21s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/2/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 3m 36s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/2/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 351 unchanged - 0 fixed = 353 total (was
351) |
| +1 :green_heart: | mvnsite | 5m 5s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 6s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 1s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 8m 53s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 33s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 20s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 5m 42s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 7m 0s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 1m 8s | | hadoop-mapreduce-examples in
the patch passed. |
| +1 :green_heart: | unit | 2m 40s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 3s | | The patch does not
generate ASF License warnings. |
| | | 234m 30s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5611 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 0d7490c92bcb 4.15.0-206-generic #217-Ubu
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720985#comment-17720985
]
ASF GitHub Bot commented on HADOOP-18724:
-
steveloughran commented on PR #5611:
URL: https://github.com/apache/hadoop/pull/5611#issuecomment-1540493289
* fixed the yetus complaints
* verified that a floating point drain threshold is ok
* fixed up abfs so it doesn't reject any must() arg of the "well known
fs.option.openfile" values, even though it doesn't (yet) process them
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721081#comment-17721081
]
ASF GitHub Bot commented on HADOOP-18724:
-
hadoop-yetus commented on PR #5611:
URL: https://github.com/apache/hadoop/pull/5611#issuecomment-1540836627
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 25s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 22m 29s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 19m 38s | | trunk passed |
| +1 :green_heart: | compile | 15m 41s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 14m 26s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 50s | | trunk passed |
| +1 :green_heart: | mvnsite | 6m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 8s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 44s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 9m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 11s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 12s | | the patch passed |
| +1 :green_heart: | compile | 14m 59s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 14m 59s | | the patch passed |
| +1 :green_heart: | compile | 17m 9s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 17m 9s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 0s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/3/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 373 unchanged - 0 fixed = 375 total (was
373) |
| +1 :green_heart: | mvnsite | 5m 39s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 30s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 16s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 10m 48s | | the patch passed |
| -1 :x: | shadedclient | 29m 37s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 5m 11s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/3/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch failed. |
| -1 :x: | unit | 0m 40s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/3/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt)
| hadoop-yarn-common in the patch failed. |
| -1 :x: | unit | 0m 41s |
[/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/3/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt)
| hadoop-mapreduce-client-core in the patch failed. |
| -1 :x: | unit | 0m 40s |
[/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/3/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-examples.txt)
| hadoop-mapreduce-examples in the patch failed. |
| -1 :x: | unit | 0m 41s |
[/patch-unit-hadoop-tools_hadoop-aws.txt](
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721090#comment-17721090
]
ASF GitHub Bot commented on HADOOP-18724:
-
hadoop-yetus commented on PR #5611:
URL: https://github.com/apache/hadoop/pull/5611#issuecomment-1540869174
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 2m 22s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 24m 31s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 22m 7s | | trunk passed |
| +1 :green_heart: | compile | 17m 29s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 16m 7s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 4m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 5m 13s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 11s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 3m 44s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 9m 4s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 52s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 10s | | the patch passed |
| +1 :green_heart: | compile | 16m 41s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 16m 41s | | the patch passed |
| +1 :green_heart: | compile | 15m 49s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 15m 49s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 3m 56s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/4/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 373 unchanged - 0 fixed = 375 total (was
373) |
| +1 :green_heart: | mvnsite | 5m 8s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 7s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 24s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 11m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 58s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 32s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 5m 20s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 6m 53s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 0m 57s | | hadoop-mapreduce-examples in
the patch passed. |
| +1 :green_heart: | unit | 2m 25s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | unit | 2m 12s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 53s | | The patch does not
generate ASF License warnings. |
| | | 266m 31s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5611 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux ed659d8b11b3 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Pers
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721091#comment-17721091
]
ASF GitHub Bot commented on HADOOP-18724:
-
hadoop-yetus commented on PR #5611:
URL: https://github.com/apache/hadoop/pull/5611#issuecomment-1540870322
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 18m 31s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 19s | | trunk passed |
| +1 :green_heart: | compile | 16m 32s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 15m 20s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 53s | | trunk passed |
| +1 :green_heart: | mvnsite | 5m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 49s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 47s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 9m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 13s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 18s | | the patch passed |
| +1 :green_heart: | compile | 15m 39s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 15m 39s | | the patch passed |
| +1 :green_heart: | compile | 15m 14s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 15m 14s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 3m 46s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/5/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 373 unchanged - 0 fixed = 375 total (was
373) |
| +1 :green_heart: | mvnsite | 5m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 21s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 26s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 10m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 52s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 44s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 5m 43s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 7m 16s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 1m 9s | | hadoop-mapreduce-examples in
the patch passed. |
| +1 :green_heart: | unit | 2m 51s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | unit | 2m 18s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 57s | | The patch does not
generate ASF License warnings. |
| | | 260m 9s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5611 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 64f44e9ff303 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Pers
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721751#comment-17721751
]
ASF GitHub Bot commented on HADOOP-18724:
-
steveloughran commented on PR #5611:
URL: https://github.com/apache/hadoop/pull/5611#issuecomment-1543896406
ok, pushed up a pr with the fixed imports; if yetus is happy will merge and
then backport
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721822#comment-17721822
]
ASF GitHub Bot commented on HADOOP-18724:
-
hadoop-yetus commented on PR #5611:
URL: https://github.com/apache/hadoop/pull/5611#issuecomment-1544169434
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 46s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 19m 27s | | trunk passed |
| +1 :green_heart: | compile | 15m 42s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 14m 32s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 6m 4s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 9s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 48s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 9m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 58s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 13s | | the patch passed |
| +1 :green_heart: | compile | 14m 59s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 14m 59s | | the patch passed |
| -1 :x: | compile | 10m 19s |
[/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/6/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt)
| root in the patch failed with JDK Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09. |
| -1 :x: | javac | 10m 19s |
[/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/6/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt)
| root in the patch failed with JDK Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 39s | | the patch passed |
| +1 :green_heart: | mvnsite | 5m 57s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 57s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 39s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 10m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 34s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 37s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 5m 42s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 6m 59s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 1m 8s | | hadoop-mapreduce-examples in
the patch passed. |
| +1 :green_heart: | unit | 2m 43s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | unit | 2m 17s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not
generate ASF License warnings. |
| | | 241m 16s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5611/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apa
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721846#comment-17721846
]
ASF GitHub Bot commented on HADOOP-18724:
-
steveloughran commented on PR #5611:
URL: https://github.com/apache/hadoop/pull/5611#issuecomment-1544332942
yarn bower build failed with network issues; 100% unrelated
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721849#comment-17721849
]
ASF GitHub Bot commented on HADOOP-18724:
-
steveloughran merged PR #5611:
URL: https://github.com/apache/hadoop/pull/5611
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721852#comment-17721852
]
ASF GitHub Bot commented on HADOOP-18724:
-
steveloughran opened a new pull request, #5647:
URL: https://github.com/apache/hadoop/pull/5647
This:
1. Adds optLong, optDouble, mustLong and mustDouble methods to the FSBuilder
interface to let callers explicitly passin long and double arguments.
2. The opt() and must() builder calls which take float/double values now
only set long values instead, so as to avoid problems related to overloaded
methods resulting in a ".0" being appended to a long value.
3. All of the relevant opt/must calls in the hadoop codebase move to the new
methods
4. And the s3a code is resilient to parse errors in is numeric options -it
will downgrade to the default.
This is nominally incompatible, but the floating-point builder methods were
never used: nothing currently expects floating point numbers.
For anyone who wants to safely set numeric builder options across all
compatible releases, convert the number to a string and then use the
opt(String, String) and must(String, String) methods.
Contributed by Steve Loughran
### How was this patch tested?
testing in progress
### For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721855#comment-17721855
]
ASF GitHub Bot commented on HADOOP-18724:
-
steveloughran commented on PR #5647:
URL: https://github.com/apache/hadoop/pull/5647#issuecomment-1544400733
test with -Dparallel-tests -DtestsThreadCount=8 -Dscale
no failures. note, recent changes in trunk and huge file upload didn't
merge, when HADOOP-18695. S3A: reject multipart copy requests when disabled
(#5548) is cherrypicked then it should copy over the move to optLong in its
changes
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721918#comment-17721918
]
ASF GitHub Bot commented on HADOOP-18724:
-
hadoop-yetus commented on PR #5647:
URL: https://github.com/apache/hadoop/pull/5647#issuecomment-1544675058
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 3m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +0 :ok: | mvndep | 28m 48s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 23m 54s | | branch-3.3 passed |
| +1 :green_heart: | compile | 17m 43s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 2m 45s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 6m 18s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 4m 54s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 9m 43s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 24m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 1s | | the patch passed |
| +1 :green_heart: | compile | 17m 10s | | the patch passed |
| -1 :x: | javac | 17m 10s |
[/results-compile-javac-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5647/1/artifact/out/results-compile-javac-root.txt)
| root generated 5 new + 1871 unchanged - 4 fixed = 1876 total (was 1875) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 45s | | the patch passed |
| +1 :green_heart: | mvnsite | 6m 19s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 44s | | the patch passed |
| +1 :green_heart: | spotbugs | 10m 42s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 43s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 4m 59s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 7m 17s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 1m 2s | | hadoop-mapreduce-examples in
the patch passed. |
| +1 :green_heart: | unit | 2m 43s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | unit | 2m 36s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 1s | | The patch does not
generate ASF License warnings. |
| | | 236m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5647/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5647 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux f6a5b4880429 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / 456c5acc153b7e8973019356856d97b5055c74cc |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~18.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5647/1/testReport/ |
| Max. process+thread count | 1571 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-aws
hadoop-tools/hadoop-azure U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5647/1/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17722083#comment-17722083
]
ASF GitHub Bot commented on HADOOP-18724:
-
steveloughran opened a new pull request, #5650:
URL: https://github.com/apache/hadoop/pull/5650
Somehow @Deprecated crept in.
### How was this patch tested?
letting yetus do it
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17722147#comment-17722147
]
ASF GitHub Bot commented on HADOOP-18724:
-
hadoop-yetus commented on PR #5650:
URL: https://github.com/apache/hadoop/pull/5650#issuecomment-1545646943
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 56s | | trunk passed |
| +1 :green_heart: | compile | 15m 34s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 14m 25s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 12s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 48s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | | the patch passed |
| +1 :green_heart: | compile | 14m 54s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 14m 54s | | the patch passed |
| +1 :green_heart: | compile | 14m 56s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 14m 56s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 5s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 4s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 51s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 17s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 36s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not
generate ASF License warnings. |
| | | 174m 35s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5650/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5650 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux efbcd47ecfdf 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / cdc77dea1bbd70d3456f759cfdad491be2f39a33 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5650/1/testReport/ |
| Max. process+thread count | 1263 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5650/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17722161#comment-17722161
]
ASF GitHub Bot commented on HADOOP-18724:
-
hadoop-yetus commented on PR #5647:
URL: https://github.com/apache/hadoop/pull/5647#issuecomment-1545700556
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +0 :ok: | mvndep | 15m 13s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 23s | | branch-3.3 passed |
| +1 :green_heart: | compile | 18m 20s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 2m 51s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 5m 41s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 4m 27s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 9m 31s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 24m 15s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 2s | | the patch passed |
| +1 :green_heart: | compile | 18m 23s | | the patch passed |
| -1 :x: | javac | 18m 23s |
[/results-compile-javac-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5647/2/artifact/out/results-compile-javac-root.txt)
| root generated 1 new + 1874 unchanged - 1 fixed = 1875 total (was 1875) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 2m 40s | | the patch passed |
| +1 :green_heart: | mvnsite | 5m 56s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 5s | | the patch passed |
| +1 :green_heart: | spotbugs | 10m 40s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 41s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 4m 59s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 7m 20s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 0m 55s | | hadoop-mapreduce-examples in
the patch passed. |
| +1 :green_heart: | unit | 2m 39s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | unit | 2m 33s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 58s | | The patch does not
generate ASF License warnings. |
| | | 220m 20s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5647/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5647 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 6dfba1738d31 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3 / 0989fc8d5a2b6ff40c41392385b0100a075efe75 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~18.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5647/2/testReport/ |
| Max. process+thread count | 3152 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-examples hadoop-tools/hadoop-aws
hadoop-tools/hadoop-azure U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5647/2/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724853#comment-17724853
]
ASF subversion and git services commented on HADOOP-18724:
--
Commit ad1e3a0f5bf7b26af9b8766dc9b897d5aa09a795 in hadoop's branch
refs/heads/feature-HADOOP-18073-s3a-sdk-upgrade from Steve Loughran
[ https://gitbox.apache.org/repos/asf?p=hadoop.git;h=ad1e3a0f5bf ]
HADOOP-18724. (followup) remove deprecation on optLong/optDouble methods (#5650)
Somehow @Deprecated crept in to the declaration of the
new FSBuilder optLong/optDouble methods.
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, fs/azure, fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.9
>
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724855#comment-17724855
]
ASF subversion and git services commented on HADOOP-18724:
--
Commit a90c7221436ca90e8ad3dd1153258d31ba4e58bc in hadoop's branch
refs/heads/feature-HADOOP-18073-s3a-sdk-upgrade from Steve Loughran
[ https://gitbox.apache.org/repos/asf?p=hadoop.git;h=a90c7221436 ]
HADOOP-18724. [FOLLOW-UP] cherrypick changes from branch-3.3 backport (#5662)
* move FileContext.copy() onto optLong()
* move FileUtil onto optLong()
This brings trunk into sync with the branch-3.3 changes
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, fs/azure, fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.9
>
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18724) Open file fails with NumberFormatException for S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-18724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724852#comment-17724852
]
ASF subversion and git services commented on HADOOP-18724:
--
Commit e76c09ac3bd3b01522c1d9b06b18c9452e774b33 in hadoop's branch
refs/heads/feature-HADOOP-18073-s3a-sdk-upgrade from Steve Loughran
[ https://gitbox.apache.org/repos/asf?p=hadoop.git;h=e76c09ac3bd ]
HADOOP-18724. Open file fails with NumberFormatException for S3AFileSystem
(#5611)
This:
1. Adds optLong, optDouble, mustLong and mustDouble
methods to the FSBuilder interface to let callers explicitly
passin long and double arguments.
2. The opt() and must() builder calls which take float/double values
now only set long values instead, so as to avoid problems
related to overloaded methods resulting in a ".0" being appended
to a long value.
3. All of the relevant opt/must calls in the hadoop codebase move to
the new methods
4. And the s3a code is resilient to parse errors in is numeric options
-it will downgrade to the default.
This is nominally incompatible, but the floating-point builder methods
were never used: nothing currently expects floating point numbers.
For anyone who wants to safely set numeric builder options across all compatible
releases, convert the number to a string and then use the opt(String, String)
and must(String, String) methods.
Contributed by Steve Loughran
> Open file fails with NumberFormatException for S3AFileSystem
>
>
> Key: HADOOP-18724
> URL: https://issues.apache.org/jira/browse/HADOOP-18724
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, fs/azure, fs/s3
>Affects Versions: 3.3.5
>Reporter: Ayush Saxena
>Assignee: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.9
>
>
> Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once
> I upgraded.
> {noformat}
> Caused by: java.lang.NumberFormatException: For input string: "5783.0"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.base/java.lang.Long.parseLong(Long.java:692)
> at java.base/java.lang.Long.parseLong(Long.java:817)
> at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
> at
> org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
> at
> org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
> at
> org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
> at
> org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
> at
> org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
