[jira] [Commented] (SPARK-26427) Upgrade Apache ORC to 1.5.4
[
https://issues.apache.org/jira/browse/SPARK-26427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749400#comment-16749400
]
Dongjoon Hyun commented on SPARK-26427:
---
It's only ORC dependency changes.
{code}
-orc-core-1.5.2-nohive.jar
-orc-mapreduce-1.5.2-nohive.jar
-orc-shims-1.5.2.jar
+orc-core-1.5.4-nohive.jar
+orc-mapreduce-1.5.4-nohive.jar
+orc-shims-1.5.4.jar
{code}
> Upgrade Apache ORC to 1.5.4
> ---
>
> Key: SPARK-26427
> URL: https://issues.apache.org/jira/browse/SPARK-26427
> Project: Spark
> Issue Type: Bug
> Components: Build
>Affects Versions: 3.0.0
>Reporter: Dongjoon Hyun
>Assignee: Dongjoon Hyun
>Priority: Major
> Fix For: 3.0.0
>
>
> This issue aims to update Apache ORC dependency to the latest version 1.5.4
> released at Dec. 20. ([Release
> Notes|https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318320&version=12344187])
> {code}
> [ORC-237] - OrcFile.mergeFiles Specified block size is less than configured
> minimum value
> [ORC-409] - Changes for extending MemoryManagerImpl
> [ORC-410] - Fix a locale-dependent test in TestCsvReader
> [ORC-416] - Avoid opening data reader when there is no stripe
> [ORC-417] - Use dynamic Apache Maven mirror link
> [ORC-419] - Ensure to call `close` at RecordReaderImpl constructor exception
> [ORC-432] - openjdk 8 has a bug that prevents surefire from working
> [ORC-435] - Ability to read stripes that are greater than 2GB
> [ORC-437] - Make acid schema checks case insensitive
> [ORC-411] - Update build to work with Java 10.
> [ORC-418] - Fix broken docker build script
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26427) Upgrade Apache ORC to 1.5.4
[
https://issues.apache.org/jira/browse/SPARK-26427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749391#comment-16749391
]
Wenchen Fan commented on SPARK-26427:
-
does it include other transitive dependences upgrade?
> Upgrade Apache ORC to 1.5.4
> ---
>
> Key: SPARK-26427
> URL: https://issues.apache.org/jira/browse/SPARK-26427
> Project: Spark
> Issue Type: Bug
> Components: Build
>Affects Versions: 3.0.0
>Reporter: Dongjoon Hyun
>Assignee: Dongjoon Hyun
>Priority: Major
> Fix For: 3.0.0
>
>
> This issue aims to update Apache ORC dependency to the latest version 1.5.4
> released at Dec. 20. ([Release
> Notes|https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318320&version=12344187])
> {code}
> [ORC-237] - OrcFile.mergeFiles Specified block size is less than configured
> minimum value
> [ORC-409] - Changes for extending MemoryManagerImpl
> [ORC-410] - Fix a locale-dependent test in TestCsvReader
> [ORC-416] - Avoid opening data reader when there is no stripe
> [ORC-417] - Use dynamic Apache Maven mirror link
> [ORC-419] - Ensure to call `close` at RecordReaderImpl constructor exception
> [ORC-432] - openjdk 8 has a bug that prevents surefire from working
> [ORC-435] - Ability to read stripes that are greater than 2GB
> [ORC-437] - Make acid schema checks case insensitive
> [ORC-411] - Update build to work with Java 10.
> [ORC-418] - Fix broken docker build script
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26427) Upgrade Apache ORC to 1.5.4
[
https://issues.apache.org/jira/browse/SPARK-26427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749223#comment-16749223
]
Dongjoon Hyun commented on SPARK-26427:
---
Hi, [~smilegator] and [~cloud_fan].
In general, we don't change the dependency at the maintenance release.
But, for this case, I'm wondering if we can have this to reduce the resource
leakage in `branch-2.4` for next Spark 2.4.1.
Can I proceed to make a backport PR?
> Upgrade Apache ORC to 1.5.4
> ---
>
> Key: SPARK-26427
> URL: https://issues.apache.org/jira/browse/SPARK-26427
> Project: Spark
> Issue Type: Bug
> Components: Build
>Affects Versions: 3.0.0
>Reporter: Dongjoon Hyun
>Assignee: Dongjoon Hyun
>Priority: Major
> Fix For: 3.0.0
>
>
> This issue aims to update Apache ORC dependency to the latest version 1.5.4
> released at Dec. 20. ([Release
> Notes|https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318320&version=12344187])
> {code}
> [ORC-237] - OrcFile.mergeFiles Specified block size is less than configured
> minimum value
> [ORC-409] - Changes for extending MemoryManagerImpl
> [ORC-410] - Fix a locale-dependent test in TestCsvReader
> [ORC-416] - Avoid opening data reader when there is no stripe
> [ORC-417] - Use dynamic Apache Maven mirror link
> [ORC-419] - Ensure to call `close` at RecordReaderImpl constructor exception
> [ORC-432] - openjdk 8 has a bug that prevents surefire from working
> [ORC-435] - Ability to read stripes that are greater than 2GB
> [ORC-437] - Make acid schema checks case insensitive
> [ORC-411] - Update build to work with Java 10.
> [ORC-418] - Fix broken docker build script
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26427) Upgrade Apache ORC to 1.5.4
[
https://issues.apache.org/jira/browse/SPARK-26427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16727286#comment-16727286
]
ASF GitHub Bot commented on SPARK-26427:
asfgit closed pull request #23364: [SPARK-26427][BUILD] Upgrade Apache ORC to
1.5.4
URL: https://github.com/apache/spark/pull/23364
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/dev/deps/spark-deps-hadoop-2.7 b/dev/deps/spark-deps-hadoop-2.7
index 71423af0789c6..1af29fcaff2aa 100644
--- a/dev/deps/spark-deps-hadoop-2.7
+++ b/dev/deps/spark-deps-hadoop-2.7
@@ -155,9 +155,9 @@ objenesis-2.5.1.jar
okhttp-3.8.1.jar
okio-1.13.0.jar
opencsv-2.3.jar
-orc-core-1.5.3-nohive.jar
-orc-mapreduce-1.5.3-nohive.jar
-orc-shims-1.5.3.jar
+orc-core-1.5.4-nohive.jar
+orc-mapreduce-1.5.4-nohive.jar
+orc-shims-1.5.4.jar
oro-2.0.8.jar
osgi-resource-locator-1.0.1.jar
paranamer-2.8.jar
diff --git a/dev/deps/spark-deps-hadoop-3.1 b/dev/deps/spark-deps-hadoop-3.1
index 93eafef045330..05f180b17a588 100644
--- a/dev/deps/spark-deps-hadoop-3.1
+++ b/dev/deps/spark-deps-hadoop-3.1
@@ -172,9 +172,9 @@ okhttp-2.7.5.jar
okhttp-3.8.1.jar
okio-1.13.0.jar
opencsv-2.3.jar
-orc-core-1.5.3-nohive.jar
-orc-mapreduce-1.5.3-nohive.jar
-orc-shims-1.5.3.jar
+orc-core-1.5.4-nohive.jar
+orc-mapreduce-1.5.4-nohive.jar
+orc-shims-1.5.4.jar
oro-2.0.8.jar
osgi-resource-locator-1.0.1.jar
paranamer-2.8.jar
diff --git a/pom.xml b/pom.xml
index 310d7de955125..de9421419edc2 100644
--- a/pom.xml
+++ b/pom.xml
@@ -132,7 +132,7 @@
2.1.0
10.12.1.1
1.10.0
-1.5.3
+1.5.4
nohive
1.6.0
9.4.12.v20180830
@@ -1740,6 +1740,10 @@
${orc.classifier}
${orc.deps.scope}
+
+javax.xml.bind
+jaxb-api
+
org.apache.hadoop
hadoop-common
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Upgrade Apache ORC to 1.5.4
> ---
>
> Key: SPARK-26427
> URL: https://issues.apache.org/jira/browse/SPARK-26427
> Project: Spark
> Issue Type: Bug
> Components: Build
>Affects Versions: 3.0.0
>Reporter: Dongjoon Hyun
>Assignee: Dongjoon Hyun
>Priority: Major
> Fix For: 3.0.0
>
>
> This issue aims to update Apache ORC dependency to the latest version 1.5.4
> released at Dec. 20. ([Release
> Notes|https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318320&version=12344187])
> {code}
> [ORC-237] - OrcFile.mergeFiles Specified block size is less than configured
> minimum value
> [ORC-409] - Changes for extending MemoryManagerImpl
> [ORC-410] - Fix a locale-dependent test in TestCsvReader
> [ORC-416] - Avoid opening data reader when there is no stripe
> [ORC-417] - Use dynamic Apache Maven mirror link
> [ORC-419] - Ensure to call `close` at RecordReaderImpl constructor exception
> [ORC-432] - openjdk 8 has a bug that prevents surefire from working
> [ORC-435] - Ability to read stripes that are greater than 2GB
> [ORC-437] - Make acid schema checks case insensitive
> [ORC-411] - Update build to work with Java 10.
> [ORC-418] - Fix broken docker build script
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26427) Upgrade Apache ORC to 1.5.4
[
https://issues.apache.org/jira/browse/SPARK-26427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16726945#comment-16726945
]
ASF GitHub Bot commented on SPARK-26427:
dongjoon-hyun opened a new pull request #23364: [SPARK-26427][BUILD] Upgrade
Apache ORC to 1.5.4
URL: https://github.com/apache/spark/pull/23364
## What changes were proposed in this pull request?
This PR aims to update Apache ORC dependency to the latest version 1.5.4
released at Dec. 20. ([Release
Notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318320&version=12344187]))
```
[ORC-237] - OrcFile.mergeFiles Specified block size is less than configured
minimum value
[ORC-409] - Changes for extending MemoryManagerImpl
[ORC-410] - Fix a locale-dependent test in TestCsvReader
[ORC-416] - Avoid opening data reader when there is no stripe
[ORC-417] - Use dynamic Apache Maven mirror link
[ORC-419] - Ensure to call `close` at RecordReaderImpl constructor exception
[ORC-432] - openjdk 8 has a bug that prevents surefire from working
[ORC-435] - Ability to read stripes that are greater than 2GB
[ORC-437] - Make acid schema checks case insensitive
[ORC-411] - Update build to work with Java 10.
[ORC-418] - Fix broken docker build script
```
## How was this patch tested?
Build and pass Jenkins.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Upgrade Apache ORC to 1.5.4
> ---
>
> Key: SPARK-26427
> URL: https://issues.apache.org/jira/browse/SPARK-26427
> Project: Spark
> Issue Type: Bug
> Components: Build
>Affects Versions: 3.0.0
>Reporter: Dongjoon Hyun
>Priority: Major
>
> This issue aims to update Apache ORC dependency to the latest version 1.5.4
> released at Dec. 20. ([Release
> Notes|https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12318320&version=12344187])
> {code}
> [ORC-237] - OrcFile.mergeFiles Specified block size is less than configured
> minimum value
> [ORC-409] - Changes for extending MemoryManagerImpl
> [ORC-410] - Fix a locale-dependent test in TestCsvReader
> [ORC-416] - Avoid opening data reader when there is no stripe
> [ORC-417] - Use dynamic Apache Maven mirror link
> [ORC-419] - Ensure to call `close` at RecordReaderImpl constructor exception
> [ORC-432] - openjdk 8 has a bug that prevents surefire from working
> [ORC-435] - Ability to read stripes that are greater than 2GB
> [ORC-437] - Make acid schema checks case insensitive
> [ORC-411] - Update build to work with Java 10.
> [ORC-418] - Fix broken docker build script
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
