[
https://issues.apache.org/jira/browse/SPARK-44124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17779460#comment-17779460
]
Steve Loughran commented on SPARK-44124:
----------------------------------------
good document
# I think you could consider cutting the kinesis module entirely. its lack of
maintenance and bug reports indicates nobody uses it.
# we don't have a timetable for a 3.4.x release. end of year for the beta phase
has been discussed -but I'm not going to manage that release. There's a lot of
other changes there, and we are still trying to stabilize that v2 code in
HADOOP-18886 with some fairly major regressions surfacing HADOOP-18945.
one strateigy
# If you cut kinesis then its only the k8s tests which use the sdk...and you
don't need to explicitly redistribute either sdk jar.
# remove the exclusion of the v1 aws-sdk from hadoop-cloud module imports and
then all hadoop releases built on v1 get that jar included in the distro
# and if anyone builds with v2 then they'll get the exact v2 release that the
s3a connector has been built with. this matters as the v2 sdk is a fairly major
migration and any bugrep against the s3a connector which doesn't use the sdk
version we released against is going to left to the submitter to troubleshoot.
> Upgrade AWS SDK to v2
> ---------------------
>
> Key: SPARK-44124
> URL: https://issues.apache.org/jira/browse/SPARK-44124
> Project: Spark
> Issue Type: Sub-task
> Components: Build
> Affects Versions: 4.0.0
> Reporter: Dongjoon Hyun
> Priority: Major
>
> Here is a design doc:
> https://docs.google.com/document/d/1nGWbGTqxuFBG2ftfYYXxzrkipINILfWCOwse36yg7Ig/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
