[
https://issues.apache.org/jira/browse/DATAFU-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17014664#comment-17014664
]
Matthew Hayes commented on DATAFU-151:
--------------------------------------
I've attached a patch:
* Now requiring Java 1.8, as Spark requires at least 1.8.
* Added properties scalaCompatVersion and sparkCompatVersion to refer to the
major.minor versions. The scalaVersion and sparkVersion refer to the specific
versions that include the build number, as in major.minor.build.
* The datafu-spark base name no longer includes the Spark version, as I don't
think it's necessary at this time. We don't have an Spark specific differences.
If we encounter issues in the future we can reintroduce the Spark major.minor
version into the base name.
* Cleaned up the datafu-spark dependencies. Hadoop is now a runtime dependency.
* Removed Scala 2.10 version from testing as it is so old I don't think we need
to support or build for it.
* Inserted "datafu" into the packages under org.apache.spark to avoid possible
conflicts in the namespace. These classes apparently have to be under this
package because we are referencing private/protected classes.
* Fixed an issue in TestScalaPythonBridge.
> datafu-spark release preparation
> --------------------------------
>
> Key: DATAFU-151
> URL: https://issues.apache.org/jira/browse/DATAFU-151
> Project: DataFu
> Issue Type: Task
> Reporter: Matthew Hayes
> Assignee: Matthew Hayes
> Priority: Major
> Attachments: DATAFU-151.patch
>
>
> There are some updates we need to make to the build scripts to support
> releasing datafu-spark. For example, only the major and minor version of
> Spark (e.g. 2.4, not 2.4.4) should be included in the package name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
