[
https://issues.apache.org/jira/browse/PHOENIX-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SonixLegend updated PHOENIX-2804:
---------------------------------
Attachment: screenshot-1.png
> Support partition parameter or repartition function for Spark plugin
> --------------------------------------------------------------------
>
> Key: PHOENIX-2804
> URL: https://issues.apache.org/jira/browse/PHOENIX-2804
> Project: Phoenix
> Issue Type: Improvement
> Affects Versions: 4.7.0
> Reporter: SonixLegend
> Fix For: 4.8.0
>
> Attachments: screenshot-1.png
>
>
> When I wanna load some hurge data from phoenix to spark dataframes via
> phoenix spark plugin, and I had set the dataframes storage level was disk
> only, but if I wanna do join query data between the dataframes, the spark
> told me "java.lang.IllegalArgumentException: Size exceeds Integer.MAX_VALUE",
> because the spark read over 2G file per one partition. Can you add the
> partition parameter or override repartition function for load data via the
> plugin? Thanks a lot.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
