We want to extract data from mysql, and calculate in sparksql.
The sql explain like below.
== Parsed Logical Plan ==
> 'Sort ['revenue DESC NULLS LAST], true
> +- 'Aggregate ['n_name], ['n_name, 'SUM(('l_extendedprice * (1 -
> 'l_discount))) AS revenue#329]
>+- 'Filter ('c_custkey = 'o_cu
We want to extract data from mysql, and calculate in sparksql.
The sql explain like below.
REGIONKEY#177,N_COMMENT#178] PushedFilters: [], ReadSchema:
struct
+- *(20) Sort [r_regionkey#203 ASC NULLS FIRST], false, 0
+- Exchange(coordinator id: 266374831)
ha
You want to use `Dataset.persist(StorageLevel.MEMORY_AND_DISK)`?
On Thu, Apr 12, 2018 at 1:12 PM, Louis Hust wrote:
> We want to extract data from mysql, and calculate in sparksql.
> The sql explain like below.
>
>
> REGIONKEY#177,N_COMMENT#178] PushedFilters: [], ReadSchema:
> struct
>