Re: How to use disk instead of just InMemoryRelation when use JDBC datasource in SPARKSQL?

You want to use `Dataset.persist(StorageLevel.MEMORY_AND_DISK)`? On Thu, Apr 12, 2018 at 1:12 PM, Louis Hust wrote: > We want to extract data from mysql, and calculate in sparksql. > The sql explain like below. > > > REGIONKEY#177,N_COMMENT#178] PushedFilters: [],

How to use disk instead of just InMemoryRelation when use JDBC datasource in SPARKSQL?

We want to extract data from mysql, and calculate in sparksql. The sql explain like below. REGIONKEY#177,N_COMMENT#178] PushedFilters: [], ReadSchema: struct +- *(20) Sort [r_regionkey#203 ASC NULLS FIRST], false,

How to use disk instead of just InMemoryRelation when use JDBC datasource in SPARKSQL?

We want to extract data from mysql, and calculate in sparksql. The sql explain like below. == Parsed Logical Plan == > 'Sort ['revenue DESC NULLS LAST], true > +- 'Aggregate ['n_name], ['n_name, 'SUM(('l_extendedprice * (1 - > 'l_discount))) AS revenue#329] >+- 'Filter ('c_custkey =