Thanks for providing the benchmark numbers! The result is very promising
and I'm looking forward to seeing more feedback from real-world workloads.
On Wed, Feb 12, 2020 at 3:43 PM Jia, Ke A wrote:
> Hi all,
>
> We have completed the Spark 3.0 Adaptive Query Execution(AQE) performance
> tests in
Even spark.dynamicAllocation.* doesn't follow 2-2, right? It follows the
mix of 1 and 2-1, though 1 is even broken there.
It doesn't matter If we just discuss about one-time decision - it may be OK
to not to be strict on consistency, though it's not ideal. The thing is
that these kind of
It's okay to just follow one prevailing style. The main point I would like
to say is the fact that we should *document* the direct relationship of
configurations.
For this case specifically, I don't think there is so much point here to
decide one hard requirement to follow for the mid-term. We
--
Sent from Gmail Mobile
--
Sent from Gmail Mobile
Hi Amogh,
Thanks for your interest in AQE work.
> Were any table stats available for TPC-DS during the runs ?
We used the default configurations and didn't set special configurations
(such as CBO) to collect the table stats both enable and disable AQE. And AQE
mainly rely on the runtime
OK I agree this is going forward as we decided the final goal and it seems
someone starts to work on. In the meanwhile we agree about documenting the
direct relationship, and which style to use is "open".
Thanks again to initiate the discussion thread - this thread led the
following thread for
any update on this? Is spark graph going to make it into Spark or no?
On Mon, Oct 14, 2019 at 12:26 PM Holden Karau wrote:
> Maybe let’s ask the folks from Lightbend who helped with the previous
> scala upgrade for their thoughts?
>
> On Mon, Oct 14, 2019 at 8:24 PM Xiao Li wrote:
>
>> 1. On
