GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/16383
[SPARK-18980][SQL] implement Aggregator with TypedImperativeAggregate
## What changes were proposed in this pull request?
Currently we implement `Aggregator` with `DeclarativeAggregate`, which will
serialize/deserialize the buffer object every time we process an input.
This PR implements `Aggregator` with `TypedImperativeAggregate` and avoids
to serialize/deserialize many times.
For simple buffer object that doesn't need serialization, we still go with
`DeclarativeAggregate`, to avoid performance regression.
## How was this patch tested?
N/A
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark aggregator
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16383.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16383
----
commit c962c52ccfc8aedf688ccb1e78564a035759f63b
Author: Wenchen Fan <[email protected]>
Date: 2016-12-22T11:52:41Z
port Aggregator to TypedImperativeAggregate
commit 0a73fe208ed7e211daf75dd2268aec91868c7ee3
Author: Wenchen Fan <[email protected]>
Date: 2016-12-22T12:01:13Z
update benchmark
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
