[
https://issues.apache.org/jira/browse/FLINK-3664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15222460#comment-15222460
]
Fabian Hueske commented on FLINK-3664:
--------------------------------------
Hi Todd, thanks for sharing your work so far. I briefly skimmed the linked
commit and it looks pretty good to me. My only comments right now would be to
add a few more tests, but that's on your to do list anyways.
I'm on vacation right now and will return in a few days. It would be nice if
you could continue with your work and open a PR when you think it is ready.
Unless nobody else has had a detailed look, I'll review it when I'm back.
Btw. this work is also relevant in the context of the currently on-going
efforts to add support for SQL as it allows to gather basic statistics on a
DataSet.
Thanks, Fabian
> Create a method to easily Summarize a DataSet
> ---------------------------------------------
>
> Key: FLINK-3664
> URL: https://issues.apache.org/jira/browse/FLINK-3664
> Project: Flink
> Issue Type: Improvement
> Reporter: Todd Lisonbee
> Attachments: DataSet-Summary-Design-March2016-v1.txt
>
>
> Here is an example:
> {code}
> /**
> * Summarize a DataSet of Tuples by collecting single pass statistics for all
> columns
> */
> public Tuple summarize()
> Dataset<Tuple3<Double, String, Boolean>> input = // [...]
> Tuple3<DoubleColumnSummary,StringColumnSummary,BooleanColumnSummary> summary
> = input.summarize()
> summary.getField(0).stddev()
> summary.getField(1).maxStringLength()
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
