[
https://issues.apache.org/jira/browse/HIVE-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879077#action_12879077
]
HBase Review Board commented on HIVE-1397:
------------------------------------------
Message from: "John Sichi" <jsi...@facebook.com>
bq. On None, John Sichi wrote:
bq. >
http://svn.apache.org/repos/asf/hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java,
line 346
bq. > <http://review.hbase.org/r/181/diff/1/?file=1351#file1351line346>
bq. >
bq. > Since eventually we would like to support histograms on non-numeric
fields such as STRING, I think we should rename this one numeric_histogram
(likewise for the Java class) to avoid confusion later when we have other
algorithms.
bq. >
bq.
bq. Mayank Lahiri wrote:
bq. It might seem a little odd since histograms are generally used as
approximations of numerical distributions. I would suggest either (a)
overloading histogram() to behave differently on STRING arguments (perhaps
STRING arguments that cause a NumberFormatException), or (b) creating a
factor_histogram() function for general strings.
bq.
bq. I could add in the code for computing frequencies of STRINGs quite
easily, although there's no way to prevent it from choking if there are too
many unique strings.
I don't think we should handle strings now, but we should rename this one to
make it clear that it only works on numeric data. And per our discussion
offline, reject attempts to use it on non-numeric data.
- John
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/181/#review224
-----------------------------------------------------------
> histogram() UDAF for a numerical column
> ---------------------------------------
>
> Key: HIVE-1397
> URL: https://issues.apache.org/jira/browse/HIVE-1397
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Mayank Lahiri
> Assignee: Mayank Lahiri
> Fix For: 0.6.0
>
> Attachments: Histogram_quality.png.jpg, HIVE-1397.1.patch
>
>
> A histogram() UDAF to generate an approximate histogram of a numerical (byte,
> short, double, long, etc.) column. The result is returned as a map of (x,y)
> histogram pairs, and can be plotted in Gnuplot using impulses (for example).
> The algorithm is currently adapted from "A streaming parallel decision tree
> algorithm" by Ben-Haim and Tom-Tov, JMLR 11 (2010), and uses space
> proportional to the number of histogram bins specified. It has no
> approximation guarantees, but seems to work well when there is a lot of data
> and a large number (e.g. 50-100) of histogram bins specified.
> A typical call might be:
> SELECT histogram(val, 10) FROM some_table;
> where the result would be a histogram with 10 bins, returned as a Hive map
> object.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
