[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-16 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961209#comment-14961209 ] Xiangrui Meng commented on SPARK-10953: --- That sounds good. I'm closing this for now since the

[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-15 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14958386#comment-14958386 ] Jihong MA commented on SPARK-10953: --- [~mengxr] merged PR9038. below are the avg elapsed time

[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-13 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14954447#comment-14954447 ] Jihong MA commented on SPARK-10953: --- had a quick run on my laptop, with a stddev implementation based

[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-12 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14953852#comment-14953852 ] Yin Huai commented on SPARK-10953: -- Yes, I think so. > Benchmark codegen vs. hand-written code for

[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14953842#comment-14953842 ] Xiangrui Meng commented on SPARK-10953: --- [~JihongMA] Any updates? I don't think it is necessary to

[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-12 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14954281#comment-14954281 ] Jihong MA commented on SPARK-10953: --- [~mengxr]as Yin indicated in the comment, we would like to merge

[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-12 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14954331#comment-14954331 ] Xiangrui Meng commented on SPARK-10953: --- We can run benchmark without #9038. I tested the following

[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-09 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950600#comment-14950600 ] Yin Huai commented on SPARK-10953: -- [~JihongMA] Yes. ImperativeAggregates are based on expressions,

[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-08 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949797#comment-14949797 ] Jihong MA commented on SPARK-10953: --- we should have a cluster for testing next Monday. we will run the

[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-06 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945532#comment-14945532 ] Jihong MA commented on SPARK-10953: --- [~mengxr] do you mean comparing an implementation which operate

[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-06 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945920#comment-14945920 ] Xiangrui Meng commented on SPARK-10953: --- The simple one is to test StatCounter with RDDs and the

[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-06 Thread Jihong MA (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14946061#comment-14946061 ] Jihong MA commented on SPARK-10953: --- it should not be too hard to put together an implementation based

[jira] [Commented] (SPARK-10953) Benchmark codegen vs. hand-written code for univariate statistics

2015-10-06 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14945409#comment-14945409 ] Xiangrui Meng commented on SPARK-10953: --- [~JihongMA] Do you have time to benchmark this? >