[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-09-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14736139#comment-14736139 ] Xiangrui Meng commented on SPARK-10199: --- Yes, please. Thanks for doing the benchmark! We will close

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-09-05 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731834#comment-14731834 ] Joseph K. Bradley commented on SPARK-10199: --- I agree that the work required for these changes

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-09-04 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731753#comment-14731753 ] Vinod KC commented on SPARK-10199: -- [~mengxr] Thanks for the suggestion. Shall I close the PR? > Avoid

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-09-04 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14731529#comment-14731529 ] Xiangrui Meng commented on SPARK-10199: --- The improvement numbers also depends on the model size. In

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-09-02 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14728427#comment-14728427 ] Vinod KC commented on SPARK-10199: -- [~mengxr] could you please check the above micro-benchmarks and give

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-09-01 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14725211#comment-14725211 ] Vinod KC commented on SPARK-10199: -- [~mengxr] I've measured the overhead of reflexion in save/load

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-31 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724138#comment-14724138 ] Feynman Liang commented on SPARK-10199: --- CC [~mengxr] [~josephkb] > Avoid using reflections for

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-31 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724438#comment-14724438 ] Xiangrui Meng commented on SPARK-10199: --- [~vinodkc] Did yo measure the entire save/load operation

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-31 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724459#comment-14724459 ] Vinod KC commented on SPARK-10199: -- [~mengxr] 1) I measured only schema inference part. Now, I will

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-30 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14721491#comment-14721491 ] Vinod KC commented on SPARK-10199: -- [~fliang] , As you suggested, 1) I've made

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-30 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14721907#comment-14721907 ] Feynman Liang commented on SPARK-10199: --- [~vinodkc] Thanks! I think these results

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14718417#comment-14718417 ] Apache Spark commented on SPARK-10199: -- User 'vinodkc' has created a pull request

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-28 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14718609#comment-14718609 ] Xiangrui Meng commented on SPARK-10199: --- What is the overhead? Do we have

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-28 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14720277#comment-14720277 ] Feynman Liang commented on SPARK-10199: --- [~vinodkc] would it be possible to get

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-26 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715148#comment-14715148 ] Feynman Liang commented on SPARK-10199: --- Awesome, thanks! You can tag that PR with

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-25 Thread Vinod KC (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712496#comment-14712496 ] Vinod KC commented on SPARK-10199: -- Sure, I'll group all the changes into single PR

[jira] [Commented] (SPARK-10199) Avoid using reflections for parquet model save

2015-08-25 Thread Feynman Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712492#comment-14712492 ] Feynman Liang commented on SPARK-10199: --- Hi [~vinodkc], I saw that you took all of