[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-05-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494708#comment-16494708 ] Felix Cheung commented on SPARK-23650: -- sorry, I really don't have time/resource to investigate

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-04-01 Thread Deepansh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16421933#comment-16421933 ] Deepansh commented on SPARK-23650: -- @[~felixcheung] following up on above discussion. > Slow SparkR udf

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-23 Thread Deepansh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411246#comment-16411246 ] Deepansh commented on SPARK-23650: -- R environment inside the thread for applying UDF is not getting

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410856#comment-16410856 ] Felix Cheung commented on SPARK-23650: -- can you clarify where you see "R environment inside the

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-18 Thread Deepansh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404213#comment-16404213 ] Deepansh commented on SPARK-23650: -- I tried reading the model in UDF, but for every new stream, the

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404198#comment-16404198 ] Felix Cheung commented on SPARK-23650: -- Is there a reason for the broadcast? Could you instead

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-18 Thread Deepansh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404183#comment-16404183 ] Deepansh commented on SPARK-23650: -- Is there any other way to implement my use case with minimum(ms)

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401525#comment-16401525 ] Felix Cheung commented on SPARK-23650: -- do you mean this?   RRunner: Times: boot = 0.010 s, init =

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-14 Thread Deepansh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398237#comment-16398237 ] Deepansh commented on SPARK-23650: -- attaching more of logs. > Slow SparkR udf (dapply) >

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398174#comment-16398174 ] Felix Cheung commented on SPARK-23650: -- I see one RRunner - do you have more of the log? > Slow

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-13 Thread Deepansh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396834#comment-16396834 ] Deepansh commented on SPARK-23650: -- I tried on local as well as yarn cluster, the result is more or less

[jira] [Commented] (SPARK-23650) Slow SparkR udf (dapply)

2018-03-13 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396607#comment-16396607 ] Felix Cheung commented on SPARK-23650: -- which system/platform are you running on? > Slow SparkR udf