Deepansh created SPARK-24405:
Summary: parameter for worker timeout in python
Key: SPARK-24405
URL: https://issues.apache.org/jira/browse/SPARK-24405
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-24405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepansh updated SPARK-24405:
-
Description:
Currently, the python worker timeout is set to 6 ms in code and cannot be
changed. I
[
https://issues.apache.org/jira/browse/SPARK-24405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepansh updated SPARK-24405:
-
Summary: parameter for python worker timeout (was: parameter for worker
timeout in python)
>
Deepansh created SPARK-24403:
Summary: reuse r worker
Key: SPARK-24403
URL: https://issues.apache.org/jira/browse/SPARK-24403
Project: Spark
Issue Type: Improvement
Components: SparkR
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16421933#comment-16421933
]
Deepansh commented on SPARK-23650:
--
@[~felixcheung] following up on above discussion.
> Slow SparkR udf
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411246#comment-16411246
]
Deepansh edited comment on SPARK-23650 at 3/23/18 1:43 PM:
---
R environment
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepansh updated SPARK-23650:
-
Attachment: packageReload.txt
> Slow SparkR udf (dapply)
>
>
>
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16411246#comment-16411246
]
Deepansh commented on SPARK-23650:
--
R environment inside the thread for applying UDF is not getting
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404213#comment-16404213
]
Deepansh edited comment on SPARK-23650 at 3/18/18 10:16 PM:
I tried reading
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepansh updated SPARK-23650:
-
Attachment: read_model_in_udf.txt
> Slow SparkR udf (dapply)
>
>
>
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404213#comment-16404213
]
Deepansh commented on SPARK-23650:
--
I tried reading the model in UDF, but for every new stream, the
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404183#comment-16404183
]
Deepansh commented on SPARK-23650:
--
Is there any other way to implement my use case with minimum(ms)
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398237#comment-16398237
]
Deepansh edited comment on SPARK-23650 at 3/15/18 4:39 AM:
---
attached more logs.
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398237#comment-16398237
]
Deepansh commented on SPARK-23650:
--
attaching more of logs.
> Slow SparkR udf (dapply)
>
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepansh updated SPARK-23650:
-
Attachment: sparkR_log2.txt
> Slow SparkR udf (dapply)
>
>
>
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepansh updated SPARK-23650:
-
Attachment: sparkRlag.txt
> Slow SparkR udf (dapply)
>
>
> Key:
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepansh updated SPARK-23650:
-
Attachment: (was: sparkRlag.txt)
> Slow SparkR udf (dapply)
>
>
>
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396834#comment-16396834
]
Deepansh edited comment on SPARK-23650 at 3/13/18 11:37 AM:
I tried on local
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepansh updated SPARK-23650:
-
Attachment: sparkRlag.txt
> Slow SparkR udf (dapply)
>
>
> Key:
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396834#comment-16396834
]
Deepansh edited comment on SPARK-23650 at 3/13/18 11:25 AM:
I tried on local
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396834#comment-16396834
]
Deepansh commented on SPARK-23650:
--
I tried on local as well as yarn cluster, the result is more or less
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepansh updated SPARK-23650:
-
Description:
For eg, I am getting streams from Kafka and I want to implement a model made in
R for
Deepansh created SPARK-23650:
Summary: Slow SparkR udf (dapply)
Key: SPARK-23650
URL: https://issues.apache.org/jira/browse/SPARK-23650
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-23650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepansh updated SPARK-23650:
-
Description:
For eg, I am getting streams from Kafka and I want to implement a model made in
R for
24 matches
Mail list logo
