[GitHub] spark issue #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in SparkR

2017-01-11 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14741 Yes - we are trying to interrupt the daemon R process. This is launched on the executor machines and communicates with the worker R processes and the JVM. --- If your project is set up for it, you

[GitHub] spark issue #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in SparkR

2017-01-06 Thread krishnakalyan3
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14741 ping @shivaram @davies, I am planning to revisit this PR. Could you please let me know which daemon process on Linux we are trying to interrupt. I am assuming its the R process?. --- If

[GitHub] spark issue #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in SparkR

2016-08-28 Thread krishnakalyan3
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14741 @shivaram thanks for the advice. Some Issue being faced by me - While reading a large file from Rstudio and trying to kill the the process using `Sys.getpid()`, I tried to interru

[GitHub] spark issue #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in SparkR

2016-08-23 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14741 The value of `what` shouldn't change if we are retrying the read. If it was `integer` it should remain `integer`. There is a corner case of what happens if we say read the first byte of the integer

[GitHub] spark issue #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in SparkR

2016-08-23 Thread krishnakalyan3
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14741 @shivaram @davies - Signature of the readBin function is `readBin(con, what, n, as.integer(size), endian)` What should the value of `what` be when an the process is interrupted i

[GitHub] spark issue #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in SparkR

2016-08-21 Thread krishnakalyan3
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14741 @shivaram I am not sure on how to go about the retry method. Could you please share some example that I could refer to?. --- If your project is set up for it, you can reply to this email an

[GitHub] spark issue #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in SparkR

2016-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14741 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat