[jira] [Updated] (SPARK-22889) CRAN checks can fail if older Spark install exists

2017-12-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22889: - Fix Version/s: 2.3.0 2.2.2 > CRAN checks can fail if older Spark install

[jira] [Commented] (SPARK-22632) Fix the behavior of timestamp values for R's DataFrame to respect session timezone

2018-01-07 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16315436#comment-16315436 ] Felix Cheung commented on SPARK-22632: -- yes, first I'd agree we should generalize this to R & Python

[jira] [Commented] (SPARK-21727) Operating on an ArrayType in a SparkR DataFrame throws error

2018-01-07 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16315434#comment-16315434 ] Felix Cheung commented on SPARK-21727: -- I think we should use is.atomic(object) ? > Operating on

[jira] [Commented] (SPARK-22918) sbt test (spark - local) fail after upgrading to 2.2.1 with: java.security.AccessControlException: access denied org.apache.derby.security.SystemPermission( "engine",

2018-01-07 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16315438#comment-16315438 ] Felix Cheung commented on SPARK-22918: -- [~sameerag] we might want to check this for 2.3.0 release >

[jira] [Resolved] (SPARK-21292) R document Catalog function metadata refresh

2018-01-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-21292. -- Resolution: Fixed Fix Version/s: 2.3.0 Target Version/s: 2.3.0 > R document

[jira] [Commented] (SPARK-21293) R document update structured streaming

2018-01-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317805#comment-16317805 ] Felix Cheung commented on SPARK-21293: -- leaving it open for the rest of items > R document update

[jira] [Resolved] (SPARK-22998) Value for SPARK_MOUNTED_CLASSPATH in executor pods is not set

2018-01-09 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-22998. -- Resolution: Fixed Assignee: Yinan Li Target Version/s: 2.3.0 > Value for

[jira] [Commented] (SPARK-21293) R document update structured streaming

2018-01-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317739#comment-16317739 ] Felix Cheung commented on SPARK-21293: -- not done: Join Operations Streaming Deduplication > R

[jira] [Updated] (SPARK-21290) R document Programmatically Specifying the Schema in SQL guide

2018-01-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-21290: - Target Version/s: (was: 2.3.0) > R document Programmatically Specifying the Schema in SQL

[jira] [Commented] (SPARK-20906) Constrained Logistic Regression for SparkR

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333654#comment-16333654 ] Felix Cheung commented on SPARK-20906: -- [~wm624] would you like to add example of this in the R

[jira] [Commented] (SPARK-22208) Improve percentile_approx by not rounding up targetError and starting from index 0

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333659#comment-16333659 ] Felix Cheung commented on SPARK-22208: -- Is this documented in the SQL programming guide/ migration

[jira] [Commented] (SPARK-23115) SparkR 2.3 QA: New R APIs and API docs

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333677#comment-16333677 ] Felix Cheung commented on SPARK-23115: -- Another pass, we should add API doc for SPARK-20906 >

[jira] [Commented] (SPARK-23107) ML, Graph 2.3 QA: API: New Scala APIs, docs

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333712#comment-16333712 ] Felix Cheung commented on SPARK-23107: -- We don't have doc on RFormula but it'll be good idea to also

[jira] [Commented] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333663#comment-16333663 ] Felix Cheung commented on SPARK-20307: -- for SPARK-20307 and SPARK-21381, do you think you can write

[jira] [Comment Edited] (SPARK-20906) Constrained Logistic Regression for SparkR

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333654#comment-16333654 ] Felix Cheung edited comment on SPARK-20906 at 1/21/18 8:54 PM: --- [~wm624]

[jira] [Comment Edited] (SPARK-23114) Spark R 2.3 QA umbrella

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333725#comment-16333725 ] Felix Cheung edited comment on SPARK-23114 at 1/21/18 11:02 PM:

[jira] [Comment Edited] (SPARK-23114) Spark R 2.3 QA umbrella

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333730#comment-16333730 ] Felix Cheung edited comment on SPARK-23114 at 1/21/18 11:03 PM: [~falaki]

[jira] [Comment Edited] (SPARK-23117) SparkR 2.3 QA: Check for new R APIs requiring example code

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333718#comment-16333718 ] Felix Cheung edited comment on SPARK-23117 at 1/21/18 10:47 PM: I did a

[jira] [Commented] (SPARK-23114) Spark R 2.3 QA umbrella

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333730#comment-16333730 ] Felix Cheung commented on SPARK-23114: -- [~falaki] [~hyukjin.kwon] About SPARK-21093, do you think

[jira] [Commented] (SPARK-23116) SparkR 2.3 QA: Update user guide for new features & APIs

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333717#comment-16333717 ] Felix Cheung commented on SPARK-23116: -- I did a pass. > SparkR 2.3 QA: Update user guide for new

[jira] [Resolved] (SPARK-23116) SparkR 2.3 QA: Update user guide for new features & APIs

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23116. -- Resolution: Fixed Assignee: Felix Cheung Fix Version/s: 2.3.0 > SparkR 2.3 QA:

[jira] [Commented] (SPARK-23117) SparkR 2.3 QA: Check for new R APIs requiring example code

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333718#comment-16333718 ] Felix Cheung commented on SPARK-23117: -- I did a pass, I think these could use an example, preferably

[jira] [Commented] (SPARK-23114) Spark R 2.3 QA umbrella

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333725#comment-16333725 ] Felix Cheung commented on SPARK-23114: -- [~sameerag] Here are some ideas for the release notes (that

[jira] [Commented] (SPARK-23118) SparkR 2.3 QA: Programming guide, migration guide, vignettes updates

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333707#comment-16333707 ] Felix Cheung commented on SPARK-23118: -- for programming guide, perhaps  SPARK-20906 But it mostly

[jira] [Resolved] (SPARK-23118) SparkR 2.3 QA: Programming guide, migration guide, vignettes updates

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23118. -- Resolution: Fixed Assignee: Felix Cheung Fix Version/s: 2.3.0 > SparkR 2.3 QA:

[jira] [Updated] (SPARK-22208) Improve percentile_approx by not rounding up targetError and starting from index 0

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-22208: - Labels: releasenotes (was: ) > Improve percentile_approx by not rounding up targetError and

[jira] [Commented] (SPARK-23108) ML, Graph 2.3 QA: API: Experimental, DeveloperApi, final, sealed audit

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333708#comment-16333708 ] Felix Cheung commented on SPARK-23108: -- >From reviewing R, it would be good to document constrained

[jira] [Comment Edited] (SPARK-20906) Constrained Logistic Regression for SparkR

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333654#comment-16333654 ] Felix Cheung edited comment on SPARK-20906 at 1/21/18 10:30 PM: [~wm624]

[jira] [Comment Edited] (SPARK-23107) ML, Graph 2.3 QA: API: New Scala APIs, docs

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333712#comment-16333712 ] Felix Cheung edited comment on SPARK-23107 at 1/21/18 11:08 PM: We don't

[jira] [Comment Edited] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2018-01-21 Thread Felix Cheung (JIRA)
, I can do that but I have a family emergency lately. It will not occur soon. Best Joseph   was (Author: monday0927!): Hi Felix, I can do that but I have a family emergency lately. It will not occur soon. Best Joseph On 1/21/18, 2:45 PM, "Felix Cheung (JIRA)" <j...@apac

[jira] [Commented] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333716#comment-16333716 ] Felix Cheung commented on SPARK-20307: -- I think [~wm624] if you have the time > SparkR: pass on

[jira] [Commented] (SPARK-21727) Operating on an ArrayType in a SparkR DataFrame throws error

2018-01-21 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333733#comment-16333733 ] Felix Cheung commented on SPARK-21727: -- how are we doing? > Operating on an ArrayType in a SparkR

[jira] [Commented] (SPARK-23117) SparkR 2.3 QA: Check for new R APIs requiring example code

2018-01-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338043#comment-16338043 ] Felix Cheung commented on SPARK-23117: -- I'm ok to sign off if we don't have example for SPARK-20307

[jira] [Assigned] (SPARK-23200) Reset configuration when restarting from checkpoints

2018-01-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-23200: Assignee: Santiago Saavedra (was: Anirudh Ramanathan) > Reset configuration when

[jira] [Commented] (SPARK-23107) ML, Graph 2.3 QA: API: New Scala APIs, docs

2018-01-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340876#comment-16340876 ] Felix Cheung commented on SPARK-23107: -- We have never had any doc for it and it’s not new in 2.3.0

[jira] [Commented] (SPARK-23107) ML, Graph 2.3 QA: API: New Scala APIs, docs

2018-01-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341232#comment-16341232 ] Felix Cheung commented on SPARK-23107: -- Thanks My bad RFormula does have a page

[jira] [Comment Edited] (SPARK-23115) SparkR 2.3 QA: New R APIs and API docs

2018-01-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328422#comment-16328422 ] Felix Cheung edited comment on SPARK-23115 at 1/17/18 8:01 AM: --- did this,

[jira] [Commented] (SPARK-23118) SparkR 2.3 QA: Programming guide, migration guide, vignettes updates

2018-01-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328430#comment-16328430 ] Felix Cheung commented on SPARK-23118: -- did this and opened SPARK-21616 > SparkR 2.3 QA:

[jira] [Commented] (SPARK-23115) SparkR 2.3 QA: New R APIs and API docs

2018-01-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328422#comment-16328422 ] Felix Cheung commented on SPARK-23115: -- did this, and opened this

[jira] [Commented] (SPARK-23114) Spark R 2.3 QA umbrella

2018-01-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328419#comment-16328419 ] Felix Cheung commented on SPARK-23114: -- sure, [~josephkb] > Spark R 2.3 QA umbrella >

[jira] [Commented] (SPARK-23065) R API doc empty in Spark 2.3.0 RC1

2018-01-13 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325229#comment-16325229 ] Felix Cheung commented on SPARK-23065: -- Was the Jekyll error failed the doc build? Just want to make

[jira] [Updated] (SPARK-23073) Fix incorrect R doc page header for generated sql functions

2018-01-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23073: - Attachment: Screen Shot 2018-01-14 at 11.11.05 AM.png > Fix incorrect R doc page header for

[jira] [Created] (SPARK-23073) Fix incorrect R doc page header for generated sql functions

2018-01-14 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-23073: Summary: Fix incorrect R doc page header for generated sql functions Key: SPARK-23073 URL: https://issues.apache.org/jira/browse/SPARK-23073 Project: Spark

[jira] [Updated] (SPARK-23073) Fix incorrect R doc page header for generated sql functions

2018-01-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23073: - Description: See title says {{asc {SparkR}}}

[jira] [Updated] (SPARK-23073) Fix incorrect R doc page header for generated sql functions

2018-01-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23073: - Component/s: (was: SpakrR) SparkR > Fix incorrect R doc page header for

[jira] [Updated] (SPARK-23073) Fix incorrect R doc page header for generated sql functions

2018-01-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23073: - Description: See title says {code} asc {SparkR} {code}

[jira] [Updated] (SPARK-23068) Jekyll doc build error does not fail build

2018-01-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23068: - Component/s: Documentation > Jekyll doc build error does not fail build >

[jira] [Updated] (SPARK-23068) Jekyll doc build error does not fail build

2018-01-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23068: - Summary: Jekyll doc build error does not fail build (was: R doc build error does not fail

[jira] [Updated] (SPARK-23073) Fix incorrect R doc page header for generated sql functions

2018-01-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23073: - Description: See title says {code} asc {SparkR} {code}

[jira] [Created] (SPARK-23068) R doc build error does not fail build

2018-01-13 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-23068: Summary: R doc build error does not fail build Key: SPARK-23068 URL: https://issues.apache.org/jira/browse/SPARK-23068 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-23065) R API doc empty in Spark 2.3.0 RC1

2018-01-13 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23065. -- Resolution: Fixed Fix Version/s: 2.3.0 Target Version/s: 2.3.0 > R API doc

[jira] [Commented] (SPARK-23065) R API doc empty in Spark 2.3.0 RC1

2018-01-13 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16325465#comment-16325465 ] Felix Cheung commented on SPARK-23065: -- I have checked the doc, looks good, but I have a few small

[jira] [Created] (SPARK-23069) R doc for describe missing text

2018-01-13 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-23069: Summary: R doc for describe missing text Key: SPARK-23069 URL: https://issues.apache.org/jira/browse/SPARK-23069 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-23063) Changes to publish the spark-kubernetes package

2018-01-13 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23063. -- Resolution: Fixed Assignee: Anirudh Ramanathan Fix Version/s: 2.3.0 > Changes

[jira] [Updated] (SPARK-23069) R doc for describe missing text

2018-01-13 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23069: - Issue Type: Documentation (was: Bug) > R doc for describe missing text >

[jira] [Resolved] (SPARK-23038) Update docker/spark-test (JDK/OS)

2018-01-13 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23038. -- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 2.3.0

[jira] [Updated] (SPARK-23038) Update docker/spark-test (JDK/OS)

2018-01-13 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23038: - Fix Version/s: 2.4.0 > Update docker/spark-test (JDK/OS) > - > >

[jira] [Commented] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

2018-01-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339522#comment-16339522 ] Felix Cheung commented on SPARK-23213: -- You can convert DataFrame into RDD But again textFile and

[jira] [Commented] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

2018-01-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339559#comment-16339559 ] Felix Cheung commented on SPARK-23213: -- If you have any specific on what you need - we should have

[jira] [Commented] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

2018-01-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339748#comment-16339748 ] Felix Cheung commented on SPARK-23213: -- To clarify we don’t support RDD in R. Anything you access

[jira] [Commented] (SPARK-23213) SparkR:::textFile(sc1,"/opt/test333") can not work on spark2.2.1

2018-01-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338987#comment-16338987 ] Felix Cheung commented on SPARK-23213: -- Try read.text instead?

[jira] [Resolved] (SPARK-23115) SparkR 2.3 QA: New R APIs and API docs

2018-01-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23115. -- Resolution: Fixed Assignee: Felix Cheung > SparkR 2.3 QA: New R APIs and API docs >

[jira] [Commented] (SPARK-23115) SparkR 2.3 QA: New R APIs and API docs

2018-01-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16337947#comment-16337947 ] Felix Cheung commented on SPARK-23115: -- done > SparkR 2.3 QA: New R APIs and API docs >

[jira] [Commented] (SPARK-21727) Operating on an ArrayType in a SparkR DataFrame throws error

2018-01-09 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16319855#comment-16319855 ] Felix Cheung commented on SPARK-21727: -- good call... if ( (is.atomic(object) && !is.raw(object)) &&

[jira] [Created] (SPARK-23065) R API doc empty in Spark 2.3.0 RC1

2018-01-12 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-23065: Summary: R API doc empty in Spark 2.3.0 RC1 Key: SPARK-23065 URL: https://issues.apache.org/jira/browse/SPARK-23065 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-23065) R API doc empty in Spark 2.3.0 RC1

2018-01-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23065: - Priority: Blocker (was: Major) > R API doc empty in Spark 2.3.0 RC1 >

[jira] [Commented] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350819#comment-16350819 ] Felix Cheung commented on SPARK-23314: -- Im running python 2 Pandas 0.22.0 Pyarrow 0.8.0 > Pandas

[jira] [Commented] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351188#comment-16351188 ] Felix Cheung commented on SPARK-23314: -- Thanks. I have isolated this to a different subset of data,

[jira] [Commented] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351333#comment-16351333 ] Felix Cheung commented on SPARK-23314: -- I've isolated this down to this particular file

[jira] [Resolved] (SPARK-23117) SparkR 2.3 QA: Check for new R APIs requiring example code

2018-01-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23117. -- Resolution: Won't Fix Assignee: Felix Cheung > SparkR 2.3 QA: Check for new R APIs

[jira] [Commented] (SPARK-23114) Spark R 2.3 QA umbrella

2018-01-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342441#comment-16342441 ] Felix Cheung commented on SPARK-23114: -- Sure! > Spark R 2.3 QA umbrella > ---

[jira] [Commented] (SPARK-23114) Spark R 2.3 QA umbrella

2018-01-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342394#comment-16342394 ] Felix Cheung commented on SPARK-23114: -- Resolving. [~sameerag] please see release note above. >

[jira] [Resolved] (SPARK-23114) Spark R 2.3 QA umbrella

2018-01-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-23114. -- Resolution: Fixed > Spark R 2.3 QA umbrella > --- > > Key:

[jira] [Commented] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349896#comment-16349896 ] Felix Cheung commented on SPARK-23314: -- data sample

[jira] [Updated] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23314: - Environment: (was: data sample

[jira] [Commented] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349897#comment-16349897 ] Felix Cheung commented on SPARK-23314: -- code   >>> flights = spark.read.option("inferSchema",

[jira] [Commented] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349898#comment-16349898 ] Felix Cheung commented on SPARK-23314: -- log [Stage

[jira] [Updated] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23314: - Description: Under  SPARK-22216 When testing pandas_udf on group bys, I saw this error with the

[jira] [Updated] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23314: - Description: Under  SPARK-22216 When testing pandas_udf on group bys, I saw this error with the

[jira] [Commented] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349899#comment-16349899 ] Felix Cheung commented on SPARK-23314: -- [~icexelloss] [~bryanc] > Pandas grouped udf on dataset

[jira] [Updated] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23314: - Description: Under  SPARK-22216 When testing pandas_udf on group bys, I saw this error with the

[jira] [Created] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-01 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-23314: Summary: Pandas grouped udf on dataset with timestamp column error Key: SPARK-23314 URL: https://issues.apache.org/jira/browse/SPARK-23314 Project: Spark

[jira] [Updated] (SPARK-23314) Pandas grouped udf on dataset with timestamp column error

2018-02-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23314: - Description: Under  SPARK-22216 When testing pandas_udf on group bys, I saw this error with the

[jira] [Updated] (SPARK-23291) SparkR : substr : In SparkR dataframe , starting and ending position arguments in "substr" is giving wrong result when the position is greater than 1

2018-01-31 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23291: - Shepherd: Felix Cheung (was: Hossein Falaki) > SparkR : substr : In SparkR dataframe , starting

[jira] [Commented] (SPARK-23285) Allow spark.executor.cores to be fractional

2018-02-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357755#comment-16357755 ] Felix Cheung commented on SPARK-23285: -- Aounds reasonable to me > Allow spark.executor.cores to

[jira] [Commented] (SPARK-22817) Use fixed testthat version for SparkR tests in AppVeyor

2018-02-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365263#comment-16365263 ] Felix Cheung commented on SPARK-22817: -- I should have caught this - we need to fix the test because

[jira] [Comment Edited] (SPARK-22817) Use fixed testthat version for SparkR tests in AppVeyor

2018-02-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365263#comment-16365263 ] Felix Cheung edited comment on SPARK-22817 at 2/15/18 9:13 AM: --- I should

[jira] [Assigned] (SPARK-23435) R tests should support latest testthat

2018-02-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-23435: Assignee: Felix Cheung > R tests should support latest testthat >

[jira] [Created] (SPARK-23435) R tests should support latest testthat

2018-02-15 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-23435: Summary: R tests should support latest testthat Key: SPARK-23435 URL: https://issues.apache.org/jira/browse/SPARK-23435 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-23435) R tests should support latest testthat

2018-02-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368351#comment-16368351 ] Felix Cheung commented on SPARK-23435: -- Working on this. Debugging a problem. > R tests should

[jira] [Updated] (SPARK-23461) vignettes should include model predictions for some ML models

2018-02-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-23461: - Description: eg.  Linear Support Vector Machine (SVM) Classifier h4. Logistic Regression Tree

[jira] [Created] (SPARK-23461) vignettes should include model predictions for some ML models

2018-02-18 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-23461: Summary: vignettes should include model predictions for some ML models Key: SPARK-23461 URL: https://issues.apache.org/jira/browse/SPARK-23461 Project: Spark

[jira] [Commented] (SPARK-21727) Operating on an ArrayType in a SparkR DataFrame throws error

2017-12-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16305157#comment-16305157 ] Felix Cheung commented on SPARK-21727: -- [~neilalex] How it is going? > Operating on an ArrayType in

[jira] [Commented] (SPARK-21616) SparkR 2.3.0 migration guide, release note

2017-12-28 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16305192#comment-16305192 ] Felix Cheung commented on SPARK-21616: -- SPARK-22315 > SparkR 2.3.0 migration guide, release note >

[jira] [Created] (SPARK-22920) R sql functions for current_date, current_timestamp, rtrim/ltrim/trim with trimString

2017-12-28 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-22920: Summary: R sql functions for current_date, current_timestamp, rtrim/ltrim/trim with trimString Key: SPARK-22920 URL: https://issues.apache.org/jira/browse/SPARK-22920

[jira] [Commented] (SPARK-17762) invokeJava fails when serialized argument list is larger than INT_MAX (2,147,483,647) bytes

2018-01-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307724#comment-16307724 ] Felix Cheung commented on SPARK-17762: -- is this still needed after SPARK-17790 is fixed? >

[jira] [Commented] (SPARK-16693) Remove R deprecated methods

2018-01-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307725#comment-16307725 ] Felix Cheung commented on SPARK-16693: -- [~shivaram]we didn't do this, not sure if we should to get

[jira] [Commented] (SPARK-16366) Time comparison failures in SparkR unit tests

2018-01-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307728#comment-16307728 ] Felix Cheung commented on SPARK-16366: -- not 100% sure, but a a similar timestamp test failure was

[jira] [Updated] (SPARK-14037) count(df) is very slow for dataframe constructed using SparkR::createDataFrame

2018-01-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-14037: - Summary: count(df) is very slow for dataframe constructed using SparkR::createDataFrame (was:

[jira] [Commented] (SPARK-16693) Remove R deprecated methods

2018-01-02 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308367#comment-16308367 ] Felix Cheung commented on SPARK-16693: -- These are all non public methods, so officially not public

<    8   9   10   11   12   13   14   15   >