Re: Does Zeppelin 0.7.1 work with Spark Structured Streaming 2.1.1?

2017-05-21 Thread Felix Cheung
I think you can have one notebook for setup - calling the source and sink as you have below; then another notebook to run the %sql to read from memory temp view and output to a visualization. This way you could have the 2nd notebook be a dashboard and run (i.e. refresh automatically) on a sched

[jira] [Updated] (SPARK-18825) Eliminate duplicate links in SparkR API doc index

2017-05-20 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-18825: - There might be some delicacy here since this might need to work without an external script when we

[jira] [Commented] (SPARK-18825) Eliminate duplicate links in SparkR API doc index

2017-05-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16018314#comment-16018314 ] Felix Cheung commented on SPARK-18825: -- handling a fork of knitr might be too

[jira] [Commented] (SPARK-18825) Eliminate duplicate links in SparkR API doc index

2017-05-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16018313#comment-16018313 ] Felix Cheung commented on SPARK-18825: -- I see about dontrun - yes, I don't

[jira] [Updated] (SPARK-20727) Skip SparkR tests when missing Hadoop winutils on CRAN windows machines

2017-05-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20727: - Issue Type: Sub-task (was: Bug) Parent: SPARK-15799 > Skip SparkR tests when miss

[jira] [Commented] (SPARK-20815) NullPointerException in RPackageUtils#checkManifestForR

2017-05-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16018300#comment-16018300 ] Felix Cheung commented on SPARK-20815: -- make sense to me. would you lik

[jira] [Commented] (SPARK-18825) Eliminate duplicate links in SparkR API doc index

2017-05-19 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16017699#comment-16017699 ] Felix Cheung commented on SPARK-18825: -- interesting - do you think knitr can

Re: Improve SparkR collect performance with Arrow

2017-05-15 Thread Felix Cheung
I can try to help. _ From: Wes McKinney Sent: Monday, May 15, 2017 12:49 PM Subject: Re: Improve SparkR collect performance with Arrow To: Dirk Eddelbuettel , , Jim Hester , Hadley Wickham , Kevin Ushey Adding Hadley and others to the conversation to advise on th

[jira] [Updated] (SPARK-19568) Must include class/method documentation for CRAN check

2017-05-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19568: - Certainly - we recognize the problem with SparkR not published in CRAN and have been actively

[jira] [Resolved] (SPARK-20726) R wrapper for SQL broadcast

2017-05-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20726. -- Resolution: Fixed Assignee: Maciej Szymkiewicz Fix Version/s: 2.3.0

[jira] [Comment Edited] (SPARK-19568) Must include class/method documentation for CRAN check

2017-05-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009830#comment-16009830 ] Felix Cheung edited comment on SPARK-19568 at 5/14/17 7:3

[jira] [Commented] (SPARK-19568) Must include class/method documentation for CRAN check

2017-05-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009831#comment-16009831 ] Felix Cheung commented on SPARK-19568: -- .. based on that, I might just resolve

[jira] [Commented] (SPARK-19568) Must include class/method documentation for CRAN check

2017-05-14 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16009830#comment-16009830 ] Felix Cheung commented on SPARK-19568: -- I checked the Rd files are generated

[jira] [Resolved] (SPARK-20704) CRAN test should run single threaded

2017-05-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20704. -- Resolution: Fixed Assignee: Felix Cheung Fix Version/s: 2.3.0

[jira] [Resolved] (SPARK-20619) StringIndexer supports multiple ways of label ordering

2017-05-12 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20619. -- Resolution: Fixed Assignee: Wayne Zhang Fix Version/s: 2.3.0

[jira] [Commented] (SPARK-20512) SparkR 2.2 QA: Programming guide, migration guide, vignettes updates

2017-05-11 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006951#comment-16006951 ] Felix Cheung commented on SPARK-20512: -- I did a QA pass on the R vigne

[jira] [Created] (SPARK-20704) CRAN test should run single threaded

2017-05-10 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20704: Summary: CRAN test should run single threaded Key: SPARK-20704 URL: https://issues.apache.org/jira/browse/SPARK-20704 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-20590) Map default input data source formats to inlined classes

2017-05-10 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005864#comment-16005864 ] Felix Cheung commented on SPARK-20590: -- When the user explicitly specifies

[jira] [Updated] (SPARK-20666) Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError

2017-05-10 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Description: seeing quite a bit of this on AppVeyor, aka Windows only,-> seems like in ot

[jira] [Resolved] (SPARK-20670) Simplify FPGrowth transform

2017-05-09 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20670. -- Resolution: Fixed Assignee: yuhao yang Fix Version/s: 2.3.0 Target

[jira] [Updated] (SPARK-20689) python doctest leaking bucketed table

2017-05-09 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20689: - Description: When trying to address build test failure in SPARK-20661 we discovered some tables

[jira] [Updated] (SPARK-20689) python doctest leaking bucketed table

2017-05-09 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20689: - Description: When trying to address build test failure in SPARK-20661 we discovered some tables

[jira] [Updated] (SPARK-20689) python doctest leaking bucketed table

2017-05-09 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20689: - Description: When trying to address build test failure in SPARK-20661 we discovered some tables

[jira] [Created] (SPARK-20689) python doctest leaking bucketed table

2017-05-09 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20689: Summary: python doctest leaking bucketed table Key: SPARK-20689 URL: https://issues.apache.org/jira/browse/SPARK-20689 Project: Spark Issue Type: Bug

Re: Hive Reserve Keyword support

2017-05-09 Thread Felix Cheung
I think you can put backticks around the name date https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL From: Jongyoul Lee Sent: Tuesday, May 9, 2017 10:33:50 AM To: users@zeppelin.apache.org Subject: Re: Hive Reserve Keyword support If it's possi

[jira] [Updated] (SPARK-20666) Flaky test - SparkListenerBus randomly failing on Windows

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Summary: Flaky test - SparkListenerBus randomly failing on Windows (was: Flaky test - random ml

[jira] [Updated] (SPARK-20666) Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError on Windows

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Summary: Flaky test - SparkListenerBus randomly failing java.lang.IllegalAccessError on Windows

[jira] [Updated] (SPARK-20666) Flaky test - SparkListenerBus randomly failing on Windows

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Description: seeing quite a bit of this on AppVeyor, aka Windows only, always only when running

[jira] [Updated] (SPARK-20666) Flaky test - SparkListenerBus randomly failing on Windows

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Component/s: Spark Core > Flaky test - SparkListenerBus randomly failing on Wind

[jira] [Updated] (SPARK-20666) Flaky test - random ml test failure on Windows

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Description: seeing quite a bit of this on AppVeyor, aka Windows only {code} Exception in

[jira] [Created] (SPARK-20666) Flaky test - random ml test failure on Windows

2017-05-08 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20666: Summary: Flaky test - random ml test failure on Windows Key: SPARK-20666 URL: https://issues.apache.org/jira/browse/SPARK-20666 Project: Spark Issue Type

[jira] [Updated] (SPARK-20666) Flaky test - random ml test failure on Windows

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20666: - Description: seeing quite a bit of this on AppVeyor, aka Windows only {code

[jira] [Resolved] (SPARK-20550) R wrappers for Dataset.alias

2017-05-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20550. -- Resolution: Fixed Assignee: Maciej Szymkiewicz Fix Version/s: 2.3.0

[jira] [Resolved] (SPARK-20626) Fix SparkR test warning on Windows with timestamp time zone

2017-05-07 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20626. -- Resolution: Fixed Assignee: Felix Cheung Fix Version/s: 2.3.0

[jira] [Updated] (SPARK-20626) Fix SparkR test warning on Windows with timestamp time zone

2017-05-07 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20626: - Summary: Fix SparkR test warning on Windows with timestamp time zone (was: Fix SparkR test on

[jira] [Created] (SPARK-20626) Fix SparkR test on Windows with timestamp time zone

2017-05-06 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20626: Summary: Fix SparkR test on Windows with timestamp time zone Key: SPARK-20626 URL: https://issues.apache.org/jira/browse/SPARK-20626 Project: Spark Issue

[jira] [Created] (SPARK-20625) SQL test running on Windows hadoop command warning

2017-05-06 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20625: Summary: SQL test running on Windows hadoop command warning Key: SPARK-20625 URL: https://issues.apache.org/jira/browse/SPARK-20625 Project: Spark Issue

[jira] [Resolved] (SPARK-20614) Use the same log4j configuration with Jenkins in AppVeyor

2017-05-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20614. -- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.3.0

[jira] [Commented] (SPARK-20520) R streaming tests failed on Windows

2017-05-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999252#comment-15999252 ] Felix Cheung commented on SPARK-20520: -- waiting for the next RC to try with fix

[jira] [Commented] (SPARK-20613) Double quotes in Windows batch script

2017-05-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998474#comment-15998474 ] Felix Cheung commented on SPARK-20613: -- [~shivaram]could you add jarretmeye

[jira] [Resolved] (SPARK-20613) Double quotes in Windows batch script

2017-05-05 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20613. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.0

[jira] [Resolved] (SPARK-20257) Fix test for directory created to work when running as R CMD check

2017-05-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20257. -- Resolution: Not A Problem Assignee: Felix Cheung We have disabled this test on CRAN, so

[jira] [Resolved] (SPARK-20015) Document R Structured Streaming (experimental) in R vignettes and R & SS programming guide, R example

2017-05-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20015. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.0 Target

[jira] [Resolved] (SPARK-20571) Flaky SparkR StructuredStreaming tests

2017-05-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20571. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.0 Target

[jira] [Commented] (SPARK-20571) Flaky SparkR StructuredStreaming tests

2017-05-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996390#comment-15996390 ] Felix Cheung commented on SPARK-20571: -- done. will monitor tests for a coupl

[jira] [Resolved] (SPARK-20544) R wrapper for input_file_name

2017-05-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20544. -- Resolution: Fixed Assignee: Maciej Szymkiewicz Fix Version/s: 2.3.0

[jira] [Resolved] (SPARK-20585) R generic hint support

2017-05-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20585. -- Resolution: Fixed Assignee: Maciej Szymkiewicz Fix Version/s: 2.3.0

[jira] [Commented] (SPARK-20520) R streaming tests failed on Windows

2017-05-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996306#comment-15996306 ] Felix Cheung commented on SPARK-20520: -- should try again after SPARK-20571

[jira] [Assigned] (SPARK-20571) Flaky SparkR StructuredStreaming tests

2017-05-04 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-20571: Assignee: Felix Cheung > Flaky SparkR StructuredStreaming te

[jira] [Resolved] (SPARK-20543) R should skip long running or non-essential tests when running on CRAN

2017-05-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20543. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.0 Target

[jira] [Commented] (SPARK-20571) Flaky SparkR StructuredStreaming tests

2017-05-03 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15995149#comment-15995149 ] Felix Cheung commented on SPARK-20571: -- Thanks, I will look into today. >

Re: Spark 2.2.0 or Spark 2.3.0?

2017-05-02 Thread Felix Cheung
Yes 2.2.0 From: kant kodali Sent: Monday, May 1, 2017 10:43:44 PM To: dev Subject: Spark 2.2.0 or Spark 2.3.0? Hi All, If I understand the Spark standard release process correctly. It looks like the official release is going to be sometime end of this month and

[jira] [Resolved] (SPARK-20532) SparkR should provide grouping and grouping_id

2017-05-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20532. -- Resolution: Fixed Assignee: Maciej Szymkiewicz Fix Version/s: 2.3.0

[jira] [Resolved] (SPARK-20192) SparkR 2.2.0 migration guide, release note

2017-05-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20192. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.0 Target

[jira] [Commented] (SPARK-20192) SparkR 2.2.0 migration guide, release note

2017-05-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992306#comment-15992306 ] Felix Cheung commented on SPARK-20192: -- SPARK-20513 for release note ins

[jira] [Resolved] (SPARK-20541) SparkR SS should support awaitTermination without timeout

2017-04-30 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20541. -- Resolution: Fixed Assignee: Felix Cheung Fix Version/s: 2.3.0

[jira] [Updated] (SPARK-20543) R should skip long running or non-essential tests when running on CRAN

2017-04-30 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20543: - Description: This is actually recommended in the CRAN policies > R should skip long running

[jira] [Assigned] (SPARK-20543) R should skip long running or non-essential tests when running on CRAN

2017-04-30 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-20543: Assignee: Felix Cheung > R should skip long running or non-essential tests when runn

[jira] [Created] (SPARK-20543) R should skip long running or non-essential tests when running on CRAN

2017-04-30 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20543: Summary: R should skip long running or non-essential tests when running on CRAN Key: SPARK-20543 URL: https://issues.apache.org/jira/browse/SPARK-20543 Project

[jira] [Resolved] (SPARK-20490) Add eqNullSafe, not and ! to SparkR

2017-04-30 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20490. -- Resolution: Fixed Assignee: Maciej Szymkiewicz Fix Version/s: 2.3.0

[jira] [Created] (SPARK-20541) SparkR SS should support awaitTermination without timeout

2017-04-30 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20541: Summary: SparkR SS should support awaitTermination without timeout Key: SPARK-20541 URL: https://issues.apache.org/jira/browse/SPARK-20541 Project: Spark

[jira] [Resolved] (SPARK-20535) R wrappers for explode_outer and posexplode_outer

2017-04-30 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20535. -- Resolution: Fixed Assignee: Maciej Szymkiewicz Fix Version/s: 2.3.0

[jira] [Resolved] (SPARK-20493) De-deuplicate parse logics for DDL-like type string in R

2017-04-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20493. -- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.3.0

[jira] [Resolved] (SPARK-20533) SparkR Wrappers Model should be private and value should be lazy

2017-04-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20533. -- Resolution: Fixed Assignee: Miao Wang Fix Version/s: 2.3.0 Target

[jira] [Commented] (SPARK-14501) spark.ml parity for fpm - frequent items

2017-04-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15989977#comment-15989977 ] Felix Cheung commented on SPARK-14501: -- all done! > spark.ml parity

[jira] [Resolved] (SPARK-19791) Add doc and example for fpgrowth

2017-04-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-19791. -- Resolution: Fixed Fix Version/s: 2.3.0 2.2.0 Target

[jira] [Resolved] (SPARK-20477) Document R bisecting k-means in R programming guide

2017-04-29 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20477. -- Resolution: Fixed Assignee: Miao Wang Fix Version/s: 2.3.0

Re: "java.lang.IllegalStateException: There is no space for new record" in GraphFrames

2017-04-28 Thread Felix Cheung
Can you allocate more memory to the executor? Also please open issue with gf on its github From: rok Sent: Friday, April 28, 2017 1:42:33 AM To: user@spark.apache.org Subject: "java.lang.IllegalStateException: There is no space for new record" in GraphFrames Wh

Re: how to create List in pyspark

2017-04-28 Thread Felix Cheung
Why no use sql functions explode and split? Would perform and be more stable then udf From: Yanbo Liang Sent: Thursday, April 27, 2017 7:34:54 AM To: Selvam Raman Cc: user Subject: Re: how to create List in pyspark ​You can try with UDF, like the following code s

Re: [VOTE] Apache Spark 2.1.1 (RC4)

2017-04-28 Thread Felix Cheung
+1 Tested R on linux and windows Previous issue with building vignettes on windows with stackoverflow in ALS still reproduce but as confirmed the issue was in 2.1.0 so this isn't a regression (and hope for the best on CRAN..) https://issues.apache.org/jira/browse/SPARK-20402 __

[jira] [Resolved] (SPARK-20478) Document LinearSVC in R programming guide

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20478. -- Resolution: Fixed Assignee: Miao Wang Fix Version/s: 2.3.0

[jira] [Commented] (SPARK-20520) R streaming tests failed on Windows

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988222#comment-15988222 ] Felix Cheung commented on SPARK-20520: -- looks like it's just running

[jira] [Updated] (SPARK-20520) R streaming tests failed on Windows

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20520: - Issue Type: Bug (was: Umbrella) > R streaming tests failed on Wind

[jira] [Updated] (SPARK-20520) R streaming tests failed on Windows

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20520: - Summary: R streaming tests failed on Windows (was: R streaming test failed on Windows) >

[jira] [Updated] (SPARK-20520) R streaming test failed on Windows

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20520: - Description: Running R CMD check on SparkR 2.2 RC1 packages {code} Failed

[jira] [Assigned] (SPARK-20520) R streaming test failed on Windows

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-20520: Assignee: Felix Cheung (was: Joseph K. Bradley) > R streaming test failed on Wind

[jira] [Updated] (SPARK-20520) R streaming test failed on Windows

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20520: - Component/s: (was: Documentation) > R streaming test failed on Wind

[jira] [Created] (SPARK-20520) R streaming test failed on Windows

2017-04-27 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20520: Summary: R streaming test failed on Windows Key: SPARK-20520 URL: https://issues.apache.org/jira/browse/SPARK-20520 Project: Spark Issue Type: Umbrella

[jira] [Updated] (SPARK-20192) SparkR 2.2.0 migration guide, release note

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20192: - Summary: SparkR 2.2.0 migration guide, release note (was: SparkR 2.2.0 release note) > Spa

[jira] [Commented] (SPARK-20512) SparkR 2.2 QA: Programming guide, migration guide, vignettes updates

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988168#comment-15988168 ] Felix Cheung commented on SPARK-20512: -- version migration section is

[jira] [Assigned] (SPARK-20192) SparkR 2.2.0 release note

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-20192: Assignee: Felix Cheung > SparkR 2.2.0 release n

[jira] [Resolved] (SPARK-20208) Document R fpGrowth support in vignettes, programming guide and code example

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20208. -- Resolution: Fixed > Document R fpGrowth support in vignettes, programming guide and c

[jira] [Assigned] (SPARK-20015) Document R Structured Streaming (experimental) in R vignettes and R & SS programming guide, R example

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-20015: Assignee: Felix Cheung > Document R Structured Streaming (experimental) in R vignet

Re: [VOTE] Superset Proposal for Apache Incubator

2017-04-27 Thread Felix Cheung
+1 (nonbinding) On Wed, Apr 26, 2017 at 11:13 PM Jeff Feng wrote: > Hello everyone, > > Thank you for checking out our proposal on Superset and for your > consideration for the Apache Incubator. So far, I believe we have 8 > binding votes and 2 non-binding votes. > > As Taylor mentioned earlier

[jira] [Commented] (SPARK-20478) Document LinearSVC in R programming guide

2017-04-27 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15986153#comment-15986153 ] Felix Cheung commented on SPARK-20478: -- where in https://raw.githubusercontent

[jira] [Updated] (SPARK-20015) Document R Structured Streaming (experimental) in R vignettes and R & SS programming guide, R example

2017-04-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20015: - Summary: Document R Structured Streaming (experimental) in R vignettes and R & SS program

[jira] [Commented] (SPARK-20477) Document R bisecting k-means in R programming guide

2017-04-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985226#comment-15985226 ] Felix Cheung commented on SPARK-20477: -- [~wangmiao1981] Would you like to add

[jira] [Updated] (SPARK-20477) Document R bisecting k-means in R programming guide

2017-04-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20477: - Issue Type: Documentation (was: Bug) > Document R bisecting k-means in R programming gu

[jira] [Commented] (SPARK-20478) Document LinearSVC in R programming guide

2017-04-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985227#comment-15985227 ] Felix Cheung commented on SPARK-20478: -- [~wangmiao1981] Would you like to add

[jira] [Reopened] (SPARK-20208) Document R fpGrowth support in vignettes, programming guide and code example

2017-04-26 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reopened SPARK-20208: -- Actually, would you mind updating the R programming guide too? > Document R fpGrowth support

[jira] [Created] (SPARK-20478) Document LinearSVC in R programming guide

2017-04-26 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20478: Summary: Document LinearSVC in R programming guide Key: SPARK-20478 URL: https://issues.apache.org/jira/browse/SPARK-20478 Project: Spark Issue Type

[jira] [Created] (SPARK-20477) Document R bisecting k-means in R programming guide

2017-04-26 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-20477: Summary: Document R bisecting k-means in R programming guide Key: SPARK-20477 URL: https://issues.apache.org/jira/browse/SPARK-20477 Project: Spark Issue

[jira] [Resolved] (SPARK-20437) R wrappers for rollup and cube

2017-04-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20437. -- Resolution: Fixed Assignee: Maciej Szymkiewicz Fix Version/s: 2.3.0

[jira] [Resolved] (SPARK-5484) Pregel should checkpoint periodically to avoid StackOverflowError

2017-04-25 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-5484. - Resolution: Fixed Assignee: dingding (was: Ankur Dave) Fix Version/s

Re: How to create a real time dashboards from spark using web socket?

2017-04-25 Thread Felix Cheung
You could have multiple notes sharing interpreter - could you have one note for setup to call streamingcontext.start and another note on a schedule just to run the select sql statement? _ From: kant kodali mailto:kanth...@gmail.com>> Sent: Tuesday, April 25, 2017 8:12

[jira] [Resolved] (SPARK-20208) Document R fpGrowth support in vignettes, programming guide and code example

2017-04-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20208. -- Resolution: Fixed Fix Version/s: 2.2.0 > Document R fpGrowth support in vignet

[jira] [Resolved] (SPARK-20438) R wrappers for split and repeat

2017-04-24 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-20438. -- Resolution: Fixed Assignee: Maciej Szymkiewicz Fix Version/s: 2.3.0

Re: [SparkR] - options around setting up SparkSession / SparkContext

2017-04-22 Thread Felix Cheung
? _ From: Vin J mailto:winjos...@gmail.com>> Sent: Saturday, April 22, 2017 12:33 AM Subject: Re: [SparkR] - options around setting up SparkSession / SparkContext To: Felix Cheung mailto:felixcheun...@hotmail.com>> Cc: mailto:dev@spark.apache.org>> This is for a noteboo

Re: Spark SQL - Global Temporary View is not behaving as expected

2017-04-22 Thread Felix Cheung
Cross session is this context is multiple spark sessions from the same spark context. Since you are running two shells, you are having different spark context. Do you have to you a temp view? Could you create a table? _ From: Hemanth Gudela mailto:hemanth.gud...@qva

Re: [SparkR] - options around setting up SparkSession / SparkContext

2017-04-21 Thread Felix Cheung
How would you handle this in Scala? If you are adding a wrapper func like getSparkSession for Scala, and have your users call it, can't you do that same in SparkR? After all, while true you don't need a SparkSession object to call the R API, someone still needs to call sparkR.session() to initi

<    10   11   12   13   14   15   16   17   18   19   >