[jira] [Assigned] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13747: Assignee: Apache Spark (was: Shixiong Zhu) > Concurrent execution in SQL doesn't work

[jira] [Commented] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734585#comment-15734585 ] Apache Spark commented on SPARK-13747: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13747: Assignee: Shixiong Zhu (was: Apache Spark) > Concurrent execution in SQL doesn't work

[jira] [Reopened] (SPARK-13747) Concurrent execution in SQL doesn't work with Scala ForkJoinPool

2016-12-08 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reopened SPARK-13747: -- This issue still exists. Reopened it. > Concurrent execution in SQL doesn't work with Scala

[jira] [Commented] (SPARK-18676) Spark 2.x query plan data size estimation can crash join queries versus 1.x

2016-12-08 Thread Michael Allman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734560#comment-15734560 ] Michael Allman commented on SPARK-18676: Ah okay. That might be a strategy to explore. > Spark

[jira] [Assigned] (SPARK-17076) Cardinality estimation of join operator

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17076: Assignee: Apache Spark > Cardinality estimation of join operator >

[jira] [Assigned] (SPARK-17076) Cardinality estimation of join operator

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17076: Assignee: (was: Apache Spark) > Cardinality estimation of join operator >

[jira] [Commented] (SPARK-17076) Cardinality estimation of join operator

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734504#comment-15734504 ] Apache Spark commented on SPARK-17076: -- User 'wzhfy' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-10413) Model should support prediction on single instance

2016-12-08 Thread Aseem Bansal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734495#comment-15734495 ] Aseem Bansal edited comment on SPARK-10413 at 12/9/16 6:39 AM: --- Hi Is

[jira] [Commented] (SPARK-10413) Model should support prediction on single instance

2016-12-08 Thread Aseem Bansal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734495#comment-15734495 ] Aseem Bansal commented on SPARK-10413: -- Hi Is anyone working on this? > Model should support

[jira] [Resolved] (SPARK-18697) Upgrade sbt plugins

2016-12-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18697. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16223

[jira] [Updated] (SPARK-18349) Update R API documentation on ml model summary

2016-12-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-18349: - Fix Version/s: 2.1.1 > Update R API documentation on ml model summary >

[jira] [Resolved] (SPARK-18349) Update R API documentation on ml model summary

2016-12-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-18349. -- Resolution: Fixed Assignee: Miao Wang Target Version/s: 2.1.1 > Update R

[jira] [Commented] (SPARK-14932) Allow DataFrame.replace() to replace values with None

2016-12-08 Thread Bravo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734426#comment-15734426 ] Bravo Zhang commented on SPARK-14932: - [~nchammas] Can your use case be done by filter?

[jira] [Commented] (SPARK-18788) Add getNumPartitions() to SparkR

2016-12-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734403#comment-15734403 ] Felix Cheung commented on SPARK-18788: -- In SparkR we don't officially support RDD - In which way

[jira] [Assigned] (SPARK-14932) Allow DataFrame.replace() to replace values with None

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14932: Assignee: (was: Apache Spark) > Allow DataFrame.replace() to replace values with None

[jira] [Commented] (SPARK-14932) Allow DataFrame.replace() to replace values with None

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734376#comment-15734376 ] Apache Spark commented on SPARK-14932: -- User 'bravo-zhang' has created a pull request for this

[jira] [Assigned] (SPARK-14932) Allow DataFrame.replace() to replace values with None

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-14932: Assignee: Apache Spark > Allow DataFrame.replace() to replace values with None >

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2016-12-08 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734318#comment-15734318 ] Takeshi Yamamuro commented on SPARK-18699: -- yea, I'm also working on large csv files now and,

[jira] [Comment Edited] (SPARK-18700) getCached in HiveMetastoreCatalog not thread safe cause driver OOM

2016-12-08 Thread Li Yuanjian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15718580#comment-15718580 ] Li Yuanjian edited comment on SPARK-18700 at 12/9/16 4:47 AM: -- Give a PR for

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

2016-12-08 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734207#comment-15734207 ] Saisai Shao commented on SPARK-13955: - Can you please check the runtime environment of launching

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

2016-12-08 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734195#comment-15734195 ] liyunzhang_intel commented on SPARK-13955: -- [~jerryshao]: yes , the archive contains

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

2016-12-08 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734184#comment-15734184 ] Saisai Shao commented on SPARK-13955: - Do you have spark-yarn_2.11 jar in your archive? > Spark in

[jira] [Commented] (SPARK-11374) skip.header.line.count is ignored in HiveContext

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734150#comment-15734150 ] Dongjoon Hyun commented on SPARK-11374: --- For this issue, there is a discussion now on the PR. It

[jira] [Commented] (SPARK-18799) Spark SQL expose interface for plug-gable parser extension

2016-12-08 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734130#comment-15734130 ] Hyukjin Kwon commented on SPARK-18799: -- Ah, there it is - https://github.com/apache/spark/pull/10801

[jira] [Commented] (SPARK-18799) Spark SQL expose interface for plug-gable parser extension

2016-12-08 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734128#comment-15734128 ] Hyukjin Kwon commented on SPARK-18799: -- Less than roughly about a year ago, I saw a PR that says

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

2016-12-08 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734082#comment-15734082 ] liyunzhang_intel commented on SPARK-13955: -- test pi in yarn-client mode by using

[jira] [Comment Edited] (SPARK-13955) Spark in yarn mode fails

2016-12-08 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734082#comment-15734082 ] liyunzhang_intel edited comment on SPARK-13955 at 12/9/16 2:40 AM: ---

[jira] [Commented] (SPARK-13955) Spark in yarn mode fails

2016-12-08 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734071#comment-15734071 ] Saisai Shao commented on SPARK-13955: - IIRC {{spark.yarn.archive}} should be worked, I tried

[jira] [Issue Comment Deleted] (SPARK-17076) Cardinality estimation of join operator

2016-12-08 Thread Ron Hu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ron Hu updated SPARK-17076: --- Comment: was deleted (was: Hi, I am out of office 11 /15 through 11/18 with very limited Internet access.

[jira] [Commented] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

2016-12-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734033#comment-15734033 ] Reynold Xin commented on SPARK-18278: - In the past few days I've given this a lot of thought. I'm

[jira] [Commented] (SPARK-18750) spark should be able to control the number of executor and should not throw stack overslow

2016-12-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734028#comment-15734028 ] Sean Owen commented on SPARK-18750: --- Yes, so the basic question is: where is the error coming from? Is

[jira] [Resolved] (SPARK-18792) SparkR vignette update: logit

2016-12-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-18792. --- Resolution: Duplicate > SparkR vignette update: logit > - > >

[jira] [Commented] (SPARK-18792) SparkR vignette update: logit

2016-12-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734024#comment-15734024 ] Xiangrui Meng commented on SPARK-18792: --- [~wangmiao1981] Please check existing sub-tasks before

[jira] [Commented] (SPARK-18792) SparkR vignette update: logit

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734013#comment-15734013 ] Apache Spark commented on SPARK-18792: -- User 'mengxr' has created a pull request for this issue:

[jira] [Updated] (SPARK-18774) Ignore non-existing files when ignoreCorruptFiles is enabled

2016-12-08 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18774: Fix Version/s: 2.1.1 > Ignore non-existing files when ignoreCorruptFiles is enabled >

[jira] [Created] (SPARK-18799) Spark SQL expose interface for plug-gable parser extension

2016-12-08 Thread Jihong MA (JIRA)
Jihong MA created SPARK-18799: - Summary: Spark SQL expose interface for plug-gable parser extension Key: SPARK-18799 URL: https://issues.apache.org/jira/browse/SPARK-18799 Project: Spark Issue

[jira] [Resolved] (SPARK-18776) Offset for FileStreamSource is not json formatted

2016-12-08 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-18776. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 16205

[jira] [Commented] (SPARK-18642) Spark SQL: Catalyst is scanning undesired columns

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733992#comment-15733992 ] Dongjoon Hyun commented on SPARK-18642: --- I see. If then, I'll record that, too. > Spark SQL:

[jira] [Updated] (SPARK-17689) _temporary files breaks the Spark SQL streaming job.

2016-12-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17689: - Target Version/s: 2.2.0 Description: Steps to reproduce: 1) Start a streaming

[jira] [Updated] (SPARK-18272) Test topic addition for subscribePattern on Kafka DStream and Structured Stream

2016-12-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18272: - Issue Type: Test (was: Bug) > Test topic addition for subscribePattern on Kafka DStream

[jira] [Updated] (SPARK-18790) Keep a general offset history of stream batches

2016-12-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18790: - Target Version/s: 2.1.0 > Keep a general offset history of stream batches >

[jira] [Updated] (SPARK-18796) StreamingQueryManager should not hold a lock when starting a query

2016-12-08 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18796: - Target Version/s: 2.1.0 > StreamingQueryManager should not hold a lock when starting a

[jira] [Commented] (SPARK-18787) spark.shuffle.io.preferDirectBufs does not completely turn off direct buffer usage by Netty

2016-12-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733902#comment-15733902 ] Sean Owen commented on SPARK-18787: --- CC [~zsxwing] The tricky thing is that I think these classes may

[jira] [Comment Edited] (SPARK-18750) spark should be able to control the number of executor and should not throw stack overslow

2016-12-08 Thread Yibing Shi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733892#comment-15733892 ] Yibing Shi edited comment on SPARK-18750 at 12/9/16 12:59 AM: -- [~srowen] The

[jira] [Commented] (SPARK-18750) spark should be able to control the number of executor and should not throw stack overslow

2016-12-08 Thread Yibing Shi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733892#comment-15733892 ] Yibing Shi commented on SPARK-18750: [~srowen] The log says: {noformat} 16/11/29 15:49:11 WARN

[jira] [Commented] (SPARK-18642) Spark SQL: Catalyst is scanning undesired columns

2016-12-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733886#comment-15733886 ] Sean Owen commented on SPARK-18642: --- [~dongjoon] ignore this if you don't know, but if you happen to

[jira] [Updated] (SPARK-18718) Skip some test failures due to path length limitation and fix tests to pass on Windows

2016-12-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18718: -- Assignee: Hyukjin Kwon > Skip some test failures due to path length limitation and fix tests to pass

[jira] [Updated] (SPARK-18615) Switch to multi-line doc to avoid a genjavadoc bug for backticks

2016-12-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18615: -- Assignee: Hyukjin Kwon > Switch to multi-line doc to avoid a genjavadoc bug for backticks >

[jira] [Updated] (SPARK-18758) StreamingQueryListener events from a StreamingQuery should be sent only to the listeners in the same session as the query

2016-12-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18758: -- Assignee: Tathagata Das > StreamingQueryListener events from a StreamingQuery should be sent only to

[jira] [Commented] (SPARK-18798) Expose the kill Executor in Yarn Mode

2016-12-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733860#comment-15733860 ] Marcelo Vanzin commented on SPARK-18798: Can you explain what you mean here? The API to kill

[jira] [Resolved] (SPARK-17859) persist should not impede with spark's ability to perform a broadcast join.

2016-12-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17859. --- Resolution: Cannot Reproduce Fix Version/s: 2.0.2 > persist should not impede with spark's

[jira] [Commented] (SPARK-18591) Replace hash-based aggregates with sort-based ones if inputs already sorted

2016-12-08 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733855#comment-15733855 ] Takeshi Yamamuro commented on SPARK-18591: -- yea, If we can, it's the best. But IIUC it's

[jira] [Commented] (SPARK-18770) Current Spark Master branch missing yarn module in pom

2016-12-08 Thread Narendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733834#comment-15733834 ] Narendra commented on SPARK-18770: -- if even i have close this is available in main pom > Current Spark

[jira] [Commented] (SPARK-9487) Use the same num. worker threads in Scala/Python unit tests

2016-12-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733832#comment-15733832 ] Sean Owen commented on SPARK-9487: -- (Which thread?) I think that if you can get all tests in one language

[jira] [Created] (SPARK-18798) Expose the kill Executor in Yarn Mode

2016-12-08 Thread Narendra (JIRA)
Narendra created SPARK-18798: Summary: Expose the kill Executor in Yarn Mode Key: SPARK-18798 URL: https://issues.apache.org/jira/browse/SPARK-18798 Project: Spark Issue Type: Improvement

[jira] [Issue Comment Deleted] (SPARK-18332) SparkR 2.1 QA: Programming guide, migration guide, vignettes updates

2016-12-08 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miao Wang updated SPARK-18332: -- Comment: was deleted (was: Update spark.logit is part of the QA work.) > SparkR 2.1 QA: Programming

[jira] [Commented] (SPARK-18332) SparkR 2.1 QA: Programming guide, migration guide, vignettes updates

2016-12-08 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733785#comment-15733785 ] Miao Wang commented on SPARK-18332: --- [~josephkb] https://github.com/apache/spark/pull/16222 This PR

[jira] [Commented] (SPARK-18795) SparkR vignette update: ksTest

2016-12-08 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733777#comment-15733777 ] Miao Wang commented on SPARK-18795: --- I will work on this one too. Thanks! Miao > SparkR vignette

[jira] [Commented] (SPARK-18792) SparkR vignette update: logit

2016-12-08 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733775#comment-15733775 ] Miao Wang commented on SPARK-18792: --- I have submitted PR for JIRA-18797, which is the same as this one.

[jira] [Commented] (SPARK-18332) SparkR 2.1 QA: Programming guide, migration guide, vignettes updates

2016-12-08 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733771#comment-15733771 ] Miao Wang commented on SPARK-18332: --- Update spark.logit is part of the QA work. > SparkR 2.1 QA:

[jira] [Updated] (SPARK-18697) Upgrade sbt plugins

2016-12-08 Thread Weiqing Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiqing Yang updated SPARK-18697: - Description: For 2.2.x, it's better to make sbt plugins up-to-date. The following sbt plugins

[jira] [Updated] (SPARK-18697) Upgrade sbt plugins

2016-12-08 Thread Weiqing Yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiqing Yang updated SPARK-18697: - Description: For 2.2.x, it's better to make sbt plugins up-to-date. The following sbt plugins

[jira] [Assigned] (SPARK-18792) SparkR vignette update: logit

2016-12-08 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-18792: - Assignee: Xiangrui Meng > SparkR vignette update: logit > -

[jira] [Commented] (SPARK-18697) Upgrade sbt plugins

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733760#comment-15733760 ] Apache Spark commented on SPARK-18697: -- User 'weiqingy' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18797) Update spark.logit in sparkr-vignettes

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18797: Assignee: Apache Spark > Update spark.logit in sparkr-vignettes >

[jira] [Assigned] (SPARK-18797) Update spark.logit in sparkr-vignettes

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18797: Assignee: (was: Apache Spark) > Update spark.logit in sparkr-vignettes >

[jira] [Commented] (SPARK-18797) Update spark.logit in sparkr-vignettes

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733746#comment-15733746 ] Apache Spark commented on SPARK-18797: -- User 'wangmiao1981' has created a pull request for this

[jira] [Created] (SPARK-18797) Update spark.logit in sparkr-vignettes

2016-12-08 Thread Miao Wang (JIRA)
Miao Wang created SPARK-18797: - Summary: Update spark.logit in sparkr-vignettes Key: SPARK-18797 URL: https://issues.apache.org/jira/browse/SPARK-18797 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-16448) RemoveAliasOnlyProject should not remove alias with metadata

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16448: -- Component/s: SQL > RemoveAliasOnlyProject should not remove alias with metadata >

[jira] [Commented] (SPARK-18590) R - Include package vignettes and help pages, build source package in Spark distribution

2016-12-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733687#comment-15733687 ] Apache Spark commented on SPARK-18590: -- User 'shivaram' has created a pull request for this issue:

[jira] [Updated] (SPARK-17239) User guide for multiclass logistic regression in spark.ml

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-17239: -- Component/s: Documentation > User guide for multiclass logistic regression in spark.ml >

[jira] [Updated] (SPARK-17213) Parquet String Pushdown for Non-Eq Comparisons Broken

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-17213: -- Component/s: SQL > Parquet String Pushdown for Non-Eq Comparisons Broken >

[jira] [Updated] (SPARK-17113) Job failure due to Executor OOM in offheap mode

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-17113: -- Component/s: Spark Core > Job failure due to Executor OOM in offheap mode >

[jira] [Updated] (SPARK-17162) Range does not support SQL generation

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-17162: -- Component/s: SQL > Range does not support SQL generation >

[jira] [Updated] (SPARK-16928) Recursive call of ColumnVector::getInt() breaks JIT inlining

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16928: -- Component/s: SQL > Recursive call of ColumnVector::getInt() breaks JIT inlining >

[jira] [Updated] (SPARK-16898) Adds argument type information for typed logical plan like MapElements, TypedFilter, and AppendColumn

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16898: -- Component/s: SQL > Adds argument type information for typed logical plan like MapElements, >

[jira] [Updated] (SPARK-16906) Adds more input type information for TypedAggregateExpression

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16906: -- Component/s: SQL > Adds more input type information for TypedAggregateExpression >

[jira] [Updated] (SPARK-16870) add "spark.sql.broadcastTimeout" into docs/sql-programming-guide.md to help people to how to fix this timeout error when it happenned

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16870: -- Component/s: Documentation > add "spark.sql.broadcastTimeout" into

[jira] [Updated] (SPARK-16853) Analysis error for DataSet typed selection

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16853: -- Component/s: SQL > Analysis error for DataSet typed selection >

[jira] [Updated] (SPARK-16818) Exchange reuse incorrectly reuses scans over different sets of partitions

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16818: -- Component/s: SQL > Exchange reuse incorrectly reuses scans over different sets of partitions >

[jira] [Updated] (SPARK-18722) Move no data rate limit from StreamExecution to ProgressReporter

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18722: -- Component/s: Structured Streaming > Move no data rate limit from StreamExecution to

[jira] [Updated] (SPARK-18370) InsertIntoHadoopFsRelationCommand should keep track of its table

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18370: -- Component/s: SQL > InsertIntoHadoopFsRelationCommand should keep track of its table >

[jira] [Updated] (SPARK-18280) Potential deadlock in `StandaloneSchedulerBackend.dead`

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18280: -- Component/s: Spark Core > Potential deadlock in `StandaloneSchedulerBackend.dead` >

[jira] [Updated] (SPARK-17994) Add back a file status cache for catalog tables

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-17994: -- Component/s: SQL > Add back a file status cache for catalog tables >

[jira] [Updated] (SPARK-18125) Spark generated code causes CompileException when groupByKey, reduceGroups and map(_._2) are used

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18125: -- Component/s: SQL > Spark generated code causes CompileException when groupByKey, reduceGroups

[jira] [Updated] (SPARK-18103) Rename *FileCatalog to *FileProvider

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18103: -- Component/s: SQL > Rename *FileCatalog to *FileProvider >

[jira] [Updated] (SPARK-17446) no total size for data source tables in InMemoryCatalog

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-17446: -- Component/s: SQL > no total size for data source tables in InMemoryCatalog >

[jira] [Updated] (SPARK-17394) should not allow specify database in table/view name after RENAME TO

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-17394: -- Component/s: SQL > should not allow specify database in table/view name after RENAME TO >

[jira] [Updated] (SPARK-17296) Spark SQL: cross join + two joins = BUG

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-17296: -- Component/s: SQL > Spark SQL: cross join + two joins = BUG >

[jira] [Updated] (SPARK-17114) Adding a 'GROUP BY 1' where first column is literal results in wrong answer

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-17114: -- Component/s: SQL > Adding a 'GROUP BY 1' where first column is literal results in wrong answer

[jira] [Updated] (SPARK-17034) Ordinal in ORDER BY or GROUP BY should be treated as an unresolved expression

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-17034: -- Component/s: SQL > Ordinal in ORDER BY or GROUP BY should be treated as an unresolved

[jira] [Updated] (SPARK-16955) Using ordinals in ORDER BY causes an analysis error when the query has a GROUP BY clause using ordinals

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16955: -- Component/s: SQL > Using ordinals in ORDER BY causes an analysis error when the query has a >

[jira] [Updated] (SPARK-16888) Implements eval method for expression AssertNotNull

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16888: -- Component/s: SQL > Implements eval method for expression AssertNotNull >

[jira] [Updated] (SPARK-16829) sparkR sc.setLogLevel doesn't work

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16829: -- Component/s: SparkR > sparkR sc.setLogLevel doesn't work > --

[jira] [Updated] (SPARK-16884) Move DataSourceScanExec out of ExistingRDD.scala file

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16884: -- Component/s: SQL > Move DataSourceScanExec out of ExistingRDD.scala file >

[jira] [Updated] (SPARK-16138) YarnAllocator tries to cancel executor requests when we have none

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-16138: -- Component/s: YARN > YarnAllocator tries to cancel executor requests when we have none >

[jira] [Updated] (SPARK-15958) Make initial buffer size for the Sorter configurable

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15958: -- Component/s: Spark Core > Make initial buffer size for the Sorter configurable >

[jira] [Updated] (SPARK-15783) Fix more flakiness: o.a.s.scheduler.BlacklistIntegrationSuite

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15783: -- Component/s: Spark Core > Fix more flakiness: o.a.s.scheduler.BlacklistIntegrationSuite >

[jira] [Updated] (SPARK-15621) BatchEvalPythonExec fails with OOM

2016-12-08 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15621: -- Component/s: SQL > BatchEvalPythonExec fails with OOM > -- > >

  1   2   3   >