[jira] [Updated] (SPARK-18243) Converge the insert path of Hive tables with data source tables

2017-01-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-18243: Assignee: Wenchen Fan > Converge the insert path of Hive tables with data source tables > -

[jira] [Resolved] (SPARK-18243) Converge the insert path of Hive tables with data source tables

2017-01-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-18243. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16517 [https://github.com/ap

[jira] [Commented] (SPARK-19115) SparkSQL unsupports the command " create external table if not exist new_tbl like old_tbl location '/warehouse/new_tbl' "

2017-01-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827547#comment-15827547 ] Xiao Li commented on SPARK-19115: - Sorry for the late reply. This sounds reasonable. Does

[jira] [Reopened] (SPARK-19115) SparkSQL unsupports the command " create external table if not exist new_tbl like old_tbl location '/warehouse/new_tbl' "

2017-01-17 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaochen Ouyang reopened SPARK-19115: - spark2.x unsupports the sql command: create external table if not exists gen_tbl like src_tb

[jira] [Updated] (SPARK-19115) SparkSQL unsupports the command " create external table if not exist new_tbl like old_tbl location '/warehouse/new_tbl' "

2017-01-17 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaochen Ouyang updated SPARK-19115: Description: spark2.0.1 unsupports the command " create external table if not exist new_tb

[jira] [Updated] (SPARK-19115) SparkSQL unsupports the command " create external table if not exist new_tbl like old_tbl location '/warehouse/new_tbl' "

2017-01-17 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaochen Ouyang updated SPARK-19115: Description: spark2.0.1 unsupports the command " create external table if not exist new_tb

[jira] [Updated] (SPARK-19115) SparkSQL unsupports the command " create external table if not exist new_tbl like old_tbl location '/warehouse/new_tbl' "

2017-01-17 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaochen Ouyang updated SPARK-19115: Description: spark2.0.1 unsupports the command " create external table if not exist new_tb

[jira] [Updated] (SPARK-19115) SparkSQL unsupports the command " create external table if not exist new_tbl like old_tbl location '/warehouse/new_tbl' "

2017-01-17 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaochen Ouyang updated SPARK-19115: Description: spark2.0.1 unsupports the command " create external table if not exist new_tb

[jira] [Updated] (SPARK-19115) SparkSQL unsupports the command " create external table if not exist new_tbl like old_tbl location '/warehouse/new_tbl' "

2017-01-17 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaochen Ouyang updated SPARK-19115: Summary: SparkSQL unsupports the command " create external table if not exist new_tbl like

[jira] [Assigned] (SPARK-19270) Add summary table to GLM summary

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19270: Assignee: (was: Apache Spark) > Add summary table to GLM summary > ---

[jira] [Assigned] (SPARK-19270) Add summary table to GLM summary

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19270: Assignee: Apache Spark > Add summary table to GLM summary > --

[jira] [Commented] (SPARK-19270) Add summary table to GLM summary

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827521#comment-15827521 ] Apache Spark commented on SPARK-19270: -- User 'actuaryzhang' has created a pull reque

[jira] [Commented] (SPARK-2868) Support named accumulators in Python

2017-01-17 Thread Kyle Heath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827519#comment-15827519 ] Kyle Heath commented on SPARK-2868: --- Short version: Is there anything I can do to help b

[jira] [Commented] (SPARK-8480) Add setName for Dataframe

2017-01-17 Thread Kaushal Prajapati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827515#comment-15827515 ] Kaushal Prajapati commented on SPARK-8480: -- For example, Using SparkContext we ca

[jira] [Closed] (SPARK-19081) spark sql use HIVE UDF throw exception when return a Map value

2017-01-17 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro closed SPARK-19081. Resolution: Resolved Fix Version/s: 1.5.0 > spark sql use HIVE UDF throw exception w

[jira] [Created] (SPARK-19270) Add summary table to GLM summary

2017-01-17 Thread Wayne Zhang (JIRA)
Wayne Zhang created SPARK-19270: --- Summary: Add summary table to GLM summary Key: SPARK-19270 URL: https://issues.apache.org/jira/browse/SPARK-19270 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-19081) spark sql use HIVE UDF throw exception when return a Map value

2017-01-17 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827510#comment-15827510 ] Takeshi Yamamuro edited comment on SPARK-19081 at 1/18/17 6:54 AM:

[jira] [Commented] (SPARK-19081) spark sql use HIVE UDF throw exception when return a Map value

2017-01-17 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827510#comment-15827510 ] Takeshi Yamamuro commented on SPARK-19081: -- Since the issue the reporter says ha

[jira] [Commented] (SPARK-19081) spark sql use HIVE UDF throw exception when return a Map value

2017-01-17 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827506#comment-15827506 ] Takeshi Yamamuro commented on SPARK-19081: -- congrats! > spark sql use HIVE UDF

[jira] [Commented] (SPARK-19081) spark sql use HIVE UDF throw exception when return a Map value

2017-01-17 Thread Davy Song (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827503#comment-15827503 ] Davy Song commented on SPARK-19081: --- @Takeshi, thanks very much. I have rewritten the u

[jira] [Created] (SPARK-19269) Scheduler Delay in spark ui

2017-01-17 Thread xdcjie (JIRA)
xdcjie created SPARK-19269: -- Summary: Scheduler Delay in spark ui Key: SPARK-19269 URL: https://issues.apache.org/jira/browse/SPARK-19269 Project: Spark Issue Type: Question Components: We

[jira] [Commented] (SPARK-18011) SparkR serialize "NA" throws exception

2017-01-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827478#comment-15827478 ] Felix Cheung commented on SPARK-18011: -- very cool, thanks for all the investigation.

[jira] [Commented] (SPARK-12347) Write script to run all MLlib examples for testing

2017-01-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827475#comment-15827475 ] Felix Cheung commented on SPARK-12347: -- [~ethanlu...@gmail.com] would you be willing

[jira] [Commented] (SPARK-18348) Improve tree ensemble model summary

2017-01-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827474#comment-15827474 ] Felix Cheung commented on SPARK-18348: -- [~yanboliang] would you like to run with thi

[jira] [Updated] (SPARK-19066) SparkR LDA doesn't set optimizer correctly

2017-01-17 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19066: - Fix Version/s: 2.1.1 > SparkR LDA doesn't set optimizer correctly > -

[jira] [Commented] (SPARK-19255) SQL Listener is causing out of memory, in case of large no of shuffle partition

2017-01-17 Thread Ashok Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827379#comment-15827379 ] Ashok Kumar commented on SPARK-19255: - @Takeshi , thanks for looking into the issue.

[jira] [Commented] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2017-01-17 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827371#comment-15827371 ] Genmao Yu commented on SPARK-19185: --- [~c...@koeninger.org] I provide a PR to give more

[jira] [Assigned] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19185: Assignee: Apache Spark > ConcurrentModificationExceptions with CachedKafkaConsumers when W

[jira] [Assigned] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19185: Assignee: (was: Apache Spark) > ConcurrentModificationExceptions with CachedKafkaConsu

[jira] [Commented] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827368#comment-15827368 ] Apache Spark commented on SPARK-19185: -- User 'uncleGen' has created a pull request f

[jira] [Commented] (SPARK-18627) Cannot connect to Hive metastore in client mode with proxy user

2017-01-17 Thread Mubashir Kazia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827347#comment-15827347 ] Mubashir Kazia commented on SPARK-18627: bq. it doesn't explain why that works in

[jira] [Commented] (SPARK-19081) spark sql use HIVE UDF throw exception when return a Map value

2017-01-17 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827332#comment-15827332 ] Takeshi Yamamuro commented on SPARK-19081: -- yea, great! Any action for this? If

[jira] [Commented] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2017-01-17 Thread Genmao Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827330#comment-15827330 ] Genmao Yu commented on SPARK-19185: --- IMHO, persisting can not cover all scene, just lik

[jira] [Commented] (SPARK-18627) Cannot connect to Hive metastore in client mode with proxy user

2017-01-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827328#comment-15827328 ] Marcelo Vanzin commented on SPARK-18627: bq. The title of this jira is also incor

[jira] [Updated] (SPARK-19250) In security cluster, spark beeline connect to hive metastore failed

2017-01-17 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] meiyoula updated SPARK-19250: - Description: 1. starting thriftserver in security mode, set hive.metastore.uris to hive metastore uri, a

[jira] [Updated] (SPARK-18627) Cannot connect to Hive metastore in client mode with proxy user

2017-01-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-18627: --- Summary: Cannot connect to Hive metastore in client mode with proxy user (was: Cannot fetch

[jira] [Updated] (SPARK-19266) Ensure DiskStore properly encrypts cached data on disk.

2017-01-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-19266: --- Summary: Ensure DiskStore properly encrypts cached data on disk. (was: DiskStore does not en

[jira] [Commented] (SPARK-19250) In security cluster, spark beeline connect to hive metastore failed

2017-01-17 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827320#comment-15827320 ] meiyoula commented on SPARK-19250: -- [~rxin] I think you may understand this bug. I find

[jira] [Updated] (SPARK-19266) DiskStore does not encrypt serialized RDD data

2017-01-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-19266: --- Priority: Minor (was: Major) > DiskStore does not encrypt serialized RDD data >

[jira] [Commented] (SPARK-19266) DiskStore does not encrypt serialized RDD data

2017-01-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827315#comment-15827315 ] Marcelo Vanzin commented on SPARK-19266: Hmm... seems {{partiallySerializedValues

[jira] [Updated] (SPARK-19250) In security cluster, spark beeline connect to hive metastore failed

2017-01-17 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] meiyoula updated SPARK-19250: - Labels: security-issue (was: ) > In security cluster, spark beeline connect to hive metastore failed > -

[jira] [Comment Edited] (SPARK-19234) AFTSurvivalRegression chokes silently or with confusing errors when any labels are zero

2017-01-17 Thread Andrew MacKinlay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827236#comment-15827236 ] Andrew MacKinlay edited comment on SPARK-19234 at 1/18/17 1:53 AM:

[jira] [Commented] (SPARK-16578) Configurable hostname for RBackend

2017-01-17 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827237#comment-15827237 ] Shivaram Venkataraman commented on SPARK-16578: --- I think its fine to remove

[jira] [Commented] (SPARK-19234) AFTSurvivalRegression chokes silently or with confusing errors when any labels are zero

2017-01-17 Thread Andrew MacKinlay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827236#comment-15827236 ] Andrew MacKinlay commented on SPARK-19234: -- [~yanbo] I presume that you are the

[jira] [Updated] (SPARK-19268) File does not exist: /tmp/temporary-157b89c1-27bb-49f3-a70c-ca1b75022b4d/state/0/2/1.delta

2017-01-17 Thread liyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyan updated SPARK-19268: -- Description: bq. ./run-example sql.streaming.JavaStructuredKafkaWordCount 192.168.3.110:9092 subscribe topic03

[jira] [Comment Edited] (SPARK-19264) Work should start driver, the same to AM of yarn

2017-01-17 Thread hustfxj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827201#comment-15827201 ] hustfxj edited comment on SPARK-19264 at 1/18/17 1:39 AM: -- So I

[jira] [Updated] (SPARK-19268) File does not exist: /tmp/temporary-157b89c1-27bb-49f3-a70c-ca1b75022b4d/state/0/2/1.delta

2017-01-17 Thread liyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyan updated SPARK-19268: -- Description: bq. ./run-example sql.streaming.JavaStructuredKafkaWordCount 192.168.3.110:9092 subscribe topic03

[jira] [Updated] (SPARK-19268) File does not exist: /tmp/temporary-157b89c1-27bb-49f3-a70c-ca1b75022b4d/state/0/2/1.delta

2017-01-17 Thread liyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyan updated SPARK-19268: -- Description: bq. ./run-example sql.streaming.JavaStructuredKafkaWordCount 192.168.3.110:9092 subscribe topic03

[jira] [Created] (SPARK-19268) File does not exist: /tmp/temporary-157b89c1-27bb-49f3-a70c-ca1b75022b4d/state/0/2/1.delta

2017-01-17 Thread liyan (JIRA)
liyan created SPARK-19268: - Summary: File does not exist: /tmp/temporary-157b89c1-27bb-49f3-a70c-ca1b75022b4d/state/0/2/1.delta Key: SPARK-19268 URL: https://issues.apache.org/jira/browse/SPARK-19268 Project:

[jira] [Commented] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2017-01-17 Thread Helena Edelson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827215#comment-15827215 ] Helena Edelson commented on SPARK-19185: I've seen this as well, the exceptions,

[jira] [Commented] (SPARK-19264) Work should start driver, the same to AM of yarn

2017-01-17 Thread hustfxj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827201#comment-15827201 ] hustfxj commented on SPARK-19264: - So I must change the last code like that {code} t

[jira] [Commented] (SPARK-19081) spark sql use HIVE UDF throw exception when return a Map value

2017-01-17 Thread Davy Song (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827199#comment-15827199 ] Davy Song commented on SPARK-19081: --- Thanks Takeshi, you are right. I test map on Datab

[jira] [Comment Edited] (SPARK-19264) Work should start driver, the same to AM of yarn

2017-01-17 Thread hustfxj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827181#comment-15827181 ] hustfxj edited comment on SPARK-19264 at 1/18/17 1:18 AM: -- [~sro

[jira] [Comment Edited] (SPARK-19264) Work should start driver, the same to AM of yarn

2017-01-17 Thread hustfxj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827181#comment-15827181 ] hustfxj edited comment on SPARK-19264 at 1/18/17 1:16 AM: -- [~sro

[jira] [Comment Edited] (SPARK-19264) Work should start driver, the same to AM of yarn

2017-01-17 Thread hustfxj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827181#comment-15827181 ] hustfxj edited comment on SPARK-19264 at 1/18/17 1:16 AM: -- [~sro

[jira] [Comment Edited] (SPARK-19264) Work should start driver, the same to AM of yarn

2017-01-17 Thread hustfxj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827181#comment-15827181 ] hustfxj edited comment on SPARK-19264 at 1/18/17 1:12 AM: -- [~sro

[jira] [Commented] (SPARK-19264) Work should start driver, the same to AM of yarn

2017-01-17 Thread hustfxj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827181#comment-15827181 ] hustfxj commented on SPARK-19264: - [~srowen] sorry, I didn't say it clearly. I means an s

[jira] [Updated] (SPARK-19264) Work should start driver, the same to AM of yarn

2017-01-17 Thread hustfxj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19264: Description: I think work can't start driver by "ProcessBuilderLike", thus we can't know the applicatio

[jira] [Commented] (SPARK-19227) Typo in `org.apache.spark.internal.config.ConfigEntry`

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827174#comment-15827174 ] Apache Spark commented on SPARK-19227: -- User 'uncleGen' has created a pull request f

[jira] [Assigned] (SPARK-19267) Fix a race condition when stopping StateStore

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19267: Assignee: Shixiong Zhu (was: Apache Spark) > Fix a race condition when stopping StateStor

[jira] [Commented] (SPARK-19267) Fix a race condition when stopping StateStore

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827135#comment-15827135 ] Apache Spark commented on SPARK-19267: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-19267) Fix a race condition when stopping StateStore

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19267: Assignee: Apache Spark (was: Shixiong Zhu) > Fix a race condition when stopping StateStor

[jira] [Updated] (SPARK-19267) Fix a race condition when stopping StateStore

2017-01-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19267: - Priority: Minor (was: Major) > Fix a race condition when stopping StateStore > -

[jira] [Updated] (SPARK-19267) Fix a race condition when stopping StateStore

2017-01-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19267: - Affects Version/s: 2.0.0 2.0.1 2.0.2

[jira] [Created] (SPARK-19267) Fix a race condition when stopping StateStore

2017-01-17 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-19267: Summary: Fix a race condition when stopping StateStore Key: SPARK-19267 URL: https://issues.apache.org/jira/browse/SPARK-19267 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-17747) WeightCol support non-double datatypes

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-17747: -- Shepherd: Joseph K. Bradley Assignee: zhengruifeng Target Versio

[jira] [Commented] (SPARK-19266) DiskStore does not encrypt serialized RDD data

2017-01-17 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827100#comment-15827100 ] Marcelo Vanzin commented on SPARK-19266: So I started down the path of looking at

[jira] [Commented] (SPARK-12347) Write script to run all MLlib examples for testing

2017-01-17 Thread Yicheng Luo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827089#comment-15827089 ] Yicheng Luo commented on SPARK-12347: - I have a half-baked improvement on my branch (

[jira] [Updated] (SPARK-13721) Add support for LATERAL VIEW OUTER explode()

2017-01-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-13721: -- Assignee: Bogdan Raducanu > Add support for LATERAL VIEW OUTER explode() >

[jira] [Commented] (SPARK-15799) Release SparkR on CRAN

2017-01-17 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827066#comment-15827066 ] Shivaram Venkataraman commented on SPARK-15799: --- The work on this is mostly

[jira] [Resolved] (SPARK-14567) Add instrumentation logs to MLlib training algorithms

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14567. --- Resolution: Fixed Fix Version/s: 2.2.0 > Add instrumentation logs to MLlib tra

[jira] [Resolved] (SPARK-13721) Add support for LATERAL VIEW OUTER explode()

2017-01-17 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-13721. --- Resolution: Fixed Fix Version/s: 2.2.0 > Add support for LATERAL VIEW OUTER ex

[jira] [Resolved] (SPARK-18206) Log instrumentation in MPC, NB, LDA, AFT, GLR, Isotonic, LinReg

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-18206. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 15671 [h

[jira] [Updated] (SPARK-7146) Should ML sharedParams be a public API?

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7146: - Target Version/s: (was: 2.2.0) > Should ML sharedParams be a public API? > -

[jira] [Updated] (SPARK-10931) PySpark ML Models should contain Param values

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-10931: -- Shepherd: (was: Joseph K. Bradley) > PySpark ML Models should contain Param values >

[jira] [Updated] (SPARK-7424) spark.ml classification, regression abstractions should add metadata to output column

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7424: - Target Version/s: (was: 2.2.0) > spark.ml classification, regression abstractions should

[jira] [Commented] (SPARK-14501) spark.ml parity for fpm - frequent items

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827052#comment-15827052 ] Joseph K. Bradley commented on SPARK-14501: --- I'm doing a general pass to encour

[jira] [Updated] (SPARK-10931) PySpark ML Models should contain Param values

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-10931: -- Target Version/s: (was: 2.2.0) > PySpark ML Models should contain Param values >

[jira] [Updated] (SPARK-15571) Pipeline unit test improvements for 2.2

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15571: -- Shepherd: Joseph K. Bradley > Pipeline unit test improvements for 2.2 > ---

[jira] [Commented] (SPARK-14659) OneHotEncoder support drop first category alphabetically in the encoded vector

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827041#comment-15827041 ] Joseph K. Bradley commented on SPARK-14659: --- I'm doing a general pass to encour

[jira] [Updated] (SPARK-14706) Python ML persistence integration test

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14706: -- Target Version/s: (was: 2.2.0) > Python ML persistence integration test > ---

[jira] [Commented] (SPARK-15799) Release SparkR on CRAN

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827038#comment-15827038 ] Joseph K. Bradley commented on SPARK-15799: --- I'm doing a general pass to encour

[jira] [Commented] (SPARK-16578) Configurable hostname for RBackend

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827036#comment-15827036 ] Joseph K. Bradley commented on SPARK-16578: --- I'm doing a general pass to encour

[jira] [Comment Edited] (SPARK-18011) SparkR serialize "NA" throws exception

2017-01-17 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827032#comment-15827032 ] Miao Wang edited comment on SPARK-18011 at 1/17/17 11:27 PM: -

[jira] [Commented] (SPARK-18011) SparkR serialize "NA" throws exception

2017-01-17 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827032#comment-15827032 ] Miao Wang commented on SPARK-18011: --- [~felixcheung] I did intensive debug in both R sid

[jira] [Created] (SPARK-19266) DiskStore does not encrypt serialized RDD data

2017-01-17 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-19266: -- Summary: DiskStore does not encrypt serialized RDD data Key: SPARK-19266 URL: https://issues.apache.org/jira/browse/SPARK-19266 Project: Spark Issue Type

[jira] [Updated] (SPARK-17455) IsotonicRegression takes non-polynomial time for some inputs

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-17455: -- Shepherd: Joseph K. Bradley > IsotonicRegression takes non-polynomial time for some inp

[jira] [Commented] (SPARK-18348) Improve tree ensemble model summary

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827023#comment-15827023 ] Joseph K. Bradley commented on SPARK-18348: --- I'm doing a general pass to enforc

[jira] [Commented] (SPARK-18569) Support R formula arithmetic

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827021#comment-15827021 ] Joseph K. Bradley commented on SPARK-18569: --- +1 for putting together a design d

[jira] [Commented] (SPARK-18618) SparkR GLM model predict should support type as a argument

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827019#comment-15827019 ] Joseph K. Bradley commented on SPARK-18618: --- [~yanboliang] Will you be willing

[jira] [Resolved] (SPARK-18917) Dataframe - Time Out Issues / Taking long time in append mode on object stores

2017-01-17 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18917. - Resolution: Fixed Assignee: Reynold Xin Fix Version/s: 2.2.0 > Dataframe - Time O

[jira] [Updated] (SPARK-3162) Train DecisionTree locally when possible

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-3162: - Target Version/s: (was: 2.2.0) > Train DecisionTree locally when possible >

[jira] [Commented] (SPARK-12347) Write script to run all MLlib examples for testing

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826997#comment-15826997 ] Joseph K. Bradley commented on SPARK-12347: --- I really want this to get in but j

[jira] [Updated] (SPARK-12347) Write script to run all MLlib examples for testing

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-12347: -- Target Version/s: (was: 2.2.0) > Write script to run all MLlib examples for testing >

[jira] [Updated] (SPARK-18613) spark.ml LDA classes should not expose spark.mllib in APIs

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18613: -- Shepherd: Joseph K. Bradley > spark.ml LDA classes should not expose spark.mllib in API

[jira] [Commented] (SPARK-18924) Improve collect/createDataFrame performance in SparkR

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826996#comment-15826996 ] Joseph K. Bradley commented on SPARK-18924: --- Per the 2.2 roadmap process, I'm g

[jira] [Updated] (SPARK-18924) Improve collect/createDataFrame performance in SparkR

2017-01-17 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18924: -- Shepherd: Xiangrui Meng > Improve collect/createDataFrame performance in SparkR > -

[jira] [Commented] (SPARK-19261) Support `ALTER TABLE table_name ADD COLUMNS(..)` statement

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826982#comment-15826982 ] Apache Spark commented on SPARK-19261: -- User 'xwu0226' has created a pull request fo

[jira] [Assigned] (SPARK-17874) Additional SSL port on HistoryServer should be configurable

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17874: Assignee: Apache Spark > Additional SSL port on HistoryServer should be configurable > ---

[jira] [Assigned] (SPARK-17874) Additional SSL port on HistoryServer should be configurable

2017-01-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17874: Assignee: (was: Apache Spark) > Additional SSL port on HistoryServer should be configu

  1   2   3   >