[jira] [Created] (SPARK-19837) Fetch failure throws a SparkException in SparkHiveWriter

2017-03-06 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-19837: --- Summary: Fetch failure throws a SparkException in SparkHiveWriter Key: SPARK-19837 URL: https://issues.apache.org/jira/browse/SPARK-19837 Project: Spark Issue

[jira] [Comment Edited] (SPARK-18832) Spark SQL: Thriftserver unable to run a registered Hive UDTF

2017-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897799#comment-15897799 ] Dongjoon Hyun edited comment on SPARK-18832 at 3/6/17 6:39 PM: --- Hi,

[jira] [Commented] (SPARK-18832) Spark SQL: Thriftserver unable to run a registered Hive UDTF

2017-03-06 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897799#comment-15897799 ] Dongjoon Hyun commented on SPARK-18832: --- Hi, [~roadster11x]. Thank you for the sample file. I

[jira] [Resolved] (SPARK-19737) New analysis rule for reporting unregistered functions without relying on relation resolution

2017-03-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-19737. Resolution: Fixed Issue resolved by pull request 17168

[jira] [Assigned] (SPARK-19737) New analysis rule for reporting unregistered functions without relying on relation resolution

2017-03-06 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-19737: -- Assignee: Cheng Lian > New analysis rule for reporting unregistered functions without relying

[jira] [Commented] (SPARK-18789) Save Data frame with Null column-- exception

2017-03-06 Thread Eugen Prokhorenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897791#comment-15897791 ] Eugen Prokhorenko commented on SPARK-18789: --- I've added a python script (instead of a Scala app

[jira] [Commented] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

2017-03-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897789#comment-15897789 ] Shixiong Zhu commented on SPARK-19831: -- Cores running in the receive method should be quick. If

[jira] [Resolved] (SPARK-17495) Hive hash implementation

2017-03-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17495. - Resolution: Fixed > Hive hash implementation > > > Key:

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

2017-03-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897730#comment-15897730 ] Marcelo Vanzin commented on SPARK-18085: bq. "A separate jar file" means we generate a new jar

[jira] [Resolved] (SPARK-15522) DataFrame Column Names That are Numbers aren't referenced correctly in SQL

2017-03-06 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15522. -- Resolution: Invalid > DataFrame Column Names That are Numbers aren't referenced correctly in

[jira] [Commented] (SPARK-15522) DataFrame Column Names That are Numbers aren't referenced correctly in SQL

2017-03-06 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897638#comment-15897638 ] Hyukjin Kwon commented on SPARK-15522: -- We can use backticks for it as below: {code} scala>

[jira] [Resolved] (SPARK-19064) Fix pip install issue with ml sub components

2017-03-06 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-19064. - Resolution: Fixed Assignee: holdenk Fix Version/s: 2.2.0 2.1.1 > Fix

[jira] [Commented] (SPARK-10764) Add optional caching to Pipelines

2017-03-06 Thread Matthieu Baechler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897465#comment-15897465 ] Matthieu Baechler commented on SPARK-10764: --- Hi, I'm curious to know how to achieve caching as

[jira] [Commented] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread bruce xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897269#comment-15897269 ] bruce xu commented on SPARK-19835: -- Thx for your response,I correct the wrong sql , hope to check

[jira] [Updated] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread bruce xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bruce xu updated SPARK-19835: - Description: when I run CAST sql such as: - create table A as select a.id, b.id,

[jira] [Updated] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread bruce xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bruce xu updated SPARK-19835: - Description: when I run CAST sql such as: - create table A as select a.id, b.id,

[jira] [Commented] (SPARK-19659) Fetch big blocks to disk when shuffle-read

2017-03-06 Thread jin xing (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897244#comment-15897244 ] jin xing commented on SPARK-19659: -- [~irashid] Thanks a lot for your comments, really helpful. I got

[jira] [Commented] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897240#comment-15897240 ] Herman van Hovell commented on SPARK-19835: --- You are doing a cross join between B and C; that

[jira] [Closed] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-19835. - Resolution: Not A Problem > Runing CTAS sql on spark2.0.1 hung >

[jira] [Updated] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-19835: -- Description: when I run CAST sql such as: {noformat} create table A as select a.id,

[jira] [Updated] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread bruce xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bruce xu updated SPARK-19835: - Description: when I run CAST sql such as: - create table A as select a.id, b.id,

[jira] [Commented] (SPARK-14409) Investigate adding a RankingEvaluator to ML

2017-03-06 Thread Danilo Ascione (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897198#comment-15897198 ] Danilo Ascione commented on SPARK-14409: Thank you [~mlnick] for taking time to thing about this.

[jira] [Commented] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897182#comment-15897182 ] Herman van Hovell commented on SPARK-19835: --- >From the looks of it your job is still running;

[jira] [Commented] (SPARK-19836) Customizable remote repository url for hive versions unit test

2017-03-06 Thread Elek, Marton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897172#comment-15897172 ] Elek, Marton commented on SPARK-19836: -- I am working on a patch for this one. Could you please

[jira] [Created] (SPARK-19836) Customizable remote repository url for hive versions unit test

2017-03-06 Thread Elek, Marton (JIRA)
Elek, Marton created SPARK-19836: Summary: Customizable remote repository url for hive versions unit test Key: SPARK-19836 URL: https://issues.apache.org/jira/browse/SPARK-19836 Project: Spark

[jira] [Assigned] (SPARK-19828) R to support JSON array in column from_json

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19828: Assignee: (was: Apache Spark) > R to support JSON array in column from_json >

[jira] [Assigned] (SPARK-19828) R to support JSON array in column from_json

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19828: Assignee: Apache Spark > R to support JSON array in column from_json >

[jira] [Commented] (SPARK-19828) R to support JSON array in column from_json

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897167#comment-15897167 ] Apache Spark commented on SPARK-19828: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread bruce xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bruce xu updated SPARK-19835: - Description: when I run CAST sql such as: - create table A as select a.id, b.id,

[jira] [Updated] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread bruce xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bruce xu updated SPARK-19835: - Description: when I run CAST sql such as: - create table A as select a.id, b.id,

[jira] [Updated] (SPARK-19835) Runing CTAS sql on spark2.0.1 hung

2017-03-06 Thread bruce xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bruce xu updated SPARK-19835: - Summary: Runing CTAS sql on spark2.0.1 hung (was: Runing CAST sql on spark hung ) > Runing CTAS sql

[jira] [Created] (SPARK-19835) Runing CAST sql on spark hung

2017-03-06 Thread bruce xu (JIRA)
bruce xu created SPARK-19835: Summary: Runing CAST sql on spark hung Key: SPARK-19835 URL: https://issues.apache.org/jira/browse/SPARK-19835 Project: Spark Issue Type: Bug Components:

[jira] [Assigned] (SPARK-19834) csv encoding/decoding error not using escape of escape

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19834: Assignee: Apache Spark > csv encoding/decoding error not using escape of escape >

[jira] [Assigned] (SPARK-19834) csv encoding/decoding error not using escape of escape

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19834: Assignee: (was: Apache Spark) > csv encoding/decoding error not using escape of

[jira] [Commented] (SPARK-19834) csv encoding/decoding error not using escape of escape

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897142#comment-15897142 ] Apache Spark commented on SPARK-19834: -- User 'ep1804' has created a pull request for this issue:

[jira] [Commented] (SPARK-19384) forget unpersist input dataset in IsotonicRegression

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897140#comment-15897140 ] Apache Spark commented on SPARK-19384: -- User 'ep1804' has created a pull request for this issue:

[jira] [Created] (SPARK-19834) csv encoding/decoding error not using escape of escape

2017-03-06 Thread Soonmok Kwon (JIRA)
Soonmok Kwon created SPARK-19834: Summary: csv encoding/decoding error not using escape of escape Key: SPARK-19834 URL: https://issues.apache.org/jira/browse/SPARK-19834 Project: Spark Issue

[jira] [Comment Edited] (SPARK-19827) spark.ml R API for PIC

2017-03-06 Thread Krishna Kalyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897106#comment-15897106 ] Krishna Kalyan edited comment on SPARK-19827 at 3/6/17 11:10 AM: - I would

[jira] [Commented] (SPARK-19827) spark.ml R API for PIC

2017-03-06 Thread Krishna Kalyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897106#comment-15897106 ] Krishna Kalyan commented on SPARK-19827: I would like to work on this. > spark.ml R API for PIC

[jira] [Commented] (SPARK-19364) Stream Blocks in Storage Persists Forever when Kinesis Checkpoints are enabled and an exception is thrown

2017-03-06 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15897012#comment-15897012 ] Takeshi Yamamuro commented on SPARK-19364: -- Do you have any way to reproduce this issue? I

[jira] [Assigned] (SPARK-19833) remove SQLConf.HIVE_VERIFY_PARTITION_PATH, we always return empty when the path does not exists

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19833: Assignee: (was: Apache Spark) > remove SQLConf.HIVE_VERIFY_PARTITION_PATH, we always

[jira] [Commented] (SPARK-19833) remove SQLConf.HIVE_VERIFY_PARTITION_PATH, we always return empty when the path does not exists

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896949#comment-15896949 ] Apache Spark commented on SPARK-19833: -- User 'windpiger' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19833) remove SQLConf.HIVE_VERIFY_PARTITION_PATH, we always return empty when the path does not exists

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19833: Assignee: Apache Spark > remove SQLConf.HIVE_VERIFY_PARTITION_PATH, we always return

[jira] [Created] (SPARK-19833) remove SQLConf.HIVE_VERIFY_PARTITION_PATH, we always return empty when the path does not exists

2017-03-06 Thread Song Jun (JIRA)
Song Jun created SPARK-19833: Summary: remove SQLConf.HIVE_VERIFY_PARTITION_PATH, we always return empty when the path does not exists Key: SPARK-19833 URL: https://issues.apache.org/jira/browse/SPARK-19833

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

2017-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19831: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) > Sending the heartbeat

[jira] [Assigned] (SPARK-19468) Dataset slow because of unnecessary shuffles

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19468: Assignee: Apache Spark > Dataset slow because of unnecessary shuffles >

[jira] [Commented] (SPARK-19468) Dataset slow because of unnecessary shuffles

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896941#comment-15896941 ] Apache Spark commented on SPARK-19468: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19468) Dataset slow because of unnecessary shuffles

2017-03-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19468: Assignee: (was: Apache Spark) > Dataset slow because of unnecessary shuffles >

[jira] [Comment Edited] (SPARK-14409) Investigate adding a RankingEvaluator to ML

2017-03-06 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896933#comment-15896933 ] Nick Pentreath edited comment on SPARK-14409 at 3/6/17 9:07 AM: I've

[jira] [Comment Edited] (SPARK-14409) Investigate adding a RankingEvaluator to ML

2017-03-06 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896933#comment-15896933 ] Nick Pentreath edited comment on SPARK-14409 at 3/6/17 9:06 AM: I've

[jira] [Commented] (SPARK-14409) Investigate adding a RankingEvaluator to ML

2017-03-06 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896933#comment-15896933 ] Nick Pentreath commented on SPARK-14409: I've thought about this a lot over the past few days,

[jira] [Commented] (SPARK-19503) Execution Plan Optimizer: avoid sort or shuffle when it does not change end result such as df.sort(...).count()

2017-03-06 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896913#comment-15896913 ] Herman van Hovell commented on SPARK-19503: --- I suppose it is fine as long as we do not change

[jira] [Updated] (SPARK-19829) The log about driver should support rolling like executor

2017-03-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19829: -- Priority: Minor (was: Major) Issue Type: Improvement (was: New Feature) Usually this is

[jira] [Commented] (SPARK-19067) mapGroupsWithState - arbitrary stateful operations with Structured Streaming (similar to DStream.mapWithState)

2017-03-06 Thread Amit Sela (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896866#comment-15896866 ] Amit Sela commented on SPARK-19067: --- It depends, will those timers be "resetable" ? So that once I

<    1   2