[jira] [Commented] (SPARK-20310) Dependency convergence error for scala-xml

2017-04-12 Thread Samik R (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967146#comment-15967146 ] Samik R commented on SPARK-20310: - Hi Sean, Thanks for your comments. You are probably thinking that I

[jira] [Commented] (SPARK-20316) In SparkSQLCLIDriver, val and var should strictly follow the Scala syntax

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967145#comment-15967145 ] Apache Spark commented on SPARK-20316: -- User 'ouyangxiaochen' has created a pull request for this

[jira] [Assigned] (SPARK-20316) In SparkSQLCLIDriver, val and var should strictly follow the Scala syntax

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20316: Assignee: Apache Spark > In SparkSQLCLIDriver, val and var should strictly follow the

[jira] [Assigned] (SPARK-20316) In SparkSQLCLIDriver, val and var should strictly follow the Scala syntax

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20316: Assignee: (was: Apache Spark) > In SparkSQLCLIDriver, val and var should strictly

[jira] [Created] (SPARK-20316) In SparkSQLCLIDriver, val and var should strictly follow the Scala syntax

2017-04-12 Thread Xiaochen Ouyang (JIRA)
Xiaochen Ouyang created SPARK-20316: --- Summary: In SparkSQLCLIDriver, val and var should strictly follow the Scala syntax Key: SPARK-20316 URL: https://issues.apache.org/jira/browse/SPARK-20316

[jira] [Commented] (SPARK-19924) Handle InvocationTargetException for all Hive Shim

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967142#comment-15967142 ] Apache Spark commented on SPARK-19924: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Commented] (SPARK-20287) Kafka Consumer should be able to subscribe to more than one topic partition

2017-04-12 Thread Stephane Maarek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966973#comment-15966973 ] Stephane Maarek commented on SPARK-20287: - [~c...@koeninger.org] How about using the subscribe

[jira] [Resolved] (SPARK-20131) Flaky Test: o.a.s.streaming.StreamingContextSuite.SPARK-18560 Receiver data should be deserialized properly

2017-04-12 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-20131. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-12 Thread Daniel Nuriyev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966939#comment-15966939 ] Daniel Nuriyev commented on SPARK-20036: Thank you, Cody. I will do as you say and report what

[jira] [Assigned] (SPARK-20315) Set ScalaUDF's deterministic to true

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20315: Assignee: Apache Spark (was: Xiao Li) > Set ScalaUDF's deterministic to true >

[jira] [Assigned] (SPARK-20315) Set ScalaUDF's deterministic to true

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20315: Assignee: Xiao Li (was: Apache Spark) > Set ScalaUDF's deterministic to true >

[jira] [Commented] (SPARK-20315) Set ScalaUDF's deterministic to true

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966938#comment-15966938 ] Apache Spark commented on SPARK-20315: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Created] (SPARK-20315) Set ScalaUDF's deterministic to true

2017-04-12 Thread Xiao Li (JIRA)
Xiao Li created SPARK-20315: --- Summary: Set ScalaUDF's deterministic to true Key: SPARK-20315 URL: https://issues.apache.org/jira/browse/SPARK-20315 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-20199) GradientBoostedTreesModel doesn't have Column Sampling Rate Paramenter

2017-04-12 Thread 颜发才
[ https://issues.apache.org/jira/browse/SPARK-20199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966926#comment-15966926 ] Yan Facai (颜发才) commented on SPARK-20199: - It's not hard, and I can work on it. However, there

[jira] [Created] (SPARK-20314) Inconsistent error handling in JSON parsing SQL functions

2017-04-12 Thread Eric Wasserman (JIRA)
Eric Wasserman created SPARK-20314: -- Summary: Inconsistent error handling in JSON parsing SQL functions Key: SPARK-20314 URL: https://issues.apache.org/jira/browse/SPARK-20314 Project: Spark

[jira] [Comment Edited] (SPARK-1809) Mesos backend doesn't respect HADOOP_CONF_DIR

2017-04-12 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966782#comment-15966782 ] Andrew Ash edited comment on SPARK-1809 at 4/12/17 11:00 PM: - I'm not using

[jira] [Closed] (SPARK-1809) Mesos backend doesn't respect HADOOP_CONF_DIR

2017-04-12 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash closed SPARK-1809. - Resolution: Unresolved Not using Mesos anymore, so closing > Mesos backend doesn't respect

[jira] [Commented] (SPARK-9103) Tracking spark's memory usage

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966736#comment-15966736 ] Apache Spark commented on SPARK-9103: - User 'jsoltren' has created a pull request for this issue:

[jira] [Commented] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

2017-04-12 Thread Jacques Nadeau (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966385#comment-15966385 ] Jacques Nadeau commented on SPARK-13534: Great, thanks [~holdenk]! > Implement Apache Arrow

[jira] [Commented] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

2017-04-12 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966339#comment-15966339 ] holdenk commented on SPARK-13534: - So I'm following along with the progress on this, I'll try and take a

[jira] [Resolved] (SPARK-20301) Flakiness in StreamingAggregationSuite

2017-04-12 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-20301. --- Resolution: Fixed Fix Version/s: 2.2.0 > Flakiness in StreamingAggregationSuite >

[jira] [Commented] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

2017-04-12 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966300#comment-15966300 ] Tejas Patil commented on SPARK-20184: - Out of curiosity, I tried out a query with ~20 columns

[jira] [Resolved] (SPARK-19570) Allow to disable hive in pyspark shell

2017-04-12 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-19570. - Resolution: Fixed Fix Version/s: 2.2.0 > Allow to disable hive in pyspark shell >

[jira] [Assigned] (SPARK-19570) Allow to disable hive in pyspark shell

2017-04-12 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-19570: --- Assignee: Jeff Zhang > Allow to disable hive in pyspark shell >

[jira] [Commented] (SPARK-13534) Implement Apache Arrow serializer for Spark DataFrame for use in DataFrame.toPandas

2017-04-12 Thread Jacques Nadeau (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966231#comment-15966231 ] Jacques Nadeau commented on SPARK-13534: Anybody know some committers we can get to look at this?

[jira] [Commented] (SPARK-19976) DirectStream API throws OffsetOutOfRange Exception

2017-04-12 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966208#comment-15966208 ] Cody Koeninger commented on SPARK-19976: What would your expected behavior be when you delete

[jira] [Commented] (SPARK-15354) Topology aware block replication strategies

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966203#comment-15966203 ] Apache Spark commented on SPARK-15354: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

2017-04-12 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966201#comment-15966201 ] holdenk commented on SPARK-20202: - Oh right, sorry I was misreading the intent of Affects Version/s. >

[jira] [Commented] (SPARK-20037) impossible to set kafka offsets using kafka 0.10 and spark 2.0.0

2017-04-12 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966199#comment-15966199 ] Cody Koeninger commented on SPARK-20037: I'd be inclined to say this is a duplicate of the issue

[jira] [Commented] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-12 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966193#comment-15966193 ] Cody Koeninger commented on SPARK-20036: fixKafkaParams is related to executor consumers, not the

[jira] [Commented] (SPARK-20287) Kafka Consumer should be able to subscribe to more than one topic partition

2017-04-12 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966180#comment-15966180 ] Cody Koeninger commented on SPARK-20287: The issue here is that the underlying new Kafka consumer

[jira] [Updated] (SPARK-20291) NaNvl(FloatType, NullType) should not be cast to NaNvl(DoubleType, DoubleType)

2017-04-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-20291: Fix Version/s: 2.0.3 > NaNvl(FloatType, NullType) should not be cast to NaNvl(DoubleType, > DoubleType)

[jira] [Updated] (SPARK-20313) Possible lack of join optimization when partitions are in the join condition

2017-04-12 Thread Albert Meltzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Albert Meltzer updated SPARK-20313: --- Description: Given two tables T1 and T2, partitioned on column part1, the following have

[jira] [Created] (SPARK-20313) Possible lack of join optimization when partitions are in the join condition

2017-04-12 Thread Albert Meltzer (JIRA)
Albert Meltzer created SPARK-20313: -- Summary: Possible lack of join optimization when partitions are in the join condition Key: SPARK-20313 URL: https://issues.apache.org/jira/browse/SPARK-20313

[jira] [Updated] (SPARK-20313) Possible lack of join optimization when partitions are in the join condition

2017-04-12 Thread Albert Meltzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Albert Meltzer updated SPARK-20313: --- Description: Given two tables T1 and T2, partitioned on column part1, the following have

[jira] [Resolved] (SPARK-20304) AssertNotNull should not include path in string representation

2017-04-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20304. - Resolution: Fixed Fix Version/s: 2.2.0 2.1.2 > AssertNotNull should not

[jira] [Resolved] (SPARK-20303) Rename createTempFunction to registerFunction

2017-04-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20303. - Resolution: Fixed Fix Version/s: 2.2.0 > Rename createTempFunction to registerFunction >

[jira] [Updated] (SPARK-20312) query optimizer calls udf with null values when it doesn't expect them

2017-04-12 Thread Albert Meltzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Albert Meltzer updated SPARK-20312: --- Description: When optimizing an outer join, spark passes an empty row to both sides to see

[jira] [Commented] (SPARK-20312) query optimizer calls udf with null values when it doesn't expect them

2017-04-12 Thread Albert Meltzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966094#comment-15966094 ] Albert Meltzer commented on SPARK-20312: During query optimization, one of the subtrees becomes:

[jira] [Commented] (SPARK-20312) query optimizer calls udf with null values when it doesn't expect them

2017-04-12 Thread Albert Meltzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966091#comment-15966091 ] Albert Meltzer commented on SPARK-20312: Query plans are as follows: {noformat} == Parsed

[jira] [Created] (SPARK-20312) query optimizer calls udf with null values when it doesn't expect them

2017-04-12 Thread Albert Meltzer (JIRA)
Albert Meltzer created SPARK-20312: -- Summary: query optimizer calls udf with null values when it doesn't expect them Key: SPARK-20312 URL: https://issues.apache.org/jira/browse/SPARK-20312 Project:

[jira] [Commented] (SPARK-20292) string representation of TreeNode is messy

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966083#comment-15966083 ] Apache Spark commented on SPARK-20292: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20292) string representation of TreeNode is messy

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20292: Assignee: Apache Spark > string representation of TreeNode is messy >

[jira] [Assigned] (SPARK-20292) string representation of TreeNode is messy

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20292: Assignee: (was: Apache Spark) > string representation of TreeNode is messy >

[jira] [Commented] (SPARK-20300) Python API for ALSModel.recommendForAllUsers,Items

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966079#comment-15966079 ] Apache Spark commented on SPARK-20300: -- User 'MLnick' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20300) Python API for ALSModel.recommendForAllUsers,Items

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20300: Assignee: (was: Apache Spark) > Python API for ALSModel.recommendForAllUsers,Items >

[jira] [Assigned] (SPARK-20300) Python API for ALSModel.recommendForAllUsers,Items

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20300: Assignee: Apache Spark > Python API for ALSModel.recommendForAllUsers,Items >

[jira] [Created] (SPARK-20311) SQL "range(N) as alias" or "range(N) alias" doesn't work

2017-04-12 Thread Juliusz Sompolski (JIRA)
Juliusz Sompolski created SPARK-20311: - Summary: SQL "range(N) as alias" or "range(N) alias" doesn't work Key: SPARK-20311 URL: https://issues.apache.org/jira/browse/SPARK-20311 Project: Spark

[jira] [Commented] (SPARK-19547) KafkaUtil throw 'No current assignment for partition' Exception

2017-04-12 Thread Omaiyma Popat (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965922#comment-15965922 ] Omaiyma Popat commented on SPARK-19547: --- Hi Team, Can you please advise a resolution on the above

[jira] [Commented] (SPARK-19451) Long values in Window function

2017-04-12 Thread Julien Champ (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965915#comment-15965915 ] Julien Champ commented on SPARK-19451: -- Any news on this bug / feature request ? > Long values in

[jira] [Commented] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

2017-04-12 Thread Paul Zaczkieiwcz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965822#comment-15965822 ] Paul Zaczkieiwcz commented on SPARK-18055: -- Actually it seems it was this issue. I forgot to

[jira] [Updated] (SPARK-20310) Dependency convergence error for scala-xml

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20310: -- Issue Type: Improvement (was: Bug) You may have a slightly earlier convergence problem in your build

[jira] [Created] (SPARK-20310) Dependency convergence error for scala-xml

2017-04-12 Thread Samik R (JIRA)
Samik R created SPARK-20310: --- Summary: Dependency convergence error for scala-xml Key: SPARK-20310 URL: https://issues.apache.org/jira/browse/SPARK-20310 Project: Spark Issue Type: Bug

[jira] [Reopened] (SPARK-20306) beeline connect spark thrift server failure

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-20306: --- > beeline connect spark thrift server failure > > >

[jira] [Resolved] (SPARK-20306) beeline connect spark thrift server failure

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20306. --- Resolution: Not A Problem > beeline connect spark thrift server failure >

[jira] [Resolved] (SPARK-20309) Repartitioning - more than the default number of partitions

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20309. --- Resolution: Not A Problem > Repartitioning - more than the default number of partitions >

[jira] [Commented] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

2017-04-12 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965735#comment-15965735 ] Fei Wang commented on SPARK-20184: -- Also use the master branch to test my test case: 1. Java version

[jira] [Closed] (SPARK-20306) beeline connect spark thrift server failure

2017-04-12 Thread sydt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sydt closed SPARK-20306. Resolution: Fixed > beeline connect spark thrift server failure > >

[jira] [Commented] (SPARK-20309) Repartitioning - more than the default number of partitions

2017-04-12 Thread balaji krishnan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965715#comment-15965715 ] balaji krishnan commented on SPARK-20309: - thanks Sean. I will send that as an email to

[jira] [Commented] (SPARK-20309) Repartitioning - more than the default number of partitions

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965714#comment-15965714 ] Sean Owen commented on SPARK-20309: --- It's not clear what you're asking, but this should go to

[jira] [Created] (SPARK-20309) Repartitioning - more than the default number of partitions

2017-04-12 Thread balaji krishnan (JIRA)
balaji krishnan created SPARK-20309: --- Summary: Repartitioning - more than the default number of partitions Key: SPARK-20309 URL: https://issues.apache.org/jira/browse/SPARK-20309 Project: Spark

[jira] [Resolved] (SPARK-20308) org.apache.spark.shuffle.FetchFailedException: Too large frame

2017-04-12 Thread Artur Sukhenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artur Sukhenko resolved SPARK-20308. Resolution: Duplicate > org.apache.spark.shuffle.FetchFailedException: Too large frame >

[jira] [Assigned] (SPARK-18692) Test Java 8 unidoc build on Jenkins master builder

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-18692: - Assignee: Hyukjin Kwon > Test Java 8 unidoc build on Jenkins master builder >

[jira] [Resolved] (SPARK-18692) Test Java 8 unidoc build on Jenkins master builder

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18692. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17477

[jira] [Commented] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

2017-04-12 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965670#comment-15965670 ] Herman van Hovell commented on SPARK-20184: --- I just tried your example using the master branch,

[jira] [Commented] (SPARK-18791) Stream-Stream Joins

2017-04-12 Thread xianyao jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965657#comment-15965657 ] xianyao jiang commented on SPARK-18791: --- when this feature will be provided? is there any idea

[jira] [Commented] (SPARK-20306) beeline connect spark thrift server failure

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965654#comment-15965654 ] Sean Owen commented on SPARK-20306: --- This is some env or config issue, not Spark. > beeline connect

[jira] [Assigned] (SPARK-20296) UnsupportedOperationChecker text on distinct aggregations differs from docs

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-20296: - Assignee: Jason Tokayer > UnsupportedOperationChecker text on distinct aggregations differs

[jira] [Resolved] (SPARK-20296) UnsupportedOperationChecker text on distinct aggregations differs from docs

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20296. --- Resolution: Fixed Fix Version/s: 2.1.2 2.2.0 Issue resolved by pull

[jira] [Created] (SPARK-20308) org.apache.spark.shuffle.FetchFailedException: Too large frame

2017-04-12 Thread Stanislav Chernichkin (JIRA)
Stanislav Chernichkin created SPARK-20308: - Summary: org.apache.spark.shuffle.FetchFailedException: Too large frame Key: SPARK-20308 URL: https://issues.apache.org/jira/browse/SPARK-20308

[jira] [Commented] (SPARK-20199) GradientBoostedTreesModel doesn't have Column Sampling Rate Paramenter

2017-04-12 Thread pralabhkumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965647#comment-15965647 ] pralabhkumar commented on SPARK-20199: -- For GBM its using Random Forest ,and to add randomness to

[jira] [Updated] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-04-12 Thread Anne Rutten (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anne Rutten updated SPARK-20307: Description: when training a model in SparkR with string variables (tested with

[jira] [Updated] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-04-12 Thread Anne Rutten (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anne Rutten updated SPARK-20307: Description: when training a model in SparkR with string variables (tested with

[jira] [Updated] (SPARK-20199) GradientBoostedTreesModel doesn't have Column Sampling Rate Paramenter

2017-04-12 Thread pralabhkumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pralabhkumar updated SPARK-20199: - Priority: Major (was: Minor) > GradientBoostedTreesModel doesn't have Column Sampling Rate

[jira] [Updated] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-04-12 Thread Anne Rutten (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anne Rutten updated SPARK-20307: Priority: Minor (was: Major) > SparkR: pass on setHandleInvalid to spark.mllib functions that use

[jira] [Updated] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-04-12 Thread Anne Rutten (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anne Rutten updated SPARK-20307: Component/s: (was: MLlib) > SparkR: pass on setHandleInvalid to spark.mllib functions that use

[jira] [Updated] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-04-12 Thread Anne Rutten (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anne Rutten updated SPARK-20307: Component/s: SparkR > SparkR: pass on setHandleInvalid to spark.mllib functions that use >

[jira] [Created] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-04-12 Thread Anne Rutten (JIRA)
Anne Rutten created SPARK-20307: --- Summary: SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer Key: SPARK-20307 URL: https://issues.apache.org/jira/browse/SPARK-20307

[jira] [Commented] (SPARK-6227) PCA and SVD for PySpark

2017-04-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965632#comment-15965632 ] Apache Spark commented on SPARK-6227: - User 'MLnick' has created a pull request for this issue:

[jira] [Commented] (SPARK-20294) _inferSchema for RDDs fails if sample returns empty RDD

2017-04-12 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965601#comment-15965601 ] Hyukjin Kwon commented on SPARK-20294: -- {quote} If we now that the rdd is not empty (the function

[jira] [Comment Edited] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

2017-04-12 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965597#comment-15965597 ] Fei Wang edited comment on SPARK-20184 at 4/12/17 9:21 AM: --- try this : 1.

[jira] [Comment Edited] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

2017-04-12 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965597#comment-15965597 ] Fei Wang edited comment on SPARK-20184 at 4/12/17 9:21 AM: --- try this : 1.

[jira] [Commented] (SPARK-20184) performance regression for complex/long sql when enable whole stage codegen

2017-04-12 Thread Fei Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965597#comment-15965597 ] Fei Wang commented on SPARK-20184: -- try this : 1. create table [code] val df = (1 to 50).map(x =>

[jira] [Commented] (SPARK-20294) _inferSchema for RDDs fails if sample returns empty RDD

2017-04-12 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-20294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965590#comment-15965590 ] João Pedro Jericó commented on SPARK-20294: --- Yes, I think that this would be a nice solution,

[jira] [Updated] (SPARK-20306) beeline connect spark thrift server failure

2017-04-12 Thread sydt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sydt updated SPARK-20306: - Description: Beeline connect spark thrift server of spark-1.6.2 with kerberos failure and error is : Error

[jira] [Updated] (SPARK-20306) beeline connect spark thrift server failure

2017-04-12 Thread sydt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sydt updated SPARK-20306: - Description: Beeline connect spark thrift server of spark-1.6.2 with kerberos failure and error is : ERROR

[jira] [Commented] (SPARK-20294) _inferSchema for RDDs fails if sample returns empty RDD

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965579#comment-15965579 ] Sean Owen commented on SPARK-20294: --- CC [~andrewor14] who might be able to comment better on this. If

[jira] [Created] (SPARK-20306) beeline connect spark thrift server failure

2017-04-12 Thread sydt (JIRA)
sydt created SPARK-20306: Summary: beeline connect spark thrift server failure Key: SPARK-20306 URL: https://issues.apache.org/jira/browse/SPARK-20306 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-20294) _inferSchema for RDDs fails if sample returns empty RDD

2017-04-12 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-20294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965568#comment-15965568 ] João Pedro Jericó edited comment on SPARK-20294 at 4/12/17 8:56 AM:

[jira] [Updated] (SPARK-20294) _inferSchema for RDDs fails if sample returns empty RDD

2017-04-12 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-20294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] João Pedro Jericó updated SPARK-20294: -- Description: Currently the _inferSchema function on

[jira] [Comment Edited] (SPARK-20294) _inferSchema for RDDs fails if sample returns empty RDD

2017-04-12 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-20294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965568#comment-15965568 ] João Pedro Jericó edited comment on SPARK-20294 at 4/12/17 8:57 AM:

[jira] [Commented] (SPARK-20294) _inferSchema for RDDs fails if sample returns empty RDD

2017-04-12 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-20294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965568#comment-15965568 ] João Pedro Jericó commented on SPARK-20294: --- Yes, if sampling ration is not given it infers for

[jira] [Updated] (SPARK-20237) Spark-1.6 current and later versions of memory management issues

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20237: -- Description: In spark-1.6 and later versions, there is a problem with its memory management

[jira] [Commented] (SPARK-20286) dynamicAllocation.executorIdleTimeout is ignored after unpersist

2017-04-12 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-20286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965556#comment-15965556 ] Miguel Pérez commented on SPARK-20286: -- I'm zero familiar with the code, but I'll try to send a

[jira] [Resolved] (SPARK-20302) Short circuit cast when from and to types are structurally the same

2017-04-12 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20302. - Resolution: Fixed Fix Version/s: 2.2.0 > Short circuit cast when from and to types are

[jira] [Updated] (SPARK-20305) Master may keep in the state of "COMPELETING_RECOVERY",then all the application registered cannot get resources, when the leader master change.

2017-04-12 Thread LvDongrong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LvDongrong updated SPARK-20305: --- Attachment: failfetchresources.PNG > Master may keep in the state of "COMPELETING_RECOVERY",then all

[jira] [Updated] (SPARK-20305) Master may keep in the state of "COMPELETING_RECOVERY",then all the application registered cannot get resources, when the leader master change.

2017-04-12 Thread LvDongrong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LvDongrong updated SPARK-20305: --- Attachment: (was: failfetchresources.PNG) > Master may keep in the state of

[jira] [Resolved] (SPARK-20298) Spelling mistake: charactor

2017-04-12 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20298. --- Resolution: Fixed Assignee: Brendan Dwyer Fix Version/s: 2.2.0 Resolved by

[jira] [Updated] (SPARK-20305) Master may keep in the state of "COMPELETING_RECOVERY",then all the application registered cannot get resources, when the leader master change.

2017-04-12 Thread LvDongrong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LvDongrong updated SPARK-20305: --- Attachment: failfetchresources.PNG > Master may keep in the state of "COMPELETING_RECOVERY",then all

[jira] [Updated] (SPARK-20305) Master may keep in the state of "COMPELETING_RECOVERY",then all the application registered cannot get resources, when the leader master change.

2017-04-12 Thread LvDongrong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LvDongrong updated SPARK-20305: --- Description: Master may keep in the state of "COMPELETING_RECOVERY",then all the application

  1   2   >