[jira] [Assigned] (SPARK-8660) Update comments that contain R statements in ml.logisticRegressionSuite

[ https://issues.apache.org/jira/browse/SPARK-8660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8660: --- Assignee: Apache Spark > Update comments that contain R statements in ml.logisticRegressionSu

[jira] [Resolved] (SPARK-8070) Improve createDataFrame in Python

[ https://issues.apache.org/jira/browse/SPARK-8070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8070. Resolution: Fixed Fix Version/s: 1.5.0 > Improve createDataFrame in Python >

[jira] [Commented] (SPARK-8706) Implement Pylint / Prospector checks for PySpark

[ https://issues.apache.org/jira/browse/SPARK-8706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606311#comment-14606311 ] Manoj Kumar commented on SPARK-8706: Mind if I hack on this? > Implement Pylint / Pro

[jira] [Commented] (SPARK-8621) crosstab exception when one of the value is empty

[ https://issues.apache.org/jira/browse/SPARK-8621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606307#comment-14606307 ] Reynold Xin commented on SPARK-8621: Talked to [~marmbrus] offline. Let's just support

[jira] [Resolved] (SPARK-8681) crosstab column names in wrong order

[ https://issues.apache.org/jira/browse/SPARK-8681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8681. Resolution: Fixed Assignee: Burak Yavuz Fix Version/s: 1.4.2 1.5.0

[jira] [Commented] (SPARK-8684) Update R version in Spark EC2 AMI

[ https://issues.apache.org/jira/browse/SPARK-8684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606293#comment-14606293 ] Shivaram Venkataraman commented on SPARK-8684: -- The create_image.sh script is

[jira] [Updated] (SPARK-8712) Window function does not work with distinct aggregations

[ https://issues.apache.org/jira/browse/SPARK-8712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-8712: Description: Seems we ignored distinct keyword. Also, for our master, I got {code} scala> Seq((1, 2, 3)).toD

[jira] [Created] (SPARK-8712) Window function does not work with distinct aggregations

Yin Huai created SPARK-8712: --- Summary: Window function does not work with distinct aggregations Key: SPARK-8712 URL: https://issues.apache.org/jira/browse/SPARK-8712 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-8633) List missing model methods in Python Pipeline API

[ https://issues.apache.org/jira/browse/SPARK-8633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606266#comment-14606266 ] Manoj Kumar commented on SPARK-8633: I think that should be it. > List missing model

[jira] [Resolved] (SPARK-7862) Query would hang when the using script has error output in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-7862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-7862. - Resolution: Fixed Issue resolved by pull request 6882 [https://github.com/apache/spark/pul

[jira] [Commented] (SPARK-8711) Add additional methods to JavaModel wrappers in trees

[ https://issues.apache.org/jira/browse/SPARK-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606256#comment-14606256 ] Apache Spark commented on SPARK-8711: - User 'MechCoder' has created a pull request for

[jira] [Assigned] (SPARK-8711) Add additional methods to JavaModel wrappers in trees

[ https://issues.apache.org/jira/browse/SPARK-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8711: --- Assignee: (was: Apache Spark) > Add additional methods to JavaModel wrappers in trees > -

[jira] [Assigned] (SPARK-8711) Add additional methods to JavaModel wrappers in trees

[ https://issues.apache.org/jira/browse/SPARK-8711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8711: --- Assignee: Apache Spark > Add additional methods to JavaModel wrappers in trees >

[jira] [Assigned] (SPARK-7977) Disallow println

[ https://issues.apache.org/jira/browse/SPARK-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7977: --- Assignee: (was: Apache Spark) > Disallow println > > > K

[jira] [Commented] (SPARK-8710) ScalaReflection.mirror should be a def

[ https://issues.apache.org/jira/browse/SPARK-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606257#comment-14606257 ] Apache Spark commented on SPARK-8710: - User 'yhuai' has created a pull request for thi

[jira] [Assigned] (SPARK-8710) ScalaReflection.mirror should be a def

[ https://issues.apache.org/jira/browse/SPARK-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8710: --- Assignee: Yin Huai (was: Apache Spark) > ScalaReflection.mirror should be a def > --

[jira] [Assigned] (SPARK-8710) ScalaReflection.mirror should be a def

[ https://issues.apache.org/jira/browse/SPARK-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8710: --- Assignee: Apache Spark (was: Yin Huai) > ScalaReflection.mirror should be a def > --

[jira] [Commented] (SPARK-7977) Disallow println

[ https://issues.apache.org/jira/browse/SPARK-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606251#comment-14606251 ] Apache Spark commented on SPARK-7977: - User 'jonalter' has created a pull request for

[jira] [Assigned] (SPARK-7977) Disallow println

[ https://issues.apache.org/jira/browse/SPARK-7977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7977: --- Assignee: Apache Spark > Disallow println > > > Key: SPARK-7

[jira] [Commented] (SPARK-8597) DataFrame partitionBy memory pressure scales extremely poorly

[ https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606248#comment-14606248 ] Michael Armbrust commented on SPARK-8597: - We could use Spark's external sort whic

[jira] [Created] (SPARK-8711) Add additional methods to JavaModel wrappers in trees

Manoj Kumar created SPARK-8711: -- Summary: Add additional methods to JavaModel wrappers in trees Key: SPARK-8711 URL: https://issues.apache.org/jira/browse/SPARK-8711 Project: Spark Issue Type: N

[jira] [Commented] (SPARK-8567) Flaky test: o.a.s.sql.hive.HiveSparkSubmitSuite --jars

[ https://issues.apache.org/jira/browse/SPARK-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606244#comment-14606244 ] Apache Spark commented on SPARK-8567: - User 'yhuai' has created a pull request for thi

[jira] [Assigned] (SPARK-8675) Executors created by LocalBackend won't get the same classpath as other executor backends

[ https://issues.apache.org/jira/browse/SPARK-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8675: --- Assignee: (was: Apache Spark) > Executors created by LocalBackend won't get the same clas

[jira] [Commented] (SPARK-8675) Executors created by LocalBackend won't get the same classpath as other executor backends

[ https://issues.apache.org/jira/browse/SPARK-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606225#comment-14606225 ] Apache Spark commented on SPARK-8675: - User 'coderplay' has created a pull request for

[jira] [Assigned] (SPARK-8675) Executors created by LocalBackend won't get the same classpath as other executor backends

[ https://issues.apache.org/jira/browse/SPARK-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8675: --- Assignee: Apache Spark > Executors created by LocalBackend won't get the same classpath as ot

[jira] [Updated] (SPARK-8710) ScalaReflection.mirror should be a def

[ https://issues.apache.org/jira/browse/SPARK-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-8710: Priority: Major (was: Critical) > ScalaReflection.mirror should be a def >

[jira] [Updated] (SPARK-8130) spark.files.useFetchCache should be off by default

[ https://issues.apache.org/jira/browse/SPARK-8130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8130: - Affects Version/s: 1.2.0 > spark.files.useFetchCache should be off by default > --

[jira] [Resolved] (SPARK-8214) math function: hex

[ https://issues.apache.org/jira/browse/SPARK-8214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8214. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 6976 [https://github.com/

[jira] [Commented] (SPARK-8597) DataFrame partitionBy memory pressure scales extremely poorly

[ https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606199#comment-14606199 ] Vlad Ionescu commented on SPARK-8597: - Hi, Regarding the third suggestion, the functi

[jira] [Commented] (SPARK-8675) Executors created by LocalBackend won't get the same classpath as other executor backends

[ https://issues.apache.org/jira/browse/SPARK-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606197#comment-14606197 ] Min Zhou commented on SPARK-8675: - [~lian cheng] I am working on it > Executors created b

[jira] [Updated] (SPARK-8710) ScalaReflection.mirror should be a def

[ https://issues.apache.org/jira/browse/SPARK-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-8710: Shepherd: Michael Armbrust > ScalaReflection.mirror should be a def > --

[jira] [Created] (SPARK-8710) ScalaReflection.mirror should be a def

Yin Huai created SPARK-8710: --- Summary: ScalaReflection.mirror should be a def Key: SPARK-8710 URL: https://issues.apache.org/jira/browse/SPARK-8710 Project: Spark Issue Type: Bug Componen

[jira] [Commented] (SPARK-8621) crosstab exception when one of the value is empty

[ https://issues.apache.org/jira/browse/SPARK-8621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606186#comment-14606186 ] Michael Armbrust commented on SPARK-8621: - Will you ever want to access the column

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606178#comment-14606178 ] Nicholas Chammas commented on SPARK-8670: - FYI: `df.stats.age` works neither on 1.

[jira] [Commented] (SPARK-6830) Memoize frequently queried vals in RDD, such as numPartitions, count etc.

[ https://issues.apache.org/jira/browse/SPARK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606176#comment-14606176 ] Perinkulam I Ganesh commented on SPARK-6830: This thought crossed our mind as

[jira] [Updated] (SPARK-8528) Add applicationId to SparkContext object in pyspark

[ https://issues.apache.org/jira/browse/SPARK-8528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-8528: -- Assignee: Vladimir Vladimirov > Add applicationId to SparkContext object in pyspark > --

[jira] [Resolved] (SPARK-8528) Add applicationId to SparkContext object in pyspark

[ https://issues.apache.org/jira/browse/SPARK-8528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-8528. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 6936 [https://github.com/

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606166#comment-14606166 ] Ted Malaska commented on SPARK-2447: Hey Andrew, https://issues.apache.org/jira/brows

[jira] [Commented] (SPARK-6129) Add a section in user guide for model evaluation

[ https://issues.apache.org/jira/browse/SPARK-6129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606165#comment-14606165 ] Seth Hendrickson commented on SPARK-6129: - If no one else has started on this, I'd

[jira] [Commented] (SPARK-5571) LDA should handle text as well

[ https://issues.apache.org/jira/browse/SPARK-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606159#comment-14606159 ] Alok Singh commented on SPARK-5571: --- Since there is already Tokenizer class. We can assu

[jira] [Commented] (SPARK-5571) LDA should handle text as well

[ https://issues.apache.org/jira/browse/SPARK-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606161#comment-14606161 ] Alok Singh commented on SPARK-5571: --- I would like to work to it if everyone is ok . > L

[jira] [Comment Edited] (SPARK-5571) LDA should handle text as well

[ https://issues.apache.org/jira/browse/SPARK-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606161#comment-14606161 ] Alok Singh edited comment on SPARK-5571 at 6/29/15 7:00 PM: I

[jira] [Commented] (SPARK-6830) Memoize frequently queried vals in RDD, such as numPartitions, count etc.

[ https://issues.apache.org/jira/browse/SPARK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606157#comment-14606157 ] Sean Owen commented on SPARK-6830: -- Is this valid? For example, consider an RDD from a fi

[jira] [Resolved] (SPARK-8235) misc function: sha1 / sha

[ https://issues.apache.org/jira/browse/SPARK-8235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8235. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 6963 [https://github.com/

[jira] [Commented] (SPARK-6830) Memoize frequently queried vals in RDD, such as numPartitions, count etc.

[ https://issues.apache.org/jira/browse/SPARK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606141#comment-14606141 ] Perinkulam I Ganesh commented on SPARK-6830: What is the reason to restrict th

[jira] [Resolved] (SPARK-8067) Add support for connecting to Hive 1.1

[ https://issues.apache.org/jira/browse/SPARK-8067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8067. - Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7026 [https:/

[jira] [Commented] (SPARK-8539) LinearRegressionSummary class for storing LR training stats

[ https://issues.apache.org/jira/browse/SPARK-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606137#comment-14606137 ] Feynman Liang commented on SPARK-8539: -- Beginning work. > LinearRegressionSummary cl

[jira] [Commented] (SPARK-8538) LinearRegressionResults class for storing LR results on data

[ https://issues.apache.org/jira/browse/SPARK-8538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606135#comment-14606135 ] Feynman Liang commented on SPARK-8538: -- Beginning work. > LinearRegressionResults cl

[jira] [Resolved] (SPARK-8066) Add support for connecting to Hive 1.0 (0.14.1)

[ https://issues.apache.org/jira/browse/SPARK-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8066. - Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7026 [https:/

[jira] [Commented] (SPARK-7674) R-like stats for ML models

[ https://issues.apache.org/jira/browse/SPARK-7674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606130#comment-14606130 ] Feynman Liang commented on SPARK-7674: -- I am going to begin work on SPARK-8538 and SP

[jira] [Commented] (SPARK-8691) Enable GZip for Web UI

[ https://issues.apache.org/jira/browse/SPARK-8691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606121#comment-14606121 ] Andrew Or commented on SPARK-8691: -- Is this the same as SPARK-7716? > Enable GZip for We

[jira] [Assigned] (SPARK-8709) Exclude hadoop-client's mockito-all dependency to fix test compilation break for Hadoop 2

[ https://issues.apache.org/jira/browse/SPARK-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8709: --- Assignee: Josh Rosen (was: Apache Spark) > Exclude hadoop-client's mockito-all dependency to

[jira] [Commented] (SPARK-8709) Exclude hadoop-client's mockito-all dependency to fix test compilation break for Hadoop 2

[ https://issues.apache.org/jira/browse/SPARK-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606116#comment-14606116 ] Apache Spark commented on SPARK-8709: - User 'JoshRosen' has created a pull request for

[jira] [Assigned] (SPARK-8709) Exclude hadoop-client's mockito-all dependency to fix test compilation break for Hadoop 2

[ https://issues.apache.org/jira/browse/SPARK-8709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8709: --- Assignee: Apache Spark (was: Josh Rosen) > Exclude hadoop-client's mockito-all dependency to

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606109#comment-14606109 ] Ted Malaska commented on SPARK-2447: There may be one. I will double check, if not I

[jira] [Commented] (SPARK-2447) Add common solution for sending upsert actions to HBase (put, deletes, and increment)

[ https://issues.apache.org/jira/browse/SPARK-2447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606095#comment-14606095 ] Andrew Purtell commented on SPARK-2447: --- bq. Yeah, I have talked a lot with TD (Spar

[jira] [Resolved] (SPARK-8692) re-order the case statements that handling catalyst data types

[ https://issues.apache.org/jira/browse/SPARK-8692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-8692. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7073 [https://github.com/

[jira] [Updated] (SPARK-8478) Harmonize UDF-related code to use uniformly UDF instead of Udf

[ https://issues.apache.org/jira/browse/SPARK-8478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8478: Shepherd: Michael Armbrust Assignee: Benjamin Fradet > Harmonize UDF-related code to use

[jira] [Commented] (SPARK-8410) Hive VersionsSuite RuntimeException

[ https://issues.apache.org/jira/browse/SPARK-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606080#comment-14606080 ] Burak Yavuz commented on SPARK-8410: Hi Joe, Could you please check whether https://gi

[jira] [Commented] (SPARK-8410) Hive VersionsSuite RuntimeException

[ https://issues.apache.org/jira/browse/SPARK-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606078#comment-14606078 ] Apache Spark commented on SPARK-8410: - User 'brkyvz' has created a pull request for th

[jira] [Assigned] (SPARK-8475) SparkSubmit with Ivy jars is very slow to load with no internet access

[ https://issues.apache.org/jira/browse/SPARK-8475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8475: --- Assignee: Apache Spark > SparkSubmit with Ivy jars is very slow to load with no internet acce

[jira] [Assigned] (SPARK-8475) SparkSubmit with Ivy jars is very slow to load with no internet access

[ https://issues.apache.org/jira/browse/SPARK-8475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8475: --- Assignee: (was: Apache Spark) > SparkSubmit with Ivy jars is very slow to load with no in

[jira] [Commented] (SPARK-8475) SparkSubmit with Ivy jars is very slow to load with no internet access

[ https://issues.apache.org/jira/browse/SPARK-8475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606079#comment-14606079 ] Apache Spark commented on SPARK-8475: - User 'brkyvz' has created a pull request for th

[jira] [Assigned] (SPARK-8410) Hive VersionsSuite RuntimeException

[ https://issues.apache.org/jira/browse/SPARK-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8410: --- Assignee: Apache Spark (was: Burak Yavuz) > Hive VersionsSuite RuntimeException > --

[jira] [Assigned] (SPARK-8410) Hive VersionsSuite RuntimeException

[ https://issues.apache.org/jira/browse/SPARK-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8410: --- Assignee: Burak Yavuz (was: Apache Spark) > Hive VersionsSuite RuntimeException > --

[jira] [Commented] (SPARK-8668) expr function to convert SQL expression into a Column

[ https://issues.apache.org/jira/browse/SPARK-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606073#comment-14606073 ] Reynold Xin commented on SPARK-8668: This is not about selectExpr, but adding a new "e

[jira] [Commented] (SPARK-8668) expr function to convert SQL expression into a Column

[ https://issues.apache.org/jira/browse/SPARK-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606067#comment-14606067 ] Tarek Auel commented on SPARK-8668: --- Hi, just to get it right: selectExpr of the dataf

[jira] [Created] (SPARK-8709) Exclude hadoop-client's mockito-all dependency to fix test compilation break for Hadoop 2

Josh Rosen created SPARK-8709: - Summary: Exclude hadoop-client's mockito-all dependency to fix test compilation break for Hadoop 2 Key: SPARK-8709 URL: https://issues.apache.org/jira/browse/SPARK-8709 Pro

[jira] [Updated] (SPARK-8705) Javascript error in the web console when `totalExecutionTime` of a task is 0

[ https://issues.apache.org/jira/browse/SPARK-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8705: - Target Version/s: 1.5.0, 1.4.2 > Javascript error in the web console when `totalExecutionTime` of a task i

[jira] [Updated] (SPARK-8705) Javascript error in the web console when `totalExecutionTime` of a task is 0

[ https://issues.apache.org/jira/browse/SPARK-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8705: - Affects Version/s: 1.4.0 > Javascript error in the web console when `totalExecutionTime` of a task is 0 >

[jira] [Updated] (SPARK-8372) History server shows incorrect information for application not started

[ https://issues.apache.org/jira/browse/SPARK-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8372: - Fix Version/s: (was: 1.4.1) (was: 1.5.0) > History server shows incorrect infor

[jira] [Commented] (SPARK-8372) History server shows incorrect information for application not started

[ https://issues.apache.org/jira/browse/SPARK-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605996#comment-14605996 ] Andrew Or commented on SPARK-8372: -- OK, per discussion on the #6827 I reverted this. > H

[jira] [Reopened] (SPARK-8372) History server shows incorrect information for application not started

[ https://issues.apache.org/jira/browse/SPARK-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reopened SPARK-8372: -- Assignee: Marcelo Vanzin (was: Carson Wang) > History server shows incorrect information for applicat

[jira] [Commented] (SPARK-8410) Hive VersionsSuite RuntimeException

[ https://issues.apache.org/jira/browse/SPARK-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605977#comment-14605977 ] Burak Yavuz commented on SPARK-8410: Hi Joe, Is it possible to delete those files (~/

[jira] [Commented] (SPARK-8475) SparkSubmit with Ivy jars is very slow to load with no internet access

[ https://issues.apache.org/jira/browse/SPARK-8475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605968#comment-14605968 ] Burak Yavuz commented on SPARK-8475: ping. I think you can go ahead with a PR for opti

[jira] [Updated] (SPARK-8567) Flaky test: o.a.s.sql.hive.HiveSparkSubmitSuite --jars

[ https://issues.apache.org/jira/browse/SPARK-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8567: - Fix Version/s: 1.4.1 > Flaky test: o.a.s.sql.hive.HiveSparkSubmitSuite --jars > --

[jira] [Closed] (SPARK-8567) Flaky test: o.a.s.sql.hive.HiveSparkSubmitSuite --jars

[ https://issues.apache.org/jira/browse/SPARK-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-8567. Resolution: Fixed Fix Version/s: 1.5.0 > Flaky test: o.a.s.sql.hive.HiveSparkSubmitSuite --jars > ---

[jira] [Updated] (SPARK-8567) Flaky test: o.a.s.sql.hive.HiveSparkSubmitSuite --jars

[ https://issues.apache.org/jira/browse/SPARK-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-8567: - Target Version/s: 1.4.1, 1.5.0 (was: 1.5.0) > Flaky test: o.a.s.sql.hive.HiveSparkSubmitSuite --jars > --

[jira] [Commented] (SPARK-3692) RBF Kernel implementation to SVM

[ https://issues.apache.org/jira/browse/SPARK-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605930#comment-14605930 ] Seth Hendrickson commented on SPARK-3692: - It looks like this JIRA will be taken c

[jira] [Commented] (SPARK-8622) Spark 1.3.1 and 1.4.0 doesn't put executor working directory on executor classpath

[ https://issues.apache.org/jira/browse/SPARK-8622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605921#comment-14605921 ] Baswaraj commented on SPARK-8622: - Thats what i mean. Jars specified by --jars are not put

[jira] [Updated] (SPARK-8686) DataFrame should support `where` with expression represented by String

[ https://issues.apache.org/jira/browse/SPARK-8686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8686: - Assignee: Kousuke Saruta > DataFrame should support `where` with expression represented by String > --

[jira] [Commented] (SPARK-8703) Add CountVectorizer as a ml transformer to convert document to words count vector

[ https://issues.apache.org/jira/browse/SPARK-8703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605902#comment-14605902 ] Liang-Chi Hsieh commented on SPARK-8703: Does org.apache.spark.mllib.feature.Hashi

[jira] [Updated] (SPARK-8707) RDD#toDebugString fails if any cached RDD has invalid partitions

[ https://issues.apache.org/jira/browse/SPARK-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson updated SPARK-8707: -- Description: Repro: {code} sc.textFile("/ThisFileDoesNotExist").cache() sc.parallelize(0 until

[jira] [Updated] (SPARK-8707) RDD#toDebugString fails if any cached RDD has invalid partitions

[ https://issues.apache.org/jira/browse/SPARK-8707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson updated SPARK-8707: -- Summary: RDD#toDebugString fails if any cached RDD has invalid partitions (was: RDD#toDebugStri

[jira] [Created] (SPARK-8708) MatrixFactorizationModel.predictAll() populates single partition only

Antony Mayi created SPARK-8708: -- Summary: MatrixFactorizationModel.predictAll() populates single partition only Key: SPARK-8708 URL: https://issues.apache.org/jira/browse/SPARK-8708 Project: Spark

[jira] [Created] (SPARK-8707) RDD#toDebugString fails if any cached RDD is invalid

Aaron Davidson created SPARK-8707: - Summary: RDD#toDebugString fails if any cached RDD is invalid Key: SPARK-8707 URL: https://issues.apache.org/jira/browse/SPARK-8707 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-8705) Javascript error in the web console when `totalExecutionTime` of a task is 0

[ https://issues.apache.org/jira/browse/SPARK-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8705: --- Assignee: Apache Spark > Javascript error in the web console when `totalExecutionTime` of a t

[jira] [Commented] (SPARK-8705) Javascript error in the web console when `totalExecutionTime` of a task is 0

[ https://issues.apache.org/jira/browse/SPARK-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605870#comment-14605870 ] Apache Spark commented on SPARK-8705: - User 'zsxwing' has created a pull request for t

[jira] [Assigned] (SPARK-8705) Javascript error in the web console when `totalExecutionTime` of a task is 0

[ https://issues.apache.org/jira/browse/SPARK-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8705: --- Assignee: (was: Apache Spark) > Javascript error in the web console when `totalExecutionT

[jira] [Created] (SPARK-8706) Implement Pylint / Prospector checks for PySpark

Josh Rosen created SPARK-8706: - Summary: Implement Pylint / Prospector checks for PySpark Key: SPARK-8706 URL: https://issues.apache.org/jira/browse/SPARK-8706 Project: Spark Issue Type: New Feat

[jira] [Resolved] (SPARK-8554) Add the SparkR document files to `.rat-excludes` for `./dev/check-license`

[ https://issues.apache.org/jira/browse/SPARK-8554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-8554. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 6947 [https://github.com/

[jira] [Updated] (SPARK-8554) Add the SparkR document files to `.rat-excludes` for `./dev/check-license`

[ https://issues.apache.org/jira/browse/SPARK-8554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-8554: -- Assignee: Yu Ishikawa > Add the SparkR document files to `.rat-excludes` for `./dev/check-license` > ---

[jira] [Commented] (SPARK-8670) Nested columns can't be referenced (but they can be selected)

[ https://issues.apache.org/jira/browse/SPARK-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605822#comment-14605822 ] Nicholas Chammas commented on SPARK-8670: - Not sure. Does Scala offer the same fle

[jira] [Updated] (SPARK-8693) profiles and goals are not printed in a nice way

[ https://issues.apache.org/jira/browse/SPARK-8693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-8693: -- Assignee: Brennon York > profiles and goals are not printed in a nice way >

[jira] [Resolved] (SPARK-8693) profiles and goals are not printed in a nice way

[ https://issues.apache.org/jira/browse/SPARK-8693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-8693. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7085 [https://github.com/

[jira] [Updated] (SPARK-8693) profiles and goals are not printed in a nice way

[ https://issues.apache.org/jira/browse/SPARK-8693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-8693: -- Affects Version/s: 1.5.0 > profiles and goals are not printed in a nice way > --

[jira] [Resolved] (SPARK-8702) Avoid massive concating strings in Javascript

[ https://issues.apache.org/jira/browse/SPARK-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-8702. --- Resolution: Fixed Fix Version/s: 1.5.0 Target Version/s: 1.5.0 > Avoid massiv

[jira] [Updated] (SPARK-8702) Avoid massive concating strings in Javascript

[ https://issues.apache.org/jira/browse/SPARK-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta updated SPARK-8702: -- Assignee: Shixiong Zhu > Avoid massive concating strings in Javascript > ---

[jira] [Commented] (SPARK-8599) Use a Random operator to handle Random distribution generating expressions

[ https://issues.apache.org/jira/browse/SPARK-8599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605777#comment-14605777 ] Burak Yavuz commented on SPARK-8599: It would be great if it works for this case as we

[jira] [Commented] (SPARK-8385) java.lang.UnsupportedOperationException: Not implemented by the TFS FileSystem implementation

[ https://issues.apache.org/jira/browse/SPARK-8385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605744#comment-14605744 ] Ángel Álvarez commented on SPARK-8385: -- I could finally reproduce this same error in

<    1   2   3   4   >