[jira] [Commented] (SPARK-11728) Replace example code in ml-ensembles.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005334#comment-15005334 ] Apache Spark commented on SPARK-11728: -- User 'yinxusen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11728) Replace example code in ml-ensembles.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11728: Assignee: (was: Apache Spark) > Replace example code in ml-ensembles.md using

[jira] [Commented] (SPARK-10673) spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

[ https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005249#comment-15005249 ] Xin Wu commented on SPARK-10673: if the default is false, {code} if (!sc.conf.verifyPartitionPath) {

[jira] [Commented] (SPARK-11553) row.getInt(i) if row[i]=null returns 0

[ https://issues.apache.org/jira/browse/SPARK-11553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005292#comment-15005292 ] Bartlomiej Alberski commented on SPARK-11553: - Thanks - good to know > row.getInt(i) if

[jira] [Commented] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005355#comment-15005355 ] mustafa elbehery commented on SPARK-5226: - Hello, I would like to use DBSCAN on spark.

[jira] [Commented] (SPARK-11337) Make example code in user guide testable

[ https://issues.apache.org/jira/browse/SPARK-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005338#comment-15005338 ] Xusen Yin commented on SPARK-11337: --- [~mengxr] Until now, all docs of ML and MLlib packages are

[jira] [Resolved] (SPARK-11573) correct 'reflective access of structural type member method should be enabled' Scala warnings

[ https://issues.apache.org/jira/browse/SPARK-11573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11573. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9550

[jira] [Updated] (SPARK-11573) correct 'reflective access of structural type member method should be enabled' Scala warnings

[ https://issues.apache.org/jira/browse/SPARK-11573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11573: -- Assignee: Gabor Liptak Priority: Trivial (was: Minor) Description: was: >

[jira] [Commented] (SPARK-11553) row.getInt(i) if row[i]=null returns 0

[ https://issues.apache.org/jira/browse/SPARK-11553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005270#comment-15005270 ] Bartlomiej Alberski commented on SPARK-11553: - Please assign me to this issue as I already

[jira] [Resolved] (SPARK-11694) Parquet logical types are not being tested properly

[ https://issues.apache.org/jira/browse/SPARK-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-11694. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9660

[jira] [Updated] (SPARK-11694) Parquet logical types are not being tested properly

[ https://issues.apache.org/jira/browse/SPARK-11694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-11694: --- Assignee: Hyukjin Kwon > Parquet logical types are not being tested properly >

[jira] [Comment Edited] (SPARK-10673) spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

[ https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005249#comment-15005249 ] Xin Wu edited comment on SPARK-10673 at 11/14/15 8:19 AM: -- if the default is

[jira] [Reopened] (SPARK-11721) The programming guide for Spark SQL in Spark 1.3.0 needs additional imports to work

[ https://issues.apache.org/jira/browse/SPARK-11721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-11721: --- > The programming guide for Spark SQL in Spark 1.3.0 needs additional imports > to work >

[jira] [Commented] (SPARK-11553) row.getInt(i) if row[i]=null returns 0

[ https://issues.apache.org/jira/browse/SPARK-11553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005284#comment-15005284 ] Sean Owen commented on SPARK-11553: --- That's clear already. We normally assign after it's fixed. >

[jira] [Assigned] (SPARK-11728) Replace example code in ml-ensembles.md using include_example

[ https://issues.apache.org/jira/browse/SPARK-11728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11728: Assignee: Apache Spark > Replace example code in ml-ensembles.md using include_example >

[jira] [Commented] (SPARK-11672) Flaky test: ml.JavaDefaultReadWriteSuite

[ https://issues.apache.org/jira/browse/SPARK-11672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005782#comment-15005782 ] Apache Spark commented on SPARK-11672: -- User 'mengxr' has created a pull request for this issue:

[jira] [Updated] (SPARK-11669) Python interface to SparkR GLM module

[ https://issues.apache.org/jira/browse/SPARK-11669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11669: -- Target Version/s: (was: 1.5.0, 1.5.1) [~shubhanshumis...@gmail.com] it doesn't make sense to target

[jira] [Updated] (SPARK-7799) Move "StreamingContext.actorStream" to a separate project and deprecate it in StreamingContext

[ https://issues.apache.org/jira/browse/SPARK-7799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7799: - Target Version/s: (was: 1.6.0) > Move "StreamingContext.actorStream" to a separate project and

[jira] [Updated] (SPARK-7441) Implement microbatch functionality so that Spark Streaming can process a large backlog of existing files discovered in batch in smaller batches

[ https://issues.apache.org/jira/browse/SPARK-7441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-7441: - Target Version/s: (was: 1.6.0) > Implement microbatch functionality so that Spark Streaming can process

[jira] [Updated] (SPARK-6227) PCA and SVD for PySpark

[ https://issues.apache.org/jira/browse/SPARK-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6227: - Target Version/s: (was: 1.6.0) > PCA and SVD for PySpark > --- > >

[jira] [Commented] (SPARK-6280) Remove Akka systemName from Spark

[ https://issues.apache.org/jira/browse/SPARK-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005396#comment-15005396 ] Sean Owen commented on SPARK-6280: -- Are this and the other Akka-related items targeted for 1.6 actually

[jira] [Commented] (SPARK-11725) Let UDF to handle null value

[ https://issues.apache.org/jira/browse/SPARK-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005399#comment-15005399 ] Jeff Zhang commented on SPARK-11725: I am on master > Let UDF to handle null value >

[jira] [Commented] (SPARK-11725) Let UDF to handle null value

[ https://issues.apache.org/jira/browse/SPARK-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005433#comment-15005433 ] Herman van Hovell commented on SPARK-11725: --- I can reproduce the {{-1}} default values on

[jira] [Updated] (SPARK-11720) Return Double.NaN instead of null for Mean and Average when count = 0

[ https://issues.apache.org/jira/browse/SPARK-11720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11720: -- Component/s: SQL > Return Double.NaN instead of null for Mean and Average when count = 0 >

[jira] [Resolved] (SPARK-11669) Python interface to SparkR GLM module

[ https://issues.apache.org/jira/browse/SPARK-11669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11669. --- Resolution: Not A Problem > Python interface to SparkR GLM module >

[jira] [Updated] (SPARK-11702) Guava ClassLoading Issue When Using Different Hive Metastore Version

[ https://issues.apache.org/jira/browse/SPARK-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11702: -- Component/s: Spark Core Got it, makes more sense now. > Guava ClassLoading Issue When Using Different

[jira] [Updated] (SPARK-10530) Kill other task attempts when one taskattempt belonging the same task is succeeded in speculation

[ https://issues.apache.org/jira/browse/SPARK-10530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10530: -- Target Version/s: (was: 1.6.0) Priority: Minor (was: Major) > Kill other task attempts

[jira] [Resolved] (SPARK-10081) Skip re-computing getMissingParentStages in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-10081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10081. --- Resolution: Won't Fix Target Version/s: (was: 1.6.0) > Skip re-computing

[jira] [Resolved] (SPARK-10526) Display cores/memory on ExecutorsTab

[ https://issues.apache.org/jira/browse/SPARK-10526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10526. --- Resolution: Won't Fix Target Version/s: (was: 1.6.0) > Display cores/memory on

[jira] [Commented] (SPARK-11725) Let UDF to handle null value

[ https://issues.apache.org/jira/browse/SPARK-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005391#comment-15005391 ] Herman van Hovell commented on SPARK-11725: --- I'd rather add a warning than prevent this from

[jira] [Updated] (SPARK-11727) split ExpressionEncoder into FlatEncoder and ProductEncoder

[ https://issues.apache.org/jira/browse/SPARK-11727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11727: -- Assignee: Wenchen Fan > split ExpressionEncoder into FlatEncoder and ProductEncoder >

[jira] [Updated] (SPARK-11732) MiMa excludes miss private classes

[ https://issues.apache.org/jira/browse/SPARK-11732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11732: -- Labels: (was: newbie) Fix Version/s: (was: 1.6.0) [~thunterdb] don't set Fix version

[jira] [Updated] (SPARK-9516) Improve Thread Dump page

[ https://issues.apache.org/jira/browse/SPARK-9516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9516: - Priority: Minor (was: Major) > Improve Thread Dump page > > >

[jira] [Updated] (SPARK-10250) Scala PairRDDFuncitons.groupByKey() should be fault-tolerant of single large groups

[ https://issues.apache.org/jira/browse/SPARK-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10250: -- Target Version/s: (was: 1.6.0) > Scala PairRDDFuncitons.groupByKey() should be fault-tolerant of

[jira] [Updated] (SPARK-9516) Improve Thread Dump page

[ https://issues.apache.org/jira/browse/SPARK-9516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9516: - Target Version/s: (was: 1.6.0) > Improve Thread Dump page > > >

[jira] [Updated] (SPARK-10062) Use tut for typechecking and running code in user guides

[ https://issues.apache.org/jira/browse/SPARK-10062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-10062: -- Target Version/s: (was: 1.6.0) > Use tut for typechecking and running code in user guides >

[jira] [Commented] (SPARK-9844) File appender race condition during SparkWorker shutdown

[ https://issues.apache.org/jira/browse/SPARK-9844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005504#comment-15005504 ] Jason Huang commented on SPARK-9844: Got the same error log in workers and my workers keep being

[jira] [Commented] (SPARK-10759) Missing Python code example in ML Programming guide

[ https://issues.apache.org/jira/browse/SPARK-10759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005522#comment-15005522 ] Nathan Davis commented on SPARK-10759: -- [~lmoos], is this in progress? I can take it > Missing

[jira] [Assigned] (SPARK-9928) LogicalLocalTable in ExistingRDD.scala is not referenced by any code else

[ https://issues.apache.org/jira/browse/SPARK-9928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9928: --- Assignee: Apache Spark > LogicalLocalTable in ExistingRDD.scala is not referenced by any

[jira] [Commented] (SPARK-9928) LogicalLocalTable in ExistingRDD.scala is not referenced by any code else

[ https://issues.apache.org/jira/browse/SPARK-9928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005525#comment-15005525 ] Apache Spark commented on SPARK-9928: - User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-9928) LogicalLocalTable in ExistingRDD.scala is not referenced by any code else

[ https://issues.apache.org/jira/browse/SPARK-9928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9928: --- Assignee: (was: Apache Spark) > LogicalLocalTable in ExistingRDD.scala is not referenced

[jira] [Commented] (SPARK-11153) Turns off Parquet filter push-down for string and binary columns

[ https://issues.apache.org/jira/browse/SPARK-11153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005542#comment-15005542 ] Mark Hamstra commented on SPARK-11153: -- Thanks. > Turns off Parquet filter push-down for string and

[jira] [Comment Edited] (SPARK-9844) File appender race condition during SparkWorker shutdown

[ https://issues.apache.org/jira/browse/SPARK-9844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005504#comment-15005504 ] Jason Huang edited comment on SPARK-9844 at 11/14/15 5:38 PM: -- Got the same

[jira] [Commented] (SPARK-11725) Let UDF to handle null value

[ https://issues.apache.org/jira/browse/SPARK-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005509#comment-15005509 ] Reynold Xin commented on SPARK-11725: - This is the problem of default value in codegen I suspect.

[jira] [Commented] (SPARK-11744) bin/pyspark --version doesn't return version and exit

[ https://issues.apache.org/jira/browse/SPARK-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005572#comment-15005572 ] Nicholas Chammas commented on SPARK-11744: -- Not sure who would be the best person to comment on

[jira] [Updated] (SPARK-11744) bin/pyspark --version doesn't return version and exit

[ https://issues.apache.org/jira/browse/SPARK-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-11744: - Description: {{bin/pyspark \-\-help}} offers a {{\-\-version}} option: {code} $

[jira] [Created] (SPARK-11744) bin/pyspark --version doesn't return version and exit

Nicholas Chammas created SPARK-11744: Summary: bin/pyspark --version doesn't return version and exit Key: SPARK-11744 URL: https://issues.apache.org/jira/browse/SPARK-11744 Project: Spark

[jira] [Updated] (SPARK-11744) bin/pyspark --version doesn't return version and exit

[ https://issues.apache.org/jira/browse/SPARK-11744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-11744: - Description: {{bin/pyspark \-\-help}} offers a {{\-\-version}} option: {code} $

[jira] [Commented] (SPARK-10673) spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

[ https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005577#comment-15005577 ] Xin Wu commented on SPARK-10673: The fix is being tested.. will submit PR shortly. >

[jira] [Updated] (SPARK-11738) Make ArrayType orderable

[ https://issues.apache.org/jira/browse/SPARK-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-11738: - Summary: Make ArrayType orderable (was: Make array orderable) > Make ArrayType orderable >

[jira] [Commented] (SPARK-11704) Optimize the Cartesian Join

[ https://issues.apache.org/jira/browse/SPARK-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005682#comment-15005682 ] Zhan Zhang commented on SPARK-11704: [~maropu] You are right. I mean fetching from network is a big

[jira] [Assigned] (SPARK-11738) Make ArrayType orderable

[ https://issues.apache.org/jira/browse/SPARK-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11738: Assignee: Apache Spark > Make ArrayType orderable > > >

[jira] [Assigned] (SPARK-11738) Make ArrayType orderable

[ https://issues.apache.org/jira/browse/SPARK-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11738: Assignee: (was: Apache Spark) > Make ArrayType orderable > >

[jira] [Commented] (SPARK-11738) Make ArrayType orderable

[ https://issues.apache.org/jira/browse/SPARK-11738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005689#comment-15005689 ] Apache Spark commented on SPARK-11738: -- User 'yhuai' has created a pull request for this issue: