[jira] [Closed] (SPARK-17842) Thread and memory leak in WindowDstream (UnionRDD ) when parallelPartition computation gets enabled.

2016-10-24 Thread Sreelal S L (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreelal S L closed SPARK-17842. --- > Thread and memory leak in WindowDstream (UnionRDD ) when parallelPartition > computation gets enabled.

[jira] [Commented] (SPARK-17842) Thread and memory leak in WindowDstream (UnionRDD ) when parallelPartition computation gets enabled.

2016-10-24 Thread Sreelal S L (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604420#comment-15604420 ] Sreelal S L commented on SPARK-17842: - Upgraded to 2.0.1. The leak is not observed .

[jira] [Resolved] (SPARK-17748) One-pass algorithm for linear regression with L1 and elastic-net penalties

2016-10-24 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-17748. - Resolution: Fixed Fix Version/s: 2.1.0 Target Version/s: 2.1.0 > One-pass algori

[jira] [Commented] (SPARK-18068) Spark SQL doesn't parse some ISO 8601 formatted dates

2016-10-24 Thread Stephane Maarek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604400#comment-15604400 ] Stephane Maarek commented on SPARK-18068: - [~hyukjin.kwon] Thanks! Didn't see th

[jira] [Updated] (SPARK-18090) NegativeArraySize exception while reading parquet when inferred type and provided type for partition column are different

2016-10-24 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-18090: Description: *Problem Description:* Reading a small parquet file (single column, single record), wi

[jira] [Updated] (SPARK-18090) NegativeArraySize exception while reading parquet when inferred type and provided type for partition column are different

2016-10-24 Thread Kapil Singh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-18090: Description: *Problem Description:* Reading a small parquet file (single column, single record), wi

[jira] [Created] (SPARK-18090) NegativeArraySize exception while reading parquet when inferred type and provided type for partition column are different

2016-10-24 Thread Kapil Singh (JIRA)
Kapil Singh created SPARK-18090: --- Summary: NegativeArraySize exception while reading parquet when inferred type and provided type for partition column are different Key: SPARK-18090 URL: https://issues.apache.org/ji

[jira] [Commented] (SPARK-18068) Spark SQL doesn't parse some ISO 8601 formatted dates

2016-10-24 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604377#comment-15604377 ] Hyukjin Kwon commented on SPARK-18068: -- [~stephane.maa...@gmail.com] As a workaroun

[jira] [Assigned] (SPARK-18089) Remove CollectLimitExec operator

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18089: Assignee: (was: Apache Spark) > Remove CollectLimitExec operator > ---

[jira] [Assigned] (SPARK-18089) Remove CollectLimitExec operator

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18089: Assignee: Apache Spark > Remove CollectLimitExec operator > --

[jira] [Commented] (SPARK-18089) Remove CollectLimitExec operator

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604351#comment-15604351 ] Apache Spark commented on SPARK-18089: -- User 'viirya' has created a pull request for

[jira] [Created] (SPARK-18089) Remove CollectLimitExec operator

2016-10-24 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-18089: --- Summary: Remove CollectLimitExec operator Key: SPARK-18089 URL: https://issues.apache.org/jira/browse/SPARK-18089 Project: Spark Issue Type: Improvemen

[jira] [Commented] (SPARK-11664) Add methods to get bisecting k-means cluster structure

2016-10-24 Thread Sijun He (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604340#comment-15604340 ] Sijun He commented on SPARK-11664: -- Hi, I was wondering what's the progress on this? [~y

[jira] [Updated] (SPARK-18088) ChiSqSelector FPR PR cleanups

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18088: -- Description: There are several cleanups I'd like to make as a follow-up to the PRs from

[jira] [Updated] (SPARK-18088) ChiSqSelector FPR PR cleanups

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18088: -- Description: There are several cleanups I'd like to make as a follow-up to the PRs from

[jira] [Commented] (SPARK-18088) ChiSqSelector FPR PR cleanups

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604317#comment-15604317 ] Joseph K. Bradley commented on SPARK-18088: --- Calling this a bug since FPR is no

[jira] [Updated] (SPARK-18088) ChiSqSelector FPR PR cleanups

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18088: -- Priority: Major (was: Minor) > ChiSqSelector FPR PR cleanups > ---

[jira] [Updated] (SPARK-18088) ChiSqSelector FPR PR cleanups

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18088: -- Issue Type: Bug (was: Improvement) > ChiSqSelector FPR PR cleanups > -

[jira] [Updated] (SPARK-18088) ChiSqSelector FPR PR cleanups

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18088: -- Description: There are several cleanups I'd like to make as a follow-up to the PRs from

[jira] [Commented] (SPARK-14914) Test Cases fail on Windows

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604304#comment-15604304 ] Apache Spark commented on SPARK-14914: -- User 'HyukjinKwon' has created a pull reques

[jira] [Commented] (SPARK-17984) Add support for numa aware feature

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604300#comment-15604300 ] Apache Spark commented on SPARK-17984: -- User 'sheepduke' has created a pull request

[jira] [Updated] (SPARK-18088) ChiSqSelector FPR PR cleanups

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18088: -- Description: There are several cleanups I'd like to make as a follow-up to the PRs from

[jira] [Created] (SPARK-18088) ChiSqSelector FPR PR cleanups

2016-10-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-18088: - Summary: ChiSqSelector FPR PR cleanups Key: SPARK-18088 URL: https://issues.apache.org/jira/browse/SPARK-18088 Project: Spark Issue Type: Improveme

[jira] [Updated] (SPARK-18019) Log instrumentation in GBTs

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18019: -- Assignee: Seth Hendrickson > Log instrumentation in GBTs > ---

[jira] [Updated] (SPARK-18019) Log instrumentation in GBTs

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18019: -- Shepherd: Joseph K. Bradley (was: Timothy Hunter) > Log instrumentation in GBTs >

[jira] [Updated] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-18078: --- Priority: Minor (was: Major) > Add option for customize zipPartition task preferred locations >

[jira] [Updated] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-18078: --- Description: `RDD.zipPartitions` task preferred locations strategy will use the intersection of corr

[jira] [Updated] (SPARK-17183) put hive serde table schema to table properties like data source table

2016-10-24 Thread Eric Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Liang updated SPARK-17183: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-17861 > put hive serde table schema to

[jira] [Created] (SPARK-18087) Optimize insert to not require REPAIR TABLE

2016-10-24 Thread Eric Liang (JIRA)
Eric Liang created SPARK-18087: -- Summary: Optimize insert to not require REPAIR TABLE Key: SPARK-18087 URL: https://issues.apache.org/jira/browse/SPARK-18087 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-18026) should not always lowercase partition columns of partition spec in parser

2016-10-24 Thread Eric Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Liang updated SPARK-18026: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-17861 > should not always lowercase pa

[jira] [Updated] (SPARK-17970) Use metastore for managing filesource table partitions as well

2016-10-24 Thread Eric Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Liang updated SPARK-17970: --- Summary: Use metastore for managing filesource table partitions as well (was: store partition spec i

[jira] [Commented] (SPARK-17894) Ensure uniqueness of TaskSetManager name

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603860#comment-15603860 ] Apache Spark commented on SPARK-17894: -- User 'kayousterhout' has created a pull requ

[jira] [Resolved] (SPARK-18028) simplify TableFileCatalog

2016-10-24 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18028. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15568 [https://githu

[jira] [Created] (SPARK-18086) Regression: Hive variables no longer work in Spark 2.0

2016-10-24 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-18086: - Summary: Regression: Hive variables no longer work in Spark 2.0 Key: SPARK-18086 URL: https://issues.apache.org/jira/browse/SPARK-18086 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-17624) Flaky test? StateStoreSuite maintenance

2016-10-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17624. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > Flaky test? StateStore

[jira] [Commented] (SPARK-14300) Scala MLlib examples code merge and clean up

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603677#comment-15603677 ] Joseph K. Bradley commented on SPARK-14300: --- [~yinxusen] It looks like quite a

[jira] [Updated] (SPARK-14300) Scala MLlib examples code merge and clean up

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14300: -- Shepherd: Joseph K. Bradley > Scala MLlib examples code merge and clean up > --

[jira] [Updated] (SPARK-17950) Match SparseVector behavior with DenseVector

2016-10-24 Thread AbderRahman Sobh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] AbderRahman Sobh updated SPARK-17950: - Description: What changes were proposed in this pull request? Simply added the __getattr

[jira] [Commented] (SPARK-18085) Scalability enhancements for the History Server

2016-10-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603596#comment-15603596 ] Marcelo Vanzin commented on SPARK-18085: Finally, I actually wrote some code for

[jira] [Commented] (SPARK-18085) Scalability enhancements for the History Server

2016-10-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603588#comment-15603588 ] Marcelo Vanzin commented on SPARK-18085: Pinging a few people who've worked / com

[jira] [Commented] (SPARK-18085) Scalability enhancements for the History Server

2016-10-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603585#comment-15603585 ] Marcelo Vanzin commented on SPARK-18085: Also, I'm not sure if we're labeling thi

[jira] [Updated] (SPARK-18085) Scalability enhancements for the History Server

2016-10-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-18085: --- Attachment: spark_hs_next_gen.pdf Here's an initial document to bootstrap the discussion of h

[jira] [Created] (SPARK-18085) Scalability enhancements for the History Server

2016-10-24 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-18085: -- Summary: Scalability enhancements for the History Server Key: SPARK-18085 URL: https://issues.apache.org/jira/browse/SPARK-18085 Project: Spark Issue Typ

[jira] [Resolved] (SPARK-11375) History Server "no histories" message to be dynamically generated by ApplicationHistoryProviders

2016-10-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-11375. Resolution: Duplicate Looks like a dupe. > History Server "no histories" message to be dyn

[jira] [Resolved] (SPARK-17894) Ensure uniqueness of TaskSetManager name

2016-10-24 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-17894. Resolution: Fixed Fix Version/s: 2.1.0 Resolved by https://github.com/apache/spark/p

[jira] [Updated] (SPARK-17894) Ensure uniqueness of TaskSetManager name

2016-10-24 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-17894: --- Assignee: Eren Avsarogullari > Ensure uniqueness of TaskSetManager name > ---

[jira] [Updated] (SPARK-18084) write.partitionBy() does not recognize nested columns that select() can access

2016-10-24 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-18084: - Issue Type: Bug (was: Improvement) > write.partitionBy() does not recognize nested colum

[jira] [Created] (SPARK-18084) write.partitionBy() does not recognize nested columns that select() can access

2016-10-24 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-18084: Summary: write.partitionBy() does not recognize nested columns that select() can access Key: SPARK-18084 URL: https://issues.apache.org/jira/browse/SPARK-18084

[jira] [Commented] (SPARK-16827) Stop reporting spill metrics as shuffle metrics

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603249#comment-15603249 ] Apache Spark commented on SPARK-16827: -- User 'dreamworks007' has created a pull requ

[jira] [Commented] (SPARK-12757) Use reference counting to prevent blocks from being evicted during reads

2016-10-24 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603211#comment-15603211 ] Nicholas Chammas commented on SPARK-12757: -- Just to link back, [~josephkb] is re

[jira] [Commented] (SPARK-18017) Changing Hadoop parameter through sparkSession.sparkContext.hadoopConfiguration doesn't work

2016-10-24 Thread Yuehua Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603181#comment-15603181 ] Yuehua Zhang commented on SPARK-18017: -- Yeah, that is what i did: "spark-submit --co

[jira] [Created] (SPARK-18081) Locality Sensitive Hashing (LSH) User Guide

2016-10-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-18081: - Summary: Locality Sensitive Hashing (LSH) User Guide Key: SPARK-18081 URL: https://issues.apache.org/jira/browse/SPARK-18081 Project: Spark Issue T

[jira] [Commented] (SPARK-17693) Fixed Insert Failure To Data Source Tables when the Schema has the Comment Field

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603074#comment-15603074 ] Apache Spark commented on SPARK-17693: -- User 'gatorsmile' has created a pull request

[jira] [Updated] (SPARK-18081) Locality Sensitive Hashing (LSH) User Guide

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18081: -- Issue Type: Documentation (was: New Feature) > Locality Sensitive Hashing (LSH) User G

[jira] [Updated] (SPARK-18083) Locality Sensitive Hashing (LSH) - BitSampling

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18083: -- Assignee: Yun Ni > Locality Sensitive Hashing (LSH) - BitSampling > ---

[jira] [Updated] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5992: - Assignee: Yun Ni > Locality Sensitive Hashing (LSH) for MLlib > --

[jira] [Updated] (SPARK-18082) Locality Sensitive Hashing (LSH) - SignRandomProjection

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18082: -- Assignee: Yun Ni > Locality Sensitive Hashing (LSH) - SignRandomProjection > --

[jira] [Updated] (SPARK-18081) Locality Sensitive Hashing (LSH) User Guide

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18081: -- Assignee: Yun Ni > Locality Sensitive Hashing (LSH) User Guide > --

[jira] [Created] (SPARK-18080) Locality Sensitive Hashing (LSH) Python API

2016-10-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-18080: - Summary: Locality Sensitive Hashing (LSH) Python API Key: SPARK-18080 URL: https://issues.apache.org/jira/browse/SPARK-18080 Project: Spark Issue T

[jira] [Commented] (SPARK-7334) Implement RandomProjection for Dimensionality Reduction

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603030#comment-15603030 ] Joseph K. Bradley commented on SPARK-7334: -- [~sebalf] I'm sorry we weren't able t

[jira] [Updated] (SPARK-7334) Implement RandomProjection for Dimensionality Reduction

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7334: - Issue Type: New Feature (was: Improvement) > Implement RandomProjection for Dimensionalit

[jira] [Updated] (SPARK-18080) Locality Sensitive Hashing (LSH) Python API

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18080: -- Component/s: PySpark ML > Locality Sensitive Hashing (LSH) Python API

[jira] [Created] (SPARK-18082) Locality Sensitive Hashing (LSH) - SignRandomProjection

2016-10-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-18082: - Summary: Locality Sensitive Hashing (LSH) - SignRandomProjection Key: SPARK-18082 URL: https://issues.apache.org/jira/browse/SPARK-18082 Project: Spark

[jira] [Created] (SPARK-18083) Locality Sensitive Hashing (LSH) - BitSampling

2016-10-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-18083: - Summary: Locality Sensitive Hashing (LSH) - BitSampling Key: SPARK-18083 URL: https://issues.apache.org/jira/browse/SPARK-18083 Project: Spark Issu

[jira] [Closed] (SPARK-7334) Implement RandomProjection for Dimensionality Reduction

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley closed SPARK-7334. Resolution: Duplicate > Implement RandomProjection for Dimensionality Reduction > --

[jira] [Commented] (SPARK-18053) ARRAY equality is broken in Spark 2.0

2016-10-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602974#comment-15602974 ] Cheng Lian commented on SPARK-18053: Yea, reproduced using 2.0. > ARRAY equality is

[jira] [Commented] (SPARK-18053) ARRAY equality is broken in Spark 2.0

2016-10-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602969#comment-15602969 ] Cheng Lian commented on SPARK-18053: Hm, the user mailing list thread said that it fa

[jira] [Commented] (SPARK-18017) Changing Hadoop parameter through sparkSession.sparkContext.hadoopConfiguration doesn't work

2016-10-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602917#comment-15602917 ] Sean Owen commented on SPARK-18017: --- You need to set it with --conf, not programmatical

[jira] [Commented] (SPARK-18073) Migrate wiki to spark.apache.org web site

2016-10-24 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602747#comment-15602747 ] Alex Bozarth commented on SPARK-18073: -- I think we should keep the Internals pages b

[jira] [Updated] (SPARK-18044) FileStreamSource should not infer partitions in every batch

2016-10-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18044: - Fix Version/s: 2.0.2 > FileStreamSource should not infer partitions in every batch >

[jira] [Updated] (SPARK-17153) [Structured streams] readStream ignores partition columns

2016-10-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17153: - Fix Version/s: 2.0.2 > [Structured streams] readStream ignores partition columns > --

[jira] [Commented] (SPARK-18017) Changing Hadoop parameter through sparkSession.sparkContext.hadoopConfiguration doesn't work

2016-10-24 Thread Yuehua Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602708#comment-15602708 ] Yuehua Zhang commented on SPARK-18017: -- Yeah, I tried that also. Not working either.

[jira] [Assigned] (SPARK-18079) CollectLimitExec.executeToIterator() should perform per-partition limits

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18079: Assignee: (was: Apache Spark) > CollectLimitExec.executeToIterator() should perform pe

[jira] [Commented] (SPARK-18079) CollectLimitExec.executeToIterator() should perform per-partition limits

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602645#comment-15602645 ] Apache Spark commented on SPARK-18079: -- User 'pwoody' has created a pull request for

[jira] [Assigned] (SPARK-18079) CollectLimitExec.executeToIterator() should perform per-partition limits

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18079: Assignee: Apache Spark > CollectLimitExec.executeToIterator() should perform per-partition

[jira] [Created] (SPARK-18079) CollectLimitExec.executeToIterator() should perform per-partition limits

2016-10-24 Thread Patrick Woody (JIRA)
Patrick Woody created SPARK-18079: - Summary: CollectLimitExec.executeToIterator() should perform per-partition limits Key: SPARK-18079 URL: https://issues.apache.org/jira/browse/SPARK-18079 Project: S

[jira] [Commented] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602525#comment-15602525 ] Apache Spark commented on SPARK-16988: -- User 'hayashidac' has created a pull request

[jira] [Assigned] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16988: Assignee: Apache Spark > spark history server log needs to be fixed to show https url when

[jira] [Commented] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602537#comment-15602537 ] Apache Spark commented on SPARK-18078: -- User 'WeichenXu123' has created a pull reque

[jira] [Assigned] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18078: Assignee: Apache Spark > Add option for customize zipPartition task preferred locations >

[jira] [Assigned] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18078: Assignee: (was: Apache Spark) > Add option for customize zipPartition task preferred l

[jira] [Assigned] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16988: Assignee: (was: Apache Spark) > spark history server log needs to be fixed to show htt

[jira] [Updated] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-18078: --- Description: `RDD.zipPartitions` task preferred locations strategy will use the intersection of corr

[jira] [Commented] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled

2016-10-24 Thread chie hayashida (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602484#comment-15602484 ] chie hayashida commented on SPARK-16988: Can I work on this issue? > spark histo

[jira] [Issue Comment Deleted] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled

2016-10-24 Thread chie hayashida (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chie hayashida updated SPARK-16988: --- Comment: was deleted (was: Can I work on it?) > spark history server log needs to be fixed t

[jira] [Created] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Weichen Xu (JIRA)
Weichen Xu created SPARK-18078: -- Summary: Add option for customize zipPartition task preferred locations Key: SPARK-18078 URL: https://issues.apache.org/jira/browse/SPARK-18078 Project: Spark I

[jira] [Commented] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled

2016-10-24 Thread chie hayashida (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602442#comment-15602442 ] chie hayashida commented on SPARK-16988: Can I work on it? > spark history serve

[jira] [Commented] (SPARK-18073) Migrate wiki to spark.apache.org web site

2016-10-24 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602432#comment-15602432 ] Shivaram Venkataraman commented on SPARK-18073: --- I think most of the list l

[jira] [Updated] (SPARK-18077) Run insert overwrite statements in spark to overwrite a partitioned table is very slow

2016-10-24 Thread J.P Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.P Feng updated SPARK-18077: - Description: Hello,all. I face a strange thing in my project. there is a table: CREATE TABLE `login4gam

[jira] [Created] (SPARK-18077) Run insert overwrite statements in spark to overwrite a partitioned table is very slow

2016-10-24 Thread J.P Feng (JIRA)
J.P Feng created SPARK-18077: Summary: Run insert overwrite statements in spark to overwrite a partitioned table is very slow Key: SPARK-18077 URL: https://issues.apache.org/jira/browse/SPARK-18077 Proje

[jira] [Assigned] (SPARK-18076) Fix default Locale used in DateFormat, NumberFormat to Locale.US

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18076: Assignee: Apache Spark > Fix default Locale used in DateFormat, NumberFormat to Locale.US

[jira] [Commented] (SPARK-18076) Fix default Locale used in DateFormat, NumberFormat to Locale.US

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602233#comment-15602233 ] Apache Spark commented on SPARK-18076: -- User 'srowen' has created a pull request for

[jira] [Assigned] (SPARK-18076) Fix default Locale used in DateFormat, NumberFormat to Locale.US

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18076: Assignee: (was: Apache Spark) > Fix default Locale used in DateFormat, NumberFormat to

[jira] [Created] (SPARK-18076) Fix default Locale used in DateFormat, NumberFormat to Locale.US

2016-10-24 Thread Sean Owen (JIRA)
Sean Owen created SPARK-18076: - Summary: Fix default Locale used in DateFormat, NumberFormat to Locale.US Key: SPARK-18076 URL: https://issues.apache.org/jira/browse/SPARK-18076 Project: Spark I

[jira] [Commented] (SPARK-9219) ClassCastException in instance of org.apache.spark.rdd.MapPartitionsRDD

2016-10-24 Thread Nick Orka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602164#comment-15602164 ] Nick Orka commented on SPARK-9219: -- Here is UDF dedicated ticket https://issues.apache.or

[jira] [Commented] (SPARK-2984) FileNotFoundException on _temporary directory

2016-10-24 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602160#comment-15602160 ] Steve Loughran commented on SPARK-2984: --- Alexy, can you describe your layout a bit m

[jira] [Commented] (SPARK-18073) Migrate wiki to spark.apache.org web site

2016-10-24 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602152#comment-15602152 ] holdenk commented on SPARK-18073: - I like the idea of migrating everything off of the wik

[jira] [Updated] (SPARK-18075) UDF doesn't work on non-local spark

2016-10-24 Thread Nick Orka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Orka updated SPARK-18075: -- Description: I have the issue with Spark 2.0.0 (spark-2.0.0-bin-hadoop2.7.tar.gz) According to this ti

[jira] [Created] (SPARK-18075) UDF doesn't work on non-local spark

2016-10-24 Thread Nick Orka (JIRA)
Nick Orka created SPARK-18075: - Summary: UDF doesn't work on non-local spark Key: SPARK-18075 URL: https://issues.apache.org/jira/browse/SPARK-18075 Project: Spark Issue Type: Bug Affects Ver

[jira] [Updated] (SPARK-18075) UDF doesn't work on non-local spark

2016-10-24 Thread Nick Orka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Orka updated SPARK-18075: -- Description: I have the issue with Spark 2.0.0 (spark-2.0.0-bin-hadoop2.7.tar.gz) Here is my pom: {code

  1   2   >