[jira] [Commented] (SPARK-14914) Test Cases fail on Windows

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15604304#comment-15604304 ] Apache Spark commented on SPARK-14914: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-17984) Add support for numa aware feature

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15604300#comment-15604300 ] Apache Spark commented on SPARK-17984: -- User 'sheepduke' has created a pull request for this issue:

[jira] [Updated] (SPARK-18088) ChiSqSelector FPR PR cleanups

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18088: -- Description: There are several cleanups I'd like to make as a follow-up to the PRs

[jira] [Created] (SPARK-18088) ChiSqSelector FPR PR cleanups

2016-10-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-18088: - Summary: ChiSqSelector FPR PR cleanups Key: SPARK-18088 URL: https://issues.apache.org/jira/browse/SPARK-18088 Project: Spark Issue Type:

[jira] [Updated] (SPARK-18019) Log instrumentation in GBTs

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18019: -- Assignee: Seth Hendrickson > Log instrumentation in GBTs > ---

[jira] [Updated] (SPARK-18019) Log instrumentation in GBTs

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18019: -- Shepherd: Joseph K. Bradley (was: Timothy Hunter) > Log instrumentation in GBTs >

[jira] [Updated] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-18078: --- Priority: Minor (was: Major) > Add option for customize zipPartition task preferred locations >

[jira] [Updated] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-18078: --- Description: `RDD.zipPartitions` task preferred locations strategy will use the intersection of

[jira] [Updated] (SPARK-17183) put hive serde table schema to table properties like data source table

2016-10-24 Thread Eric Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Liang updated SPARK-17183: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-17861 > put hive serde table schema

[jira] [Created] (SPARK-18087) Optimize insert to not require REPAIR TABLE

2016-10-24 Thread Eric Liang (JIRA)
Eric Liang created SPARK-18087: -- Summary: Optimize insert to not require REPAIR TABLE Key: SPARK-18087 URL: https://issues.apache.org/jira/browse/SPARK-18087 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-18026) should not always lowercase partition columns of partition spec in parser

2016-10-24 Thread Eric Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Liang updated SPARK-18026: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-17861 > should not always lowercase

[jira] [Updated] (SPARK-17970) Use metastore for managing filesource table partitions as well

2016-10-24 Thread Eric Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Liang updated SPARK-17970: --- Summary: Use metastore for managing filesource table partitions as well (was: store partition spec

[jira] [Commented] (SPARK-17894) Ensure uniqueness of TaskSetManager name

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603860#comment-15603860 ] Apache Spark commented on SPARK-17894: -- User 'kayousterhout' has created a pull request for this

[jira] [Resolved] (SPARK-18028) simplify TableFileCatalog

2016-10-24 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18028. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15568

[jira] [Created] (SPARK-18086) Regression: Hive variables no longer work in Spark 2.0

2016-10-24 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-18086: - Summary: Regression: Hive variables no longer work in Spark 2.0 Key: SPARK-18086 URL: https://issues.apache.org/jira/browse/SPARK-18086 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-17624) Flaky test? StateStoreSuite maintenance

2016-10-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17624. -- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 > Flaky test?

[jira] [Commented] (SPARK-14300) Scala MLlib examples code merge and clean up

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603677#comment-15603677 ] Joseph K. Bradley commented on SPARK-14300: --- [~yinxusen] It looks like quite a few of these are

[jira] [Updated] (SPARK-14300) Scala MLlib examples code merge and clean up

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-14300: -- Shepherd: Joseph K. Bradley > Scala MLlib examples code merge and clean up >

[jira] [Updated] (SPARK-17950) Match SparseVector behavior with DenseVector

2016-10-24 Thread AbderRahman Sobh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] AbderRahman Sobh updated SPARK-17950: - Description: What changes were proposed in this pull request? Simply added the

[jira] [Commented] (SPARK-18085) Scalability enhancements for the History Server

2016-10-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603596#comment-15603596 ] Marcelo Vanzin commented on SPARK-18085: Finally, I actually wrote some code for the first two

[jira] [Commented] (SPARK-18085) Scalability enhancements for the History Server

2016-10-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603588#comment-15603588 ] Marcelo Vanzin commented on SPARK-18085: Pinging a few people who've worked / complained about

[jira] [Commented] (SPARK-18085) Scalability enhancements for the History Server

2016-10-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603585#comment-15603585 ] Marcelo Vanzin commented on SPARK-18085: Also, I'm not sure if we're labeling things as

[jira] [Updated] (SPARK-18085) Scalability enhancements for the History Server

2016-10-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-18085: --- Attachment: spark_hs_next_gen.pdf Here's an initial document to bootstrap the discussion of

[jira] [Created] (SPARK-18085) Scalability enhancements for the History Server

2016-10-24 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-18085: -- Summary: Scalability enhancements for the History Server Key: SPARK-18085 URL: https://issues.apache.org/jira/browse/SPARK-18085 Project: Spark Issue

[jira] [Resolved] (SPARK-11375) History Server "no histories" message to be dynamically generated by ApplicationHistoryProviders

2016-10-24 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-11375. Resolution: Duplicate Looks like a dupe. > History Server "no histories" message to be

[jira] [Resolved] (SPARK-17894) Ensure uniqueness of TaskSetManager name

2016-10-24 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-17894. Resolution: Fixed Fix Version/s: 2.1.0 Resolved by

[jira] [Updated] (SPARK-17894) Ensure uniqueness of TaskSetManager name

2016-10-24 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-17894: --- Assignee: Eren Avsarogullari > Ensure uniqueness of TaskSetManager name >

[jira] [Updated] (SPARK-18084) write.partitionBy() does not recognize nested columns that select() can access

2016-10-24 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-18084: - Issue Type: Bug (was: Improvement) > write.partitionBy() does not recognize nested

[jira] [Created] (SPARK-18084) write.partitionBy() does not recognize nested columns that select() can access

2016-10-24 Thread Nicholas Chammas (JIRA)
Nicholas Chammas created SPARK-18084: Summary: write.partitionBy() does not recognize nested columns that select() can access Key: SPARK-18084 URL: https://issues.apache.org/jira/browse/SPARK-18084

[jira] [Commented] (SPARK-16827) Stop reporting spill metrics as shuffle metrics

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603249#comment-15603249 ] Apache Spark commented on SPARK-16827: -- User 'dreamworks007' has created a pull request for this

[jira] [Commented] (SPARK-12757) Use reference counting to prevent blocks from being evicted during reads

2016-10-24 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603211#comment-15603211 ] Nicholas Chammas commented on SPARK-12757: -- Just to link back, [~josephkb] is reporting that

[jira] [Commented] (SPARK-18017) Changing Hadoop parameter through sparkSession.sparkContext.hadoopConfiguration doesn't work

2016-10-24 Thread Yuehua Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603181#comment-15603181 ] Yuehua Zhang commented on SPARK-18017: -- Yeah, that is what i did: "spark-submit --conf

[jira] [Created] (SPARK-18081) Locality Sensitive Hashing (LSH) User Guide

2016-10-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-18081: - Summary: Locality Sensitive Hashing (LSH) User Guide Key: SPARK-18081 URL: https://issues.apache.org/jira/browse/SPARK-18081 Project: Spark Issue

[jira] [Updated] (SPARK-18081) Locality Sensitive Hashing (LSH) User Guide

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18081: -- Issue Type: Documentation (was: New Feature) > Locality Sensitive Hashing (LSH) User

[jira] [Updated] (SPARK-18083) Locality Sensitive Hashing (LSH) - BitSampling

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18083: -- Assignee: Yun Ni > Locality Sensitive Hashing (LSH) - BitSampling >

[jira] [Updated] (SPARK-5992) Locality Sensitive Hashing (LSH) for MLlib

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5992: - Assignee: Yun Ni > Locality Sensitive Hashing (LSH) for MLlib >

[jira] [Updated] (SPARK-18082) Locality Sensitive Hashing (LSH) - SignRandomProjection

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18082: -- Assignee: Yun Ni > Locality Sensitive Hashing (LSH) - SignRandomProjection >

[jira] [Updated] (SPARK-18081) Locality Sensitive Hashing (LSH) User Guide

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18081: -- Assignee: Yun Ni > Locality Sensitive Hashing (LSH) User Guide >

[jira] [Created] (SPARK-18080) Locality Sensitive Hashing (LSH) Python API

2016-10-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-18080: - Summary: Locality Sensitive Hashing (LSH) Python API Key: SPARK-18080 URL: https://issues.apache.org/jira/browse/SPARK-18080 Project: Spark Issue

[jira] [Commented] (SPARK-7334) Implement RandomProjection for Dimensionality Reduction

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603030#comment-15603030 ] Joseph K. Bradley commented on SPARK-7334: -- [~sebalf] I'm sorry we weren't able to get your PR

[jira] [Updated] (SPARK-7334) Implement RandomProjection for Dimensionality Reduction

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7334: - Issue Type: New Feature (was: Improvement) > Implement RandomProjection for

[jira] [Updated] (SPARK-18080) Locality Sensitive Hashing (LSH) Python API

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-18080: -- Component/s: PySpark ML > Locality Sensitive Hashing (LSH) Python API

[jira] [Created] (SPARK-18082) Locality Sensitive Hashing (LSH) - SignRandomProjection

2016-10-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-18082: - Summary: Locality Sensitive Hashing (LSH) - SignRandomProjection Key: SPARK-18082 URL: https://issues.apache.org/jira/browse/SPARK-18082 Project: Spark

[jira] [Created] (SPARK-18083) Locality Sensitive Hashing (LSH) - BitSampling

2016-10-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-18083: - Summary: Locality Sensitive Hashing (LSH) - BitSampling Key: SPARK-18083 URL: https://issues.apache.org/jira/browse/SPARK-18083 Project: Spark

[jira] [Closed] (SPARK-7334) Implement RandomProjection for Dimensionality Reduction

2016-10-24 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley closed SPARK-7334. Resolution: Duplicate > Implement RandomProjection for Dimensionality Reduction >

[jira] [Commented] (SPARK-18053) ARRAY equality is broken in Spark 2.0

2016-10-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602974#comment-15602974 ] Cheng Lian commented on SPARK-18053: Yea, reproduced using 2.0. > ARRAY equality is broken in Spark

[jira] [Commented] (SPARK-18053) ARRAY equality is broken in Spark 2.0

2016-10-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602969#comment-15602969 ] Cheng Lian commented on SPARK-18053: Hm, the user mailing list thread said that it fails under 2.0

[jira] [Commented] (SPARK-18017) Changing Hadoop parameter through sparkSession.sparkContext.hadoopConfiguration doesn't work

2016-10-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602917#comment-15602917 ] Sean Owen commented on SPARK-18017: --- You need to set it with --conf, not programmatically, I'd imagine.

[jira] [Commented] (SPARK-18073) Migrate wiki to spark.apache.org web site

2016-10-24 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602747#comment-15602747 ] Alex Bozarth commented on SPARK-18073: -- I think we should keep the Internals pages but (open some

[jira] [Updated] (SPARK-18044) FileStreamSource should not infer partitions in every batch

2016-10-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18044: - Fix Version/s: 2.0.2 > FileStreamSource should not infer partitions in every batch >

[jira] [Updated] (SPARK-17153) [Structured streams] readStream ignores partition columns

2016-10-24 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-17153: - Fix Version/s: 2.0.2 > [Structured streams] readStream ignores partition columns >

[jira] [Commented] (SPARK-18017) Changing Hadoop parameter through sparkSession.sparkContext.hadoopConfiguration doesn't work

2016-10-24 Thread Yuehua Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602708#comment-15602708 ] Yuehua Zhang commented on SPARK-18017: -- Yeah, I tried that also. Not working either... > Changing

[jira] [Assigned] (SPARK-18079) CollectLimitExec.executeToIterator() should perform per-partition limits

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18079: Assignee: (was: Apache Spark) > CollectLimitExec.executeToIterator() should perform

[jira] [Commented] (SPARK-18079) CollectLimitExec.executeToIterator() should perform per-partition limits

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602645#comment-15602645 ] Apache Spark commented on SPARK-18079: -- User 'pwoody' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18079) CollectLimitExec.executeToIterator() should perform per-partition limits

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18079: Assignee: Apache Spark > CollectLimitExec.executeToIterator() should perform

[jira] [Created] (SPARK-18079) CollectLimitExec.executeToIterator() should perform per-partition limits

2016-10-24 Thread Patrick Woody (JIRA)
Patrick Woody created SPARK-18079: - Summary: CollectLimitExec.executeToIterator() should perform per-partition limits Key: SPARK-18079 URL: https://issues.apache.org/jira/browse/SPARK-18079 Project:

[jira] [Commented] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602525#comment-15602525 ] Apache Spark commented on SPARK-16988: -- User 'hayashidac' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16988: Assignee: Apache Spark > spark history server log needs to be fixed to show https url

[jira] [Commented] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602537#comment-15602537 ] Apache Spark commented on SPARK-18078: -- User 'WeichenXu123' has created a pull request for this

[jira] [Assigned] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18078: Assignee: Apache Spark > Add option for customize zipPartition task preferred locations >

[jira] [Assigned] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18078: Assignee: (was: Apache Spark) > Add option for customize zipPartition task preferred

[jira] [Assigned] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16988: Assignee: (was: Apache Spark) > spark history server log needs to be fixed to show

[jira] [Updated] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-18078: --- Description: `RDD.zipPartitions` task preferred locations strategy will use the intersection of

[jira] [Commented] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled

2016-10-24 Thread chie hayashida (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602484#comment-15602484 ] chie hayashida commented on SPARK-16988: Can I work on this issue? > spark history server log

[jira] [Issue Comment Deleted] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled

2016-10-24 Thread chie hayashida (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chie hayashida updated SPARK-16988: --- Comment: was deleted (was: Can I work on it?) > spark history server log needs to be fixed

[jira] [Created] (SPARK-18078) Add option for customize zipPartition task preferred locations

2016-10-24 Thread Weichen Xu (JIRA)
Weichen Xu created SPARK-18078: -- Summary: Add option for customize zipPartition task preferred locations Key: SPARK-18078 URL: https://issues.apache.org/jira/browse/SPARK-18078 Project: Spark

[jira] [Commented] (SPARK-16988) spark history server log needs to be fixed to show https url when ssl is enabled

2016-10-24 Thread chie hayashida (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602442#comment-15602442 ] chie hayashida commented on SPARK-16988: Can I work on it? > spark history server log needs to

[jira] [Commented] (SPARK-18073) Migrate wiki to spark.apache.org web site

2016-10-24 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602432#comment-15602432 ] Shivaram Venkataraman commented on SPARK-18073: --- I think most of the list looks good to me.

[jira] [Updated] (SPARK-18077) Run insert overwrite statements in spark to overwrite a partitioned table is very slow

2016-10-24 Thread J.P Feng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.P Feng updated SPARK-18077: - Description: Hello,all. I face a strange thing in my project. there is a table: CREATE TABLE

[jira] [Created] (SPARK-18077) Run insert overwrite statements in spark to overwrite a partitioned table is very slow

2016-10-24 Thread J.P Feng (JIRA)
J.P Feng created SPARK-18077: Summary: Run insert overwrite statements in spark to overwrite a partitioned table is very slow Key: SPARK-18077 URL: https://issues.apache.org/jira/browse/SPARK-18077

[jira] [Assigned] (SPARK-18076) Fix default Locale used in DateFormat, NumberFormat to Locale.US

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18076: Assignee: Apache Spark > Fix default Locale used in DateFormat, NumberFormat to Locale.US

[jira] [Commented] (SPARK-18076) Fix default Locale used in DateFormat, NumberFormat to Locale.US

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602233#comment-15602233 ] Apache Spark commented on SPARK-18076: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18076) Fix default Locale used in DateFormat, NumberFormat to Locale.US

2016-10-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18076: Assignee: (was: Apache Spark) > Fix default Locale used in DateFormat, NumberFormat

[jira] [Created] (SPARK-18076) Fix default Locale used in DateFormat, NumberFormat to Locale.US

2016-10-24 Thread Sean Owen (JIRA)
Sean Owen created SPARK-18076: - Summary: Fix default Locale used in DateFormat, NumberFormat to Locale.US Key: SPARK-18076 URL: https://issues.apache.org/jira/browse/SPARK-18076 Project: Spark

[jira] [Commented] (SPARK-9219) ClassCastException in instance of org.apache.spark.rdd.MapPartitionsRDD

2016-10-24 Thread Nick Orka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602164#comment-15602164 ] Nick Orka commented on SPARK-9219: -- Here is UDF dedicated ticket

[jira] [Commented] (SPARK-2984) FileNotFoundException on _temporary directory

2016-10-24 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602160#comment-15602160 ] Steve Loughran commented on SPARK-2984: --- Alexy, can you describe your layout a bit more # are you

[jira] [Commented] (SPARK-18073) Migrate wiki to spark.apache.org web site

2016-10-24 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602152#comment-15602152 ] holdenk commented on SPARK-18073: - I like the idea of migrating everything off of the wiki - the fact its

[jira] [Updated] (SPARK-18075) UDF doesn't work on non-local spark

2016-10-24 Thread Nick Orka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Orka updated SPARK-18075: -- Description: I have the issue with Spark 2.0.0 (spark-2.0.0-bin-hadoop2.7.tar.gz) According to this

[jira] [Created] (SPARK-18075) UDF doesn't work on non-local spark

2016-10-24 Thread Nick Orka (JIRA)
Nick Orka created SPARK-18075: - Summary: UDF doesn't work on non-local spark Key: SPARK-18075 URL: https://issues.apache.org/jira/browse/SPARK-18075 Project: Spark Issue Type: Bug Affects

[jira] [Updated] (SPARK-18075) UDF doesn't work on non-local spark

2016-10-24 Thread Nick Orka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Orka updated SPARK-18075: -- Description: I have the issue with Spark 2.0.0 (spark-2.0.0-bin-hadoop2.7.tar.gz) Here is my pom:

[jira] [Commented] (SPARK-2984) FileNotFoundException on _temporary directory

2016-10-24 Thread Alexey Balchunas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602045#comment-15602045 ] Alexey Balchunas commented on SPARK-2984: - I'm getting a similar exception on Spark 1.6.0: {code}

[jira] [Created] (SPARK-18074) UDFs don't work on non-local environment

2016-10-24 Thread Alberto Andreotti (JIRA)
Alberto Andreotti created SPARK-18074: - Summary: UDFs don't work on non-local environment Key: SPARK-18074 URL: https://issues.apache.org/jira/browse/SPARK-18074 Project: Spark Issue

[jira] [Created] (SPARK-18073) Migrate wiki to spark.apache.org web site

2016-10-24 Thread Sean Owen (JIRA)
Sean Owen created SPARK-18073: - Summary: Migrate wiki to spark.apache.org web site Key: SPARK-18073 URL: https://issues.apache.org/jira/browse/SPARK-18073 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-9219) ClassCastException in instance of org.apache.spark.rdd.MapPartitionsRDD

2016-10-24 Thread Alberto Andreotti (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601892#comment-15601892 ] Alberto Andreotti commented on SPARK-9219: -- Please paste the ticket number here. Thanks. >

[jira] [Commented] (SPARK-9219) ClassCastException in instance of org.apache.spark.rdd.MapPartitionsRDD

2016-10-24 Thread Nick Orka (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601836#comment-15601836 ] Nick Orka commented on SPARK-9219: -- This UDF functionality doesn't work on non-local environment. I would

[jira] [Commented] (SPARK-15487) Spark Master UI to reverse proxy Application and Workers UI

2016-10-24 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601827#comment-15601827 ] Cédric Hernalsteens commented on SPARK-15487: - Sure, this one (reverse proxy to access the

[jira] [Closed] (SPARK-18052) Spark Job failing with org.apache.spark.rpc.RpcTimeoutException

2016-10-24 Thread Srikanth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth closed SPARK-18052. Resolution: Not A Bug > Spark Job failing with org.apache.spark.rpc.RpcTimeoutException >

[jira] [Commented] (SPARK-15487) Spark Master UI to reverse proxy Application and Workers UI

2016-10-24 Thread Matthew Farrellee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601794#comment-15601794 ] Matthew Farrellee commented on SPARK-15487: --- well, unless you're putting another proxy in front

[jira] [Commented] (SPARK-17935) Add KafkaForeachWriter in external kafka-0.8.0 for structured streaming module

2016-10-24 Thread zhangxinyu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601783#comment-15601783 ] zhangxinyu commented on SPARK-17935: I write a short deasign doc of KafkaSink for kafka-0.10 as

[jira] [Commented] (SPARK-17935) Add KafkaForeachWriter in external kafka-0.8.0 for structured streaming module

2016-10-24 Thread zhangxinyu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601778#comment-15601778 ] zhangxinyu commented on SPARK-17935: h2. KafkaSink Design Doc h4. Goal Output results to kafka

[jira] [Updated] (SPARK-18065) Spark 2 allows filter/where on columns not in current schema

2016-10-24 Thread Matthew Scruggs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Scruggs updated SPARK-18065: Priority: Minor (was: Major) > Spark 2 allows filter/where on columns not in current

[jira] [Commented] (SPARK-15487) Spark Master UI to reverse proxy Application and Workers UI

2016-10-24 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601746#comment-15601746 ] Cédric Hernalsteens commented on SPARK-15487: - @Matthew : that would be too easy ;)

[jira] [Commented] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

2016-10-24 Thread Dhananjay Patkar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601625#comment-15601625 ] Dhananjay Patkar commented on SPARK-4105: - I see this error intermittently. I am using spark

[jira] [Resolved] (SPARK-17810) Default spark.sql.warehouse.dir is relative to local FS but can resolve as HDFS path

2016-10-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17810. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.2 Issue resolved by pull

[jira] [Assigned] (SPARK-17847) Reduce shuffled data size of GaussianMixture & copy the implementation from mllib to ml

2016-10-24 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang reassigned SPARK-17847: --- Assignee: Yanbo Liang > Reduce shuffled data size of GaussianMixture & copy the

[jira] [Updated] (SPARK-17847) Reduce shuffled data size of GaussianMixture & copy the implementation from mllib to ml

2016-10-24 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17847: Description: Copy {{GaussianMixture}} implementation from mllib to ml, then we can add new

[jira] [Updated] (SPARK-17847) Copy GaussianMixture implementation from mllib to ml

2016-10-24 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17847: Description: Copy {{GaussianMixture}} implementation from mllib to ml, then we can add new

[jira] [Updated] (SPARK-17847) Reduce shuffled data size of GaussianMixture & copy the implementation from mllib to ml

2016-10-24 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-17847: Summary: Reduce shuffled data size of GaussianMixture & copy the implementation from mllib to ml

[jira] [Commented] (SPARK-18017) Changing Hadoop parameter through sparkSession.sparkContext.hadoopConfiguration doesn't work

2016-10-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601473#comment-15601473 ] Sean Owen commented on SPARK-18017: --- Ah, try spark.hadoop.fs.s3n.block.size=... > Changing Hadoop

[jira] [Commented] (SPARK-18067) SortMergeJoin adds shuffle if join predicates have non partitioned columns

2016-10-24 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601455#comment-15601455 ] Herman van Hovell commented on SPARK-18067: --- [~tejasp] This makes sense to me. However there

  1   2   >