[jira] [Resolved] (SPARK-12044) Fix usage of isnan, isNaN

[ https://issues.apache.org/jira/browse/SPARK-12044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-12044. --- Resolution: Fixed Assignee: Yanbo Liang Fix Version/s: 2.0.0

[jira] [Commented] (SPARK-12136) rddToFileName does not properly handle prefix and suffix parameters

[ https://issues.apache.org/jira/browse/SPARK-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043735#comment-15043735 ] Naveen Kumar Minchu commented on SPARK-12136: - can I pick these one I am new person to

[jira] [Commented] (SPARK-8118) Turn off noisy log output produced by Parquet 1.7.0

[ https://issues.apache.org/jira/browse/SPARK-8118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043326#comment-15043326 ] Sean Owen commented on SPARK-8118: -- I think this is mostly a Parquet problem right? in theory the log

[jira] [Updated] (SPARK-12096) remove the old constraint in word2vec

[ https://issues.apache.org/jira/browse/SPARK-12096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12096: -- Assignee: yuhao yang Priority: Trivial (was: Minor) > remove the old constraint in word2vec >

[jira] [Resolved] (SPARK-12096) remove the old constraint in word2vec

[ https://issues.apache.org/jira/browse/SPARK-12096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12096. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10103

[jira] [Assigned] (SPARK-12041) Add columnSimilarities to IndexedRowMatrix for PySpark

[ https://issues.apache.org/jira/browse/SPARK-12041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12041: Assignee: Apache Spark > Add columnSimilarities to IndexedRowMatrix for PySpark >

[jira] [Assigned] (SPARK-12041) Add columnSimilarities to IndexedRowMatrix for PySpark

[ https://issues.apache.org/jira/browse/SPARK-12041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12041: Assignee: (was: Apache Spark) > Add columnSimilarities to IndexedRowMatrix for

[jira] [Commented] (SPARK-12041) Add columnSimilarities to IndexedRowMatrix for PySpark

[ https://issues.apache.org/jira/browse/SPARK-12041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043302#comment-15043302 ] Apache Spark commented on SPARK-12041: -- User 'vectorijk' has created a pull request for this issue:

[jira] [Updated] (SPARK-12153) Word2Vec uses a fixed length for sentences which is not reasonable for reality, and similarity functions and fields are not accessible

[ https://issues.apache.org/jira/browse/SPARK-12153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YongGang Cao updated SPARK-12153: - Priority: Major (was: Minor) > Word2Vec uses a fixed length for sentences which is not

[jira] [Updated] (SPARK-11994) Word2VecModel load and save cause SparkException when model is bigger than spark.kryoserializer.buffer.max

[ https://issues.apache.org/jira/browse/SPARK-11994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11994: -- Assignee: Antonio Murgia > Word2VecModel load and save cause SparkException when model is bigger than

[jira] [Assigned] (SPARK-11824) WebUI does not render descriptions with 'bad' HTML, throws console error

[ https://issues.apache.org/jira/browse/SPARK-11824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11824: Assignee: (was: Apache Spark) > WebUI does not render descriptions with 'bad' HTML,

[jira] [Updated] (SPARK-11824) WebUI does not render descriptions with 'bad' HTML, throws console error

[ https://issues.apache.org/jira/browse/SPARK-11824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11824: -- Issue Type: Improvement (was: Bug) > WebUI does not render descriptions with 'bad' HTML, throws

[jira] [Commented] (SPARK-12033) Build Spark Error:UNRESOLVED DEPENDENCIES org.eclipse.paho#org.eclipse.paho.client.mqttv3;1.0.1: not found

[ https://issues.apache.org/jira/browse/SPARK-12033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043375#comment-15043375 ] Sean Owen commented on SPARK-12033: --- I don't think it actually has to do with the Eclipse repo; it's

[jira] [Commented] (SPARK-11824) WebUI does not render descriptions with 'bad' HTML, throws console error

[ https://issues.apache.org/jira/browse/SPARK-11824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043374#comment-15043374 ] Apache Spark commented on SPARK-11824: -- User 'srowen' has created a pull request for this issue:

[jira] [Commented] (SPARK-12033) Build Spark Error:UNRESOLVED DEPENDENCIES org.eclipse.paho#org.eclipse.paho.client.mqttv3;1.0.1: not found

[ https://issues.apache.org/jira/browse/SPARK-12033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043380#comment-15043380 ] YONG FENG commented on SPARK-12033: --- I just try to show my experience how to resolve it in case anyone

[jira] [Commented] (SPARK-12134) Spark Streaming Kinesis: Allow a mechanism to bypass validateRegion

[ https://issues.apache.org/jira/browse/SPARK-12134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043445#comment-15043445 ] Brian London commented on SPARK-12134: -- That's an interesting idea (although the {return} is

[jira] [Resolved] (SPARK-6830) Memoize frequently queried vals in RDD, such as numPartitions, count etc.

[ https://issues.apache.org/jira/browse/SPARK-6830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6830. -- Resolution: Won't Fix OK I infer this is "wont fix" > Memoize frequently queried vals in RDD, such as

[jira] [Created] (SPARK-12158) [R] [SQL] Fix 'sample' functions that break R unit test cases

Xiao Li created SPARK-12158: --- Summary: [R] [SQL] Fix 'sample' functions that break R unit test cases Key: SPARK-12158 URL: https://issues.apache.org/jira/browse/SPARK-12158 Project: Spark Issue

[jira] [Resolved] (SPARK-11988) Update JPMML to 1.2.7

[ https://issues.apache.org/jira/browse/SPARK-11988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11988. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: (was: ) Issue resolved

[jira] [Commented] (SPARK-12033) Build Spark Error:UNRESOLVED DEPENDENCIES org.eclipse.paho#org.eclipse.paho.client.mqttv3;1.0.1: not found

[ https://issues.apache.org/jira/browse/SPARK-12033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043370#comment-15043370 ] YONG FENG commented on SPARK-12033: --- Just show the solution in case others meet the same issue. It

[jira] [Created] (SPARK-12157) Support numpy types as return values of Python UDFs

Justin Uang created SPARK-12157: --- Summary: Support numpy types as return values of Python UDFs Key: SPARK-12157 URL: https://issues.apache.org/jira/browse/SPARK-12157 Project: Spark Issue

[jira] [Resolved] (SPARK-11994) Word2VecModel load and save cause SparkException when model is bigger than spark.kryoserializer.buffer.max

[ https://issues.apache.org/jira/browse/SPARK-11994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11994. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 9989

[jira] [Commented] (SPARK-12158) [R] [SQL] Fix 'sample' functions that break R unit test cases

[ https://issues.apache.org/jira/browse/SPARK-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043424#comment-15043424 ] Apache Spark commented on SPARK-12158: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12158) [R] [SQL] Fix 'sample' functions that break R unit test cases

[ https://issues.apache.org/jira/browse/SPARK-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12158: Assignee: Apache Spark > [R] [SQL] Fix 'sample' functions that break R unit test cases >

[jira] [Assigned] (SPARK-12158) [R] [SQL] Fix 'sample' functions that break R unit test cases

[ https://issues.apache.org/jira/browse/SPARK-12158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12158: Assignee: (was: Apache Spark) > [R] [SQL] Fix 'sample' functions that break R unit

[jira] [Commented] (SPARK-4502) Spark SQL reads unneccesary nested fields from Parquet

[ https://issues.apache.org/jira/browse/SPARK-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043438#comment-15043438 ] Alosh Bennett commented on SPARK-4502: -- I'm interested in this one. Consider this schema for books

[jira] [Comment Edited] (SPARK-12134) Spark Streaming Kinesis: Allow a mechanism to bypass validateRegion

[ https://issues.apache.org/jira/browse/SPARK-12134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043445#comment-15043445 ] Brian London edited comment on SPARK-12134 at 12/5/15 6:55 PM: --- That's an

[jira] [Comment Edited] (SPARK-12134) Spark Streaming Kinesis: Allow a mechanism to bypass validateRegion

[ https://issues.apache.org/jira/browse/SPARK-12134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043445#comment-15043445 ] Brian London edited comment on SPARK-12134 at 12/5/15 6:55 PM: --- That's an

[jira] [Commented] (SPARK-11314) Add service API and test service for Yarn Cluster schedulers

[ https://issues.apache.org/jira/browse/SPARK-11314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043241#comment-15043241 ] Steve Loughran commented on SPARK-11314: Sorry about this: I hadn't caught the issue you had to

[jira] [Updated] (SPARK-12159) Add user guide section for IndexToString transformer

[ https://issues.apache.org/jira/browse/SPARK-12159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-12159: Description: Add a user guide section for the IndexToString transformer as reported on the mailing list (

[jira] [Assigned] (SPARK-12160) spark.mllib should use SQLContext.getOrCreate

[ https://issues.apache.org/jira/browse/SPARK-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-12160: - Assignee: Joseph K. Bradley > spark.mllib should use SQLContext.getOrCreate >

[jira] [Resolved] (SPARK-9972) Add `struct`, `encode` and `decode` function in SparkR

[ https://issues.apache.org/jira/browse/SPARK-9972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-9972. -- Resolution: Fixed Assignee: Sun Rui Fix Version/s: 2.0.0

[jira] [Created] (SPARK-12160) spark.mllib should use SQLContext.getOrCreate

Joseph K. Bradley created SPARK-12160: - Summary: spark.mllib should use SQLContext.getOrCreate Key: SPARK-12160 URL: https://issues.apache.org/jira/browse/SPARK-12160 Project: Spark

[jira] [Assigned] (SPARK-12160) spark.mllib should use SQLContext.getOrCreate

[ https://issues.apache.org/jira/browse/SPARK-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12160: Assignee: Apache Spark (was: Joseph K. Bradley) > spark.mllib should use

[jira] [Commented] (SPARK-12160) spark.mllib should use SQLContext.getOrCreate

[ https://issues.apache.org/jira/browse/SPARK-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043581#comment-15043581 ] Apache Spark commented on SPARK-12160: -- User 'jkbradley' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12160) spark.mllib should use SQLContext.getOrCreate

[ https://issues.apache.org/jira/browse/SPARK-12160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12160: Assignee: Joseph K. Bradley (was: Apache Spark) > spark.mllib should use

[jira] [Resolved] (SPARK-11715) R support corr for Column Aggregration

[ https://issues.apache.org/jira/browse/SPARK-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-11715. --- Resolution: Fixed Assignee: Felix Cheung Fix Version/s: 2.0.0

[jira] [Commented] (SPARK-12157) Support numpy types as return values of Python UDFs

[ https://issues.apache.org/jira/browse/SPARK-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043594#comment-15043594 ] Justin Uang commented on SPARK-12157: - Good question, scala types would be good enough for this

[jira] [Commented] (SPARK-12159) Add user guide section for IndexToString transformer

[ https://issues.apache.org/jira/browse/SPARK-12159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043605#comment-15043605 ] Benjamin Fradet commented on SPARK-12159: - I've started working on this. > Add user guide

[jira] [Resolved] (SPARK-11774) Implement "struct", "encode","decode" in SparkR

[ https://issues.apache.org/jira/browse/SPARK-11774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-11774. --- Resolution: Fixed Assignee: Sun Rui Fix Version/s: 2.0.0

[jira] [Resolved] (SPARK-12115) Change numPartitions() in RDD to be "getNumPartitions" to be consistent with pyspark/scala

[ https://issues.apache.org/jira/browse/SPARK-12115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-12115. --- Resolution: Fixed Assignee: Yanbo Liang Fix Version/s: 2.0.0

[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame

[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043625#comment-15043625 ] Michael Lawrence commented on SPARK-12148: -- Not only would the constructor symbol conflict be

[jira] [Created] (SPARK-12156) SPARK_EXECUTOR_INSTANCES is not effective

KaiXinXIaoLei created SPARK-12156: - Summary: SPARK_EXECUTOR_INSTANCES is not effective Key: SPARK-12156 URL: https://issues.apache.org/jira/browse/SPARK-12156 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12156) SPARK_EXECUTOR_INSTANCES is not effective

[ https://issues.apache.org/jira/browse/SPARK-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042773#comment-15042773 ] Apache Spark commented on SPARK-12156: -- User 'KaiXinXiaoLei' has created a pull request for this

[jira] [Assigned] (SPARK-12156) SPARK_EXECUTOR_INSTANCES is not effective

[ https://issues.apache.org/jira/browse/SPARK-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12156: Assignee: (was: Apache Spark) > SPARK_EXECUTOR_INSTANCES is not effective >

[jira] [Assigned] (SPARK-12156) SPARK_EXECUTOR_INSTANCES is not effective

[ https://issues.apache.org/jira/browse/SPARK-12156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12156: Assignee: Apache Spark > SPARK_EXECUTOR_INSTANCES is not effective >

[jira] [Assigned] (SPARK-12153) Word2Vec uses a fixed length for sentences which is not reasonable for reality, and similarity functions and fields are not accessible

[ https://issues.apache.org/jira/browse/SPARK-12153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12153: Assignee: (was: Apache Spark) > Word2Vec uses a fixed length for sentences which is

[jira] [Assigned] (SPARK-12153) Word2Vec uses a fixed length for sentences which is not reasonable for reality, and similarity functions and fields are not accessible

[ https://issues.apache.org/jira/browse/SPARK-12153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12153: Assignee: Apache Spark > Word2Vec uses a fixed length for sentences which is not

[jira] [Commented] (SPARK-12153) Word2Vec uses a fixed length for sentences which is not reasonable for reality, and similarity functions and fields are not accessible

[ https://issues.apache.org/jira/browse/SPARK-12153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043533#comment-15043533 ] Apache Spark commented on SPARK-12153: -- User 'ygcao' has created a pull request for this issue:

[jira] [Created] (SPARK-12159) Add user guide section for IndexToString transformer

Joseph K. Bradley created SPARK-12159: - Summary: Add user guide section for IndexToString transformer Key: SPARK-12159 URL: https://issues.apache.org/jira/browse/SPARK-12159 Project: Spark

[jira] [Commented] (SPARK-12157) Support numpy types as return values of Python UDFs

[ https://issues.apache.org/jira/browse/SPARK-12157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043544#comment-15043544 ] Joseph K. Bradley commented on SPARK-12157: --- To confirm: Is this just for scalar types? I'm

[jira] [Commented] (SPARK-12015) Auto convert int to Double when required in pyspark.ml

[ https://issues.apache.org/jira/browse/SPARK-12015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043630#comment-15043630 ] Joseph K. Bradley commented on SPARK-12015: --- This is a bit different. SPARK-7675 is for Param

[jira] [Commented] (SPARK-11250) Generate different alias for columns with same name during join

[ https://issues.apache.org/jira/browse/SPARK-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043646#comment-15043646 ] Apache Spark commented on SPARK-11250: -- User 'NarineK' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11250) Generate different alias for columns with same name during join

[ https://issues.apache.org/jira/browse/SPARK-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11250: Assignee: Apache Spark (was: Narine Kokhlikyan) > Generate different alias for columns

[jira] [Commented] (SPARK-11250) Generate different alias for columns with same name during join

[ https://issues.apache.org/jira/browse/SPARK-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043647#comment-15043647 ] Narine Kokhlikyan commented on SPARK-11250: --- Hi there, I've created a pull request for the

[jira] [Updated] (SPARK-12161) SparkSQL Cache Matching Improvement

[ https://issues.apache.org/jira/browse/SPARK-12161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiang Chen updated SPARK-12161: --- Summary: SparkSQL Cache Matching Improvement (was: SparkSQL Cache Matching improvement) > SparkSQL

[jira] [Updated] (SPARK-12161) SparkSQL Cache Matching improvement

[ https://issues.apache.org/jira/browse/SPARK-12161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiang Chen updated SPARK-12161: --- Summary: SparkSQL Cache Matching improvement (was: SparkSQL Cache Match improvement) > SparkSQL

[jira] [Updated] (SPARK-12006) GaussianMixture.train crashes if an initial model is not None

[ https://issues.apache.org/jira/browse/SPARK-12006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-12006: -- Summary: GaussianMixture.train crashes if an initial model is not None (was:

[jira] [Commented] (SPARK-12040) Add toJson/fromJson to Vector/Vectors for PySpark

[ https://issues.apache.org/jira/browse/SPARK-12040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043638#comment-15043638 ] Joseph K. Bradley commented on SPARK-12040: --- Is this needed? It was needed in Scala for

[jira] [Comment Edited] (SPARK-11250) Generate different alias for columns with same name during join

[ https://issues.apache.org/jira/browse/SPARK-11250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043647#comment-15043647 ] Narine Kokhlikyan edited comment on SPARK-11250 at 12/6/15 2:04 AM:

[jira] [Created] (SPARK-12161) SparkSQL Cache Match improvement

Jiang Chen created SPARK-12161: -- Summary: SparkSQL Cache Match improvement Key: SPARK-12161 URL: https://issues.apache.org/jira/browse/SPARK-12161 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-12161) SparkSQL Cache Matching Improvement

[ https://issues.apache.org/jira/browse/SPARK-12161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15043675#comment-15043675 ] Apache Spark commented on SPARK-12161: -- User 'codingjaguar' has created a pull request for this

[jira] [Updated] (SPARK-12161) SparkSQL Cache Matching Improvement

[ https://issues.apache.org/jira/browse/SPARK-12161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiang Chen updated SPARK-12161: --- Description: Right now SparkSQL CacheManager can only map incoming query with cached query when

[jira] [Assigned] (SPARK-12161) SparkSQL Cache Matching Improvement

[ https://issues.apache.org/jira/browse/SPARK-12161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12161: Assignee: Apache Spark > SparkSQL Cache Matching Improvement >

[jira] [Assigned] (SPARK-12161) SparkSQL Cache Matching Improvement

[ https://issues.apache.org/jira/browse/SPARK-12161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12161: Assignee: (was: Apache Spark) > SparkSQL Cache Matching Improvement >