[jira] [Created] (SPARK-17763) JacksonParser silently parses null as 0 when the field is not nullable

Hyukjin Kwon created SPARK-17763: Summary: JacksonParser silently parses null as 0 when the field is not nullable Key: SPARK-17763 URL: https://issues.apache.org/jira/browse/SPARK-17763 Project: Spark

[jira] [Assigned] (SPARK-17763) JacksonParser silently parses null as 0 when the field is not nullable

[ https://issues.apache.org/jira/browse/SPARK-17763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17763: Assignee: Apache Spark > JacksonParser silently parses null as 0 when the field is not nul

[jira] [Assigned] (SPARK-17763) JacksonParser silently parses null as 0 when the field is not nullable

[ https://issues.apache.org/jira/browse/SPARK-17763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17763: Assignee: (was: Apache Spark) > JacksonParser silently parses null as 0 when the field

[jira] [Commented] (SPARK-17763) JacksonParser silently parses null as 0 when the field is not nullable

[ https://issues.apache.org/jira/browse/SPARK-17763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15541825#comment-15541825 ] Apache Spark commented on SPARK-17763: -- User 'HyukjinKwon' has created a pull reques

[jira] [Created] (SPARK-17764) to_json function for parsing Structs to json Strings

Hyukjin Kwon created SPARK-17764: Summary: to_json function for parsing Structs to json Strings Key: SPARK-17764 URL: https://issues.apache.org/jira/browse/SPARK-17764 Project: Spark Issue Ty

[jira] [Commented] (SPARK-17764) to_json function for parsing Structs to json Strings

[ https://issues.apache.org/jira/browse/SPARK-17764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15541842#comment-15541842 ] Hyukjin Kwon commented on SPARK-17764: -- Hi [~marmbrus], if no one is working one thi

[jira] [Commented] (SPARK-17734) inner equi-join shorthand that returns Datasets, like DataFrame already has

[ https://issues.apache.org/jira/browse/SPARK-17734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15541870#comment-15541870 ] Dongjoon Hyun commented on SPARK-17734: --- Hi, [~pdxleif]. How do you think about the

[jira] [Commented] (SPARK-17465) Inappropriate memory management in `org.apache.spark.storage.MemoryStore` may lead to memory leak

[ https://issues.apache.org/jira/browse/SPARK-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15541946#comment-15541946 ] Xing Shi commented on SPARK-17465: -- Resolved. In every task, method _currentUnrollMemor

[jira] [Comment Edited] (SPARK-17465) Inappropriate memory management in `org.apache.spark.storage.MemoryStore` may lead to memory leak

[ https://issues.apache.org/jira/browse/SPARK-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15541946#comment-15541946 ] Xing Shi edited comment on SPARK-17465 at 10/3/16 9:17 AM: --- Res

[jira] [Comment Edited] (SPARK-17465) Inappropriate memory management in `org.apache.spark.storage.MemoryStore` may lead to memory leak

[ https://issues.apache.org/jira/browse/SPARK-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15541946#comment-15541946 ] Xing Shi edited comment on SPARK-17465 at 10/3/16 9:16 AM: --- Res

[jira] [Resolved] (SPARK-17598) User-friendly name for Spark Thrift Server in web UI

[ https://issues.apache.org/jira/browse/SPARK-17598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17598. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15268 [https://github.co

[jira] [Updated] (SPARK-17598) User-friendly name for Spark Thrift Server in web UI

[ https://issues.apache.org/jira/browse/SPARK-17598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17598: -- Assignee: Alex Bozarth Priority: Trivial (was: Minor) > User-friendly name for Spark Thrift Server

[jira] [Commented] (SPARK-13288) [1.6.0] Memory leak in Spark streaming

[ https://issues.apache.org/jira/browse/SPARK-13288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15541970#comment-15541970 ] Xing Shi commented on SPARK-13288: -- This problem is just similar to [SPARK-17465|https:

[jira] [Updated] (SPARK-17718) Make loss function formulation label note clearer in MLlib docs

[ https://issues.apache.org/jira/browse/SPARK-17718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17718: -- Summary: Make loss function formulation label note clearer in MLlib docs (was: Update MLib Classificat

[jira] [Updated] (SPARK-17718) Make loss function formulation label note clearer in MLlib docs

[ https://issues.apache.org/jira/browse/SPARK-17718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17718: -- Assignee: Sean Owen Priority: Trivial (was: Minor) > Make loss function formulation label note cle

[jira] [Commented] (SPARK-17718) Make loss function formulation label note clearer in MLlib docs

[ https://issues.apache.org/jira/browse/SPARK-17718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542004#comment-15542004 ] Apache Spark commented on SPARK-17718: -- User 'srowen' has created a pull request for

[jira] [Resolved] (SPARK-17736) Update R README for rmarkdown, pandoc

[ https://issues.apache.org/jira/browse/SPARK-17736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17736. --- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15309 [https://github.co

[jira] [Updated] (SPARK-17736) Update R README for rmarkdown, pandoc

[ https://issues.apache.org/jira/browse/SPARK-17736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17736: -- Assignee: Jagadeesan A S Priority: Trivial (was: Major) > Update R README for rmarkdown, pandoc >

[jira] [Updated] (SPARK-17736) Update R README for rmarkdown, pandoc

[ https://issues.apache.org/jira/browse/SPARK-17736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17736: -- Fix Version/s: 2.0.2 > Update R README for rmarkdown, pandoc > - >

[jira] [Created] (SPARK-17765) org.apache.spark.mllib.linalg.VectorUDT cannot be cast to org.apache.spark.sql.types.StructType

Alexander Shorin created SPARK-17765: Summary: org.apache.spark.mllib.linalg.VectorUDT cannot be cast to org.apache.spark.sql.types.StructType Key: SPARK-17765 URL: https://issues.apache.org/jira/browse/SPARK-

[jira] [Resolved] (SPARK-17512) Specifying remote files for Python based Spark jobs in Yarn cluster mode not working

[ https://issues.apache.org/jira/browse/SPARK-17512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17512. --- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 2.1.0 2.0.1

[jira] [Created] (SPARK-17766) Write ahead corruption on a toy project

Nadav Samet created SPARK-17766: --- Summary: Write ahead corruption on a toy project Key: SPARK-17766 URL: https://issues.apache.org/jira/browse/SPARK-17766 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

Alex Liu created SPARK-17767: Summary: Spark SQL ExternalCatalog API custom implementation support Key: SPARK-17767 URL: https://issues.apache.org/jira/browse/SPARK-17767 Project: Spark Issue Ty

[jira] [Updated] (SPARK-17766) Write ahead log corruption on a toy project

[ https://issues.apache.org/jira/browse/SPARK-17766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nadav Samet updated SPARK-17766: Summary: Write ahead log corruption on a toy project (was: Write ahead corruption on a toy project

[jira] [Updated] (SPARK-17766) Write ahead log corruption on a toy project

[ https://issues.apache.org/jira/browse/SPARK-17766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nadav Samet updated SPARK-17766: Description: Write ahead log seems to get corrupted when the application is stopped abruptly (Ctrl

[jira] [Commented] (SPARK-17728) UDFs are run too many times

[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542689#comment-15542689 ] Jacob Eisinger commented on SPARK-17728: Thanks for the explanation and the trick

[jira] [Comment Edited] (SPARK-17728) UDFs are run too many times

[ https://issues.apache.org/jira/browse/SPARK-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542689#comment-15542689 ] Jacob Eisinger edited comment on SPARK-17728 at 10/3/16 3:37 PM: --

[jira] [Created] (SPARK-17768) Small {Sum,Count,Mean}Evaluator problems and suboptimalities

Sean Owen created SPARK-17768: - Summary: Small {Sum,Count,Mean}Evaluator problems and suboptimalities Key: SPARK-17768 URL: https://issues.apache.org/jira/browse/SPARK-17768 Project: Spark Issue

[jira] [Commented] (SPARK-17766) Write ahead log corruption on a toy project

[ https://issues.apache.org/jira/browse/SPARK-17766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542709#comment-15542709 ] Sean Owen commented on SPARK-17766: --- Where is the corruption here though.. this is just

[jira] [Commented] (SPARK-17766) Write ahead log corruption on a toy project

[ https://issues.apache.org/jira/browse/SPARK-17766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542790#comment-15542790 ] Nadav Samet commented on SPARK-17766: - Can you elaborate on how to turn off kryo regi

[jira] [Commented] (SPARK-17766) Write ahead log corruption on a toy project

[ https://issues.apache.org/jira/browse/SPARK-17766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542799#comment-15542799 ] Sean Owen commented on SPARK-17766: --- Yes, you would need to turn it off. That may let t

[jira] [Commented] (SPARK-17074) generate histogram information for column

[ https://issues.apache.org/jira/browse/SPARK-17074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542830#comment-15542830 ] Srinath commented on SPARK-17074: - IMO if you can get reasonable error bounds (as Tim poi

[jira] [Commented] (SPARK-17749) Unresolved columns when nesting SQL join clauses

[ https://issues.apache.org/jira/browse/SPARK-17749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542850#comment-15542850 ] Andreas Damm commented on SPARK-17749: -- Combining the two ON clauses into one by and

[jira] [Commented] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542862#comment-15542862 ] Dongjoon Hyun commented on SPARK-17767: --- +1 > Spark SQL ExternalCatalog API custom

[jira] [Comment Edited] (SPARK-12985) Spark Hive thrift server big decimal data issue

[ https://issues.apache.org/jira/browse/SPARK-12985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542877#comment-15542877 ] Dongjoon Hyun edited comment on SPARK-12985 at 10/3/16 5:05 PM: ---

[jira] [Commented] (SPARK-12985) Spark Hive thrift server big decimal data issue

[ https://issues.apache.org/jira/browse/SPARK-12985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542877#comment-15542877 ] Dongjoon Hyun commented on SPARK-12985: --- Hi, [~alexliu68] and [~adrian-wang]. I rea

[jira] [Resolved] (SPARK-17073) generate basic stats for column

[ https://issues.apache.org/jira/browse/SPARK-17073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17073. - Resolution: Fixed Assignee: Zhenhua Wang Fix Version/s: 2.1.0 > generate basic st

[jira] [Commented] (SPARK-12985) Spark Hive thrift server big decimal data issue

[ https://issues.apache.org/jira/browse/SPARK-12985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542909#comment-15542909 ] Alex Liu commented on SPARK-12985: -- It's ok to close it, It could be fixed from Simba si

[jira] [Commented] (SPARK-5484) Pregel should checkpoint periodically to avoid StackOverflowError

[ https://issues.apache.org/jira/browse/SPARK-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542942#comment-15542942 ] Shreya Agarwal commented on SPARK-5484: --- Hi, I am running a Spark 2.0 cluster and w

[jira] [Commented] (SPARK-17766) Write ahead log corruption on a toy project

[ https://issues.apache.org/jira/browse/SPARK-17766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542956#comment-15542956 ] Nadav Samet commented on SPARK-17766: - I don't believe there's anything environmental

[jira] [Comment Edited] (SPARK-17766) Write ahead log corruption on a toy project

[ https://issues.apache.org/jira/browse/SPARK-17766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542956#comment-15542956 ] Nadav Samet edited comment on SPARK-17766 at 10/3/16 5:43 PM: -

[jira] [Commented] (SPARK-17749) Unresolved columns when nesting SQL join clauses

[ https://issues.apache.org/jira/browse/SPARK-17749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542964#comment-15542964 ] Dongjoon Hyun commented on SPARK-17749: --- Good for you! > Unresolved columns when n

[jira] [Commented] (SPARK-15777) Catalog federation

[ https://issues.apache.org/jira/browse/SPARK-15777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542995#comment-15542995 ] Frederick Reiss commented on SPARK-15777: - Thanks for posting this design doc. So

[jira] [Commented] (SPARK-17752) Spark returns incorrect result when 'collect()'ing a cached Dataset with many columns

[ https://issues.apache.org/jira/browse/SPARK-17752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543008#comment-15543008 ] Shivaram Venkataraman commented on SPARK-17752: --- Thanks for the bug report

[jira] [Commented] (SPARK-12985) Spark Hive thrift server big decimal data issue

[ https://issues.apache.org/jira/browse/SPARK-12985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543011#comment-15543011 ] Dongjoon Hyun commented on SPARK-12985: --- Thank you, [~alexliu68] > Spark Hive thri

[jira] [Closed] (SPARK-12985) Spark Hive thrift server big decimal data issue

[ https://issues.apache.org/jira/browse/SPARK-12985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-12985. - Resolution: Not A Problem > Spark Hive thrift server big decimal data issue > ---

[jira] [Resolved] (SPARK-17718) Make loss function formulation label note clearer in MLlib docs

[ https://issues.apache.org/jira/browse/SPARK-17718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-17718. - Resolution: Fixed Fix Version/s: 2.1.0 > Make loss function formulation label note clearer in MLli

[jira] [Commented] (SPARK-17709) spark 2.0 join - column resolution error

[ https://issues.apache.org/jira/browse/SPARK-17709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543046#comment-15543046 ] Dilip Biswal commented on SPARK-17709: -- Hi Ashish, Thanks a lot.. will try and get b

[jira] [Commented] (SPARK-17752) Spark returns incorrect result when 'collect()'ing a cached Dataset with many columns

[ https://issues.apache.org/jira/browse/SPARK-17752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543045#comment-15543045 ] Kevin Ushey commented on SPARK-17752: - I can confirm that everything is okay with Spa

[jira] [Commented] (SPARK-17742) Spark Launcher does not get failed state in Listener

[ https://issues.apache.org/jira/browse/SPARK-17742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543048#comment-15543048 ] Marcelo Vanzin commented on SPARK-17742: This code in "LocalSchedulerBackend" cau

[jira] [Updated] (SPARK-17720) Static configurations in SQL

[ https://issues.apache.org/jira/browse/SPARK-17720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17720: Summary: Static configurations in SQL (was: introduce global SQL conf) > Static configurations in

[jira] [Commented] (SPARK-10634) The spark sql fails if the where clause contains a string with " in it.

[ https://issues.apache.org/jira/browse/SPARK-10634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543059#comment-15543059 ] Apache Spark commented on SPARK-10634: -- User 'dilipbiswal' has created a pull reques

[jira] [Updated] (SPARK-17720) Static configurations in SQL

[ https://issues.apache.org/jira/browse/SPARK-17720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17720: Description: Spark SQL has two kinds of configuration parameters: dynamic configs and static confi

[jira] [Assigned] (SPARK-17761) Simplify InternalRow hierarchy

[ https://issues.apache.org/jira/browse/SPARK-17761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17761: Assignee: Herman van Hovell (was: Apache Spark) > Simplify InternalRow hierarchy > --

[jira] [Assigned] (SPARK-17761) Simplify InternalRow hierarchy

[ https://issues.apache.org/jira/browse/SPARK-17761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17761: Assignee: Apache Spark (was: Herman van Hovell) > Simplify InternalRow hierarchy > --

[jira] [Commented] (SPARK-17761) Simplify InternalRow hierarchy

[ https://issues.apache.org/jira/browse/SPARK-17761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543079#comment-15543079 ] Apache Spark commented on SPARK-17761: -- User 'hvanhovell' has created a pull request

[jira] [Commented] (SPARK-17758) Spark Aggregate function LAST returns null on an empty partition

[ https://issues.apache.org/jira/browse/SPARK-17758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543097#comment-15543097 ] Franck Tago commented on SPARK-17758: - I tested the behavior of the min and max funct

[jira] [Commented] (SPARK-10634) The spark sql fails if the where clause contains a string with " in it.

[ https://issues.apache.org/jira/browse/SPARK-10634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543104#comment-15543104 ] Apache Spark commented on SPARK-10634: -- User 'dilipbiswal' has created a pull reques

[jira] [Updated] (SPARK-17766) Write ahead log exception on a toy project

[ https://issues.apache.org/jira/browse/SPARK-17766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17766: -- Priority: Minor (was: Major) Summary: Write ahead log exception on a toy project (was: Write ahea

[jira] [Created] (SPARK-17769) Some FetchFailure refactoring in the DAGScheduler

Mark Hamstra created SPARK-17769: Summary: Some FetchFailure refactoring in the DAGScheduler Key: SPARK-17769 URL: https://issues.apache.org/jira/browse/SPARK-17769 Project: Spark Issue Type:

[jira] [Commented] (SPARK-17769) Some FetchFailure refactoring in the DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-17769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543138#comment-15543138 ] Apache Spark commented on SPARK-17769: -- User 'markhamstra' has created a pull reques

[jira] [Assigned] (SPARK-17769) Some FetchFailure refactoring in the DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-17769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17769: Assignee: Mark Hamstra (was: Apache Spark) > Some FetchFailure refactoring in the DAGSche

[jira] [Assigned] (SPARK-17769) Some FetchFailure refactoring in the DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-17769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17769: Assignee: Apache Spark (was: Mark Hamstra) > Some FetchFailure refactoring in the DAGSche

[jira] [Commented] (SPARK-17709) spark 2.0 join - column resolution error

[ https://issues.apache.org/jira/browse/SPARK-17709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543141#comment-15543141 ] Dilip Biswal commented on SPARK-17709: -- @ashrowty Hi Ashish, in your example, the co

[jira] [Updated] (SPARK-17758) Spark Aggregate function LAST returns null on an empty partition

[ https://issues.apache.org/jira/browse/SPARK-17758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17758: -- Description: My Environment Spark 2.0.0 I have included the physical plan of my app

[jira] [Updated] (SPARK-17752) Spark returns incorrect result when 'collect()'ing a cached Dataset with many columns

[ https://issues.apache.org/jira/browse/SPARK-17752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-17752: -- Fix Version/s: 2.1.0 2.0.1 > Spark returns incorrect result

[jira] [Commented] (SPARK-17758) Spark Aggregate function LAST returns null on an empty partition

[ https://issues.apache.org/jira/browse/SPARK-17758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543175#comment-15543175 ] Herman van Hovell commented on SPARK-17758: --- Why should last return a non null

[jira] [Commented] (SPARK-17752) Spark returns incorrect result when 'collect()'ing a cached Dataset with many columns

[ https://issues.apache.org/jira/browse/SPARK-17752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543179#comment-15543179 ] Shivaram Venkataraman commented on SPARK-17752: --- I don't know what the plan

[jira] [Resolved] (SPARK-17752) Spark returns incorrect result when 'collect()'ing a cached Dataset with many columns

[ https://issues.apache.org/jira/browse/SPARK-17752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-17752. --- Resolution: Fixed > Spark returns incorrect result when 'collect()'ing a cach

[jira] [Updated] (SPARK-17764) to_json function for parsing Structs to json Strings

[ https://issues.apache.org/jira/browse/SPARK-17764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17764: - Target Version/s: 2.1.0 > to_json function for parsing Structs to json Strings >

[jira] [Commented] (SPARK-17764) to_json function for parsing Structs to json Strings

[ https://issues.apache.org/jira/browse/SPARK-17764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543229#comment-15543229 ] Michael Armbrust commented on SPARK-17764: -- I would start by looking at the impl

[jira] [Commented] (SPARK-10637) DataFrames: saving with nested User Data Types

[ https://issues.apache.org/jira/browse/SPARK-10637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543246#comment-15543246 ] Apache Spark commented on SPARK-10637: -- User 'dilipbiswal' has created a pull reques

[jira] [Commented] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543319#comment-15543319 ] Dongjoon Hyun commented on SPARK-17767: --- It seems that custom external catalog supp

[jira] [Commented] (SPARK-15777) Catalog federation

[ https://issues.apache.org/jira/browse/SPARK-15777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543329#comment-15543329 ] Yan commented on SPARK-15777: - 1) Currently the rules are applied on a per-session basis. Rig

[jira] [Commented] (SPARK-16589) Chained cartesian produces incorrect number of records

[ https://issues.apache.org/jira/browse/SPARK-16589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543349#comment-15543349 ] holdenk commented on SPARK-16589: - Is this something you are still investigating/working

[jira] [Created] (SPARK-17770) Make ObjectType SQL Type Public

Aleksander Eskilson created SPARK-17770: --- Summary: Make ObjectType SQL Type Public Key: SPARK-17770 URL: https://issues.apache.org/jira/browse/SPARK-17770 Project: Spark Issue Type: Wis

[jira] [Created] (SPARK-17771) Allow start-master/slave scripts to start in the foreground

Mike Ihbe created SPARK-17771: - Summary: Allow start-master/slave scripts to start in the foreground Key: SPARK-17771 URL: https://issues.apache.org/jira/browse/SPARK-17771 Project: Spark Issue

[jira] [Updated] (SPARK-17771) Allow start-master/slave scripts to start in the foreground

[ https://issues.apache.org/jira/browse/SPARK-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Ihbe updated SPARK-17771: -- Description: Based on conversation from this thread: http://apache-spark-developers-list.1001551.n3.na

[jira] [Commented] (SPARK-10063) Remove DirectParquetOutputCommitter

[ https://issues.apache.org/jira/browse/SPARK-10063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543412#comment-15543412 ] Ai Deng commented on SPARK-10063: - You can add this line for your SparkContext, and this

[jira] [Resolved] (SPARK-17679) Remove unnecessary Py4J ListConverter patch

[ https://issues.apache.org/jira/browse/SPARK-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-17679. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15254 [https://github.

[jira] [Resolved] (SPARK-17771) Allow start-master/slave scripts to start in the foreground

[ https://issues.apache.org/jira/browse/SPARK-17771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17771. --- Resolution: Fixed > Allow start-master/slave scripts to start in the foreground > ---

[jira] [Assigned] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17767: Assignee: (was: Apache Spark) > Spark SQL ExternalCatalog API custom implementation su

[jira] [Assigned] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17767: Assignee: Apache Spark > Spark SQL ExternalCatalog API custom implementation support > ---

[jira] [Commented] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543461#comment-15543461 ] Apache Spark commented on SPARK-17767: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Commented] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543466#comment-15543466 ] Dongjoon Hyun commented on SPARK-17767: --- Hi, [~alexliu68]. I made a PR for this iss

[jira] [Commented] (SPARK-16532) Provide a REST API for submitting and tracking status of jobs

[ https://issues.apache.org/jira/browse/SPARK-16532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543469#comment-15543469 ] Joao Vasques commented on SPARK-16532: -- Yes, the idea is not about having a whole ne

[jira] [Created] (SPARK-17772) Add helper testing methods for instance weighting

Seth Hendrickson created SPARK-17772: Summary: Add helper testing methods for instance weighting Key: SPARK-17772 URL: https://issues.apache.org/jira/browse/SPARK-17772 Project: Spark Iss

[jira] [Commented] (SPARK-17758) Spark Aggregate function LAST returns null on an empty partition

[ https://issues.apache.org/jira/browse/SPARK-17758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543488#comment-15543488 ] Franck Tago commented on SPARK-17758: - [~hvanhovell] It should return on null in this

[jira] [Commented] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543539#comment-15543539 ] Alex Liu commented on SPARK-17767: -- It looks good to me. How about hive thrift server,

[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543555#comment-15543555 ] Don Drake commented on SPARK-16845: --- I just hit this bug as well. Are there any sugges

[jira] [Commented] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543613#comment-15543613 ] Dongjoon Hyun commented on SPARK-17767: --- Thank you for confirming. For Spark Thrif

[jira] [Updated] (SPARK-17759) FairSchedulableBuilder should avoid to create duplicate fair scheduler-pools.

[ https://issues.apache.org/jira/browse/SPARK-17759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-17759: --- Affects Version/s: (was: 2.1.0) 2.0.0 > FairSchedulableBuilder sho

[jira] [Comment Edited] (SPARK-17758) Spark Aggregate function LAST returns null on an empty partition

[ https://issues.apache.org/jira/browse/SPARK-17758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543488#comment-15543488 ] Franck Tago edited comment on SPARK-17758 at 10/3/16 11:04 PM:

[jira] [Comment Edited] (SPARK-17758) Spark Aggregate function LAST returns null on an empty partition

[ https://issues.apache.org/jira/browse/SPARK-17758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543488#comment-15543488 ] Franck Tago edited comment on SPARK-17758 at 10/3/16 11:04 PM:

[jira] [Updated] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17767: Issue Type: Sub-task (was: Improvement) Parent: SPARK-15777 > Spark SQL ExternalCatalog AP

[jira] [Commented] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543733#comment-15543733 ] Reynold Xin commented on SPARK-17767: - Guys - while I think this is very useful to do

[jira] [Comment Edited] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543733#comment-15543733 ] Reynold Xin edited comment on SPARK-17767 at 10/3/16 11:22 PM:

[jira] [Closed] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-17767. --- Resolution: Later > Spark SQL ExternalCatalog API custom implementation support > ---

[jira] [Commented] (SPARK-17758) Spark Aggregate function LAST returns null on an empty partition

[ https://issues.apache.org/jira/browse/SPARK-17758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543738#comment-15543738 ] Herman van Hovell commented on SPARK-17758: --- Yeah, you have a point there. It s

[jira] [Commented] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support

[ https://issues.apache.org/jira/browse/SPARK-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543802#comment-15543802 ] Dongjoon Hyun commented on SPARK-17767: --- Yep. I closed the PR. > Spark SQL Externa

  1   2   >