[jira] [Commented] (SPARK-24665) Add SQLConf in PySpark to manage all sql configs

[ https://issues.apache.org/jira/browse/SPARK-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524603#comment-16524603 ] Apache Spark commented on SPARK-24665: -- User 'xuanyuanking' has created a pull request for this

[jira] [Assigned] (SPARK-24665) Add SQLConf in PySpark to manage all sql configs

[ https://issues.apache.org/jira/browse/SPARK-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24665: Assignee: (was: Apache Spark) > Add SQLConf in PySpark to manage all sql configs >

[jira] [Assigned] (SPARK-24665) Add SQLConf in PySpark to manage all sql configs

[ https://issues.apache.org/jira/browse/SPARK-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24665: Assignee: Apache Spark > Add SQLConf in PySpark to manage all sql configs >

[jira] [Created] (SPARK-24665) Add SQLConf in PySpark to manage all sql configs

Li Yuanjian created SPARK-24665: --- Summary: Add SQLConf in PySpark to manage all sql configs Key: SPARK-24665 URL: https://issues.apache.org/jira/browse/SPARK-24665 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24642) Add a function which infers schema from a JSON column

[ https://issues.apache.org/jira/browse/SPARK-24642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524590#comment-16524590 ] Reynold Xin commented on SPARK-24642: - Do we want this as an aggregate function? I'm thinking it's

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524554#comment-16524554 ] Xiao Li commented on SPARK-24530: - [~hyukjin.kwon]  Thanks for helping this! > Sphinx doesn't render

[jira] [Commented] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524544#comment-16524544 ] Hyukjin Kwon commented on SPARK-24530: -- Will take a look on the weekends. Please go ahead if anyone

[jira] [Comment Edited] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524544#comment-16524544 ] Hyukjin Kwon edited comment on SPARK-24530 at 6/27/18 4:01 AM: --- Will take

[jira] [Commented] (SPARK-21335) support un-aliased subquery

[ https://issues.apache.org/jira/browse/SPARK-21335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524527#comment-16524527 ] Apache Spark commented on SPARK-21335: -- User 'cnZach' has created a pull request for this issue:

[jira] [Commented] (SPARK-23927) High-order function: sequence

[ https://issues.apache.org/jira/browse/SPARK-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524519#comment-16524519 ] Apache Spark commented on SPARK-23927: -- User 'ueshin' has created a pull request for this issue:

[jira] [Updated] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and pyspark.ml docs are broken

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24530: -- Summary: Sphinx doesn't render autodoc_docstring_signature correctly (with Python 2?) and

[jira] [Updated] (SPARK-24530) Sphinx doesn't render autodoc_docstring_signature correctly (using Python 2?)

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-24530: -- Summary: Sphinx doesn't render autodoc_docstring_signature correctly (using Python 2?) (was:

[jira] [Commented] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524510#comment-16524510 ] Xiangrui Meng commented on SPARK-24530: --- Confirmed that macOS, python 3, and Sphinx v1.6.6 can

[jira] [Resolved] (SPARK-23927) High-order function: sequence

[ https://issues.apache.org/jira/browse/SPARK-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-23927. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21155

[jira] [Assigned] (SPARK-23927) High-order function: sequence

[ https://issues.apache.org/jira/browse/SPARK-23927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-23927: - Assignee: Alex Vayda > High-order function: sequence > - >

[jira] [Resolved] (SPARK-24605) size(null) should return null

[ https://issues.apache.org/jira/browse/SPARK-24605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24605. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21598

[jira] [Assigned] (SPARK-24605) size(null) should return null

[ https://issues.apache.org/jira/browse/SPARK-24605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24605: --- Assignee: Maxim Gekk > size(null) should return null > - > >

[jira] [Created] (SPARK-24664) Column support name getter

zhengruifeng created SPARK-24664: Summary: Column support name getter Key: SPARK-24664 URL: https://issues.apache.org/jira/browse/SPARK-24664 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-23014) Migrate MemorySink fully to v2

[ https://issues.apache.org/jira/browse/SPARK-23014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524481#comment-16524481 ] Richard Yu commented on SPARK-23014: Hi [~joseph.torres] Are you still working on this PR? It seems

[jira] [Resolved] (SPARK-24659) GenericArrayData.equals should respect element type differences

[ https://issues.apache.org/jira/browse/SPARK-24659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-24659. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21643

[jira] [Assigned] (SPARK-24659) GenericArrayData.equals should respect element type differences

[ https://issues.apache.org/jira/browse/SPARK-24659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-24659: --- Assignee: Kris Mok > GenericArrayData.equals should respect element type differences >

[jira] [Updated] (SPARK-24447) Pyspark RowMatrix.columnSimilarities() loses spark context

[ https://issues.apache.org/jira/browse/SPARK-24447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Perry Chu updated SPARK-24447: -- Description: The RDD behind the CoordinateMatrix returned by RowMatrix.columnSimilarities() appears

[jira] [Updated] (SPARK-24447) Pyspark RowMatrix.columnSimilarities() loses spark context

[ https://issues.apache.org/jira/browse/SPARK-24447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Perry Chu updated SPARK-24447: -- Description: The RDD behind the CoordinateMatrix returned by RowMatrix.columnSimilarities() appears

[jira] [Updated] (SPARK-24447) Pyspark RowMatrix.columnSimilarities() loses spark context

[ https://issues.apache.org/jira/browse/SPARK-24447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Perry Chu updated SPARK-24447: -- Priority: Minor (was: Major) > Pyspark RowMatrix.columnSimilarities() loses spark context >

[jira] [Created] (SPARK-24663) Flaky test: StreamingContextSuite "stop slow receiver gracefully"

Marcelo Vanzin created SPARK-24663: -- Summary: Flaky test: StreamingContextSuite "stop slow receiver gracefully" Key: SPARK-24663 URL: https://issues.apache.org/jira/browse/SPARK-24663 Project: Spark

[jira] [Commented] (SPARK-24208) Cannot resolve column in self join after applying Pandas UDF

[ https://issues.apache.org/jira/browse/SPARK-24208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524341#comment-16524341 ] Stu (Michael Stewart) commented on SPARK-24208: --- [~hyukjin.kwon] I can confirm I ran into

[jira] [Assigned] (SPARK-6237) Support uploading blocks > 2GB as a stream

[ https://issues.apache.org/jira/browse/SPARK-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-6237: - Assignee: Imran Rashid > Support uploading blocks > 2GB as a stream >

[jira] [Resolved] (SPARK-6237) Support uploading blocks > 2GB as a stream

[ https://issues.apache.org/jira/browse/SPARK-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-6237. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21346

[jira] [Created] (SPARK-24662) Structured Streaming should support LIMIT

Mukul Murthy created SPARK-24662: Summary: Structured Streaming should support LIMIT Key: SPARK-24662 URL: https://issues.apache.org/jira/browse/SPARK-24662 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-24423) Add a new option `query` for JDBC sources

[ https://issues.apache.org/jira/browse/SPARK-24423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24423. - Resolution: Fixed Assignee: Dilip Biswal Fix Version/s: 2.4.0 > Add a new option

[jira] [Comment Edited] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524233#comment-16524233 ] Dongjoon Hyun edited comment on SPARK-24530 at 6/26/18 9:49 PM:

[jira] [Commented] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524233#comment-16524233 ] Dongjoon Hyun commented on SPARK-24530: --- [~mengxr] and [~hyukjin.kwon]. My environment is macOS,

[jira] [Resolved] (SPARK-24658) Remove workaround for ANTLR bug

[ https://issues.apache.org/jira/browse/SPARK-24658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24658. - Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 2.4.0 > Remove workaround for

[jira] [Assigned] (SPARK-24537) Add array_remove / array_zip / map_from_arrays / array_distinct

[ https://issues.apache.org/jira/browse/SPARK-24537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24537: Assignee: Apache Spark > Add array_remove / array_zip / map_from_arrays / array_distinct

[jira] [Commented] (SPARK-24537) Add array_remove / array_zip / map_from_arrays / array_distinct

[ https://issues.apache.org/jira/browse/SPARK-24537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524114#comment-16524114 ] Apache Spark commented on SPARK-24537: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24537) Add array_remove / array_zip / map_from_arrays / array_distinct

[ https://issues.apache.org/jira/browse/SPARK-24537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24537: Assignee: (was: Apache Spark) > Add array_remove / array_zip / map_from_arrays /

[jira] [Issue Comment Deleted] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-24631: --- Comment: was deleted (was: User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523918#comment-16523918 ] Marcelo Vanzin commented on SPARK-24631: Sorry for the noise, pasted the wrong bug number in my

[jira] [Assigned] (SPARK-24653) Flaky test "JoinSuite.test SortMergeJoin (with spill)"

[ https://issues.apache.org/jira/browse/SPARK-24653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24653: Assignee: Apache Spark > Flaky test "JoinSuite.test SortMergeJoin (with spill)" >

[jira] [Assigned] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24631: -- Assignee: Marcelo Vanzin > Cannot up cast column from bigint to smallint as it may

[jira] [Commented] (SPARK-24653) Flaky test "JoinSuite.test SortMergeJoin (with spill)"

[ https://issues.apache.org/jira/browse/SPARK-24653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523916#comment-16523916 ] Apache Spark commented on SPARK-24653: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24631: -- Assignee: (was: Marcelo Vanzin) > Cannot up cast column from bigint to smallint

[jira] [Assigned] (SPARK-24653) Flaky test "JoinSuite.test SortMergeJoin (with spill)"

[ https://issues.apache.org/jira/browse/SPARK-24653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24653: Assignee: (was: Apache Spark) > Flaky test "JoinSuite.test SortMergeJoin (with

[jira] [Comment Edited] (SPARK-6305) Add support for log4j 2.x to Spark

[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523888#comment-16523888 ] Hari Sekhon edited comment on SPARK-6305 at 6/26/18 3:47 PM: - Log4j 2.x would

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523888#comment-16523888 ] Hari Sekhon commented on SPARK-6305: Log4j 2.x would really help with Spark logging integration to

[jira] [Updated] (SPARK-24661) Window API - using multiple fields for partitioning with WindowSpec API and dataset that is cached causes org.apache.spark.sql.catalyst.errors.package$TreeNodeException

[ https://issues.apache.org/jira/browse/SPARK-24661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mavashev updated SPARK-24661: --- Description: Steps to reproduce: Creating a data set:   {code:java} List

[jira] [Created] (SPARK-24661) Window API - using multiple fields for partitioning with WindowSpec API and dataset that is cached causes org.apache.spark.sql.catalyst.errors.package$TreeNodeException

David Mavashev created SPARK-24661: -- Summary: Window API - using multiple fields for partitioning with WindowSpec API and dataset that is cached causes org.apache.spark.sql.catalyst.errors.package$TreeNodeException Key:

[jira] [Assigned] (SPARK-24660) SHS is not showing properly errors when downloading logs

[ https://issues.apache.org/jira/browse/SPARK-24660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24660: Assignee: Apache Spark > SHS is not showing properly errors when downloading logs >

[jira] [Commented] (SPARK-24660) SHS is not showing properly errors when downloading logs

[ https://issues.apache.org/jira/browse/SPARK-24660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523788#comment-16523788 ] Apache Spark commented on SPARK-24660: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24660) SHS is not showing properly errors when downloading logs

[ https://issues.apache.org/jira/browse/SPARK-24660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24660: Assignee: (was: Apache Spark) > SHS is not showing properly errors when downloading

[jira] [Created] (SPARK-24660) SHS is not showing properly errors when downloading logs

Marco Gaido created SPARK-24660: --- Summary: SHS is not showing properly errors when downloading logs Key: SPARK-24660 URL: https://issues.apache.org/jira/browse/SPARK-24660 Project: Spark Issue

[jira] [Assigned] (SPARK-24659) GenericArrayData.equals should respect element type differences

[ https://issues.apache.org/jira/browse/SPARK-24659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24659: Assignee: Apache Spark > GenericArrayData.equals should respect element type differences

[jira] [Assigned] (SPARK-24659) GenericArrayData.equals should respect element type differences

[ https://issues.apache.org/jira/browse/SPARK-24659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24659: Assignee: (was: Apache Spark) > GenericArrayData.equals should respect element type

[jira] [Commented] (SPARK-24659) GenericArrayData.equals should respect element type differences

[ https://issues.apache.org/jira/browse/SPARK-24659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523578#comment-16523578 ] Apache Spark commented on SPARK-24659: -- User 'rednaxelafx' has created a pull request for this

[jira] [Created] (SPARK-24659) GenericArrayData.equals should respect element type differences

Kris Mok created SPARK-24659: Summary: GenericArrayData.equals should respect element type differences Key: SPARK-24659 URL: https://issues.apache.org/jira/browse/SPARK-24659 Project: Spark

[jira] [Commented] (SPARK-18649) sc.textFile(my_file).collect() raises socket.timeout on large files

[ https://issues.apache.org/jira/browse/SPARK-18649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523479#comment-16523479 ] Andrei Gorlanov commented on SPARK-18649: - Hello, I am going to take care of it. >

[jira] [Commented] (SPARK-24347) df.alias() in python API should not clear metadata by default

[ https://issues.apache.org/jira/browse/SPARK-24347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523476#comment-16523476 ] Ruben Berenguel commented on SPARK-24347: - Pinging [~hyukjin.kwon], too :) > df.alias() in

[jira] [Commented] (SPARK-24458) Invalid PythonUDF check_1(), requires attributes from more than one child

[ https://issues.apache.org/jira/browse/SPARK-24458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523475#comment-16523475 ] Ruben Berenguel commented on SPARK-24458: - Oh, big facepalm, thanks [~hyukjin.kwon]. My

[jira] [Assigned] (SPARK-22425) add output files information to EventLogger

[ https://issues.apache.org/jira/browse/SPARK-22425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22425: Assignee: (was: Apache Spark) > add output files information to EventLogger >

[jira] [Commented] (SPARK-22425) add output files information to EventLogger

[ https://issues.apache.org/jira/browse/SPARK-22425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523459#comment-16523459 ] Apache Spark commented on SPARK-22425: -- User 'voidfunction' has created a pull request for this

[jira] [Assigned] (SPARK-22425) add output files information to EventLogger

[ https://issues.apache.org/jira/browse/SPARK-22425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22425: Assignee: Apache Spark > add output files information to EventLogger >

[jira] [Commented] (SPARK-24458) Invalid PythonUDF check_1(), requires attributes from more than one child

[ https://issues.apache.org/jira/browse/SPARK-24458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523443#comment-16523443 ] Hyukjin Kwon commented on SPARK-24458: -- I usually just checkout on the tag, for example, {{git

[jira] [Commented] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523433#comment-16523433 ] Hyukjin Kwon commented on SPARK-24530: -- I have another computer: macOS, Python 2.7.14, Sphinx 1.7.2

[jira] [Commented] (SPARK-24530) pyspark.ml doesn't generate class docs correctly

[ https://issues.apache.org/jira/browse/SPARK-24530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523431#comment-16523431 ] Hyukjin Kwon commented on SPARK-24530: -- macOS, Python 2.7.14, Sphinx 1.4.1 shows: {code} class

[jira] [Commented] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523417#comment-16523417 ] Hyukjin Kwon commented on SPARK-24570: -- So you are saying {code} == SQL == SHOW TABLE EXTENDED

[jira] [Commented] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523405#comment-16523405 ] Hyukjin Kwon commented on SPARK-24647: -- (please avoid to set a fix version which is usually set

[jira] [Updated] (SPARK-24647) Sink Should Return OffsetSeqs For ProgressReporting

[ https://issues.apache.org/jira/browse/SPARK-24647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24647: - Fix Version/s: (was: 2.4.0) > Sink Should Return OffsetSeqs For ProgressReporting >

[jira] [Commented] (SPARK-24643) from_json should accept an aggregate function as schema

[ https://issues.apache.org/jira/browse/SPARK-24643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523403#comment-16523403 ] Hyukjin Kwon commented on SPARK-24643: -- SPARK-24642 is not added yet though ... > from_json

[jira] [Commented] (SPARK-24644) Pyarrow exception while running pandas_udf on pyspark 2.3.1

[ https://issues.apache.org/jira/browse/SPARK-24644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523399#comment-16523399 ] Hyukjin Kwon commented on SPARK-24644: -- Can you clarify the environment, in particular, PyArrow and

[jira] [Resolved] (SPARK-24649) SparkUDF.unapply is not backwards compatable

[ https://issues.apache.org/jira/browse/SPARK-24649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24649. -- Resolution: Invalid catalysis is considered as an internal API, and subject to change between

[jira] [Commented] (SPARK-24651) Add ability to write null values while writing JSON

[ https://issues.apache.org/jira/browse/SPARK-24651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523383#comment-16523383 ] Hyukjin Kwon commented on SPARK-24651: -- I think it's basically a duplicate of SPARK-23773. > Add

[jira] [Commented] (SPARK-24650) GroupingSet

[ https://issues.apache.org/jira/browse/SPARK-24650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523351#comment-16523351 ] Hyukjin Kwon commented on SPARK-24650: -- Please avoid to set a blocker which is usually reserved for

[jira] [Updated] (SPARK-24650) GroupingSet

[ https://issues.apache.org/jira/browse/SPARK-24650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24650: - Priority: Major (was: Blocker) > GroupingSet > --- > > Key:

[jira] [Commented] (SPARK-24528) Missing optimization for Aggregations/Windowing on a bucketed table

[ https://issues.apache.org/jira/browse/SPARK-24528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523344#comment-16523344 ] Ohad Raviv commented on SPARK-24528: Hi, well it took me some time to get to it, but here are my

[jira] [Commented] (SPARK-24631) Cannot up cast column from bigint to smallint as it may truncate

[ https://issues.apache.org/jira/browse/SPARK-24631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523284#comment-16523284 ] Sivakumar commented on SPARK-24631: --- The table that I queried was a view. That view was pointing to a

[jira] [Updated] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24570: - Description: An end-user SQL client tool (ie in the screenshot) can list tables from hiveserver2 and major DBs

[jira] [Updated] (SPARK-24570) SparkSQL - show schemas/tables in dropdowns of SQL client tools (ie Squirrel SQL, DBVisualizer.etc)

[ https://issues.apache.org/jira/browse/SPARK-24570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated SPARK-24570: - Description: An end-user SQL client tool (ie in the screenshot) can list tables from hiveserver2 and major DBs