[jira] [Assigned] (SPARK-26134) Upgrading Hadoop to 2.7.4 to fix java.version problem

[ https://issues.apache.org/jira/browse/SPARK-26134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-26134: - Assignee: Takanobu Asanuma > Upgrading Hadoop to 2.7.4 to fix java.version problem >

[jira] [Resolved] (SPARK-26134) Upgrading Hadoop to 2.7.4 to fix java.version problem

[ https://issues.apache.org/jira/browse/SPARK-26134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26134. --- Resolution: Fixed Fix Version/s: 3.0.0 This is resolved via 

[jira] [Updated] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26118: -- Fix Version/s: 2.3.3 2.2.3 > Make Jetty's requestHeaderSize configurable

[jira] [Commented] (SPARK-24553) Job UI redirect causing http 302 error

[ https://issues.apache.org/jira/browse/SPARK-24553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695601#comment-16695601 ] Apache Spark commented on SPARK-24553: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Commented] (SPARK-24553) Job UI redirect causing http 302 error

[ https://issues.apache.org/jira/browse/SPARK-24553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695604#comment-16695604 ] Apache Spark commented on SPARK-24553: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Commented] (SPARK-23410) Unable to read jsons in charset different from UTF-8

[ https://issues.apache.org/jira/browse/SPARK-23410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695565#comment-16695565 ] Hyukjin Kwon commented on SPARK-23410: -- [~x1q1j1], can you point me out the flink pr? > Unable to

[jira] [Commented] (SPARK-23410) Unable to read jsons in charset different from UTF-8

[ https://issues.apache.org/jira/browse/SPARK-23410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695539#comment-16695539 ] xuqianjin commented on SPARK-23410: --- [~maxgekk] I want to support utf-16 and utf-32 with BOMs because

[jira] [Commented] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet leads to OOM errors

[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695526#comment-16695526 ] Hyukjin Kwon commented on SPARK-26116: -- Please describe that fact in the JIRA as well. > Spark SQL

[jira] [Reopened] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet leads to OOM errors

[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-26116: -- > Spark SQL - Sort when writing partitioned parquet leads to OOM errors >

[jira] [Assigned] (SPARK-26099) Verification of the corrupt column in from_csv/from_json

[ https://issues.apache.org/jira/browse/SPARK-26099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26099: Assignee: Maxim Gekk > Verification of the corrupt column in from_csv/from_json >

[jira] [Resolved] (SPARK-26085) Key attribute of primitive type under typed aggregation should be named as "key" too

[ https://issues.apache.org/jira/browse/SPARK-26085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26085. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23054

[jira] [Resolved] (SPARK-26099) Verification of the corrupt column in from_csv/from_json

[ https://issues.apache.org/jira/browse/SPARK-26099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26099. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23070

[jira] [Assigned] (SPARK-26085) Key attribute of primitive type under typed aggregation should be named as "key" too

[ https://issues.apache.org/jira/browse/SPARK-26085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26085: --- Assignee: Liang-Chi Hsieh > Key attribute of primitive type under typed aggregation should

[jira] [Commented] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695491#comment-16695491 ] Apache Spark commented on SPARK-26118: -- User 'attilapiros' has created a pull request for this

[jira] [Commented] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695493#comment-16695493 ] Apache Spark commented on SPARK-26118: -- User 'attilapiros' has created a pull request for this

[jira] [Resolved] (SPARK-25935) Prevent null rows from JSON parser

[ https://issues.apache.org/jira/browse/SPARK-25935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25935. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22938

[jira] [Assigned] (SPARK-25935) Prevent null rows from JSON parser

[ https://issues.apache.org/jira/browse/SPARK-25935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25935: --- Assignee: Maxim Gekk > Prevent null rows from JSON parser >

[jira] [Commented] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695449#comment-16695449 ] Apache Spark commented on SPARK-26118: -- User 'attilapiros' has created a pull request for this

[jira] [Commented] (SPARK-26118) Make Jetty's requestHeaderSize configurable in Spark

[ https://issues.apache.org/jira/browse/SPARK-26118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695448#comment-16695448 ] Apache Spark commented on SPARK-26118: -- User 'attilapiros' has created a pull request for this

[jira] [Comment Edited] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695427#comment-16695427 ] Ruslan Dautkhanov edited comment on SPARK-26019 at 11/22/18 12:42 AM:

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695427#comment-16695427 ] Ruslan Dautkhanov commented on SPARK-26019: --- Thank you [~irashid] I confirm that swapping

[jira] [Assigned] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26019: Assignee: (was: Apache Spark) > pyspark/accumulators.py: "TypeError: object of type

[jira] [Commented] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695421#comment-16695421 ] Apache Spark commented on SPARK-26019: -- User 'Tagar' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26019: Assignee: Apache Spark > pyspark/accumulators.py: "TypeError: object of type 'NoneType'

[jira] [Reopened] (SPARK-26019) pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" in authenticate_and_accum_updates()

[ https://issues.apache.org/jira/browse/SPARK-26019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Dautkhanov reopened SPARK-26019: --- > pyspark/accumulators.py: "TypeError: object of type 'NoneType' has no len()" > in

[jira] [Resolved] (SPARK-26106) Prioritizes ML unittests over the doctests in PySpark

[ https://issues.apache.org/jira/browse/SPARK-26106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26106. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23078

[jira] [Assigned] (SPARK-26106) Prioritizes ML unittests over the doctests in PySpark

[ https://issues.apache.org/jira/browse/SPARK-26106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26106: Assignee: Hyukjin Kwon > Prioritizes ML unittests over the doctests in PySpark >

[jira] [Resolved] (SPARK-25957) Skip building spark-r docker image if spark distribution does not have R support

[ https://issues.apache.org/jira/browse/SPARK-25957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah resolved SPARK-25957. Resolution: Fixed Fix Version/s: 3.0.0 > Skip building spark-r docker image if spark

[jira] [Commented] (SPARK-22865) Publish Official Apache Spark Docker images

[ https://issues.apache.org/jira/browse/SPARK-22865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695338#comment-16695338 ] Andrew Korzhuev commented on SPARK-22865: - I've added builds for vanilla 2.3.1, 2.3.2 and 2.4.0

[jira] [Assigned] (SPARK-26127) Remove deprecated setters from tree regression and classification models

[ https://issues.apache.org/jira/browse/SPARK-26127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26127: - Assignee: Marco Gaido > Remove deprecated setters from tree regression and classification

[jira] [Resolved] (SPARK-26127) Remove deprecated setters from tree regression and classification models

[ https://issues.apache.org/jira/browse/SPARK-26127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26127. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23093

[jira] [Comment Edited] (SPARK-22865) Publish Official Apache Spark Docker images

[ https://issues.apache.org/jira/browse/SPARK-22865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695338#comment-16695338 ] Andrew Korzhuev edited comment on SPARK-22865 at 11/21/18 11:00 PM:

[jira] [Comment Edited] (SPARK-22865) Publish Official Apache Spark Docker images

[ https://issues.apache.org/jira/browse/SPARK-22865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695338#comment-16695338 ] Andrew Korzhuev edited comment on SPARK-22865 at 11/21/18 11:00 PM:

[jira] [Comment Edited] (SPARK-22865) Publish Official Apache Spark Docker images

[ https://issues.apache.org/jira/browse/SPARK-22865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695338#comment-16695338 ] Andrew Korzhuev edited comment on SPARK-22865 at 11/21/18 10:59 PM:

[jira] [Commented] (SPARK-26129) Instrumentation for query planning time

[ https://issues.apache.org/jira/browse/SPARK-26129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695316#comment-16695316 ] Apache Spark commented on SPARK-26129: -- User 'rxin' has created a pull request for this issue:

[jira] [Commented] (SPARK-26075) Cannot broadcast the table that is larger than 8GB : Spark 2.3

[ https://issues.apache.org/jira/browse/SPARK-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695261#comment-16695261 ] Maxim Gekk commented on SPARK-26075: The restriction of 8GB still exists

[jira] [Commented] (SPARK-26069) Flaky test: RpcIntegrationSuite.sendRpcWithStreamFailures

[ https://issues.apache.org/jira/browse/SPARK-26069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695201#comment-16695201 ] Apache Spark commented on SPARK-26069: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25993) Add test cases for resolution of ORC table location

[ https://issues.apache.org/jira/browse/SPARK-25993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25993: Assignee: Apache Spark > Add test cases for resolution of ORC table location >

[jira] [Commented] (SPARK-25993) Add test cases for resolution of ORC table location

[ https://issues.apache.org/jira/browse/SPARK-25993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695194#comment-16695194 ] Apache Spark commented on SPARK-25993: -- User 'kevinyu98' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25993) Add test cases for resolution of ORC table location

[ https://issues.apache.org/jira/browse/SPARK-25993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25993: Assignee: (was: Apache Spark) > Add test cases for resolution of ORC table location

[jira] [Commented] (SPARK-25153) Improve error messages for columns with dots/periods

[ https://issues.apache.org/jira/browse/SPARK-25153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695162#comment-16695162 ] Bradley LaVigne commented on SPARK-25153: - I'll take a crack at this one; I took a look at the

[jira] [Updated] (SPARK-26143) Shuffle shuffle default storage level

[ https://issues.apache.org/jira/browse/SPARK-26143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Avi minsky updated SPARK-26143: --- Summary: Shuffle shuffle default storage level (was: Shuffle shuffle default persist type) >

[jira] [Updated] (SPARK-26143) Shuffle shuffle default persist type

[ https://issues.apache.org/jira/browse/SPARK-26143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Avi minsky updated SPARK-26143: --- Description: Currently developer can set storage level explicitly only on persist command but

[jira] [Created] (SPARK-26143) Shuffle shuffle default persist type

Avi minsky created SPARK-26143: -- Summary: Shuffle shuffle default persist type Key: SPARK-26143 URL: https://issues.apache.org/jira/browse/SPARK-26143 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-26066) Moving truncatedString to sql/catalyst

[ https://issues.apache.org/jira/browse/SPARK-26066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-26066. --- Resolution: Fixed Assignee: Maxim Gekk This is resolved via 

[jira] [Updated] (SPARK-26066) Moving truncatedString to sql/catalyst

[ https://issues.apache.org/jira/browse/SPARK-26066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-26066: -- Fix Version/s: 3.0.0 > Moving truncatedString to sql/catalyst >

[jira] [Commented] (SPARK-18180) pyspark.sql.Row does not serialize well to json

[ https://issues.apache.org/jira/browse/SPARK-18180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694971#comment-16694971 ] Oleg V Korchagin commented on SPARK-18180: -- I'll take a look on this if there are no

[jira] [Updated] (SPARK-25919) Date value corrupts when tables are "ParquetHiveSerDe" formatted and target table is Partitioned

[ https://issues.apache.org/jira/browse/SPARK-25919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pawan updated SPARK-25919: -- Priority: Blocker (was: Major) > Date value corrupts when tables are "ParquetHiveSerDe" formatted and target

[jira] [Assigned] (SPARK-26141) Enable custom shuffle metrics implementation in shuffle write

[ https://issues.apache.org/jira/browse/SPARK-26141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26141: Assignee: Reynold Xin (was: Apache Spark) > Enable custom shuffle metrics

[jira] [Updated] (SPARK-26141) Enable custom shuffle metrics implementation in shuffle write

[ https://issues.apache.org/jira/browse/SPARK-26141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26141: Summary: Enable custom shuffle metrics implementation in shuffle write (was: Enable passing in

[jira] [Commented] (SPARK-26141) Enable custom shuffle metrics implementation in shuffle write

[ https://issues.apache.org/jira/browse/SPARK-26141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694878#comment-16694878 ] Apache Spark commented on SPARK-26141: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26141) Enable custom shuffle metrics implementation in shuffle write

[ https://issues.apache.org/jira/browse/SPARK-26141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26141: Assignee: Apache Spark (was: Reynold Xin) > Enable custom shuffle metrics

[jira] [Created] (SPARK-26142) Implement shuffle read metrics in SQL

Reynold Xin created SPARK-26142: --- Summary: Implement shuffle read metrics in SQL Key: SPARK-26142 URL: https://issues.apache.org/jira/browse/SPARK-26142 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-8288) ScalaReflection should also try apply methods defined in companion objects when inferring schema from a Product type

[ https://issues.apache.org/jira/browse/SPARK-8288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8288. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23062

[jira] [Updated] (SPARK-26140) Enable custom shuffle metrics reporter in shuffle reader

[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Enable custom shuffle metrics reporter in shuffle reader (was: Enable custom shuffle

[jira] [Updated] (SPARK-26140) Enable custom shuffle metrics implementation in shuffle reader

[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Enable custom shuffle metrics implementation in shuffle reader (was: Enable custom

[jira] [Updated] (SPARK-26140) Enable custom shuffle metrics reporter into shuffle reader

[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Enable custom shuffle metrics reporter into shuffle reader (was: Enable passing in a

[jira] [Created] (SPARK-26141) Enable passing in custom shuffle metrics implementation in shuffle write

Reynold Xin created SPARK-26141: --- Summary: Enable passing in custom shuffle metrics implementation in shuffle write Key: SPARK-26141 URL: https://issues.apache.org/jira/browse/SPARK-26141 Project:

[jira] [Resolved] (SPARK-26109) Duration in the task summary metrics table and the task table are different

[ https://issues.apache.org/jira/browse/SPARK-26109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26109. --- Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 2.3.3

[jira] [Resolved] (SPARK-26129) Instrumentation for query planning time

[ https://issues.apache.org/jira/browse/SPARK-26129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-26129. - Resolution: Fixed Fix Version/s: 3.0.0 > Instrumentation for query planning time >

[jira] [Assigned] (SPARK-8288) ScalaReflection should also try apply methods defined in companion objects when inferring schema from a Product type

[ https://issues.apache.org/jira/browse/SPARK-8288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-8288: Assignee: Drew Robb > ScalaReflection should also try apply methods defined in companion objects

[jira] [Assigned] (SPARK-26109) Duration in the task summary metrics table and the task table are different

[ https://issues.apache.org/jira/browse/SPARK-26109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26109: - Assignee: shahid > Duration in the task summary metrics table and the task table are different

[jira] [Resolved] (SPARK-25678) SPIP: Adding support in Spark for HPC cluster manager (PBS Professional)

[ https://issues.apache.org/jira/browse/SPARK-25678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25678. --- Resolution: Won't Fix If there is more work to be done to make resource managers pluggable, I'd put

[jira] [Commented] (SPARK-26140) Enable passing in a custom shuffle metrics reporter into shuffle reader

[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694815#comment-16694815 ] Apache Spark commented on SPARK-26140: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26140) Enable passing in a custom shuffle metrics reporter into shuffle reader

[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26140: Assignee: Apache Spark (was: Reynold Xin) > Enable passing in a custom shuffle metrics

[jira] [Assigned] (SPARK-26140) Enable passing in a custom shuffle metrics reporter into shuffle reader

[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26140: Assignee: Reynold Xin (was: Apache Spark) > Enable passing in a custom shuffle metrics

[jira] [Commented] (SPARK-26140) Enable passing in a custom shuffle metrics reporter into shuffle reader

[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694813#comment-16694813 ] Apache Spark commented on SPARK-26140: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-26140) Pull TempShuffleReadMetrics creation out of shuffle layer

Reynold Xin created SPARK-26140: --- Summary: Pull TempShuffleReadMetrics creation out of shuffle layer Key: SPARK-26140 URL: https://issues.apache.org/jira/browse/SPARK-26140 Project: Spark

[jira] [Updated] (SPARK-26140) Enable passing in a custom shuffle metrics reporter into shuffle reader

[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Enable passing in a custom shuffle metrics reporter into shuffle reader (was: Allow

[jira] [Updated] (SPARK-26140) Allow passing in a custom shuffle metrics reporter into shuffle reader

[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Allow passing in a custom shuffle metrics reporter into shuffle reader (was: Pull

[jira] [Created] (SPARK-26139) Support passing shuffle metrics to exchange operator

Reynold Xin created SPARK-26139: --- Summary: Support passing shuffle metrics to exchange operator Key: SPARK-26139 URL: https://issues.apache.org/jira/browse/SPARK-26139 Project: Spark Issue

[jira] [Assigned] (SPARK-26138) LimitPushDown cross join requires maybeBushLocalLimit

[ https://issues.apache.org/jira/browse/SPARK-26138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26138: Assignee: Apache Spark > LimitPushDown cross join requires maybeBushLocalLimit >

[jira] [Assigned] (SPARK-26138) LimitPushDown cross join requires maybeBushLocalLimit

[ https://issues.apache.org/jira/browse/SPARK-26138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26138: Assignee: (was: Apache Spark) > LimitPushDown cross join requires

[jira] [Commented] (SPARK-26138) LimitPushDown cross join requires maybeBushLocalLimit

[ https://issues.apache.org/jira/browse/SPARK-26138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694712#comment-16694712 ] Apache Spark commented on SPARK-26138: -- User 'guoxiaolongzte' has created a pull request for this

[jira] [Created] (SPARK-26138) LimitPushDown cross join requires maybeBushLocalLimit

guoxiaolong created SPARK-26138: --- Summary: LimitPushDown cross join requires maybeBushLocalLimit Key: SPARK-26138 URL: https://issues.apache.org/jira/browse/SPARK-26138 Project: Spark Issue

[jira] [Assigned] (SPARK-26121) [Structured Streaming] Allow users to define prefix of Kafka's consumer group (group.id)

[ https://issues.apache.org/jira/browse/SPARK-26121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26121: Assignee: (was: Apache Spark) > [Structured Streaming] Allow users to define prefix

[jira] [Commented] (SPARK-26121) [Structured Streaming] Allow users to define prefix of Kafka's consumer group (group.id)

[ https://issues.apache.org/jira/browse/SPARK-26121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694558#comment-16694558 ] Apache Spark commented on SPARK-26121: -- User 'zouzias' has created a pull request for this issue:

[jira] [Assigned] (SPARK-26121) [Structured Streaming] Allow users to define prefix of Kafka's consumer group (group.id)

[ https://issues.apache.org/jira/browse/SPARK-26121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26121: Assignee: Apache Spark > [Structured Streaming] Allow users to define prefix of Kafka's

[jira] [Commented] (SPARK-26116) Spark SQL - Sort when writing partitioned parquet leads to OOM errors

[ https://issues.apache.org/jira/browse/SPARK-26116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694544#comment-16694544 ] Pierre Lienhart commented on SPARK-26116: - Ok so I started from a situation where I have the

[jira] [Assigned] (SPARK-26137) Linux file separator is hard coded in DependencyUtils used in deploy process

[ https://issues.apache.org/jira/browse/SPARK-26137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26137: Assignee: Apache Spark > Linux file separator is hard coded in DependencyUtils used in

[jira] [Assigned] (SPARK-26137) Linux file separator is hard coded in DependencyUtils used in deploy process

[ https://issues.apache.org/jira/browse/SPARK-26137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26137: Assignee: (was: Apache Spark) > Linux file separator is hard coded in

[jira] [Commented] (SPARK-26137) Linux file separator is hard coded in DependencyUtils used in deploy process

[ https://issues.apache.org/jira/browse/SPARK-26137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694521#comment-16694521 ] Apache Spark commented on SPARK-26137: -- User 'markpavey' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-25829) Duplicated map keys are not handled consistently

[ https://issues.apache.org/jira/browse/SPARK-25829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16663118#comment-16663118 ] Wenchen Fan edited comment on SPARK-25829 at 11/21/18 10:06 AM: More

[jira] [Resolved] (SPARK-25599) Stateful aggregation in PySpark

[ https://issues.apache.org/jira/browse/SPARK-25599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25599. -- Resolution: Duplicate > Stateful aggregation in PySpark > --- > >

[jira] [Created] (SPARK-26137) Linux file separator is hard coded in DependencyUtils used in deploy process

Mark Pavey created SPARK-26137: -- Summary: Linux file separator is hard coded in DependencyUtils used in deploy process Key: SPARK-26137 URL: https://issues.apache.org/jira/browse/SPARK-26137 Project:

[jira] [Commented] (SPARK-26136) Row.getAs return null value in some condition

[ https://issues.apache.org/jira/browse/SPARK-26136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694470#comment-16694470 ] Hyukjin Kwon commented on SPARK-26136: -- That's minor tho. Please reopen and go ahead for a PR if

[jira] [Commented] (SPARK-26136) Row.getAs return null value in some condition

[ https://issues.apache.org/jira/browse/SPARK-26136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1669#comment-1669 ] Charlie Feng commented on SPARK-26136: -- Thanks for quick feedback. > Row.getAs return null value

[jira] [Commented] (SPARK-26136) Row.getAs return null value in some condition

[ https://issues.apache.org/jira/browse/SPARK-26136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694464#comment-16694464 ] Charlie Feng commented on SPARK-26136: -- And I'm thinking when row.getAs() can't infer the type

[jira] [Issue Comment Deleted] (SPARK-26136) Row.getAs return null value in some condition

[ https://issues.apache.org/jira/browse/SPARK-26136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charlie Feng updated SPARK-26136: - Comment: was deleted (was: And I'm thinking when row.getAs() can't infer the type correctly, it

[jira] [Commented] (SPARK-26136) Row.getAs return null value in some condition

[ https://issues.apache.org/jira/browse/SPARK-26136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694459#comment-16694459 ] Charlie Feng commented on SPARK-26136: -- And I'm thinking when row.getAs() can't infer the type

[jira] [Resolved] (SPARK-26136) Row.getAs return null value in some condition

[ https://issues.apache.org/jira/browse/SPARK-26136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26136. -- Resolution: Invalid For questions, please ask to mailing list next time. When filing an

[jira] [Commented] (SPARK-26136) Row.getAs return null value in some condition

[ https://issues.apache.org/jira/browse/SPARK-26136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694427#comment-16694427 ] Hyukjin Kwon commented on SPARK-26136: -- Type should be specified {{row.getAs[String]("A")}};

[jira] [Updated] (SPARK-26136) Row.getAs return null value in some condition

[ https://issues.apache.org/jira/browse/SPARK-26136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26136: - Description: {{Row.getAs("fieldName")}} will return null value when all below conditions met:

[jira] [Updated] (SPARK-26136) Row.getAs return null value in some condition

[ https://issues.apache.org/jira/browse/SPARK-26136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26136: - Description: {{Row.getAs("fieldName")}} will return null value when all below conditions met:

[jira] [Reopened] (SPARK-26108) Support custom lineSep in CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-26108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-26108: -- > Support custom lineSep in CSV datasource > > >

[jira] [Resolved] (SPARK-26102) Common CSV/JSON functions tests

[ https://issues.apache.org/jira/browse/SPARK-26102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26102. -- Resolution: Won't Fix > Common CSV/JSON functions tests > --- > >

[jira] [Assigned] (SPARK-26108) Support custom lineSep in CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-26108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26108: Assignee: (was: Apache Spark) > Support custom lineSep in CSV datasource >

[jira] [Updated] (SPARK-26136) Row.getAs return null value in some condition

[ https://issues.apache.org/jira/browse/SPARK-26136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26136: - Docs Text: (was: import org.apache.spark.sql.SparkSession object FlatMapGetAsBug { def

[jira] [Assigned] (SPARK-26108) Support custom lineSep in CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-26108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26108: Assignee: Apache Spark > Support custom lineSep in CSV datasource >

[jira] [Resolved] (SPARK-26108) Support custom lineSep in CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-26108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26108. -- Resolution: Won't Fix > Support custom lineSep in CSV datasource >

  1   2   >