[jira] [Assigned] (SPARK-24135) [K8s] Executors that fail to start up because of init-container errors are not retried and limit the executor pool size

[ https://issues.apache.org/jira/browse/SPARK-24135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-24135: -- Assignee: (was: Marcelo Vanzin) > [K8s] Executors that fail to start up because

[jira] [Assigned] (SPARK-26995) Running Spark in Docker image with Alpine Linux 3.9.0 throws errors when using snappy

[ https://issues.apache.org/jira/browse/SPARK-26995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-26995: -- Assignee: Luca Canali > Running Spark in Docker image with Alpine Linux 3.9.0 throws

[jira] [Reopened] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi reopened SPARK-27027: --- > from_avro function does not deserialize the Avro record of a struct column > type correctly

[jira] [Commented] (SPARK-27005) Design sketch: Accelerator-aware scheduling

[ https://issues.apache.org/jira/browse/SPARK-27005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783550#comment-16783550 ] Xingbo Jiang commented on SPARK-27005: -- I updated the above document, so the Spark internal shall

[jira] [Comment Edited] (SPARK-27005) Design sketch: Accelerator-aware scheduling

[ https://issues.apache.org/jira/browse/SPARK-27005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779503#comment-16779503 ] Xingbo Jiang edited comment on SPARK-27005 at 3/4/19 4:44 PM: -- *API Changes

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783546#comment-16783546 ] Hien Luu commented on SPARK-27027: -- Hi [~hyukjin.kwon],  here is another data point.  This issue is

[jira] [Assigned] (SPARK-26016) Encoding not working when using a map / mapPartitions call

[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26016: Assignee: Apache Spark > Encoding not working when using a map / mapPartitions call >

[jira] [Assigned] (SPARK-26016) Encoding not working when using a map / mapPartitions call

[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26016: Assignee: (was: Apache Spark) > Encoding not working when using a map /

[jira] [Commented] (SPARK-26881) Scaling issue with Gramian computation for RowMatrix: too many results sent to driver

[ https://issues.apache.org/jira/browse/SPARK-26881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783526#comment-16783526 ] Rafael RENAUDIN-AVINO commented on SPARK-26881: --- Sure, just started working on it. Was

[jira] [Commented] (SPARK-26016) Encoding not working when using a map / mapPartitions call

[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783513#comment-16783513 ] Sean Owen commented on SPARK-26016: --- [~maxgekk] yeah I'm thinking of parts like

[jira] [Comment Edited] (SPARK-26809) insert overwrite directory + concat function => error

[ https://issues.apache.org/jira/browse/SPARK-26809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783494#comment-16783494 ] Ajith S edited comment on SPARK-26809 at 3/4/19 4:23 PM: - This is because 

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783509#comment-16783509 ] Gabor Somogyi commented on SPARK-27027: --- > Does that imply the bug is in the implementation of

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783505#comment-16783505 ] Gabor Somogyi commented on SPARK-27027: --- [~hyukjin.kwon] seems like the issue comes only with

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783504#comment-16783504 ] Hien Luu commented on SPARK-27027: -- Thanks for digging [~gsomogyi]. Your comment about running spark

[jira] [Comment Edited] (SPARK-27037) Pyspark Row .asDict() cannot handle MapType with a Struct as the key or value

[ https://issues.apache.org/jira/browse/SPARK-27037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783460#comment-16783460 ] Tanjin Panna edited comment on SPARK-27037 at 3/4/19 3:32 PM: --

[jira] [Assigned] (SPARK-27049) Support handling partition values in the abstraction of file source V2

[ https://issues.apache.org/jira/browse/SPARK-27049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27049: Assignee: (was: Apache Spark) > Support handling partition values in the abstraction

[jira] [Assigned] (SPARK-27049) Support handling partition values in the abstraction of file source V2

[ https://issues.apache.org/jira/browse/SPARK-27049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27049: Assignee: Apache Spark > Support handling partition values in the abstraction of file

[jira] [Commented] (SPARK-26809) insert overwrite directory + concat function => error

[ https://issues.apache.org/jira/browse/SPARK-26809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783494#comment-16783494 ] Ajith S commented on SPARK-26809: - This is because 

[jira] [Created] (SPARK-27049) Support handling partition values in the abstraction of file source V2

Gengliang Wang created SPARK-27049: -- Summary: Support handling partition values in the abstraction of file source V2 Key: SPARK-27049 URL: https://issues.apache.org/jira/browse/SPARK-27049 Project:

[jira] [Assigned] (SPARK-19712) EXISTS and Left Semi join do not produce the same plan

[ https://issues.apache.org/jira/browse/SPARK-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19712: Assignee: (was: Apache Spark) > EXISTS and Left Semi join do not produce the same

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783478#comment-16783478 ] Gabor Somogyi commented on SPARK-27027: --- What I've found is really interesting. The following

[jira] [Assigned] (SPARK-19712) EXISTS and Left Semi join do not produce the same plan

[ https://issues.apache.org/jira/browse/SPARK-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19712: Assignee: Apache Spark > EXISTS and Left Semi join do not produce the same plan >

[jira] [Resolved] (SPARK-27046) Remove SPARK-19185 related references from documentation since its resolved

[ https://issues.apache.org/jira/browse/SPARK-27046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27046. --- Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 Issue resolved by pull

[jira] [Assigned] (SPARK-27046) Remove SPARK-19185 related references from documentation since its resolved

[ https://issues.apache.org/jira/browse/SPARK-27046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27046: - Assignee: Gabor Somogyi > Remove SPARK-19185 related references from documentation since its

[jira] [Commented] (SPARK-27037) Pyspark Row .asDict() cannot handle MapType with a Struct as the key or value

[ https://issues.apache.org/jira/browse/SPARK-27037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783460#comment-16783460 ] Tanjin Panna commented on SPARK-27037: -- [~hyukjin.kwon] what about the key to the map? You can see

[jira] [Commented] (SPARK-26972) Issue with CSV import and inferSchema set to true

[ https://issues.apache.org/jira/browse/SPARK-26972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783440#comment-16783440 ] Sean Owen commented on SPARK-26972: --- One explanation for your comment about "multiline" support is

[jira] [Comment Edited] (SPARK-26994) Enhance StructField to accept number format or date format

[ https://issues.apache.org/jira/browse/SPARK-26994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783425#comment-16783425 ] Murali Aakula edited comment on SPARK-26994 at 3/4/19 2:39 PM: --- We could

[jira] [Updated] (SPARK-27046) Remove SPARK-19185 related references from documentation since its resolved

[ https://issues.apache.org/jira/browse/SPARK-27046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-27046: -- Component/s: Documentation > Remove SPARK-19185 related references from documentation since

[jira] [Commented] (SPARK-26994) Enhance StructField to accept number format or date format

[ https://issues.apache.org/jira/browse/SPARK-26994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783425#comment-16783425 ] Murali Aakula commented on SPARK-26994: --- We could have defined that format conversion in the

[jira] [Resolved] (SPARK-27040) Avoid using unnecessary JoinRow in FileFormat

[ https://issues.apache.org/jira/browse/SPARK-27040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27040. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23953

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783416#comment-16783416 ] Gabor Somogyi commented on SPARK-27027: --- OK, started to dig and write it here when I've found

[jira] [Assigned] (SPARK-27040) Avoid using unnecessary JoinRow in FileFormat

[ https://issues.apache.org/jira/browse/SPARK-27040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27040: --- Assignee: Gengliang Wang > Avoid using unnecessary JoinRow in FileFormat >

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783411#comment-16783411 ] Hyukjin Kwon commented on SPARK-27027: -- I haven't looked through which one resolved this. It would

[jira] [Resolved] (SPARK-26965) Makes ElementAt nullability more precise

[ https://issues.apache.org/jira/browse/SPARK-26965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26965. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23867

[jira] [Assigned] (SPARK-26965) Makes ElementAt nullability more precise

[ https://issues.apache.org/jira/browse/SPARK-26965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26965: --- Assignee: Takeshi Yamamuro > Makes ElementAt nullability more precise >

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783388#comment-16783388 ] Gabor Somogyi commented on SPARK-27027: --- [~hyukjin.kwon] excellent. Can you put the commit here

[jira] [Updated] (SPARK-27048) A way to execute functions on Executor Startup and Executor Exit in Standalone

[ https://issues.apache.org/jira/browse/SPARK-27048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ross Brigoli updated SPARK-27048: - Component/s: (was: Spark Core) > A way to execute functions on Executor Startup and

[jira] [Updated] (SPARK-27048) A way to execute functions on Executor Startup and Executor Exit in Standalone

[ https://issues.apache.org/jira/browse/SPARK-27048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ross Brigoli updated SPARK-27048: - Description: *Background* We have a Spark Standalone ETL workload that is heavily dependent on

[jira] [Created] (SPARK-27048) A way to execute functions on Executor Startup and Executor Exit in Standalone

Ross Brigoli created SPARK-27048: Summary: A way to execute functions on Executor Startup and Executor Exit in Standalone Key: SPARK-27048 URL: https://issues.apache.org/jira/browse/SPARK-27048

[jira] [Commented] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783363#comment-16783363 ] Apache Spark commented on SPARK-19185: -- User 'gaborgsomogyi' has created a pull request for this

[jira] [Updated] (SPARK-27047) Document stop-slave.sh in spark-standalone

[ https://issues.apache.org/jira/browse/SPARK-27047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27047: Affects Version/s: (was: 2.3.3) > Document stop-slave.sh in spark-standalone >

[jira] [Assigned] (SPARK-27047) Document stop-slave.sh in spark-standalone

[ https://issues.apache.org/jira/browse/SPARK-27047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27047: Assignee: (was: Apache Spark) > Document stop-slave.sh in spark-standalone >

[jira] [Assigned] (SPARK-27047) Document stop-slave.sh in spark-standalone

[ https://issues.apache.org/jira/browse/SPARK-27047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27047: Assignee: Apache Spark > Document stop-slave.sh in spark-standalone >

[jira] [Created] (SPARK-27047) Document stop-slave.sh in spark-standalone

Ajith S created SPARK-27047: --- Summary: Document stop-slave.sh in spark-standalone Key: SPARK-27047 URL: https://issues.apache.org/jira/browse/SPARK-27047 Project: Spark Issue Type: Documentation

[jira] [Assigned] (SPARK-27046) Remove SPARK-19185 related references from documentation since its resolved

[ https://issues.apache.org/jira/browse/SPARK-27046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27046: Assignee: Apache Spark > Remove SPARK-19185 related references from documentation since

[jira] [Assigned] (SPARK-27046) Remove SPARK-19185 related references from documentation since its resolved

[ https://issues.apache.org/jira/browse/SPARK-27046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27046: Assignee: (was: Apache Spark) > Remove SPARK-19185 related references from

[jira] [Assigned] (SPARK-27045) SQL tab in UI shows callsite instead of actual SQL

[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27045: Assignee: Apache Spark > SQL tab in UI shows callsite instead of actual SQL >

[jira] [Created] (SPARK-27046) Remove SPARK-19185 related references from documentation since its resolved

Gabor Somogyi created SPARK-27046: - Summary: Remove SPARK-19185 related references from documentation since its resolved Key: SPARK-27046 URL: https://issues.apache.org/jira/browse/SPARK-27046

[jira] [Commented] (SPARK-27045) SQL tab in UI shows callsite instead of actual SQL

[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783351#comment-16783351 ] Ajith S commented on SPARK-27045: - I will be working on this issue > SQL tab in UI shows callsite

[jira] [Commented] (SPARK-27039) toPandas with Arrow swallows maxResultSize errors

[ https://issues.apache.org/jira/browse/SPARK-27039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783352#comment-16783352 ] peay commented on SPARK-27039: -- Oops, sorry, I've edited the title. I meant _Arrow_, not Avro. Maybe

[jira] [Updated] (SPARK-27039) toPandas with Arrow swallows maxResultSize errors

[ https://issues.apache.org/jira/browse/SPARK-27039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] peay updated SPARK-27039: - Summary: toPandas with Arrow swallows maxResultSize errors (was: toPandas with Avro swallows maxResultSize

[jira] [Updated] (SPARK-27045) SQL tab in UI shows callsite instead of actual SQL

[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27045: Description: When we run sql in spark ( for example via thrift server), the SparkUI SQL tab must show

[jira] [Updated] (SPARK-27045) SQL tab in UI shows callsite instead of actual SQL

[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27045: Attachment: image-2019-03-04-18-24-27-469.png > SQL tab in UI shows callsite instead of actual SQL >

[jira] [Assigned] (SPARK-27045) SQL tab in UI shows callsite instead of actual SQL

[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27045: Assignee: (was: Apache Spark) > SQL tab in UI shows callsite instead of actual SQL >

[jira] [Updated] (SPARK-27045) SQL tab in UI shows callsite instead of actual SQL

[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27045: Description: When we run sql in spark ( for example via thrift server), the SparkUI SQL tab must show

[jira] [Updated] (SPARK-27045) SQL tab in UI shows callsite instead of actual SQL

[ https://issues.apache.org/jira/browse/SPARK-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-27045: Attachment: image-2019-03-04-18-24-54-053.png > SQL tab in UI shows callsite instead of actual SQL >

[jira] [Created] (SPARK-27045) SQL tab in UI shows callsite instead of actual SQL

Ajith S created SPARK-27045: --- Summary: SQL tab in UI shows callsite instead of actual SQL Key: SPARK-27045 URL: https://issues.apache.org/jira/browse/SPARK-27045 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783337#comment-16783337 ] Gabor Somogyi commented on SPARK-26727: --- I think the trick is not executing things in a loop but

[jira] [Commented] (SPARK-26972) Issue with CSV import and inferSchema set to true

[ https://issues.apache.org/jira/browse/SPARK-26972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783262#comment-16783262 ] Jean Georges Perrin commented on SPARK-26972: - [~srowen], with all respect, why do you see

[jira] [Commented] (SPARK-26727) CREATE OR REPLACE VIEW query fails with TableAlreadyExistsException

[ https://issues.apache.org/jira/browse/SPARK-26727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783332#comment-16783332 ] Udbhav Agrawal commented on SPARK-26727: thanks [~gsomogyi] for the info. I was trying to

[jira] [Commented] (SPARK-26972) Issue with CSV import and inferSchema set to true

[ https://issues.apache.org/jira/browse/SPARK-26972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783270#comment-16783270 ] Hyukjin Kwon commented on SPARK-26972: -- Basically because the behaviour of Spark 2.4 looks more

[jira] [Resolved] (SPARK-19712) EXISTS and Left Semi join do not produce the same plan

[ https://issues.apache.org/jira/browse/SPARK-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19712. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23750

[jira] [Commented] (SPARK-19712) EXISTS and Left Semi join do not produce the same plan

[ https://issues.apache.org/jira/browse/SPARK-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783259#comment-16783259 ] Wenchen Fan commented on SPARK-19712: - The issue is not fully resolved, [~dkbiswal] will submit

[jira] [Reopened] (SPARK-19712) EXISTS and Left Semi join do not produce the same plan

[ https://issues.apache.org/jira/browse/SPARK-19712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-19712: - > EXISTS and Left Semi join do not produce the same plan >

[jira] [Created] (SPARK-27044) Maven dependency resolution does not support classifiers

Mathias Herberts created SPARK-27044: Summary: Maven dependency resolution does not support classifiers Key: SPARK-27044 URL: https://issues.apache.org/jira/browse/SPARK-27044 Project: Spark

[jira] [Commented] (SPARK-27039) toPandas with Avro swallows maxResultSize errors

[ https://issues.apache.org/jira/browse/SPARK-27039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783247#comment-16783247 ] Liang-Chi Hsieh commented on SPARK-27039: - Not sure if I miss it, but I don't see avro usage.

[jira] [Commented] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783227#comment-16783227 ] Ajith S commented on SPARK-26961: - The problem is here org.apache.spark.util.MutableURLClassLoader

[jira] [Comment Edited] (SPARK-27025) Speed up toLocalIterator

[ https://issues.apache.org/jira/browse/SPARK-27025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783220#comment-16783220 ] Erik van Oosten edited comment on SPARK-27025 at 3/4/19 10:36 AM: --

[jira] [Commented] (SPARK-27025) Speed up toLocalIterator

[ https://issues.apache.org/jira/browse/SPARK-27025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783220#comment-16783220 ] Erik van Oosten commented on SPARK-27025: - [~hyukjin.kwon] maybe I misunderstood Sean's comment.

[jira] [Commented] (SPARK-23901) Data Masking Functions

[ https://issues.apache.org/jira/browse/SPARK-23901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783210#comment-16783210 ] Gourav commented on SPARK-23901: the data masking functions are used widely in the industry for data

[jira] [Updated] (SPARK-27042) Query fails if task is failing due to corrupt cached Kafka producer

[ https://issues.apache.org/jira/browse/SPARK-27042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-27042: -- Description: If a task is failing due to a corrupt cached KafkaProducer and the task is

[jira] [Updated] (SPARK-27042) Query fails if task is failing due to corrupt cached Kafka producer

[ https://issues.apache.org/jira/browse/SPARK-27042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-27042: -- Description: If a task is failing due to a cached corrupted KafkaProducer and the task is

[jira] [Issue Comment Deleted] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-26961: Comment: was deleted (was: The problem is here org.apache.spark.util.MutableURLClassLoader (entire

[jira] [Comment Edited] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783161#comment-16783161 ] Ajith S edited comment on SPARK-26961 at 3/4/19 9:47 AM: - The problem is here

[jira] [Comment Edited] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783161#comment-16783161 ] Ajith S edited comment on SPARK-26961 at 3/4/19 9:40 AM: - The problem is here

[jira] [Assigned] (SPARK-27043) Nested schema pruning benchmark for ORC

[ https://issues.apache.org/jira/browse/SPARK-27043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27043: Assignee: (was: Apache Spark) > Nested schema pruning benchmark for ORC >

[jira] [Commented] (SPARK-26961) Found Java-level deadlock in Spark Driver

[ https://issues.apache.org/jira/browse/SPARK-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783161#comment-16783161 ] Ajith S commented on SPARK-26961: - The problem is here org.apache.spark.util.MutableURLClassLoader

[jira] [Assigned] (SPARK-27042) Query fails if task is failing due to corrupt cached Kafka producer

[ https://issues.apache.org/jira/browse/SPARK-27042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27042: Assignee: Apache Spark > Query fails if task is failing due to corrupt cached Kafka

[jira] [Assigned] (SPARK-27042) Query fails if task is failing due to corrupt cached Kafka producer

[ https://issues.apache.org/jira/browse/SPARK-27042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27042: Assignee: (was: Apache Spark) > Query fails if task is failing due to corrupt cached

[jira] [Assigned] (SPARK-27043) Nested schema pruning benchmark for ORC

[ https://issues.apache.org/jira/browse/SPARK-27043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27043: Assignee: Apache Spark > Nested schema pruning benchmark for ORC >

[jira] [Created] (SPARK-27043) Nested schema pruning benchmark for ORC

Liang-Chi Hsieh created SPARK-27043: --- Summary: Nested schema pruning benchmark for ORC Key: SPARK-27043 URL: https://issues.apache.org/jira/browse/SPARK-27043 Project: Spark Issue Type:

[jira] [Updated] (SPARK-27042) Query fails if task is failing due to corrupt cached Kafka producer

[ https://issues.apache.org/jira/browse/SPARK-27042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-27042: -- Description: If a task is failing due to a cached Kafka producer and the task is retried in

[jira] [Created] (SPARK-27042) Query fails if task is failing due to corrupt cached Kafka producer

Gabor Somogyi created SPARK-27042: - Summary: Query fails if task is failing due to corrupt cached Kafka producer Key: SPARK-27042 URL: https://issues.apache.org/jira/browse/SPARK-27042 Project: Spark

[jira] [Assigned] (SPARK-27041) large partition data cause pyspark with python2.x oom

[ https://issues.apache.org/jira/browse/SPARK-27041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27041: Assignee: Apache Spark > large partition data cause pyspark with python2.x oom >

[jira] [Assigned] (SPARK-27041) large partition data cause pyspark with python2.x oom

[ https://issues.apache.org/jira/browse/SPARK-27041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27041: Assignee: (was: Apache Spark) > large partition data cause pyspark with python2.x

[jira] [Created] (SPARK-27041) large partition data cause pyspark with python2.x oom

David Yang created SPARK-27041: -- Summary: large partition data cause pyspark with python2.x oom Key: SPARK-27041 URL: https://issues.apache.org/jira/browse/SPARK-27041 Project: Spark Issue

[jira] [Commented] (SPARK-27014) Support removal of jars and Spark binaries from Mesos driver and executor sandboxes

[ https://issues.apache.org/jira/browse/SPARK-27014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783113#comment-16783113 ] Hyukjin Kwon commented on SPARK-27014: -- 2.5.0 is not released and it will be 3.0.0. Also please

[jira] [Assigned] (SPARK-27040) Avoid using unnecessary JoinRow in FileFormat

[ https://issues.apache.org/jira/browse/SPARK-27040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27040: Assignee: Apache Spark > Avoid using unnecessary JoinRow in FileFormat >

[jira] [Assigned] (SPARK-27040) Avoid using unnecessary JoinRow in FileFormat

[ https://issues.apache.org/jira/browse/SPARK-27040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-27040: Assignee: (was: Apache Spark) > Avoid using unnecessary JoinRow in FileFormat >

[jira] [Created] (SPARK-27040) Avoid using unnecessary JoinRow in FileFormat

Gengliang Wang created SPARK-27040: -- Summary: Avoid using unnecessary JoinRow in FileFormat Key: SPARK-27040 URL: https://issues.apache.org/jira/browse/SPARK-27040 Project: Spark Issue

[jira] [Commented] (SPARK-27025) Speed up toLocalIterator

[ https://issues.apache.org/jira/browse/SPARK-27025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783122#comment-16783122 ] Hyukjin Kwon commented on SPARK-27025: -- It's one use case. How common is that use case? If not,

[jira] [Commented] (SPARK-26983) Spark PassThroughSuite,ColumnVectorSuite failure on bigendian

[ https://issues.apache.org/jira/browse/SPARK-26983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783116#comment-16783116 ] Hyukjin Kwon commented on SPARK-26983: -- There are many big endian issues. Please find and link them

[jira] [Updated] (SPARK-27014) Support removal of jars and Spark binaries from Mesos driver and executor sandboxes

[ https://issues.apache.org/jira/browse/SPARK-27014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27014: - Affects Version/s: (was: 2.5.0) 3.0.0 > Support removal of jars and

[jira] [Updated] (SPARK-27014) Support removal of jars and Spark binaries from Mesos driver and executor sandboxes

[ https://issues.apache.org/jira/browse/SPARK-27014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-27014: - Fix Version/s: (was: 2.5.0) (was: 3.0.0) > Support removal of jars

[jira] [Resolved] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-27027. -- Resolution: Cannot Reproduce > from_avro function does not deserialize the Avro record of a

[jira] [Commented] (SPARK-27027) from_avro function does not deserialize the Avro record of a struct column type correctly

[ https://issues.apache.org/jira/browse/SPARK-27027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783111#comment-16783111 ] Hyukjin Kwon commented on SPARK-27027: -- This is fixed in the current master: {code}

[jira] [Commented] (SPARK-27025) Speed up toLocalIterator

[ https://issues.apache.org/jira/browse/SPARK-27025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783110#comment-16783110 ] Erik van Oosten commented on SPARK-27025: - Thanks Sean, that is very useful. In my use case the

[jira] [Commented] (SPARK-27020) Unable to insert data with partial dynamic partition with Spark & Hive 3

[ https://issues.apache.org/jira/browse/SPARK-27020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783098#comment-16783098 ] Hyukjin Kwon commented on SPARK-27020: -- Yes, please make it self-contained reproducer if possible

[jira] [Commented] (SPARK-27025) Speed up toLocalIterator

[ https://issues.apache.org/jira/browse/SPARK-27025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783088#comment-16783088 ] Hyukjin Kwon commented on SPARK-27025: -- Yes, I think it should better be implemented in application

[jira] [Created] (SPARK-27039) toPandas with Avro swallows maxResultSize errors

peay created SPARK-27039: Summary: toPandas with Avro swallows maxResultSize errors Key: SPARK-27039 URL: https://issues.apache.org/jira/browse/SPARK-27039 Project: Spark Issue Type: Bug

<    1   2   3   >