[jira] [Commented] (SPARK-25137) NumberFormatException` when starting spark-shell from Mac terminal

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583356#comment-16583356 ] Apache Spark commented on SPARK-25137: -- User 'vinodkc' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25137) NumberFormatException` when starting spark-shell from Mac terminal

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25137: Assignee: (was: Apache Spark) > NumberFormatException` when starting spark-shell

[jira] [Assigned] (SPARK-25137) NumberFormatException` when starting spark-shell from Mac terminal

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25137: Assignee: Apache Spark > NumberFormatException` when starting spark-shell from Mac

[jira] [Created] (SPARK-25137) NumberFormatException` when starting spark-shell from Mac terminal

2018-08-16 Thread Vinod KC (JIRA)
Vinod KC created SPARK-25137: Summary: NumberFormatException` when starting spark-shell from Mac terminal Key: SPARK-25137 URL: https://issues.apache.org/jira/browse/SPARK-25137 Project: Spark

[jira] [Resolved] (SPARK-25122) Deduplication of supports equals code

2018-08-16 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25122. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22110

[jira] [Assigned] (SPARK-25122) Deduplication of supports equals code

2018-08-16 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25122: --- Assignee: Marek Novotny > Deduplication of supports equals code >

[jira] [Commented] (SPARK-25130) [Python] Wrong timestamp returned by toPandas

2018-08-16 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583320#comment-16583320 ] Hyukjin Kwon commented on SPARK-25130: -- If the problem is specific to some date range, it looks

[jira] [Updated] (SPARK-25130) [Python] Wrong timestamp returned by toPandas

2018-08-16 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25130: - Component/s: (was: Input/Output) PySpark > [Python] Wrong timestamp

[jira] [Commented] (SPARK-25132) Spark returns NULL for a column whose Hive metastore schema and Parquet schema are in different letter cases

2018-08-16 Thread Chenxiao Mao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583305#comment-16583305 ] Chenxiao Mao commented on SPARK-25132: -- I'm working on this. > Spark returns NULL for a column

[jira] [Resolved] (SPARK-23940) High-order function: transform_values(map, function) → map

2018-08-16 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-23940. --- Resolution: Fixed Assignee: Neha Patil Fix Version/s: 2.4.0 Issue resolved

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2018-08-16 Thread Vijay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583290#comment-16583290 ] Vijay commented on SPARK-6305: -- Hello, I have a question and need help. I am using Spark 2 version. My

[jira] [Commented] (SPARK-25040) Empty string for double and float types should be nulls in JSON

2018-08-16 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583260#comment-16583260 ] Hyukjin Kwon commented on SPARK-25040: -- We will fix this by disallowing empty string as nulls in

[jira] [Updated] (SPARK-25040) Empty string for double and float types should be nulls in JSON

2018-08-16 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25040: - Target Version/s: 3.0.0 > Empty string for double and float types should be nulls in JSON >

[jira] [Commented] (SPARK-24299) UI for Structured Streaming Monitoring

2018-08-16 Thread David Winters (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583207#comment-16583207 ] David Winters commented on SPARK-24299: --- +1 The "Streaming" tab in the Spark UI is the main way

[jira] [Updated] (SPARK-25135) insert datasource table may all null when select from view

2018-08-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25135: Priority: Blocker (was: Major) > insert datasource table may all null when select from view >

[jira] [Updated] (SPARK-25135) insert datasource table may all null when select from view

2018-08-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25135: Target Version/s: 2.3.2 > insert datasource table may all null when select from view >

[jira] [Updated] (SPARK-25135) insert datasource table may all null when select from view

2018-08-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25135: Labels: correctness (was: ) > insert datasource table may all null when select from view >

[jira] [Commented] (SPARK-24838) Support uncorrelated IN/EXISTS subqueries for more operators

2018-08-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583171#comment-16583171 ] Xiao Li commented on SPARK-24838: - [~dkbiswal] Can you submit a fix? > Support uncorrelated IN/EXISTS

[jira] [Resolved] (SPARK-25031) The schema of MapType can not be printed correctly

2018-08-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25031. - Resolution: Fixed Assignee: Hao Ren Fix Version/s: 2.4.0 > The schema of MapType can

[jira] [Commented] (SPARK-25132) Spark returns NULL for a column whose Hive metastore schema and Parquet schema are in different letter cases

2018-08-16 Thread yucai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583145#comment-16583145 ] yucai commented on SPARK-25132: --- [~cloud_fan] [~smilegator] [~budde] [~ekhliang], do you have any insight?

[jira] [Assigned] (SPARK-24555) logNumExamples in KMeans/BiKM/GMM/AFT/NB

2018-08-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-24555: - Assignee: zhengruifeng > logNumExamples in KMeans/BiKM/GMM/AFT/NB >

[jira] [Resolved] (SPARK-24555) logNumExamples in KMeans/BiKM/GMM/AFT/NB

2018-08-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24555. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21561

[jira] [Updated] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

2018-08-16 Thread Robert Reid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Reid updated SPARK-25136: Description: I have been attempting to work around the issue first discussed at the end of

[jira] [Created] (SPARK-25136) unable to use HDFS checkpoint directories after driver restart

2018-08-16 Thread Robert Reid (JIRA)
Robert Reid created SPARK-25136: --- Summary: unable to use HDFS checkpoint directories after driver restart Key: SPARK-25136 URL: https://issues.apache.org/jira/browse/SPARK-25136 Project: Spark

[jira] [Assigned] (SPARK-25032) Create table is failing, after dropping the database . It is not falling back to default database

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25032: Assignee: Apache Spark > Create table is failing, after dropping the database . It is

[jira] [Assigned] (SPARK-25032) Create table is failing, after dropping the database . It is not falling back to default database

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25032: Assignee: (was: Apache Spark) > Create table is failing, after dropping the database

[jira] [Commented] (SPARK-25032) Create table is failing, after dropping the database . It is not falling back to default database

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583105#comment-16583105 ] Apache Spark commented on SPARK-25032: -- User 'bomeng' has created a pull request for this issue:

[jira] [Commented] (SPARK-24771) Upgrade AVRO version from 1.7.7 to 1.8

2018-08-16 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583077#comment-16583077 ] Steve Loughran commented on SPARK-24771: All the wire stuff (e.g. to HDFS is protobuf).

[jira] [Commented] (SPARK-24771) Upgrade AVRO version from 1.7.7 to 1.8

2018-08-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583009#comment-16583009 ] Marcelo Vanzin commented on SPARK-24771: Do the Hadoop services use Avro for their protocol? I

[jira] [Resolved] (SPARK-23654) Cut jets3t as a dependency of spark-core

2018-08-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23654. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22081

[jira] [Commented] (SPARK-23938) High-order function: map_zip_with(map, map, function) → map

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582973#comment-16582973 ] Apache Spark commented on SPARK-23938: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-24924) Add mapping for built-in Avro data source

2018-08-16 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582948#comment-16582948 ] Dongjoon Hyun commented on SPARK-24924: --- [~Gengliang.Wang] . Ur, the latest consensus isn't

[jira] [Commented] (SPARK-23817) Migrate ORC file format read path to data source V2

2018-08-16 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582942#comment-16582942 ] Dongjoon Hyun commented on SPARK-23817: --- Thanks, [~Gengliang.Wang] ! I'm looking forward to see

[jira] [Updated] (SPARK-23817) Migrate ORC file format read path to data source V2

2018-08-16 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23817: -- Fix Version/s: (was: 2.4.0) > Migrate ORC file format read path to data source V2 >

[jira] [Commented] (SPARK-22236) CSV I/O: does not respect RFC 4180

2018-08-16 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582933#comment-16582933 ] Steve Loughran commented on SPARK-22236: bq. But is the implication that we can never change the

[jira] [Commented] (SPARK-22236) CSV I/O: does not respect RFC 4180

2018-08-16 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582930#comment-16582930 ] Steve Loughran commented on SPARK-22236: bq. One can always repartition after reading, and if

[jira] [Assigned] (SPARK-25135) insert datasource table may all null when select from view

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25135: Assignee: Apache Spark > insert datasource table may all null when select from view >

[jira] [Commented] (SPARK-25135) insert datasource table may all null when select from view

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582923#comment-16582923 ] Apache Spark commented on SPARK-25135: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25135) insert datasource table may all null when select from view

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25135: Assignee: (was: Apache Spark) > insert datasource table may all null when select

[jira] [Commented] (SPARK-24771) Upgrade AVRO version from 1.7.7 to 1.8

2018-08-16 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582911#comment-16582911 ] Steve Loughran commented on SPARK-24771: Linking to the previous PR, as that's got the

[jira] [Updated] (SPARK-25135) insert datasource table may all null when select from view

2018-08-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-25135: Affects Version/s: (was: 2.4.0) 2.3.0 2.3.1 >

[jira] [Created] (SPARK-25135) insert datasource table may all null when select from view

2018-08-16 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-25135: --- Summary: insert datasource table may all null when select from view Key: SPARK-25135 URL: https://issues.apache.org/jira/browse/SPARK-25135 Project: Spark

[jira] [Updated] (SPARK-24909) Spark scheduler can hang when fetch failures, executor lost, task running on lost executor, and multiple stage attempts

2018-08-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-24909: -- Description: The DAGScheduler can hang if the executor was lost (due to fetch failure) and

[jira] [Updated] (SPARK-24909) Spark scheduler can hang when fetch failures, executor lost, task running on lost executor, and multiple stage attempts

2018-08-16 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-24909: -- Description: The DAGScheduler can hang if the executor was lost (due to fetch failure) and

[jira] [Assigned] (SPARK-23817) Migrate ORC file format read path to data source V2

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23817: Assignee: Gengliang Wang (was: Apache Spark) > Migrate ORC file format read path to

[jira] [Assigned] (SPARK-23817) Migrate ORC file format read path to data source V2

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23817: Assignee: Apache Spark (was: Gengliang Wang) > Migrate ORC file format read path to

[jira] [Commented] (SPARK-24882) data source v2 API improvement

2018-08-16 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582760#comment-16582760 ] Ryan Blue commented on SPARK-24882: --- I'm fine separating the write modifications into their own PR; I

[jira] [Assigned] (SPARK-25134) Csv column pruning with checking of headers throws incorrect error

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25134: Assignee: Apache Spark > Csv column pruning with checking of headers throws incorrect

[jira] [Assigned] (SPARK-25134) Csv column pruning with checking of headers throws incorrect error

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25134: Assignee: (was: Apache Spark) > Csv column pruning with checking of headers throws

[jira] [Commented] (SPARK-25134) Csv column pruning with checking of headers throws incorrect error

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582740#comment-16582740 ] Apache Spark commented on SPARK-25134: -- User 'koertkuipers' has created a pull request for this

[jira] [Commented] (SPARK-24665) Add SQLConf in PySpark to manage all sql configs

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582733#comment-16582733 ] Apache Spark commented on SPARK-24665: -- User 'xuanyuanking' has created a pull request for this

[jira] [Created] (SPARK-25134) Csv column pruning with checking of headers throws incorrect error

2018-08-16 Thread koert kuipers (JIRA)
koert kuipers created SPARK-25134: - Summary: Csv column pruning with checking of headers throws incorrect error Key: SPARK-25134 URL: https://issues.apache.org/jira/browse/SPARK-25134 Project: Spark

[jira] [Commented] (SPARK-25078) Standalone does not work with spark.authenticate.secret and deploy-mode=cluster

2018-08-16 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582686#comment-16582686 ] Imran Rashid commented on SPARK-25078: -- it might be as simple as updating

[jira] [Resolved] (SPARK-25120) EventLogListener may miss driver SparkListenerBlockManagerAdded event

2018-08-16 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25120. -- Resolution: Duplicate > EventLogListener may miss driver SparkListenerBlockManagerAdded event

[jira] [Commented] (SPARK-25132) Spark returns NULL for a column whose Hive metastore schema and Parquet schema are in different letter cases

2018-08-16 Thread yucai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582667#comment-16582667 ] yucai commented on SPARK-25132: --- If Spark allows data source case insensitive, query t2 should return

[jira] [Commented] (SPARK-25133) Documentaion: AVRO data source guide

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582612#comment-16582612 ] Apache Spark commented on SPARK-25133: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-25133) Documentaion: AVRO data source guide

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25133: Assignee: Apache Spark > Documentaion: AVRO data source guide >

[jira] [Assigned] (SPARK-25133) Documentaion: AVRO data source guide

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25133: Assignee: (was: Apache Spark) > Documentaion: AVRO data source guide >

[jira] [Created] (SPARK-25133) Documentaion: AVRO data source guide

2018-08-16 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-25133: -- Summary: Documentaion: AVRO data source guide Key: SPARK-25133 URL: https://issues.apache.org/jira/browse/SPARK-25133 Project: Spark Issue Type:

[jira] [Commented] (SPARK-25132) Spark returns NULL for a column whose Hive metastore schema and Parquet schema are in different letter cases

2018-08-16 Thread Chenxiao Mao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582606#comment-16582606 ] Chenxiao Mao commented on SPARK-25132: -- We found this PR

[jira] [Resolved] (SPARK-23932) High-order function: zip_with(array, array, function) → array

2018-08-16 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-23932. --- Resolution: Fixed Assignee: Sandeep Singh Fix Version/s: 2.4.0 Issue

[jira] [Commented] (SPARK-25093) CodeFormatter could avoid creating regex object again and again

2018-08-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582543#comment-16582543 ] Marco Gaido commented on SPARK-25093: - [~igreenfi] do you want to submit a PR for this? Otherwise I

[jira] [Created] (SPARK-25132) Spark returns NULL for a column whose Hive metastore schema and Parquet schema are in different letter cases

2018-08-16 Thread Chenxiao Mao (JIRA)
Chenxiao Mao created SPARK-25132: Summary: Spark returns NULL for a column whose Hive metastore schema and Parquet schema are in different letter cases Key: SPARK-25132 URL:

[jira] [Commented] (SPARK-25031) The schema of MapType can not be printed correctly

2018-08-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582531#comment-16582531 ] Marco Gaido commented on SPARK-25031: - ^ kindly ping [~smilegator] > The schema of MapType can not

[jira] [Updated] (SPARK-25071) BuildSide is coming not as expected with join queries

2018-08-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25071: -- Issue Type: Improvement (was: Bug) This isn't my area, so I can't comment. I think this could be

[jira] [Assigned] (SPARK-25131) Event logs missing applicationAttemptId for SparkListenerApplicationStart

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25131: Assignee: Apache Spark > Event logs missing applicationAttemptId for

[jira] [Updated] (SPARK-25131) Event logs missing applicationAttemptId for SparkListenerApplicationStart

2018-08-16 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-25131: Priority: Minor (was: Major) > Event logs missing applicationAttemptId for SparkListenerApplicationStart

[jira] [Issue Comment Deleted] (SPARK-25131) Event logs missing applicationAttemptId for SparkListenerApplicationStart

2018-08-16 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated SPARK-25131: Comment: was deleted (was: A draft fix https://github.com/apache/spark/pull/22120) > Event logs missing

[jira] [Assigned] (SPARK-25131) Event logs missing applicationAttemptId for SparkListenerApplicationStart

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25131: Assignee: (was: Apache Spark) > Event logs missing applicationAttemptId for

[jira] [Comment Edited] (SPARK-25125) Spark SQL percentile_approx takes longer than Hive version for large datasets

2018-08-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582383#comment-16582383 ] Marco Gaido edited comment on SPARK-25125 at 8/16/18 1:07 PM: -- I think his

[jira] [Comment Edited] (SPARK-25125) Spark SQL percentile_approx takes longer than Hive version for large datasets

2018-08-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582383#comment-16582383 ] Marco Gaido edited comment on SPARK-25125 at 8/16/18 1:07 PM: -- I think this

[jira] [Commented] (SPARK-25131) Event logs missing applicationAttemptId for SparkListenerApplicationStart

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582481#comment-16582481 ] Apache Spark commented on SPARK-25131: -- User 'ajithme' has created a pull request for this issue:

[jira] [Commented] (SPARK-25131) Event logs missing applicationAttemptId for SparkListenerApplicationStart

2018-08-16 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582480#comment-16582480 ] Ajith S commented on SPARK-25131: - A draft fix https://github.com/apache/spark/pull/22120 > Event logs

[jira] [Created] (SPARK-25131) Event logs missing applicationAttemptId for SparkListenerApplicationStart

2018-08-16 Thread Ajith S (JIRA)
Ajith S created SPARK-25131: --- Summary: Event logs missing applicationAttemptId for SparkListenerApplicationStart Key: SPARK-25131 URL: https://issues.apache.org/jira/browse/SPARK-25131 Project: Spark

[jira] [Comment Edited] (SPARK-25071) BuildSide is coming not as expected with join queries

2018-08-16 Thread Sujith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582419#comment-16582419 ] Sujith edited comment on SPARK-25071 at 8/16/18 12:11 PM: -- [~hyukjin.kwon]  

[jira] [Commented] (SPARK-25071) BuildSide is coming not as expected with join queries

2018-08-16 Thread Sujith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582419#comment-16582419 ] Sujith commented on SPARK-25071: [~hyukjin.kwon]  [~ZenWzh]a [~srowen] I have a doubt based on above

[jira] [Commented] (SPARK-25125) Spark SQL percentile_approx takes longer than Hive version for large datasets

2018-08-16 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582383#comment-16582383 ] Marco Gaido commented on SPARK-25125: - I think his may be a duplicate of SPARK-25125. [~myali] may

[jira] [Updated] (SPARK-25130) Wrong timestamp returned by toPandas

2018-08-16 Thread Anton Daitche (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Daitche updated SPARK-25130: -- Description: The code snippet {code:java} import datetime df =

[jira] [Updated] (SPARK-25130) [Python] Wrong timestamp returned by toPandas

2018-08-16 Thread Anton Daitche (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Daitche updated SPARK-25130: -- Summary: [Python] Wrong timestamp returned by toPandas (was: Wrong timestamp returned by

[jira] [Created] (SPARK-25130) Wrong timestamp returned by toPandas

2018-08-16 Thread Anton Daitche (JIRA)
Anton Daitche created SPARK-25130: - Summary: Wrong timestamp returned by toPandas Key: SPARK-25130 URL: https://issues.apache.org/jira/browse/SPARK-25130 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-23939) High-order function: transform_keys(map, function) → map

2018-08-16 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-23939. --- Resolution: Fixed Assignee: Neha Patil Fix Version/s: 2.4.0 Issue resolved

[jira] [Assigned] (SPARK-25129) Revert mapping of com.databricks.spark.avro

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25129: Assignee: Apache Spark > Revert mapping of com.databricks.spark.avro >

[jira] [Commented] (SPARK-25129) Revert mapping of com.databricks.spark.avro

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582035#comment-16582035 ] Apache Spark commented on SPARK-25129: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-25129) Revert mapping of com.databricks.spark.avro

2018-08-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25129: Assignee: (was: Apache Spark) > Revert mapping of com.databricks.spark.avro >

[jira] [Comment Edited] (SPARK-23194) from_json in FAILFAST mode doesn't fail fast, instead it just returns nulls

2018-08-16 Thread Denis Bolshakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582025#comment-16582025 ] Denis Bolshakov edited comment on SPARK-23194 at 8/16/18 6:33 AM: --

[jira] [Created] (SPARK-25129) Revert mapping of com.databricks.spark.avro

2018-08-16 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-25129: -- Summary: Revert mapping of com.databricks.spark.avro Key: SPARK-25129 URL: https://issues.apache.org/jira/browse/SPARK-25129 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23194) from_json in FAILFAST mode doesn't fail fast, instead it just returns nulls

2018-08-16 Thread Denis Bolshakov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582025#comment-16582025 ] Denis Bolshakov commented on SPARK-23194: - [~cloud_fan], [~hyukjin.kwon], do you have any

[jira] [Resolved] (SPARK-25082) Documentation for Spark Function expm1 is incomplete

2018-08-16 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25082. -- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22115

[jira] [Assigned] (SPARK-25082) Documentation for Spark Function expm1 is incomplete

2018-08-16 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25082: Assignee: Bo Meng > Documentation for Spark Function expm1 is incomplete >

[jira] [Commented] (SPARK-24918) Executor Plugin API

2018-08-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582009#comment-16582009 ] Felix Cheung commented on SPARK-24918: -- I'd tend to agree with opt in - too many mistakes with

[jira] [Commented] (SPARK-23817) Migrate ORC file format read path to data source V2

2018-08-16 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582005#comment-16582005 ] Gengliang Wang commented on SPARK-23817: [~dongjoon] Thanks! This issue is still open. >