[jira] [Created] (SPARK-26070) another implicit type coercion bug

2018-11-15 Thread Ohad Raviv (JIRA)
Ohad Raviv created SPARK-26070: -- Summary: another implicit type coercion bug Key: SPARK-26070 URL: https://issues.apache.org/jira/browse/SPARK-26070 Project: Spark Issue Type: Bug Comp

[jira] [Assigned] (SPARK-26070) another implicit type coercion bug

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26070: Assignee: Apache Spark > another implicit type coercion bug > ---

[jira] [Assigned] (SPARK-26070) another implicit type coercion bug

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26070: Assignee: (was: Apache Spark) > another implicit type coercion bug >

[jira] [Commented] (SPARK-26070) another implicit type coercion bug

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687635#comment-16687635 ] Apache Spark commented on SPARK-26070: -- User 'uzadude' has created a pull request f

[jira] [Commented] (SPARK-26021) -0.0 and 0.0 not treated consistently, doesn't match Hive

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687659#comment-16687659 ] Apache Spark commented on SPARK-26021: -- User 'adoron' has created a pull request fo

[jira] [Commented] (SPARK-26021) -0.0 and 0.0 not treated consistently, doesn't match Hive

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687660#comment-16687660 ] Apache Spark commented on SPARK-26021: -- User 'adoron' has created a pull request fo

[jira] [Assigned] (SPARK-26021) -0.0 and 0.0 not treated consistently, doesn't match Hive

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26021: Assignee: (was: Apache Spark) > -0.0 and 0.0 not treated consistently, doesn't match

[jira] [Assigned] (SPARK-26021) -0.0 and 0.0 not treated consistently, doesn't match Hive

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26021: Assignee: Apache Spark > -0.0 and 0.0 not treated consistently, doesn't match Hive >

[jira] [Resolved] (SPARK-26014) Deprecate R < 3.4 support

2018-11-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26014. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23012 [https://gi

[jira] [Commented] (SPARK-25992) Accumulators giving KeyError in pyspark

2018-11-15 Thread Abdeali Kothari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687726#comment-16687726 ] Abdeali Kothari commented on SPARK-25992: - [~hyukjin.kwon] I tried a fair bit an

[jira] [Assigned] (SPARK-25974) Optimizes Generates bytecode for ordering based on the given order

2018-11-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-25974: --- Assignee: caoxuewen > Optimizes Generates bytecode for ordering based on the given order >

[jira] [Resolved] (SPARK-25974) Optimizes Generates bytecode for ordering based on the given order

2018-11-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25974. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22976 [https://gith

[jira] [Created] (SPARK-26071) disallow map as map key

2018-11-15 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-26071: --- Summary: disallow map as map key Key: SPARK-26071 URL: https://issues.apache.org/jira/browse/SPARK-26071 Project: Spark Issue Type: Bug Components: S

[jira] [Created] (SPARK-26072) remove invalid comment as we don't use it anymore

2018-11-15 Thread caoxuewen (JIRA)
caoxuewen created SPARK-26072: - Summary: remove invalid comment as we don't use it anymore Key: SPARK-26072 URL: https://issues.apache.org/jira/browse/SPARK-26072 Project: Spark Issue Type: Impro

[jira] [Resolved] (SPARK-26018) Support Scalar subqueries in predicate push down to datasources

2018-11-15 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Gaido resolved SPARK-26018. - Resolution: Won't Fix This may be very hard to do as we now add filters to datasources from logi

[jira] [Commented] (SPARK-26073) remove invalid comment as we don't use it anymore

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687856#comment-16687856 ] Apache Spark commented on SPARK-26073: -- User 'heary-cao' has created a pull request

[jira] [Created] (SPARK-26073) remove invalid comment as we don't use it anymore

2018-11-15 Thread caoxuewen (JIRA)
caoxuewen created SPARK-26073: - Summary: remove invalid comment as we don't use it anymore Key: SPARK-26073 URL: https://issues.apache.org/jira/browse/SPARK-26073 Project: Spark Issue Type: Impro

[jira] [Assigned] (SPARK-26073) remove invalid comment as we don't use it anymore

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26073: Assignee: (was: Apache Spark) > remove invalid comment as we don't use it anymore > -

[jira] [Assigned] (SPARK-26073) remove invalid comment as we don't use it anymore

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26073: Assignee: Apache Spark > remove invalid comment as we don't use it anymore >

[jira] [Commented] (SPARK-26073) remove invalid comment as we don't use it anymore

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687857#comment-16687857 ] Apache Spark commented on SPARK-26073: -- User 'heary-cao' has created a pull request

[jira] [Assigned] (SPARK-26071) disallow map as map key

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26071: Assignee: Wenchen Fan (was: Apache Spark) > disallow map as map key > --

[jira] [Commented] (SPARK-26071) disallow map as map key

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687893#comment-16687893 ] Apache Spark commented on SPARK-26071: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-26071) disallow map as map key

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26071: Assignee: Apache Spark (was: Wenchen Fan) > disallow map as map key > --

[jira] [Commented] (SPARK-26071) disallow map as map key

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687882#comment-16687882 ] Apache Spark commented on SPARK-26071: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-23207) Shuffle+Repartition on an DataFrame could lead to incorrect answers

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687899#comment-16687899 ] Apache Spark commented on SPARK-23207: -- User 'ueshin' has created a pull request fo

[jira] [Created] (SPARK-26074) AsyncEventQueue.stop hangs when eventQueue is full

2018-11-15 Thread liupengcheng (JIRA)
liupengcheng created SPARK-26074: Summary: AsyncEventQueue.stop hangs when eventQueue is full Key: SPARK-26074 URL: https://issues.apache.org/jira/browse/SPARK-26074 Project: Spark Issue Type

[jira] [Commented] (SPARK-26074) AsyncEventQueue.stop hangs when eventQueue is full

2018-11-15 Thread liupengcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687926#comment-16687926 ] liupengcheng commented on SPARK-26074: -- I will submit a PR soon > AsyncEventQueue.

[jira] [Resolved] (SPARK-26057) Table joining is broken in Spark 2.4

2018-11-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26057. - Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 Issue resolved by pull re

[jira] [Assigned] (SPARK-26057) Table joining is broken in Spark 2.4

2018-11-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26057: --- Assignee: Marco Gaido > Table joining is broken in Spark 2.4 >

[jira] [Issue Comment Deleted] (SPARK-26074) AsyncEventQueue.stop hangs when eventQueue is full

2018-11-15 Thread liupengcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-26074: - Comment: was deleted (was: I will submit a PR soon) > AsyncEventQueue.stop hangs when eventQueu

[jira] [Updated] (SPARK-26074) AsyncEventQueue.stop hangs when eventQueue is full

2018-11-15 Thread liupengcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng updated SPARK-26074: - Affects Version/s: (was: 2.4.0) (was: 2.3.0) > AsyncEventQueue.st

[jira] [Resolved] (SPARK-26055) InterfaceStability annotations should be retained at runtime

2018-11-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26055. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23029 [https://gith

[jira] [Commented] (SPARK-26041) catalyst cuts out some columns from dataframes: org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute

2018-11-15 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687951#comment-16687951 ] Marco Gaido commented on SPARK-26041: - [~Tagar] I don't have you table definitions s

[jira] [Commented] (SPARK-26074) AsyncEventQueue.stop hangs when eventQueue is full

2018-11-15 Thread liupengcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687966#comment-16687966 ] liupengcheng commented on SPARK-26074: -- Sorry, i mischecked the code, Seems already

[jira] [Resolved] (SPARK-26074) AsyncEventQueue.stop hangs when eventQueue is full

2018-11-15 Thread liupengcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liupengcheng resolved SPARK-26074. -- Resolution: Duplicate Fix Version/s: 2.3.1 2.4.0 > AsyncEventQueue.s

[jira] [Commented] (SPARK-26063) CatalystDataToAvro gives "UnresolvedException: Invalid call to dataType on unresolved object" when requested for numberedTreeString

2018-11-15 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687971#comment-16687971 ] Marco Gaido commented on SPARK-26063: - I think this was fixed in SPARK-25883. But we

[jira] [Commented] (SPARK-26063) CatalystDataToAvro gives "UnresolvedException: Invalid call to dataType on unresolved object" when requested for numberedTreeString

2018-11-15 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687985#comment-16687985 ] Gengliang Wang commented on SPARK-26063: Hi [~mgaido], thanks for the investigat

[jira] [Commented] (SPARK-25883) Override method `prettyName` in `from_avro`/`to_avro`

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688011#comment-16688011 ] Apache Spark commented on SPARK-25883: -- User 'gengliangwang' has created a pull req

[jira] [Resolved] (SPARK-26072) remove invalid comment as we don't use it anymore

2018-11-15 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-26072. - Resolution: Duplicate > remove invalid comment as we don't use it anymore >

[jira] [Created] (SPARK-26075) Cannot broadcast the table that is larger than 8GB : Spark 2.3

2018-11-15 Thread Neeraj Bhadani (JIRA)
Neeraj Bhadani created SPARK-26075: -- Summary: Cannot broadcast the table that is larger than 8GB : Spark 2.3 Key: SPARK-26075 URL: https://issues.apache.org/jira/browse/SPARK-26075 Project: Spark

[jira] [Updated] (SPARK-22450) Safely register class for mllib

2018-11-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22450: -- Priority: Minor (was: Major) > Safely register class for mllib > --- > >

[jira] [Commented] (SPARK-23266) Matrix Inversion on BlockMatrix

2018-11-15 Thread Jose Saray (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688225#comment-16688225 ] Jose Saray commented on SPARK-23266: Hi Chandan, we have a chat by mail in June, I t

[jira] [Commented] (SPARK-26076) Revise ambiguous error message from load-spark-env.sh

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688252#comment-16688252 ] Apache Spark commented on SPARK-26076: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-26076) Revise ambiguous error message from load-spark-env.sh

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26076: Assignee: (was: Apache Spark) > Revise ambiguous error message from load-spark-env.sh

[jira] [Assigned] (SPARK-26076) Revise ambiguous error message from load-spark-env.sh

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26076: Assignee: Apache Spark > Revise ambiguous error message from load-spark-env.sh >

[jira] [Created] (SPARK-26076) Revise ambiguous error message from load-spark-env.sh

2018-11-15 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-26076: -- Summary: Revise ambiguous error message from load-spark-env.sh Key: SPARK-26076 URL: https://issues.apache.org/jira/browse/SPARK-26076 Project: Spark Iss

[jira] [Created] (SPARK-26077) Reserved SQL words are not escaped by JDBC writer for table name

2018-11-15 Thread Eugene Golovan (JIRA)
Eugene Golovan created SPARK-26077: -- Summary: Reserved SQL words are not escaped by JDBC writer for table name Key: SPARK-26077 URL: https://issues.apache.org/jira/browse/SPARK-26077 Project: Spark

[jira] [Updated] (SPARK-26044) Aggregate Metrics table is not sorted properly based on executor ID

2018-11-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26044: -- Priority: Minor (was: Major) > Aggregate Metrics table is not sorted properly based on executor ID >

[jira] [Resolved] (SPARK-26044) Aggregate Metrics table is not sorted properly based on executor ID

2018-11-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26044. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23024 [https://github.c

[jira] [Assigned] (SPARK-26044) Aggregate Metrics table is not sorted properly based on executor ID

2018-11-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26044: - Assignee: shahid > Aggregate Metrics table is not sorted properly based on executor ID > --

[jira] [Assigned] (SPARK-26011) pyspark app with "spark.jars.packages" config does not work

2018-11-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26011: - Assignee: shanyu zhao > pyspark app with "spark.jars.packages" config does not work > -

[jira] [Resolved] (SPARK-26011) pyspark app with "spark.jars.packages" config does not work

2018-11-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26011. --- Resolution: Fixed Fix Version/s: 2.4.1 3.0.0 2.3.3 Issu

[jira] [Commented] (SPARK-23886) update query.status

2018-11-15 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688335#comment-16688335 ] Gabor Somogyi commented on SPARK-23886: --- [~efim.poberezkin] as I see this Jira is

[jira] [Created] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

2018-11-15 Thread Arttu Voutilainen (JIRA)
Arttu Voutilainen created SPARK-26078: - Summary: WHERE .. IN fails to filter rows when used in combination with UNION Key: SPARK-26078 URL: https://issues.apache.org/jira/browse/SPARK-26078 Projec

[jira] [Commented] (SPARK-20236) Overwrite a partitioned data source table should only overwrite related partitions

2018-11-15 Thread Mikalai Surta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688371#comment-16688371 ] Mikalai Surta commented on SPARK-20236: --- Could you please explain why it doesn't w

[jira] [Commented] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

2018-11-15 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688460#comment-16688460 ] Matt Cheah commented on SPARK-26078: As per other correctness issues we have seen as

[jira] [Updated] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

2018-11-15 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-26078: --- Priority: Blocker (was: Major) > WHERE .. IN fails to filter rows when used in combination with UNI

[jira] [Updated] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

2018-11-15 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-26078: --- Labels: Correctness correctness (was: Correctness) > WHERE .. IN fails to filter rows when used in

[jira] [Updated] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

2018-11-15 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-26078: --- Target Version/s: 2.4.1, 3.0.0 > WHERE .. IN fails to filter rows when used in combination with UNIO

[jira] [Updated] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

2018-11-15 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-26078: --- Labels: correctness (was: Correctness correctness) > WHERE .. IN fails to filter rows when used in

[jira] [Updated] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

2018-11-15 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah updated SPARK-26078: --- Labels: Correctness (was: ) > WHERE .. IN fails to filter rows when used in combination with UNION

[jira] [Updated] (SPARK-26078) WHERE .. IN fails to filter rows when used in combination with UNION

2018-11-15 Thread Arttu Voutilainen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arttu Voutilainen updated SPARK-26078: -- Description: Hey, We encountered a case where Spark SQL does not seem to handle WHERE

[jira] [Commented] (SPARK-23886) update query.status

2018-11-15 Thread Efim Poberezkin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688509#comment-16688509 ] Efim Poberezkin commented on SPARK-23886: - [~gsomogyi] sure, feel free to take t

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

2018-11-15 Thread Sean Shubin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688533#comment-16688533 ] Sean Shubin commented on SPARK-14220: - So far I have not been able to get it working

[jira] [Created] (SPARK-26079) Flaky test: StreamingQueryListenersConfSuite

2018-11-15 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-26079: -- Summary: Flaky test: StreamingQueryListenersConfSuite Key: SPARK-26079 URL: https://issues.apache.org/jira/browse/SPARK-26079 Project: Spark Issue Type:

[jira] [Commented] (SPARK-26079) Flaky test: StreamingQueryListenersConfSuite

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688561#comment-16688561 ] Apache Spark commented on SPARK-26079: -- User 'vanzin' has created a pull request fo

[jira] [Assigned] (SPARK-26079) Flaky test: StreamingQueryListenersConfSuite

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26079: Assignee: (was: Apache Spark) > Flaky test: StreamingQueryListenersConfSuite > --

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

2018-11-15 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688564#comment-16688564 ] Stavros Kontopoulos commented on SPARK-14220: - That one was fixed here: [htt

[jira] [Comment Edited] (SPARK-14220) Build and test Spark against Scala 2.12

2018-11-15 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688564#comment-16688564 ] Stavros Kontopoulos edited comment on SPARK-14220 at 11/15/18 7:45 PM: ---

[jira] [Assigned] (SPARK-26079) Flaky test: StreamingQueryListenersConfSuite

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26079: Assignee: Apache Spark > Flaky test: StreamingQueryListenersConfSuite > -

[jira] [Created] (SPARK-26080) Unable to run worker.py on Windows

2018-11-15 Thread Hayden Jeune (JIRA)
Hayden Jeune created SPARK-26080: Summary: Unable to run worker.py on Windows Key: SPARK-26080 URL: https://issues.apache.org/jira/browse/SPARK-26080 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-26081) Do not write empty files by text datasources

2018-11-15 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-26081: -- Summary: Do not write empty files by text datasources Key: SPARK-26081 URL: https://issues.apache.org/jira/browse/SPARK-26081 Project: Spark Issue Type: Improvem

[jira] [Commented] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688654#comment-16688654 ] Apache Spark commented on SPARK-23128: -- User 'justinuang' has created a pull reques

[jira] [Commented] (SPARK-23128) A new approach to do adaptive execution in Spark SQL

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688653#comment-16688653 ] Apache Spark commented on SPARK-23128: -- User 'justinuang' has created a pull reques

[jira] [Commented] (SPARK-26081) Do not write empty files by text datasources

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688664#comment-16688664 ] Apache Spark commented on SPARK-26081: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-26081) Do not write empty files by text datasources

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26081: Assignee: Apache Spark > Do not write empty files by text datasources > -

[jira] [Assigned] (SPARK-26081) Do not write empty files by text datasources

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26081: Assignee: (was: Apache Spark) > Do not write empty files by text datasources > --

[jira] [Created] (SPARK-26082) Misnaming of spark.mesos.fetch(er)Cache.enable in MesosClusterScheduler

2018-11-15 Thread Martin Loncaric (JIRA)
Martin Loncaric created SPARK-26082: --- Summary: Misnaming of spark.mesos.fetch(er)Cache.enable in MesosClusterScheduler Key: SPARK-26082 URL: https://issues.apache.org/jira/browse/SPARK-26082 Project

[jira] [Updated] (SPARK-26082) Misnaming of spark.mesos.fetch(er)Cache.enable in MesosClusterScheduler

2018-11-15 Thread Martin Loncaric (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Loncaric updated SPARK-26082: Description: Currently in [docs|https://spark.apache.org/docs/latest/running-on-mesos.html

[jira] [Updated] (SPARK-26082) Misnaming of spark.mesos.fetch(er)Cache.enable in MesosClusterScheduler

2018-11-15 Thread Martin Loncaric (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Loncaric updated SPARK-26082: Description: Currently in [docs|https://spark.apache.org/docs/latest/running-on-mesos.html

[jira] [Updated] (SPARK-26082) Misnaming of spark.mesos.fetch(er)Cache.enable in MesosClusterScheduler

2018-11-15 Thread Martin Loncaric (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Loncaric updated SPARK-26082: Description: Currently in [docs|https://spark.apache.org/docs/latest/running-on-mesos.html

[jira] [Updated] (SPARK-25035) Replicating disk-stored blocks should avoid memory mapping

2018-11-15 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25035: - Labels: memory-analysis (was: ) > Replicating disk-stored blocks should avoid memory mapping >

[jira] [Created] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

2018-11-15 Thread Qi Shao (JIRA)
Qi Shao created SPARK-26083: --- Summary: Pyspark command is not working properly with default Docker Image build Key: SPARK-26083 URL: https://issues.apache.org/jira/browse/SPARK-26083 Project: Spark

[jira] [Updated] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

2018-11-15 Thread Qi Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Shao updated SPARK-26083: Description: When I try to run {code:java} ./bin/pyspark{code} in a pod in Kubernetes(image built without

[jira] [Updated] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

2018-11-15 Thread Qi Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Shao updated SPARK-26083: Description: When I try to run {code:java} ./bin/pyspark{code} in a pod in Kubernetes(image built without

[jira] [Assigned] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26083: Assignee: (was: Apache Spark) > Pyspark command is not working properly with default

[jira] [Commented] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688741#comment-16688741 ] Apache Spark commented on SPARK-26083: -- User 'AzureQ' has created a pull request fo

[jira] [Assigned] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26083: Assignee: Apache Spark > Pyspark command is not working properly with default Docker Imag

[jira] [Commented] (SPARK-26083) Pyspark command is not working properly with default Docker Image build

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688743#comment-16688743 ] Apache Spark commented on SPARK-26083: -- User 'AzureQ' has created a pull request fo

[jira] [Commented] (SPARK-26084) AggregateExpression.references fails on unresolved expression trees

2018-11-15 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688760#comment-16688760 ] Simeon Simeonov commented on SPARK-26084: - /cc [~maropu] [~hvanhovell] who worke

[jira] [Created] (SPARK-26084) AggregateExpression.references fails on unresolved expression trees

2018-11-15 Thread Simeon Simeonov (JIRA)
Simeon Simeonov created SPARK-26084: --- Summary: AggregateExpression.references fails on unresolved expression trees Key: SPARK-26084 URL: https://issues.apache.org/jira/browse/SPARK-26084 Project: Sp

[jira] [Assigned] (SPARK-26035) Break large streaming/tests.py files into smaller files

2018-11-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26035: Assignee: Hyukjin Kwon > Break large streaming/tests.py files into smaller files > --

[jira] [Resolved] (SPARK-26035) Break large streaming/tests.py files into smaller files

2018-11-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26035. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23034 [https://gi

[jira] [Updated] (SPARK-25883) Override method `prettyName` in `from_avro`/`to_avro`

2018-11-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25883: - Fix Version/s: 2.4.1 > Override method `prettyName` in `from_avro`/`to_avro` > -

[jira] [Assigned] (SPARK-25957) Skip building spark-r docker image if spark distribution does not have R support

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25957: Assignee: (was: Apache Spark) > Skip building spark-r docker image if spark distribut

[jira] [Assigned] (SPARK-25957) Skip building spark-r docker image if spark distribution does not have R support

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25957: Assignee: Apache Spark > Skip building spark-r docker image if spark distribution does no

[jira] [Commented] (SPARK-25957) Skip building spark-r docker image if spark distribution does not have R support

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688870#comment-16688870 ] Apache Spark commented on SPARK-25957: -- User 'ramaddepally' has created a pull requ

[jira] [Commented] (SPARK-25957) Skip building spark-r docker image if spark distribution does not have R support

2018-11-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688871#comment-16688871 ] Apache Spark commented on SPARK-25957: -- User 'ramaddepally' has created a pull requ

[jira] [Commented] (SPARK-25957) Skip building spark-r docker image if spark distribution does not have R support

2018-11-15 Thread Nagaram Prasad Addepally (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688869#comment-16688869 ] Nagaram Prasad Addepally commented on SPARK-25957: -- Posted PR [https://

[jira] [Commented] (SPARK-26080) Unable to run worker.py on Windows

2018-11-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1665#comment-1665 ] Hyukjin Kwon commented on SPARK-26080: -- We should fix this. > Unable to run worker

  1   2   >