[jira] [Updated] (SPARK-30467) On Federal Information Processing Standard (FIPS) enabled cluster, Spark Workers are not able to connect to Master.

2020-01-08 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHOBHIT SHUKLA updated SPARK-30467: --- Description: On _*Federal Information Processing Standard*_ (FIPS) enabled clusters, Spark

[jira] [Updated] (SPARK-30467) On Federal Information Processing Standard (FIPS) enabled cluster, Spark Workers are not able to connect to Master.

2020-01-08 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHOBHIT SHUKLA updated SPARK-30467: --- Description: On _*Federal Information Processing Standard*_ (FIPS) enabled clusters, Spark

[jira] [Updated] (SPARK-30467) On Federal Information Processing Standard (FIPS) enabled cluster, Spark Workers are not able to connect to Master.

2020-01-08 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHOBHIT SHUKLA updated SPARK-30467: --- Description: On _*Federal Information Processing Standard*_ (FIPS) enabled clusters, Spark

[jira] [Updated] (SPARK-30467) On Federal Information Processing Standard (FIPS) enabled cluster, Spark Workers are not able to connect to Master.

2020-01-08 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHOBHIT SHUKLA updated SPARK-30467: --- Description: On _*Federal Information Processing Standard*_ (FIPS) enabled clusters, Spark

[jira] [Updated] (SPARK-30467) On Federal Information Processing Standard (FIPS) enabled cluster, Spark Workers are not able to connect to Master.

2020-01-08 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHOBHIT SHUKLA updated SPARK-30467: --- Description: On _*Federal Information Processing Standard*_ (FIPS) enabled clusters, Spark

[jira] [Updated] (SPARK-30467) On Federal Information Processing Standard (FIPS) enabled cluster, Spark Workers are not able to connect to Master.

2020-01-08 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHOBHIT SHUKLA updated SPARK-30467: --- Description: On _*Federal Information Processing Standard*_ (FIPS) enabled clusters, Spark

[jira] [Updated] (SPARK-30467) On Federal Information Processing Standard (FIPS) enabled cluster, Spark Workers are not able to connect to Master.

2020-01-08 Thread SHOBHIT SHUKLA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHOBHIT SHUKLA updated SPARK-30467: --- Description: On _*Federal Information Processing Standard*_ (FIPS) enabled clusters, Spark

[jira] [Created] (SPARK-30467) On Federal Information Processing Standard (FIPS) enabled cluster, Spark Workers are not able to connect to Master.

2020-01-08 Thread SHOBHIT SHUKLA (Jira)
SHOBHIT SHUKLA created SPARK-30467: -- Summary: On Federal Information Processing Standard (FIPS) enabled cluster, Spark Workers are not able to connect to Master. Key: SPARK-30467 URL:

[jira] [Updated] (SPARK-30410) Calculating size of table having large number of partitions causes flooding logs

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-30410: - Fix Version/s: 2.4.5 > Calculating size of table having large number of partitions

[jira] [Commented] (SPARK-30421) Dropped columns still available for filtering

2020-01-08 Thread Tobias Hermann (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011422#comment-17011422 ] Tobias Hermann commented on SPARK-30421: [~cloud_fan] I think it is an issue because it means

[jira] [Commented] (SPARK-28478) Optimizer rule to remove unnecessary explicit null checks for null-intolerant expressions (e.g. if(x is null, x, f(x)))

2020-01-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011392#comment-17011392 ] Wenchen Fan commented on SPARK-28478: - makes sense to me. Look forward to your PR! > Optimizer rule

[jira] [Commented] (SPARK-30421) Dropped columns still available for filtering

2020-01-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011390#comment-17011390 ] Wenchen Fan commented on SPARK-30421: - is this really an issue? The current semantic seems like: you

[jira] [Comment Edited] (SPARK-30261) Should not change owner of hive table for some commands like 'alter' operation

2020-01-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011368#comment-17011368 ] Jungtaek Lim edited comment on SPARK-30261 at 1/9/20 3:28 AM: -- Just removed

[jira] [Commented] (SPARK-30261) Should not change owner of hive table for some commands like 'alter' operation

2020-01-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011368#comment-17011368 ] Jungtaek Lim commented on SPARK-30261: -- Just removed fixed versions, as it's filled when resolving

[jira] [Updated] (SPARK-30261) Should not change owner of hive table for some commands like 'alter' operation

2020-01-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-30261: - Fix Version/s: (was: 2.4.3) (was: 2.3.0) (was:

[jira] [Updated] (SPARK-30462) Structured Streaming _spark_metadata fills up Spark Driver memory when having lots of objects

2020-01-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-30462: - Affects Version/s: 3.0.0 > Structured Streaming _spark_metadata fills up Spark Driver memory

[jira] [Commented] (SPARK-28123) String Functions: Add support btrim

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011353#comment-17011353 ] Takeshi Yamamuro commented on SPARK-28123: -- Instead of adding a new func for binary, better to

[jira] [Commented] (SPARK-28076) String Functions: SUBSTRING support regular expression

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011348#comment-17011348 ] Takeshi Yamamuro commented on SPARK-28076: -- Yea, this features is implementation-specific and I

[jira] [Assigned] (SPARK-30183) Disallow to specify reserved properties in CREATE NAMESPACE syntax

2020-01-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30183: --- Assignee: Kent Yao > Disallow to specify reserved properties in CREATE NAMESPACE syntax >

[jira] [Resolved] (SPARK-28076) String Functions: SUBSTRING support regular expression

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28076. -- Resolution: Won't Fix > String Functions: SUBSTRING support regular expression >

[jira] [Resolved] (SPARK-30183) Disallow to specify reserved properties in CREATE NAMESPACE syntax

2020-01-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30183. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26806

[jira] [Resolved] (SPARK-28317) Built-in Mathematical Functions: SCALE

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28317. -- Resolution: Won't Fix > Built-in Mathematical Functions: SCALE >

[jira] [Commented] (SPARK-28317) Built-in Mathematical Functions: SCALE

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011344#comment-17011344 ] Takeshi Yamamuro commented on SPARK-28317: -- It seems this is pg-specific, so I'll close for

[jira] [Resolved] (SPARK-30464) Explicitly note that we don't add "pandas compatible" aliases

2020-01-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30464. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27142

[jira] [Assigned] (SPARK-30464) Explicitly note that we don't add "pandas compatible" aliases

2020-01-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30464: Assignee: Hyukjin Kwon > Explicitly note that we don't add "pandas compatible" aliases >

[jira] [Resolved] (SPARK-28078) String Functions: Add support other 4 REGEXP functions

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28078. -- Resolution: Won't Fix > String Functions: Add support other 4 REGEXP functions >

[jira] [Commented] (SPARK-28078) String Functions: Add support other 4 REGEXP functions

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011339#comment-17011339 ] Takeshi Yamamuro commented on SPARK-28078: -- Since this is pg-specific, I'll close this for now.

[jira] [Resolved] (SPARK-27976) Add built-in Array Functions: array_append

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-27976. -- Resolution: Later > Add built-in Array Functions: array_append >

[jira] [Updated] (SPARK-30465) bytesToString util adapts to negative number

2020-01-08 Thread SongXun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SongXun updated SPARK-30465: Description: when the input of bytesToString is negative number, the string returned  is unfriendly.

[jira] [Created] (SPARK-30466) remove dependency on jackson-mapper-asl-1.9.13 and jackson-core-asl-1.9.13

2020-01-08 Thread Michael Burgener (Jira)
Michael Burgener created SPARK-30466: Summary: remove dependency on jackson-mapper-asl-1.9.13 and jackson-core-asl-1.9.13 Key: SPARK-30466 URL: https://issues.apache.org/jira/browse/SPARK-30466

[jira] [Commented] (SPARK-27976) Add built-in Array Functions: array_append

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011335#comment-17011335 ] Takeshi Yamamuro commented on SPARK-27976: -- See: 

[jira] [Created] (SPARK-30465) bytesToString util adapts to negative number

2020-01-08 Thread SongXun (Jira)
SongXun created SPARK-30465: --- Summary: bytesToString util adapts to negative number Key: SPARK-30465 URL: https://issues.apache.org/jira/browse/SPARK-30465 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-30462) Structured Streaming _spark_metadata fills up Spark Driver memory when having lots of objects

2020-01-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011325#comment-17011325 ] Jungtaek Lim commented on SPARK-30462: -- I'm seeing many requests to fix up Spark metadata for

[jira] [Resolved] (SPARK-28122) Binary String Functions: SHA functions

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28122. -- Resolution: Won't Fix > Binary String Functions: SHA functions >

[jira] [Commented] (SPARK-28122) Binary String Functions: SHA functions

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011317#comment-17011317 ] Takeshi Yamamuro commented on SPARK-28122: -- [~aman_omer] You're still working on this. Since

[jira] [Resolved] (SPARK-28087) String Functions: Add support split_part

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28087. -- Resolution: Won't Fix > String Functions: Add support split_part >

[jira] [Commented] (SPARK-28087) String Functions: Add support split_part

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011316#comment-17011316 ] Takeshi Yamamuro commented on SPARK-28087: -- I'll close this because the corresponding pr has

[jira] [Created] (SPARK-30464) Explicitly note that we don't add "pandas compatible" aliases

2020-01-08 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-30464: Summary: Explicitly note that we don't add "pandas compatible" aliases Key: SPARK-30464 URL: https://issues.apache.org/jira/browse/SPARK-30464 Project: Spark

[jira] [Created] (SPARK-30463) Move test cases for 'pandas' sub-package

2020-01-08 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-30463: Summary: Move test cases for 'pandas' sub-package Key: SPARK-30463 URL: https://issues.apache.org/jira/browse/SPARK-30463 Project: Spark Issue Type: Test

[jira] [Assigned] (SPARK-30434) Move pandas related functionalities into 'pandas' sub-package

2020-01-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-30434: Assignee: Hyukjin Kwon > Move pandas related functionalities into 'pandas' sub-package >

[jira] [Resolved] (SPARK-30434) Move pandas related functionalities into 'pandas' sub-package

2020-01-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30434. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27109

[jira] [Assigned] (SPARK-30440) Flaky test: org.apache.spark.scheduler.TaskSetManagerSuite.reset

2020-01-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30440: - Assignee: Ajith S > Flaky test: org.apache.spark.scheduler.TaskSetManagerSuite.reset >

[jira] [Resolved] (SPARK-30440) Flaky test: org.apache.spark.scheduler.TaskSetManagerSuite.reset

2020-01-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30440. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27115

[jira] [Reopened] (SPARK-26739) Standardized Join Types for DataFrames

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro reopened SPARK-26739: -- > Standardized Join Types for DataFrames > -- > >

[jira] [Commented] (SPARK-26739) Standardized Join Types for DataFrames

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011067#comment-17011067 ] Takeshi Yamamuro commented on SPARK-26739: -- Ah, I didn't notice that. ok, I'll reopen this. >

[jira] [Commented] (SPARK-26739) Standardized Join Types for DataFrames

2020-01-08 Thread Jeff Evans (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17011019#comment-17011019 ] Jeff Evans commented on SPARK-26739: [~maropu], I have an active PR for this, which is just awaiting

[jira] [Resolved] (SPARK-30417) SPARK-29976 calculation of slots wrong for Standalone Mode

2020-01-08 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-30417. -- Fix Version/s: 3.0.0 Assignee: Yuchen Huo Resolution: Fixed Resolved by

[jira] [Commented] (SPARK-30377) Make Regressors extend abstract class Regressor

2020-01-08 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010858#comment-17010858 ] Huaxin Gao commented on SPARK-30377: Hi [~yanboliang], a quick question: is there any reason that 

[jira] [Resolved] (SPARK-30446) Accelerator aware scheduling checkResourcesPerTask doesn't cover all cases

2020-01-08 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-30446. --- Resolution: Duplicate this is superceded by SPARK-30448 > Accelerator aware scheduling

[jira] [Commented] (SPARK-29988) Adjust Jenkins jobs for `hive-1.2/2.3` combination

2020-01-08 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010844#comment-17010844 ] Shane Knapp commented on SPARK-29988: - ok, i'll get on this today.  sorry about the delay – the end

[jira] [Assigned] (SPARK-30281) 'archive' option in FileStreamSource misses to consider partitioned and recursive option

2020-01-08 Thread Marcelo Masiero Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin reassigned SPARK-30281: -- Assignee: Jungtaek Lim > 'archive' option in FileStreamSource misses

[jira] [Resolved] (SPARK-30281) 'archive' option in FileStreamSource misses to consider partitioned and recursive option

2020-01-08 Thread Marcelo Masiero Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30281. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-30445) Accelerator aware scheduling handle setting configs to 0 better

2020-01-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30445. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27118

[jira] [Assigned] (SPARK-30445) Accelerator aware scheduling handle setting configs to 0 better

2020-01-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30445: - Assignee: Thomas Graves > Accelerator aware scheduling handle setting configs to 0

[jira] [Commented] (SPARK-30411) saveAsTable does not honor spark.hadoop.hive.warehouse.subdir.inherit.perms

2020-01-08 Thread Sanket Reddy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010799#comment-17010799 ] Sanket Reddy commented on SPARK-30411: -- spark.sql.orc.impl=hive does not work. [~yumwang]

[jira] [Commented] (SPARK-27249) Developers API for Transformers beyond UnaryTransformer

2020-01-08 Thread Nick Afshartous (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010756#comment-17010756 ] Nick Afshartous commented on SPARK-27249: - I could try and look into this. Could someone

[jira] [Created] (SPARK-30462) Structured Streaming _spark_metadata fills up Spark Driver memory when having lots of objects

2020-01-08 Thread Vladimir Yankov (Jira)
Vladimir Yankov created SPARK-30462: --- Summary: Structured Streaming _spark_metadata fills up Spark Driver memory when having lots of objects Key: SPARK-30462 URL:

[jira] [Commented] (SPARK-30461) details link when expanded should change the status from collapse to expand state

2020-01-08 Thread Ankit Raj Boudh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010744#comment-17010744 ] Ankit Raj Boudh commented on SPARK-30461: - I will raise the pr for this  > details link when

[jira] [Commented] (SPARK-30460) Spark checkpoint failing after some run with S3 path

2020-01-08 Thread Sachin Pasalkar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010742#comment-17010742 ] Sachin Pasalkar commented on SPARK-30460: - I had a long discussion with AWS folks but they are

[jira] [Created] (SPARK-30461) details link when expanded should change the status from collapse to expand state

2020-01-08 Thread ABHISHEK KUMAR GUPTA (Jira)
ABHISHEK KUMAR GUPTA created SPARK-30461: Summary: details link when expanded should change the status from collapse to expand state Key: SPARK-30461 URL:

[jira] [Created] (SPARK-30460) Spark checkpoint failing after some run with S3 path

2020-01-08 Thread Sachin Pasalkar (Jira)
Sachin Pasalkar created SPARK-30460: --- Summary: Spark checkpoint failing after some run with S3 path Key: SPARK-30460 URL: https://issues.apache.org/jira/browse/SPARK-30460 Project: Spark

[jira] [Created] (SPARK-30459) Fix ignoreMissingFiles/ignoreCorruptFiles in DSv2

2020-01-08 Thread wuyi (Jira)
wuyi created SPARK-30459: Summary: Fix ignoreMissingFiles/ignoreCorruptFiles in DSv2 Key: SPARK-30459 URL: https://issues.apache.org/jira/browse/SPARK-30459 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-30410) Calculating size of table having large number of partitions causes flooding logs

2020-01-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-30410: - Priority: Trivial (was: Minor) > Calculating size of table having large number of partitions

[jira] [Assigned] (SPARK-30410) Calculating size of table having large number of partitions causes flooding logs

2020-01-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30410: Assignee: Zhenhua Wang > Calculating size of table having large number of partitions

[jira] [Resolved] (SPARK-30410) Calculating size of table having large number of partitions causes flooding logs

2020-01-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30410. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27079

[jira] [Commented] (SPARK-30049) SQL fails to parse when comment contains an unmatched quote character

2020-01-08 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010698#comment-17010698 ] Thomas Graves commented on SPARK-30049: --- that Pr doesn't seem to be in yet. I'm not sure the

[jira] [Commented] (SPARK-30295) Remove Hive dependencies from SparkSQLCLI

2020-01-08 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010697#comment-17010697 ] Thomas Graves commented on SPARK-30295: --- So what do you mean by unnecessary dependencies?  I

[jira] [Assigned] (SPARK-30215) Remove PrunedInMemoryFileIndex and merge its functionality into InMemoryFileIndex

2020-01-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30215: --- Assignee: Hu Fuwang > Remove PrunedInMemoryFileIndex and merge its functionality into >

[jira] [Updated] (SPARK-30458) The Executor Computing Time in Time Line of Stage Page is Wrong

2020-01-08 Thread SongXun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SongXun updated SPARK-30458: Description: The Executor Computing Time in Time Line of Stage Page is Wrong. It includes the Scheduler

[jira] [Updated] (SPARK-30458) The Executor Computing Time in Time Line of Stage Page is Wrong

2020-01-08 Thread SongXun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SongXun updated SPARK-30458: Attachment: Snipaste_2020-01-08_19-04-33.png > The Executor Computing Time in Time Line of Stage Page is

[jira] [Assigned] (SPARK-30457) Use PeriodicRDDCheckpointer instead of NodeIdCache

2020-01-08 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng reassigned SPARK-30457: Assignee: zhengruifeng > Use PeriodicRDDCheckpointer instead of NodeIdCache >

[jira] [Created] (SPARK-30458) The Executor Computing Time in Time Line of Stage Page is Wrong

2020-01-08 Thread SongXun (Jira)
SongXun created SPARK-30458: --- Summary: The Executor Computing Time in Time Line of Stage Page is Wrong Key: SPARK-30458 URL: https://issues.apache.org/jira/browse/SPARK-30458 Project: Spark Issue

[jira] [Updated] (SPARK-30458) The Executor Computing Time in Time Line of Stage Page is Wrong

2020-01-08 Thread SongXun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SongXun updated SPARK-30458: Priority: Minor (was: Major) > The Executor Computing Time in Time Line of Stage Page is Wrong >

[jira] [Created] (SPARK-30457) Use PeriodicRDDCheckpointer instead of NodeIdCache

2020-01-08 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-30457: Summary: Use PeriodicRDDCheckpointer instead of NodeIdCache Key: SPARK-30457 URL: https://issues.apache.org/jira/browse/SPARK-30457 Project: Spark Issue

[jira] [Created] (SPARK-30456) Trim partition property exception

2020-01-08 Thread yangfan (Jira)
yangfan created SPARK-30456: --- Summary: Trim partition property exception Key: SPARK-30456 URL: https://issues.apache.org/jira/browse/SPARK-30456 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-30196) Bump lz4-java version to 1.7.0

2020-01-08 Thread Lars Francke (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010475#comment-17010475 ] Lars Francke commented on SPARK-30196: -- Sorry I didn't have time to check yet but it seems as if

[jira] [Comment Edited] (SPARK-30421) Dropped columns still available for filtering

2020-01-08 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010271#comment-17010271 ] Takeshi Yamamuro edited comment on SPARK-30421 at 1/8/20 8:38 AM: --

[jira] [Resolved] (SPARK-30402) Support push down for filters with cast expression

2020-01-08 Thread wuyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-30402. -- Resolution: Won't Do > Support push down for filters with cast expression >