[jira] [Commented] (SPARK-30196) Bump lz4-java version to 1.7.0

2020-01-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010438#comment-17010438 ] Hyukjin Kwon commented on SPARK-30196: -- (y) > Bump lz4-java version to 1.7.0 > ---

[jira] [Resolved] (SPARK-30425) FileScan of Data Source V2 doesn't implement Partition Pruning

2020-01-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30425. -- Resolution: Duplicate > FileScan of Data Source V2 doesn't implement Partition Pruning > -

[jira] [Commented] (SPARK-30455) Select All should unselect after un-selecting any selected item from list.

2020-01-07 Thread Ankit Raj Boudh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010424#comment-17010424 ] Ankit Raj Boudh commented on SPARK-30455: - i will raise for this. > Select All

[jira] [Comment Edited] (SPARK-30455) Select All should unselect after un-selecting any selected item from list.

2020-01-07 Thread Ankit Raj Boudh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010424#comment-17010424 ] Ankit Raj Boudh edited comment on SPARK-30455 at 1/8/20 7:06 AM: -

[jira] [Created] (SPARK-30455) Select All should unselect after un-selecting any selected item from list.

2020-01-07 Thread Ankit Raj Boudh (Jira)
Ankit Raj Boudh created SPARK-30455: --- Summary: Select All should unselect after un-selecting any selected item from list. Key: SPARK-30455 URL: https://issues.apache.org/jira/browse/SPARK-30455 Proj

[jira] [Commented] (SPARK-30425) FileScan of Data Source V2 doesn't implement Partition Pruning

2020-01-07 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010414#comment-17010414 ] Gengliang Wang commented on SPARK-30425: [~sandeep.katta2007]Yes [~jerrychenhf]

[jira] [Commented] (SPARK-30454) Null Dereference in HiveSQLException

2020-01-07 Thread pavithra ramachandran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010412#comment-17010412 ] pavithra ramachandran commented on SPARK-30454: --- I shall raise the PR > N

[jira] [Created] (SPARK-30454) Null Dereference in HiveSQLException

2020-01-07 Thread pavithra ramachandran (Jira)
pavithra ramachandran created SPARK-30454: - Summary: Null Dereference in HiveSQLException Key: SPARK-30454 URL: https://issues.apache.org/jira/browse/SPARK-30454 Project: Spark Issue

[jira] [Commented] (SPARK-30425) FileScan of Data Source V2 doesn't implement Partition Pruning

2020-01-07 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010408#comment-17010408 ] Sandeep Katta commented on SPARK-30425: --- [~gengliang] > FileScan of Data Source V

[jira] [Commented] (SPARK-30425) FileScan of Data Source V2 doesn't implement Partition Pruning

2020-01-07 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010407#comment-17010407 ] Sandeep Katta commented on SPARK-30425: --- is this duplicate of [SPARK-30428|https:

[jira] [Commented] (SPARK-28478) Optimizer rule to remove unnecessary explicit null checks for null-intolerant expressions (e.g. if(x is null, x, f(x)))

2020-01-07 Thread David Vrba (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010387#comment-17010387 ] David Vrba commented on SPARK-28478: [~cloud_fan] what do you think about this? Is i

[jira] [Updated] (SPARK-30408) orderBy in sortBy clause is removed by EliminateSorts

2020-01-07 Thread APeng Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] APeng Zhang updated SPARK-30408: Description: OrderBy in sortBy clause will be removed by EliminateSorts. code to reproduce: {code

[jira] [Updated] (SPARK-30427) Add config item for limiting partition number when calculating statistics through File System

2020-01-07 Thread Hu Fuwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hu Fuwang updated SPARK-30427: -- Description: Currently, when spark need to calculate the statistics (eg. sizeInBytes) of table partit

[jira] [Updated] (SPARK-30427) Add config item for limiting partition number when calculating statistics through File System

2020-01-07 Thread Hu Fuwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hu Fuwang updated SPARK-30427: -- Summary: Add config item for limiting partition number when calculating statistics through File System

[jira] [Commented] (SPARK-30196) Bump lz4-java version to 1.7.0

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010373#comment-17010373 ] Takeshi Yamamuro commented on SPARK-30196: -- Yea, it seems yes (I can reproduce

[jira] [Commented] (SPARK-28137) Data Type Formatting Functions: `to_number`

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010351#comment-17010351 ] Takeshi Yamamuro commented on SPARK-28137: -- See: [https://github.com/apache/spa

[jira] [Resolved] (SPARK-28137) Data Type Formatting Functions: `to_number`

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28137. -- Resolution: Won't Fix > Data Type Formatting Functions: `to_number` >

[jira] [Commented] (SPARK-29878) Improper cache strategies in GraphX

2020-01-07 Thread Dong Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010317#comment-17010317 ] Dong Wang commented on SPARK-29878: --- So are these unnecessary caches tolerable? These

[jira] [Commented] (SPARK-30125) Remove PostgreSQL dialect

2020-01-07 Thread Yuanjian Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010304#comment-17010304 ] Yuanjian Li commented on SPARK-30125: - Also link #26940 with this Jira. > Remove Po

[jira] [Resolved] (SPARK-30450) Exclude .git folder for python linter

2020-01-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30450. --- Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed Issue resolved by pul

[jira] [Resolved] (SPARK-30429) WideSchemaBenchmark fails with OOM

2020-01-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30429. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27117 [https://

[jira] [Assigned] (SPARK-30429) WideSchemaBenchmark fails with OOM

2020-01-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30429: - Assignee: L. C. Hsieh > WideSchemaBenchmark fails with OOM > --

[jira] [Resolved] (SPARK-30453) Update AppVeyor R version to 3.6.2

2020-01-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30453. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27124 [https://

[jira] [Assigned] (SPARK-30453) Update AppVeyor R version to 3.6.2

2020-01-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30453: - Assignee: Hyukjin Kwon > Update AppVeyor R version to 3.6.2 > -

[jira] [Resolved] (SPARK-30302) Complete info for show create table for views

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-30302. -- Fix Version/s: 3.0.0 Assignee: Zhenhua Wang Resolution: Fixed Resolved

[jira] [Commented] (SPARK-30421) Dropped columns still available for filtering

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010271#comment-17010271 ] Takeshi Yamamuro commented on SPARK-30421: -- Based on the current implementation

[jira] [Resolved] (SPARK-30381) GBT reuse treePoints for all trees

2020-01-07 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-30381. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27103 [https://gi

[jira] [Created] (SPARK-30453) Update AppVeyor R version to 3.6.2

2020-01-07 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-30453: Summary: Update AppVeyor R version to 3.6.2 Key: SPARK-30453 URL: https://issues.apache.org/jira/browse/SPARK-30453 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-28264) Revisiting Python / pandas UDF

2020-01-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28264: Assignee: Hyukjin Kwon (was: Reynold Xin) > Revisiting Python / pandas UDF > ---

[jira] [Commented] (SPARK-30421) Dropped columns still available for filtering

2020-01-07 Thread Aman Omer (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010227#comment-17010227 ] Aman Omer commented on SPARK-30421: --- cc [~maropu] > Dropped columns still available f

[jira] [Commented] (SPARK-26249) Extension Points Enhancements to inject a rule in order and to add a batch

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010223#comment-17010223 ] Takeshi Yamamuro commented on SPARK-26249: -- I'll close this because the corresp

[jira] [Resolved] (SPARK-26249) Extension Points Enhancements to inject a rule in order and to add a batch

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-26249. -- Resolution: Won't Fix > Extension Points Enhancements to inject a rule in order and to

[jira] [Resolved] (SPARK-28825) Document EXPLAIN Statement in SQL Reference.

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28825. -- Fix Version/s: 3.0.0 Assignee: pavithra ramachandran Resolution: Fixed

[jira] [Updated] (SPARK-28825) Document EXPLAIN Statement in SQL Reference.

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-28825: - Affects Version/s: (was: 2.4.3) 3.0.0 > Document EXPLAIN Stat

[jira] [Resolved] (SPARK-24884) Implement regexp_extract_all

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-24884. -- Resolution: Won't Fix > Implement regexp_extract_all > >

[jira] [Commented] (SPARK-24884) Implement regexp_extract_all

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010193#comment-17010193 ] Takeshi Yamamuro commented on SPARK-24884: -- I'll close this because the corresp

[jira] [Created] (SPARK-30452) Add predict and numFeatures in Python IsotonicRegressionModel

2020-01-07 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-30452: -- Summary: Add predict and numFeatures in Python IsotonicRegressionModel Key: SPARK-30452 URL: https://issues.apache.org/jira/browse/SPARK-30452 Project: Spark Is

[jira] [Resolved] (SPARK-29167) Metrics of Analyzer/Optimizer use Scientific counting is not human readable

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-29167. -- Resolution: Won't Fix > Metrics of Analyzer/Optimizer use Scientific counting is not h

[jira] [Commented] (SPARK-29167) Metrics of Analyzer/Optimizer use Scientific counting is not human readable

2020-01-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010187#comment-17010187 ] Takeshi Yamamuro commented on SPARK-29167: -- I'll close this because no committe

[jira] [Commented] (SPARK-30417) SPARK-29976 calculation of slots wrong for Standalone Mode

2020-01-07 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010156#comment-17010156 ] Xingbo Jiang commented on SPARK-30417: -- Good catch! `max(conf.get(EXECUTOR_CORES) /

[jira] [Commented] (SPARK-30417) SPARK-29976 calculation of slots wrong for Standalone Mode

2020-01-07 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010146#comment-17010146 ] Thomas Graves commented on SPARK-30417: --- The only way for standalone mode would be

[jira] [Commented] (SPARK-30417) SPARK-29976 calculation of slots wrong for Standalone Mode

2020-01-07 Thread Yuchen Huo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010142#comment-17010142 ] Yuchen Huo commented on SPARK-30417: [~tgraves] Sure. Is there a more stable way to

[jira] [Resolved] (SPARK-30382) start-thriftserver throws ClassNotFoundException

2020-01-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30382. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27042 [https://

[jira] [Assigned] (SPARK-30382) start-thriftserver throws ClassNotFoundException

2020-01-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30382: - Assignee: Ajith S > start-thriftserver throws ClassNotFoundException >

[jira] [Created] (SPARK-30451) Stage level Sched: ExecutorResourceRequests/TaskResourceRequests should have functions to remove requests

2020-01-07 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-30451: - Summary: Stage level Sched: ExecutorResourceRequests/TaskResourceRequests should have functions to remove requests Key: SPARK-30451 URL: https://issues.apache.org/jira/browse/S

[jira] [Updated] (SPARK-30450) Exclude .git folder for python linter

2020-01-07 Thread Yin Huai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-30450: - Affects Version/s: (was: 2.4.4) 3.0.0 > Exclude .git folder for python linter

[jira] [Updated] (SPARK-30450) Exclude .git folder for python linter

2020-01-07 Thread Yin Huai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-30450: - Priority: Minor (was: Major) > Exclude .git folder for python linter >

[jira] [Assigned] (SPARK-30450) Exclude .git folder for python linter

2020-01-07 Thread Yin Huai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai reassigned SPARK-30450: Assignee: Eric Chang > Exclude .git folder for python linter > --

[jira] [Created] (SPARK-30450) Exclude .git folder for python linter

2020-01-07 Thread Eric Chang (Jira)
Eric Chang created SPARK-30450: -- Summary: Exclude .git folder for python linter Key: SPARK-30450 URL: https://issues.apache.org/jira/browse/SPARK-30450 Project: Spark Issue Type: Bug C

[jira] [Commented] (SPARK-2984) FileNotFoundException on _temporary directory

2020-01-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009988#comment-17009988 ] Steve Loughran commented on SPARK-2984: --- bq. As part of your recommendation, is it

[jira] [Commented] (SPARK-30448) accelerator aware scheduling enforce cores as limiting resource

2020-01-07 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009989#comment-17009989 ] Thomas Graves commented on SPARK-30448: --- Note this actually overlaps with https:/

[jira] [Commented] (SPARK-30442) Write mode ignored when using CodecStreams

2020-01-07 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009963#comment-17009963 ] Maxim Gekk commented on SPARK-30442: > This can cause issues, particularly with aws

[jira] [Updated] (SPARK-30449) Introducing get_dummies method in pyspark

2020-01-07 Thread Krishna Kumar Tiwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar Tiwari updated SPARK-30449: - Flags: Important > Introducing get_dummies method in pyspark > -

[jira] [Commented] (SPARK-30449) Introducing get_dummies method in pyspark

2020-01-07 Thread Krishna Kumar Tiwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009924#comment-17009924 ] Krishna Kumar Tiwari commented on SPARK-30449: -- I am already working on thi

[jira] [Updated] (SPARK-30449) Introducing get_dummies method in pyspark

2020-01-07 Thread Krishna Kumar Tiwari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krishna Kumar Tiwari updated SPARK-30449: - Issue Type: Task (was: New Feature) > Introducing get_dummies method in pyspark

[jira] [Created] (SPARK-30449) Introducing get_dummies method in pyspark

2020-01-07 Thread Krishna Kumar Tiwari (Jira)
Krishna Kumar Tiwari created SPARK-30449: Summary: Introducing get_dummies method in pyspark Key: SPARK-30449 URL: https://issues.apache.org/jira/browse/SPARK-30449 Project: Spark Iss

[jira] [Commented] (SPARK-30448) accelerator aware scheduling enforce cores as limiting resource

2020-01-07 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009901#comment-17009901 ] Thomas Graves commented on SPARK-30448: --- note there are other calculations through

[jira] [Commented] (SPARK-30446) Accelerator aware scheduling checkResourcesPerTask doesn't cover all cases

2020-01-07 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009896#comment-17009896 ] Thomas Graves commented on SPARK-30446: --- working on this > Accelerator aware sche

[jira] [Created] (SPARK-30448) accelerator aware scheduling enforce cores as limiting resource

2020-01-07 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-30448: - Summary: accelerator aware scheduling enforce cores as limiting resource Key: SPARK-30448 URL: https://issues.apache.org/jira/browse/SPARK-30448 Project: Spark

[jira] [Assigned] (SPARK-30039) CREATE FUNCTION should look up catalog/table like v2 commands

2020-01-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30039: --- Assignee: Pablo Langa Blanco > CREATE FUNCTION should look up catalog/table like v2 comman

[jira] [Resolved] (SPARK-30039) CREATE FUNCTION should look up catalog/table like v2 commands

2020-01-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30039. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26890 [https://gith

[jira] [Updated] (SPARK-30443) "Managed memory leak detected" even with no calls to take() or limit()

2020-01-07 Thread Luke Richter (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Richter updated SPARK-30443: - Description: Our Spark code is causing a "Managed memory leak detected" warning to appear, even

[jira] [Updated] (SPARK-30443) "Managed memory leak detected" even with no calls to take() or limit()

2020-01-07 Thread Luke Richter (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Richter updated SPARK-30443: - Attachment: a.csv.zip b.csv.zip c.csv.zip > "Managed memory leak

[jira] [Resolved] (SPARK-30437) Uneven spaces for some fields in EXPLAIN FORMATTED

2020-01-07 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30437. -- Resolution: Won't Fix > Uneven spaces for some fields in EXPLAIN FORMATTED > -

[jira] [Updated] (SPARK-27604) Enhance constant and constraint propagation

2020-01-07 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-27604: --- Issue Type: Improvement (was: Bug) > Enhance constant and constraint propagation >

[jira] [Updated] (SPARK-27604) Enhance constant and constraint propagation

2020-01-07 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-27604: --- Description: There is some room for improvement as constant propagation could allow substitution of

[jira] [Created] (SPARK-30447) Constant propagation nullability issue

2020-01-07 Thread Peter Toth (Jira)
Peter Toth created SPARK-30447: -- Summary: Constant propagation nullability issue Key: SPARK-30447 URL: https://issues.apache.org/jira/browse/SPARK-30447 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-30446) Accelerator aware scheduling checkResourcesPerTask doesn't cover all cases

2020-01-07 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009830#comment-17009830 ] Thomas Graves commented on SPARK-30446: --- Yeah so running on standalone if you set

[jira] [Resolved] (SPARK-30431) Update SqlBase.g4 to create commentSpec pattern as same as locationSpec

2020-01-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30431. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27102 [https://gith

[jira] [Assigned] (SPARK-30431) Update SqlBase.g4 to create commentSpec pattern as same as locationSpec

2020-01-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30431: --- Assignee: Kent Yao > Update SqlBase.g4 to create commentSpec pattern as same as locationSpe

[jira] [Commented] (SPARK-30446) Accelerator aware scheduling checkResourcesPerTask doesn't cover all cases

2020-01-07 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009742#comment-17009742 ] Thomas Graves commented on SPARK-30446: --- I think there may also be issues in it wi

[jira] [Created] (SPARK-30446) Accelerator aware scheduling checkResourcesPerTask doesn't cover all cases

2020-01-07 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-30446: - Summary: Accelerator aware scheduling checkResourcesPerTask doesn't cover all cases Key: SPARK-30446 URL: https://issues.apache.org/jira/browse/SPARK-30446 Project:

[jira] [Created] (SPARK-30445) Accelerator aware scheduling handle setting configs to 0 better

2020-01-07 Thread Thomas Graves (Jira)
Thomas Graves created SPARK-30445: - Summary: Accelerator aware scheduling handle setting configs to 0 better Key: SPARK-30445 URL: https://issues.apache.org/jira/browse/SPARK-30445 Project: Spark

[jira] [Resolved] (SPARK-30338) Avoid unnecessary InternalRow copies in ParquetRowConverter

2020-01-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30338. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26993 [https://gith

[jira] [Commented] (SPARK-30429) WideSchemaBenchmark fails with OOM

2020-01-07 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009468#comment-17009468 ] L. C. Hsieh commented on SPARK-30429: - Thanks for pinging me. Looking into this. >