[jira] [Commented] (SPARK-30955) Exclude Generate output when aliasing in nested column pruning

2020-02-25 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045211#comment-17045211 ] Peter Toth commented on SPARK-30955: This affects only 3.0, doesn't it? > Exclude Generate output

[jira] [Commented] (SPARK-30932) ML 3.0 QA: API: Java compatibility, docs

2020-02-25 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045168#comment-17045168 ] zhengruifeng commented on SPARK-30932: -- I check the output result and do not find outstanding

[jira] [Created] (SPARK-30955) Exclude Generate output when aliasing in nested column pruning

2020-02-25 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-30955: --- Summary: Exclude Generate output when aliasing in nested column pruning Key: SPARK-30955 URL: https://issues.apache.org/jira/browse/SPARK-30955 Project: Spark

[jira] [Created] (SPARK-30954) TreeModelWrappers class name do not correspond to file name

2020-02-25 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-30954: Summary: TreeModelWrappers class name do not correspond to file name Key: SPARK-30954 URL: https://issues.apache.org/jira/browse/SPARK-30954 Project: Spark

[jira] [Updated] (SPARK-30932) ML 3.0 QA: API: Java compatibility, docs

2020-02-25 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30932: - Attachment: (was: signature.diff) > ML 3.0 QA: API: Java compatibility, docs >

[jira] [Updated] (SPARK-30932) ML 3.0 QA: API: Java compatibility, docs

2020-02-25 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30932: - Attachment: (was: added_ml_class) > ML 3.0 QA: API: Java compatibility, docs >

[jira] [Updated] (SPARK-30932) ML 3.0 QA: API: Java compatibility, docs

2020-02-25 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30932: - Attachment: signature.diff common_ml_class added_ml_class

[jira] [Updated] (SPARK-30932) ML 3.0 QA: API: Java compatibility, docs

2020-02-25 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-30932: - Attachment: (was: 1_process_script.sh) > ML 3.0 QA: API: Java compatibility, docs >

[jira] [Created] (SPARK-30953) InsertAdaptiveSparkPlan should also skip v2 command

2020-02-25 Thread wuyi (Jira)
wuyi created SPARK-30953: Summary: InsertAdaptiveSparkPlan should also skip v2 command Key: SPARK-30953 URL: https://issues.apache.org/jira/browse/SPARK-30953 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-30950) Setting version to 3.1.0-SNAPSHOT

2020-02-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30950. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27698

[jira] [Updated] (SPARK-30904) Thrift RowBasedSet serialization throws NullPointerException on NULL BigDecimal

2020-02-25 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30904: Fix Version/s: (was: 3.1.0) > Thrift RowBasedSet serialization throws NullPointerException on NULL >

[jira] [Updated] (SPARK-30893) Expressions should not change its data type/behavior after it's created

2020-02-25 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-30893: Target Version/s: 3.0.0 > Expressions should not change its data type/behavior after it's created >

[jira] [Updated] (SPARK-30952) Grouped pandas_udf crashed when a group returned an empty DataFrame

2020-02-25 Thread bruce_zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bruce_zhao updated SPARK-30952: --- Description: We are trying to apply three-sigma rule in grouped data to detect anomaly data. We

[jira] [Created] (SPARK-30952) Grouped pandas_udf crashed when a group returned an empty DataFrame

2020-02-25 Thread bruce_zhao (Jira)
bruce_zhao created SPARK-30952: -- Summary: Grouped pandas_udf crashed when a group returned an empty DataFrame Key: SPARK-30952 URL: https://issues.apache.org/jira/browse/SPARK-30952 Project: Spark

[jira] [Commented] (SPARK-30855) Issue using 'explode' function followed by a (*)star expand selection of resulting struct

2020-02-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045100#comment-17045100 ] Dongjoon Hyun commented on SPARK-30855: --- Sorry, but we don't have a snapshot distribution. >

[jira] [Resolved] (SPARK-24472) Orc RecordReaderFactory throws IndexOutOfBoundsException

2020-02-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-24472. --- Resolution: Duplicate This is fixed at `3.0.0-preview2` by SPARK-23710. > Orc

[jira] [Updated] (SPARK-24472) Orc RecordReaderFactory throws IndexOutOfBoundsException

2020-02-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-24472: -- Affects Version/s: 2.4.5 > Orc RecordReaderFactory throws IndexOutOfBoundsException >

[jira] [Commented] (SPARK-24472) Orc RecordReaderFactory throws IndexOutOfBoundsException

2020-02-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045096#comment-17045096 ] Dongjoon Hyun commented on SPARK-24472: --- I confirmed that this is fixed at 3.0.0-preview2. {code}

[jira] [Comment Edited] (SPARK-27148) Support CURRENT_TIME and LOCALTIME when ANSI mode enabled

2020-02-25 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045078#comment-17045078 ] YoungGyu Chun edited comment on SPARK-27148 at 2/26/20 2:27 AM:

[jira] [Updated] (SPARK-30950) Setting version to 3.1.0-SNAPSHOT

2020-02-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30950: -- Issue Type: Improvement (was: Story) > Setting version to 3.1.0-SNAPSHOT >

[jira] [Commented] (SPARK-27148) Support CURRENT_TIME and LOCALTIME when ANSI mode enabled

2020-02-25 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045078#comment-17045078 ] YoungGyu Chun commented on SPARK-27148: --- [~maropu] Thank you :) > Support CURRENT_TIME and

[jira] [Created] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

2020-02-25 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-30951: - Summary: Potential data loss for legacy applications after switch to proleptic Gregorian calendar Key: SPARK-30951 URL: https://issues.apache.org/jira/browse/SPARK-30951

[jira] [Commented] (SPARK-27148) Support CURRENT_TIME and LOCALTIME when ANSI mode enabled

2020-02-25 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045066#comment-17045066 ] Takeshi Yamamuro commented on SPARK-27148: -- Yea, feel free to take this over. > Support

[jira] [Commented] (SPARK-27148) Support CURRENT_TIME and LOCALTIME when ANSI mode enabled

2020-02-25 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045060#comment-17045060 ] YoungGyu Chun commented on SPARK-27148: --- [~maropu] I am wondering if I can work on this. >

[jira] [Commented] (SPARK-29969) parse_url function result in incorrect result

2020-02-25 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045059#comment-17045059 ] YoungGyu Chun commented on SPARK-29969: --- [~xiaoxigua] is this issue is relating to Spark? The

[jira] [Created] (SPARK-30950) Setting version to 3.1.0-SNAPSHOT

2020-02-25 Thread Xiao Li (Jira)
Xiao Li created SPARK-30950: --- Summary: Setting version to 3.1.0-SNAPSHOT Key: SPARK-30950 URL: https://issues.apache.org/jira/browse/SPARK-30950 Project: Spark Issue Type: Story

[jira] [Resolved] (SPARK-29333) Sample weight in RandomForestRegressor

2020-02-25 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-29333. Resolution: Fixed > Sample weight in RandomForestRegressor >

[jira] [Commented] (SPARK-29333) Sample weight in RandomForestRegressor

2020-02-25 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045038#comment-17045038 ] Huaxin Gao commented on SPARK-29333: This is resolved in 

[jira] [Commented] (SPARK-30561) start spark applications without a 30second startup penalty

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045015#comment-17045015 ] t oo commented on SPARK-30561: -- [~rxin] looks like you committed val SLEEP_TIME_SECS = 5 in BlockManager,

[jira] [Commented] (SPARK-30561) start spark applications without a 30second startup penalty

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045013#comment-17045013 ] t oo commented on SPARK-30561: -- [~pwendell] looks like you added the sleep(5000), you don't happen to

[jira] [Commented] (SPARK-24295) Purge Structured streaming FileStreamSinkLog metadata compact file data.

2020-02-25 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045011#comment-17045011 ] Jungtaek Lim commented on SPARK-24295: -- [~iqbal_khattra] [~alfredo-gimenez-bv] Hi, if you're open

[jira] [Commented] (SPARK-29995) Structured Streaming file-sink log grow indefinitely

2020-02-25 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17045009#comment-17045009 ] Jungtaek Lim commented on SPARK-29995: -- [~zhangliming] Hi, if you're open to try out something on

[jira] [Resolved] (SPARK-30943) Show "batch ID" in tool tip string for Structured Streaming UI graphs

2020-02-25 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-30943. -- Fix Version/s: 3.0.0 Assignee: Jungtaek Lim Resolution: Fixed > Show "batch

[jira] [Commented] (SPARK-30610) spark worker graceful shutdown

2020-02-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044974#comment-17044974 ] t oo commented on SPARK-30610: -- >From [~holdenkarau]: In its present state it could help, you'd call decom

[jira] [Commented] (SPARK-30949) Driver cores in kubernetes are coupled with container resources, not spark.driver.cores

2020-02-25 Thread Onur Satici (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044970#comment-17044970 ] Onur Satici commented on SPARK-30949: - Hi [~dongjoon], sorry my mistake, changed it to improvement,

[jira] [Updated] (SPARK-30949) Driver cores in kubernetes are coupled with container resources, not spark.driver.cores

2020-02-25 Thread Onur Satici (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Satici updated SPARK-30949: Issue Type: Improvement (was: Dependency upgrade) > Driver cores in kubernetes are coupled with

[jira] [Commented] (SPARK-30949) Driver cores in kubernetes are coupled with container resources, not spark.driver.cores

2020-02-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044963#comment-17044963 ] Dongjoon Hyun commented on SPARK-30949: --- Hi, [~onursatici]. Thank you for filing a JIRA and making

[jira] [Resolved] (SPARK-30918) improve the splitting of skewed partitions

2020-02-25 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-30918. - Fix Version/s: 3.0.0 Resolution: Fixed > improve the splitting of skewed partitions >

[jira] [Comment Edited] (SPARK-18879) Spark SQL support for Hive hooks regressed

2020-02-25 Thread NITISH SHARMA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044931#comment-17044931 ] NITISH SHARMA edited comment on SPARK-18879 at 2/25/20 9:59 PM: I have a

[jira] [Commented] (SPARK-18879) Spark SQL support for Hive hooks regressed

2020-02-25 Thread NITISH SHARMA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044931#comment-17044931 ] NITISH SHARMA commented on SPARK-18879: --- I have a similar requirement where i am looking to update

[jira] [Updated] (SPARK-30378) FM support getter of training params

2020-02-25 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-30378: --- Summary: FM support getter of training params (was: FM support getter of training prams) > FM

[jira] [Commented] (SPARK-30934) ML, GraphX 3.0 QA: Programming guide update and migration guide

2020-02-25 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044908#comment-17044908 ] Huaxin Gao commented on SPARK-30934: cc [~podongfeng] I will change migration guide for the

[jira] [Comment Edited] (SPARK-30928) ML, GraphX 3.0 QA: API: Binary incompatible changes

2020-02-25 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043874#comment-17043874 ] Huaxin Gao edited comment on SPARK-30928 at 2/25/20 9:28 PM: -   I audited

[jira] [Commented] (SPARK-24472) Orc RecordReaderFactory throws IndexOutOfBoundsException

2020-02-25 Thread Shreyas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044744#comment-17044744 ] Shreyas commented on SPARK-24472: - Now that SPARK-23710 has been fixed, can this one be attended to?

[jira] [Commented] (SPARK-30855) Issue using 'explode' function followed by a (*)star expand selection of resulting struct

2020-02-25 Thread Benoit Roy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044741#comment-17044741 ] Benoit Roy commented on SPARK-30855: Do you know if there is any distro available of the latest

[jira] [Comment Edited] (SPARK-30928) ML, GraphX 3.0 QA: API: Binary incompatible changes

2020-02-25 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043874#comment-17043874 ] Huaxin Gao edited comment on SPARK-30928 at 2/25/20 6:51 PM: -   I audited

[jira] [Comment Edited] (SPARK-30928) ML, GraphX 3.0 QA: API: Binary incompatible changes

2020-02-25 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043874#comment-17043874 ] Huaxin Gao edited comment on SPARK-30928 at 2/25/20 6:47 PM: - I audited all

[jira] [Comment Edited] (SPARK-30928) ML, GraphX 3.0 QA: API: Binary incompatible changes

2020-02-25 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043874#comment-17043874 ] Huaxin Gao edited comment on SPARK-30928 at 2/25/20 6:43 PM: - I audited all

[jira] [Comment Edited] (SPARK-30928) ML, GraphX 3.0 QA: API: Binary incompatible changes

2020-02-25 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043874#comment-17043874 ] Huaxin Gao edited comment on SPARK-30928 at 2/25/20 6:40 PM: - I audited all

[jira] [Commented] (SPARK-30855) Issue using 'explode' function followed by a (*)star expand selection of resulting struct

2020-02-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044718#comment-17044718 ] Dongjoon Hyun commented on SPARK-30855: --- Thanks, [~benoitroy] and [~petertoth]. > Issue using

[jira] [Resolved] (SPARK-30942) Fix the warning for requiring cores to be limiting resource

2020-02-25 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-30942. --- Fix Version/s: 3.1.0 Assignee: Thomas Graves Resolution: Fixed > Fix the

[jira] [Updated] (SPARK-30949) Driver cores in kubernetes are coupled with container resources, not spark.driver.cores

2020-02-25 Thread Onur Satici (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Satici updated SPARK-30949: Description: Drivers submitted in kubernetes cluster mode set the parallelism of various

[jira] [Created] (SPARK-30949) Driver cores in kubernetes are coupled with container resources, not spark.driver.cores

2020-02-25 Thread Onur Satici (Jira)
Onur Satici created SPARK-30949: --- Summary: Driver cores in kubernetes are coupled with container resources, not spark.driver.cores Key: SPARK-30949 URL: https://issues.apache.org/jira/browse/SPARK-30949

[jira] [Comment Edited] (SPARK-8665) Update ALS documentation to include performance tips

2020-02-25 Thread Dmitry Goldenberg (Jira)
[ https://issues.apache.org/jira/browse/SPARK-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044607#comment-17044607 ] Dmitry Goldenberg edited comment on SPARK-8665 at 2/25/20 4:04 PM: --- Any

[jira] [Commented] (SPARK-8665) Update ALS documentation to include performance tips

2020-02-25 Thread Dmitry Goldenberg (Jira)
[ https://issues.apache.org/jira/browse/SPARK-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044607#comment-17044607 ] Dmitry Goldenberg commented on SPARK-8665: -- Any reason why this is marked as resolved?

[jira] [Updated] (SPARK-30948) Sharing the same external shuffle service

2020-02-25 Thread 10000kang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 1kang updated SPARK-30948: -- Affects Version/s: (was: 1.0.0) 1.6.0 > Sharing the same external shuffle

[jira] [Updated] (SPARK-30948) Sharing the same external shuffle service

2020-02-25 Thread 10000kang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 1kang updated SPARK-30948: -- Affects Version/s: (was: 2.4.3) 1.0.0 > Sharing the same external shuffle

[jira] [Updated] (SPARK-30948) Sharing the same external shuffle service

2020-02-25 Thread wankang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wankang updated SPARK-30948: Description:  Multiple spark tasks sharing the same external shuffle service will affect each other. if

[jira] [Created] (SPARK-30948) Sharing the same external shuffle service

2020-02-25 Thread wankang (Jira)
wankang created SPARK-30948: --- Summary: Sharing the same external shuffle service Key: SPARK-30948 URL: https://issues.apache.org/jira/browse/SPARK-30948 Project: Spark Issue Type: Question

[jira] [Commented] (SPARK-30322) Add stage level scheduling docs

2020-02-25 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044561#comment-17044561 ] Thomas Graves commented on SPARK-30322: --- Document the yarn priority behavior > Add stage level

[jira] [Resolved] (SPARK-30869) Convert dates to/from timestamps in microseconds precision

2020-02-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30869. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27618

[jira] [Assigned] (SPARK-30869) Convert dates to/from timestamps in microseconds precision

2020-02-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30869: --- Assignee: Maxim Gekk > Convert dates to/from timestamps in microseconds precision >

[jira] [Commented] (SPARK-30855) Issue using 'explode' function followed by a (*)star expand selection of resulting struct

2020-02-25 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044530#comment-17044530 ] Peter Toth commented on SPARK-30855: Yes, it's been merged. > Issue using 'explode' function

[jira] [Assigned] (SPARK-30919) Make interval multiply and divide's overflow behavior consisitent with other interval operations

2020-02-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30919: --- Assignee: Kent Yao > Make interval multiply and divide's overflow behavior consisitent

[jira] [Resolved] (SPARK-30919) Make interval multiply and divide's overflow behavior consisitent with other interval operations

2020-02-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30919. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27672

[jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark

2020-02-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044514#comment-17044514 ] Steve Loughran commented on SPARK-6305: --- bq. Also note that Log4j 2 now offers experimental support

[jira] [Commented] (SPARK-27148) Support CURRENT_TIME and LOCALTIME when ANSI mode enabled

2020-02-25 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044470#comment-17044470 ] YoungGyu Chun commented on SPARK-27148: --- [~pavithraramachandran] are you still working on? >

[jira] [Commented] (SPARK-3875) Add TEMP DIRECTORY configuration

2020-02-25 Thread Shyam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-3875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044467#comment-17044467 ] Shyam commented on SPARK-3875: -- [~srowen] ,  I am still getting this error ,  its suggested this is the fix 

[jira] [Created] (SPARK-30947) Don't log accelerate resources when it's empty

2020-02-25 Thread wuyi (Jira)
wuyi created SPARK-30947: Summary: Don't log accelerate resources when it's empty Key: SPARK-30947 URL: https://issues.apache.org/jira/browse/SPARK-30947 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-30946) FileStreamSourceLog/FileStreamSinkLog: leverage UnsafeRow type to serialize/deserialize entry

2020-02-25 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-30946: Summary: FileStreamSourceLog/FileStreamSinkLog: leverage UnsafeRow type to serialize/deserialize entry Key: SPARK-30946 URL: https://issues.apache.org/jira/browse/SPARK-30946

[jira] [Commented] (SPARK-30855) Issue using 'explode' function followed by a (*)star expand selection of resulting struct

2020-02-25 Thread Benoit Roy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044453#comment-17044453 ] Benoit Roy commented on SPARK-30855: Awesome.  Thanks for contributing the PR.  I see that this PR

[jira] [Updated] (SPARK-30476) NullPointerException when Insert data to hive mongo external table by spark-sql

2020-02-25 Thread XiongCheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiongCheng updated SPARK-30476: --- Attachment: (was: SPARK-30885-master.patch) > NullPointerException when Insert data to hive

[jira] [Updated] (SPARK-30476) NullPointerException when Insert data to hive mongo external table by spark-sql

2020-02-25 Thread XiongCheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiongCheng updated SPARK-30476: --- Attachment: SPARK-30885-master.patch > NullPointerException when Insert data to hive mongo external

[jira] [Commented] (SPARK-28024) Incorrect numeric values when out of range

2020-02-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044374#comment-17044374 ] Wenchen Fan commented on SPARK-28024: - I've lowered the priority from blocker to major. >

[jira] [Updated] (SPARK-28024) Incorrect numeric values when out of range

2020-02-25 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-28024: Priority: Major (was: Blocker) > Incorrect numeric values when out of range >

[jira] [Commented] (SPARK-30096) drop function does not delete the jars file from tmp folder- session is not getting clear

2020-02-25 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044353#comment-17044353 ] Sandeep Katta commented on SPARK-30096: --- [~abhishek.akg] I have cross checked the behaviour, those

[jira] [Updated] (SPARK-30096) drop function does not delete the jars file from tmp folder- session is not getting clear

2020-02-25 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Katta updated SPARK-30096: -- Attachment: screenshot-1.png > drop function does not delete the jars file from tmp folder-

[jira] [Reopened] (SPARK-30476) NullPointerException when Insert data to hive mongo external table by spark-sql

2020-02-25 Thread XiongCheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiongCheng reopened SPARK-30476: > NullPointerException when Insert data to hive mongo external table by > spark-sql >

[jira] [Updated] (SPARK-30476) NullPointerException when Insert data to hive mongo external table by spark-sql

2020-02-25 Thread XiongCheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiongCheng updated SPARK-30476: --- Issue Type: Bug (was: Improvement) > NullPointerException when Insert data to hive mongo external

[jira] [Commented] (SPARK-30476) NullPointerException when Insert data to hive mongo external table by spark-sql

2020-02-25 Thread XiongCheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044322#comment-17044322 ] XiongCheng commented on SPARK-30476: The root of this problem is that SPARK does not continue to

[jira] [Updated] (SPARK-30476) NullPointerException when Insert data to hive mongo external table by spark-sql

2020-02-25 Thread XiongCheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiongCheng updated SPARK-30476: --- Priority: Minor (was: Major) > NullPointerException when Insert data to hive mongo external table

[jira] [Updated] (SPARK-30476) NullPointerException when Insert data to hive mongo external table by spark-sql

2020-02-25 Thread XiongCheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiongCheng updated SPARK-30476: --- Issue Type: Improvement (was: Bug) > NullPointerException when Insert data to hive mongo external

[jira] [Updated] (SPARK-30476) NullPointerException when Insert data to hive mongo external table by spark-sql

2020-02-25 Thread XiongCheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiongCheng updated SPARK-30476: --- Attachment: SPARK-30885-master.patch > NullPointerException when Insert data to hive mongo external

[jira] [Updated] (SPARK-30841) Add version information to the configuration of SQL

2020-02-25 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-30841: --- Summary: Add version information to the configuration of SQL (was: Display the version of Spark

[jira] [Resolved] (SPARK-30830) method setQueryTimeout not support

2020-02-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-30830. -- Resolution: Not A Problem > method setQueryTimeout not support >

[jira] [Comment Edited] (SPARK-30830) method setQueryTimeout not support

2020-02-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044205#comment-17044205 ] Hyukjin Kwon edited comment on SPARK-30830 at 2/25/20 8:02 AM: --- That

[jira] [Commented] (SPARK-30830) method setQueryTimeout not support

2020-02-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044205#comment-17044205 ] Hyukjin Kwon commented on SPARK-30830: -- That should be fixed in the JDBC driver. Spark will also