[jira] [Resolved] (SPARK-27356) File source V2: return actual schema in method `FileScan.readSchema`

2019-04-04 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-27356. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 24284

[jira] [Assigned] (SPARK-27356) File source V2: return actual schema in method `FileScan.readSchema`

2019-04-04 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-27356: --- Assignee: Gengliang Wang > File source V2: return actual schema in method

[jira] [Resolved] (SPARK-27342) Optimize limit 0 queries

2019-04-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-27342. - Resolution: Fixed Assignee: Aayushmaan Jain Fix Version/s: 3.0.0 > Optimize limit 0

[jira] [Closed] (SPARK-26057) Table joining is broken in Spark 2.4

2019-04-04 Thread Pavel Parkhomenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Parkhomenko closed SPARK-26057. - Thank you very much, it seems fine in 2.4.1 > Table joining is broken in Spark 2.4 >

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Description: Add an expression encoder for objects defined by properties (ie. I call

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Description: Add an expression encoder for objects defined by properties (ie. I call

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Description: Add an expression encoder for objects defined by properties (ie. I call

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Description: Add an expression encoder for objects defined by properties (ie. I call

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Description: Add an expression encoder for objects (ie. Avro) defined by properties

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Description: Add an expression encoder for objects (ie. Avro) defined by properties

[jira] [Commented] (SPARK-27395) Improve EXPLAIN command

2019-04-04 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810431#comment-16810431 ] Dilip Biswal commented on SPARK-27395: -- [~smilegator] I would like to look into this. > Improve

[jira] [Updated] (SPARK-27395) Improve EXPLAIN command

2019-04-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-27395: Description: Currently, when the query is complex or the output schema is long, the outputs of our

[jira] [Created] (SPARK-27395) Improve EXPLAIN command

2019-04-04 Thread Xiao Li (JIRA)
Xiao Li created SPARK-27395: --- Summary: Improve EXPLAIN command Key: SPARK-27395 URL: https://issues.apache.org/jira/browse/SPARK-27395 Project: Spark Issue Type: Umbrella Components: SQL

[jira] [Updated] (SPARK-27216) Upgrade RoaringBitmap to 0.7.45 to fix Kryo unsafe ser/dser issue

2019-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-27216: -- Fix Version/s: 2.4.2 2.3.4 > Upgrade RoaringBitmap to 0.7.45 to fix Kryo unsafe

[jira] [Assigned] (SPARK-24345) Improve ParseError stop location when offending symbol is a token

2019-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-24345: - Assignee: Ruben Fiszel > Improve ParseError stop location when offending symbol is a token >

[jira] [Resolved] (SPARK-24345) Improve ParseError stop location when offending symbol is a token

2019-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24345. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21334

[jira] [Created] (SPARK-27394) The staleness of UI may last minutes or hours when no tasks start or finish

2019-04-04 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-27394: Summary: The staleness of UI may last minutes or hours when no tasks start or finish Key: SPARK-27394 URL: https://issues.apache.org/jira/browse/SPARK-27394 Project:

[jira] [Assigned] (SPARK-27394) The staleness of UI may last minutes or hours when no tasks start or finish

2019-04-04 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-27394: Assignee: Shixiong Zhu > The staleness of UI may last minutes or hours when no tasks

[jira] [Created] (SPARK-27393) Show ReusedSubquery in the plan when the subquery is reused

2019-04-04 Thread Xiao Li (JIRA)
Xiao Li created SPARK-27393: --- Summary: Show ReusedSubquery in the plan when the subquery is reused Key: SPARK-27393 URL: https://issues.apache.org/jira/browse/SPARK-27393 Project: Spark Issue

[jira] [Created] (SPARK-27392) TestHive test tables should be placed in shared test state, not per session

2019-04-04 Thread Eric Liang (JIRA)
Eric Liang created SPARK-27392: -- Summary: TestHive test tables should be placed in shared test state, not per session Key: SPARK-27392 URL: https://issues.apache.org/jira/browse/SPARK-27392 Project:

[jira] [Created] (SPARK-27391) deadlock in ContinuousExecution unit tests

2019-04-04 Thread Jose Torres (JIRA)
Jose Torres created SPARK-27391: --- Summary: deadlock in ContinuousExecution unit tests Key: SPARK-27391 URL: https://issues.apache.org/jira/browse/SPARK-27391 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Description: Add an expression encoder for objects (ie. Avro) defined by properties

[jira] [Created] (SPARK-27390) Fix package name mismatch

2019-04-04 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-27390: - Summary: Fix package name mismatch Key: SPARK-27390 URL: https://issues.apache.org/jira/browse/SPARK-27390 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-27390) Fix package name mismatch

2019-04-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27390: -- Component/s: Tests > Fix package name mismatch > - > >

[jira] [Updated] (SPARK-27389) pyspark test failures w/ "UnknownTimeZoneError: 'US/Pacific-New'"

2019-04-04 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-27389: - Summary: pyspark test failures w/ "UnknownTimeZoneError: 'US/Pacific-New'" (was: Odd failures

[jira] [Resolved] (SPARK-27382) Update Spark 2.4.x testing in HiveExternalCatalogVersionsSuite

2019-04-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-27382. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 3.0.0

[jira] [Created] (SPARK-27389) Odd failures w/ "UnknownTimeZoneError: 'US/Pacific-New'"

2019-04-04 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-27389: Summary: Odd failures w/ "UnknownTimeZoneError: 'US/Pacific-New'" Key: SPARK-27389 URL: https://issues.apache.org/jira/browse/SPARK-27389 Project: Spark

[jira] [Updated] (SPARK-27389) Odd failures w/ "UnknownTimeZoneError: 'US/Pacific-New'"

2019-04-04 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-27389: - Component/s: PySpark > Odd failures w/ "UnknownTimeZoneError: 'US/Pacific-New'" >

[jira] [Comment Edited] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810220#comment-16810220 ] Taoufik DACHRAOUI edited comment on SPARK-27388 at 4/4/19 8:40 PM: ---

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Affects Version/s: (was: 2.3.3) (was: 2.3.2)

[jira] [Commented] (SPARK-27332) Filter Pushdown duplicates expensive ScalarSubquery (discarding result)

2019-04-04 Thread David Stewart Zink (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810249#comment-16810249 ] David Stewart Zink commented on SPARK-27332: I'm a little confused by that comment chain.

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Description: Add an expression encoder for objects (ie. Avro) defined by properties

[jira] [Updated] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Taoufik DACHRAOUI updated SPARK-27388: -- Description: Add an expression encoder for objects (ie. Avro) defined by properties

[jira] [Comment Edited] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810220#comment-16810220 ] Taoufik DACHRAOUI edited comment on SPARK-27388 at 4/4/19 7:42 PM: ---

[jira] [Comment Edited] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810220#comment-16810220 ] Taoufik DACHRAOUI edited comment on SPARK-27388 at 4/4/19 7:41 PM: ---

[jira] [Commented] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810220#comment-16810220 ] Taoufik DACHRAOUI commented on SPARK-27388: --- Hi,   You can find the implementation in 

[jira] [Created] (SPARK-27388) expression encoder for avro like objects

2019-04-04 Thread Taoufik DACHRAOUI (JIRA)
Taoufik DACHRAOUI created SPARK-27388: - Summary: expression encoder for avro like objects Key: SPARK-27388 URL: https://issues.apache.org/jira/browse/SPARK-27388 Project: Spark Issue

[jira] [Commented] (SPARK-27387) Replace sqlutils assertPandasEqual with Pandas assert_frame_equal in tests

2019-04-04 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810200#comment-16810200 ] Bryan Cutler commented on SPARK-27387: -- I can work on this > Replace sqlutils assertPandasEqual

[jira] [Commented] (SPARK-27387) Replace sqlutils assertPandasEqual with Pandas assert_frame_equal in tests

2019-04-04 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810197#comment-16810197 ] Bryan Cutler commented on SPARK-27387: -- This can be done after the upgrade of pyarrow version to

[jira] [Created] (SPARK-27387) Replace sqlutils assertPandasEqual with Pandas assert_frame_equal in tests

2019-04-04 Thread Bryan Cutler (JIRA)
Bryan Cutler created SPARK-27387: Summary: Replace sqlutils assertPandasEqual with Pandas assert_frame_equal in tests Key: SPARK-27387 URL: https://issues.apache.org/jira/browse/SPARK-27387 Project:

[jira] [Commented] (SPARK-21784) Add ALTER TABLE ADD CONSTRANT DDL to support defining primary key and foreign keys

2019-04-04 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810124#comment-16810124 ] Ruslan Dautkhanov commented on SPARK-21784: --- Any chance this can be part of Spark 3.0 release?

[jira] [Created] (SPARK-27386) Improve partition transform parsing

2019-04-04 Thread Ryan Blue (JIRA)
Ryan Blue created SPARK-27386: - Summary: Improve partition transform parsing Key: SPARK-27386 URL: https://issues.apache.org/jira/browse/SPARK-27386 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-27276) Increase the minimum pyarrow version to 0.12.1

2019-04-04 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810024#comment-16810024 ] Bryan Cutler commented on SPARK-27276: -- I think we should use 0.12.1, there was a bug fix

[jira] [Updated] (SPARK-27276) Increase the minimum pyarrow version to 0.12.1

2019-04-04 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler updated SPARK-27276: - Summary: Increase the minimum pyarrow version to 0.12.1 (was: Increase the minimum pyarrow

[jira] [Commented] (SPARK-26393) Different behaviors of date_add when calling it inside expr

2019-04-04 Thread Brandon Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810015#comment-16810015 ] Brandon Bradley commented on SPARK-26393: - I'm experiencing this as well; except I get

[jira] [Commented] (SPARK-27353) PySpark Row __repr__ bug

2019-04-04 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810009#comment-16810009 ] Bryan Cutler commented on SPARK-27353: -- Works for me out of master, can you provide a script to

[jira] [Commented] (SPARK-27375) cache not working after discretizer.fit(df).transform(df)

2019-04-04 Thread Zhenyi Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809978#comment-16809978 ] Zhenyi Lin commented on SPARK-27375: Good to know that it is good in higher version. The physical

[jira] [Commented] (SPARK-27234) Continuous Streaming does not support python UDFs

2019-04-04 Thread venki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809924#comment-16809924 ] venki commented on SPARK-27234: --- FWIW i had a similar issue with python UDF not working w/ Continuous

[jira] [Commented] (SPARK-25994) SPIP: Property Graphs, Cypher Queries, and Algorithms

2019-04-04 Thread Martin Junghanns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809896#comment-16809896 ] Martin Junghanns commented on SPARK-25994: -- [~kanjilal] This might interest you: 

[jira] [Commented] (SPARK-27364) User-facing APIs for GPU-aware scheduling

2019-04-04 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809893#comment-16809893 ] Thomas Graves commented on SPARK-27364: --- working on this, will post a basic design when I have

[jira] [Updated] (SPARK-27024) Executor interface for cluster managers to support GPU resources

2019-04-04 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-27024: -- Description: The executor interface shall deal with the resources allocated to the executor

[jira] [Updated] (SPARK-27024) Executor interface for cluster managers to support GPU resources

2019-04-04 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-27024: -- Description: The executor interface shall deal with the resources allocated to the executor

[jira] [Commented] (SPARK-27299) Design: Property graph construction, save/load, and query APIs

2019-04-04 Thread Martin Junghanns (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809874#comment-16809874 ] Martin Junghanns commented on SPARK-27299: -- Done. I will open a draft PR that demonstrates the

[jira] [Commented] (SPARK-27332) Filter Pushdown duplicates expensive ScalarSubquery (discarding result)

2019-04-04 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809720#comment-16809720 ] Liang-Chi Hsieh commented on SPARK-27332: - I think this should be already fixed by SPARK-25482.

[jira] [Comment Edited] (SPARK-27385) use mapPartitionsWIthIndex in Dataframe

2019-04-04 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809706#comment-16809706 ] Liang-Chi Hsieh edited comment on SPARK-27385 at 4/4/19 10:39 AM: -- I

[jira] [Commented] (SPARK-27385) use mapPartitionsWIthIndex in Dataframe

2019-04-04 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809706#comment-16809706 ] Liang-Chi Hsieh commented on SPARK-27385: - I don't really see

[jira] [Commented] (SPARK-27375) cache not working after discretizer.fit(df).transform(df)

2019-04-04 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809704#comment-16809704 ] Liang-Chi Hsieh commented on SPARK-27375: - This is the physical plan when using current master

[jira] [Commented] (SPARK-27363) Mesos support for GPU-aware scheduling

2019-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809647#comment-16809647 ] Sean Owen commented on SPARK-27363: --- I'd open it to any Mesos contributors, though as you say, not

[jira] [Updated] (SPARK-27358) Update jquery to 1.12.x to pick up security fixes

2019-04-04 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-27358: -- Description: jquery 1.11.1 is affected by a CVE: https://www.cvedetails.com/cve/CVE-2016-7103/ This

[jira] [Updated] (SPARK-27385) use mapPartitionsWIthIndex in Dataframe

2019-04-04 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-27385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 范晓亮 updated SPARK-27385: Component/s: (was: Spark Core) SQL > use mapPartitionsWIthIndex in Dataframe >

[jira] [Created] (SPARK-27385) use mapPartitionsWIthIndex in Dataframe

2019-04-04 Thread JIRA
范晓亮 created SPARK-27385: --- Summary: use mapPartitionsWIthIndex in Dataframe Key: SPARK-27385 URL: https://issues.apache.org/jira/browse/SPARK-27385 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-27278) Optimize GetMapValue when the map is a foldable and the key is not

2019-04-04 Thread Huon Wilson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809554#comment-16809554 ] Huon Wilson commented on SPARK-27278: - That does seem to be what added this optimisation, but it

[jira] [Created] (SPARK-27384) File source V2: Prune unnecessary partition columns

2019-04-04 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-27384: -- Summary: File source V2: Prune unnecessary partition columns Key: SPARK-27384 URL: https://issues.apache.org/jira/browse/SPARK-27384 Project: Spark

[jira] [Comment Edited] (SPARK-27278) Optimize GetMapValue when the map is a foldable and the key is not

2019-04-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809541#comment-16809541 ] Dongjoon Hyun edited comment on SPARK-27278 at 4/4/19 6:06 AM: --- What you

[jira] [Updated] (SPARK-27278) Optimize GetMapValue when the map is a foldable and the key is not

2019-04-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-27278: -- Priority: Minor (was: Major) > Optimize GetMapValue when the map is a foldable and the key

[jira] [Commented] (SPARK-27298) Dataset except operation gives different results(dataset count) on Spark 2.3.0 Windows and Spark 2.3.0 Linux environment

2019-04-04 Thread Mahima Khatri (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809542#comment-16809542 ] Mahima Khatri commented on SPARK-27298: --- I downloaded "spark-2.3.3-bin-hadoop2.7" and tested and

[jira] [Commented] (SPARK-27278) Optimize GetMapValue when the map is a foldable and the key is not

2019-04-04 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809541#comment-16809541 ] Dongjoon Hyun commented on SPARK-27278: --- What you mean by the half is very old commit

[jira] [Updated] (SPARK-27298) Dataset except operation gives different results(dataset count) on Spark 2.3.0 Windows and Spark 2.3.0 Linux environment

2019-04-04 Thread Mahima Khatri (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahima Khatri updated SPARK-27298: -- Attachment: console-reslt-2.3.3-linux.txt > Dataset except operation gives different

[jira] [Updated] (SPARK-27298) Dataset except operation gives different results(dataset count) on Spark 2.3.0 Windows and Spark 2.3.0 Linux environment

2019-04-04 Thread Mahima Khatri (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahima Khatri updated SPARK-27298: -- Attachment: console-result-2.3.3-windows.txt > Dataset except operation gives different