[jira] [Commented] (SPARK-26175) PySpark cannot terminate worker process if user program reads from stdin

2018-11-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16699337#comment-16699337 ] Xiao Li commented on SPARK-26175: - cc [~hyukjin.kwon] [~bryanc] [~icexelloss] > PySpark cannot

[jira] [Updated] (SPARK-26175) PySpark cannot terminate worker process if user program reads from stdin

2018-11-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26175: Target Version/s: 3.0.0 > PySpark cannot terminate worker process if user program reads from stdin >

[jira] [Updated] (SPARK-26176) Verify column name when creating table via `STORED AS`

2018-11-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26176: Issue Type: Bug (was: Test) > Verify column name when creating table via `STORED AS` >

[jira] [Updated] (SPARK-26176) Verify column name when creating table via `STORED AS`

2018-11-26 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26176: Labels: starter (was: ) > Verify column name when creating table via `STORED AS` >

[jira] [Created] (SPARK-26176) Verify column name when creating table via `STORED AS`

2018-11-26 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26176: --- Summary: Verify column name when creating table via `STORED AS` Key: SPARK-26176 URL: https://issues.apache.org/jira/browse/SPARK-26176 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-25860) Replace Literal(null, _) with FalseLiteral whenever possible

2018-11-25 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-25860: --- Assignee: Anton Okolnychyi > Replace Literal(null, _) with FalseLiteral whenever possible >

[jira] [Created] (SPARK-26169) Create DataFrameSetOperationsSuite

2018-11-25 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26169: --- Summary: Create DataFrameSetOperationsSuite Key: SPARK-26169 URL: https://issues.apache.org/jira/browse/SPARK-26169 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-26168) Update the code comments in Expression and Aggregate

2018-11-25 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26168: --- Summary: Update the code comments in Expression and Aggregate Key: SPARK-26168 URL: https://issues.apache.org/jira/browse/SPARK-26168 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-26140) Enable custom shuffle metrics implementation in shuffle reader

2018-11-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26140. - Resolution: Fixed Fix Version/s: 3.0.0 > Enable custom shuffle metrics implementation in shuffle

[jira] [Commented] (SPARK-26022) PySpark Comparison with Pandas

2018-11-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684283#comment-16684283 ] Xiao Li commented on SPARK-26022: - [~hyukjin.kwon] Could you lead this effort to help the community

[jira] [Created] (SPARK-26022) PySpark Comparison with Pandas

2018-11-12 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26022: --- Summary: PySpark Comparison with Pandas Key: SPARK-26022 URL: https://issues.apache.org/jira/browse/SPARK-26022 Project: Spark Issue Type: Documentation

[jira] [Resolved] (SPARK-26005) Upgrade ANTRL to 4.7.1

2018-11-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-26005. - Resolution: Fixed Fix Version/s: 3.0.0 > Upgrade ANTRL to 4.7.1 > -- > >

[jira] [Updated] (SPARK-25914) Separate projection from grouping and aggregate in logical Aggregate

2018-11-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25914: Target Version/s: 3.0.0 > Separate projection from grouping and aggregate in logical Aggregate >

[jira] [Created] (SPARK-26005) Upgrade ANTRL to 4.7.1

2018-11-11 Thread Xiao Li (JIRA)
Xiao Li created SPARK-26005: --- Summary: Upgrade ANTRL to 4.7.1 Key: SPARK-26005 URL: https://issues.apache.org/jira/browse/SPARK-26005 Project: Spark Issue Type: Bug Components: SQL

[jira] [Resolved] (SPARK-25102) Write Spark version to ORC/Parquet file metadata

2018-11-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25102. - Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 3.0.0 > Write Spark version to

[jira] [Created] (SPARK-25993) Add test cases for resolution of ORC table location

2018-11-09 Thread Xiao Li (JIRA)
Xiao Li created SPARK-25993: --- Summary: Add test cases for resolution of ORC table location Key: SPARK-25993 URL: https://issues.apache.org/jira/browse/SPARK-25993 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-25993) Add test cases for resolution of ORC table location

2018-11-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25993: Labels: starter (was: ) > Add test cases for resolution of ORC table location >

[jira] [Resolved] (SPARK-25979) Window function: allow parentheses around window reference

2018-11-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25979. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 3.0.0 2.4.1

[jira] [Resolved] (SPARK-25988) Keep names unchanged when deduplicating the column names in Analyzer

2018-11-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25988. - Resolution: Fixed Fix Version/s: 3.0.0 2.4.1 > Keep names unchanged when

[jira] [Created] (SPARK-25988) Keep names unchanged when deduplicating the column names in Analyzer

2018-11-08 Thread Xiao Li (JIRA)
Xiao Li created SPARK-25988: --- Summary: Keep names unchanged when deduplicating the column names in Analyzer Key: SPARK-25988 URL: https://issues.apache.org/jira/browse/SPARK-25988 Project: Spark

[jira] [Commented] (SPARK-25966) "EOF Reached the end of stream with bytes left to read" while reading/writing to Parquets

2018-11-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680901#comment-16680901 ] Xiao Li commented on SPARK-25966: - Thank you for reporting this. I think this is not an issue. Please

[jira] [Updated] (SPARK-25986) Banning throw new OutOfMemoryErrors

2018-11-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25986: Description: Adding a linter rule to ban the construction of new OutOfMemoryErrors and then make sure

[jira] [Created] (SPARK-25986) Banning throw new OutOfMemoryErrors

2018-11-08 Thread Xiao Li (JIRA)
Xiao Li created SPARK-25986: --- Summary: Banning throw new OutOfMemoryErrors Key: SPARK-25986 URL: https://issues.apache.org/jira/browse/SPARK-25986 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-25985) Verify the SPARK-24613 Cache with UDF could not be matched with subsequent dependent caches

2018-11-08 Thread Xiao Li (JIRA)
Xiao Li created SPARK-25985: --- Summary: Verify the SPARK-24613 Cache with UDF could not be matched with subsequent dependent caches Key: SPARK-25985 URL: https://issues.apache.org/jira/browse/SPARK-25985

[jira] [Commented] (SPARK-25966) "EOF Reached the end of stream with bytes left to read" while reading/writing to Parquets

2018-11-07 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678527#comment-16678527 ] Xiao Li commented on SPARK-25966: - Do you still have the file that fail your job? Can you use the

[jira] [Updated] (SPARK-24561) User-defined window functions with pandas udf (bounded window)

2018-11-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24561: Target Version/s: 3.0.0 > User-defined window functions with pandas udf (bounded window) >

[jira] [Resolved] (SPARK-25913) Unary SparkPlan nodes should extend UnaryExecNode

2018-11-04 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25913. - Resolution: Fixed Assignee: Maxim Gekk Fix Version/s: 3.0.0 > Unary SparkPlan nodes

[jira] [Updated] (SPARK-23549) Spark SQL unexpected behavior when comparing timestamp to date

2018-11-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23549: Labels: release_notes (was: ) > Spark SQL unexpected behavior when comparing timestamp to date >

[jira] [Updated] (SPARK-25769) UnresolvedAttribute.sql() incorrectly escapes nested columns

2018-11-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25769: Labels: (was: sql) > UnresolvedAttribute.sql() incorrectly escapes nested columns >

[jira] [Assigned] (SPARK-25916) Add `resultExpressions` in logical `Aggregate`

2018-11-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-25916: --- Assignee: Dilip Biswal (was: Xiao Li) > Add `resultExpressions` in logical `Aggregate` >

[jira] [Assigned] (SPARK-25916) Add `resultExpressions` in logical `Aggregate`

2018-11-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-25916: --- Assignee: Xiao Li > Add `resultExpressions` in logical `Aggregate` >

[jira] [Assigned] (SPARK-25915) Replace grouping expressions with references in `aggregateExpressions` of logical `Aggregate`

2018-11-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-25915: --- Assignee: Dilip Biswal > Replace grouping expressions with references in `aggregateExpressions` of

[jira] [Assigned] (SPARK-25914) Separate projection from grouping and aggregate in logical Aggregate

2018-11-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-25914: --- Assignee: Dilip Biswal > Separate projection from grouping and aggregate in logical Aggregate >

[jira] [Resolved] (SPARK-25899) Flaky test: CoarseGrainedSchedulerBackendSuite.compute max number of concurrent tasks can be launched

2018-10-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25899. - Resolution: Fixed Fix Version/s: 3.0.0 2.4.1 > Flaky test:

[jira] [Resolved] (SPARK-25883) Override method `prettyName` in `from_avro`/`to_avro`

2018-10-31 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25883. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 3.0.0 > Override method

[jira] [Resolved] (SPARK-25862) Remove rangeBetween APIs introduced in SPARK-21608

2018-10-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25862. - Resolution: Fixed Fix Version/s: 3.0.0 > Remove rangeBetween APIs introduced in SPARK-21608 >

[jira] [Updated] (SPARK-25767) Error reported in Spark logs when using the org.apache.spark:spark-sql_2.11:2.3.2 Java library

2018-10-30 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25767: Component/s: (was: Java API) SQL > Error reported in Spark logs when using the >

[jira] [Resolved] (SPARK-25179) Document the features that require Pyarrow 0.10

2018-10-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25179. - Resolution: Fixed Fix Version/s: 2.4.0 > Document the features that require Pyarrow 0.10 >

[jira] [Updated] (SPARK-25674) If the records are incremented by more than 1 at a time,the number of bytes might rarely ever get updated

2018-10-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25674: Fix Version/s: (was: 2.4.1) (was: 3.0.0) 2.4.0 > If the

[jira] [Updated] (SPARK-25636) spark-submit swallows the failure reason when there is an error connecting to master

2018-10-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25636: Fix Version/s: (was: 2.4.1) (was: 3.0.0) 2.4.0 >

[jira] [Updated] (SPARK-25677) Configuring zstd compression in JDBC throwing IllegalArgumentException Exception

2018-10-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25677: Fix Version/s: (was: 2.4.1) (was: 3.0.0) 2.4.0 >

[jira] [Updated] (SPARK-25639) Add documentation on foreachBatch, and multiple watermark policy

2018-10-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25639: Fix Version/s: (was: 2.4.1) 2.4.0 > Add documentation on foreachBatch, and

[jira] [Updated] (SPARK-24787) Events being dropped at an alarming rate due to hsync being slow for eventLogging

2018-10-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24787: Fix Version/s: (was: 2.4.1) (was: 3.0.0) 2.4.0 > Events

[jira] [Updated] (SPARK-25805) Flaky test: DataFrameSuite.SPARK-25159 unittest failure

2018-10-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25805: Fix Version/s: (was: 2.4.1) (was: 3.0.0) 2.4.0 > Flaky

[jira] [Resolved] (SPARK-25816) Functions does not resolve Columns correctly

2018-10-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25816. - Resolution: Fixed Fix Version/s: 2.4.0 2.3.3 > Functions does not resolve

[jira] [Updated] (SPARK-25795) Fix CSV SparkR SQL Example

2018-10-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25795: Fix Version/s: (was: 2.4.1) (was: 3.0.0) 2.4.0 > Fix CSV

[jira] [Updated] (SPARK-25803) The -n option to docker-image-tool.sh causes other options to be ignored

2018-10-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25803: Fix Version/s: (was: 2.4.1) (was: 3.0.0) 2.4.0 > The -n

[jira] [Updated] (SPARK-25697) When zstd compression enabled in progress application is throwing Error in UI

2018-10-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25697: Fix Version/s: (was: 2.4.1) 2.4.0 > When zstd compression enabled in progress

[jira] [Assigned] (SPARK-25816) Functions does not resolve Columns correctly

2018-10-28 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-25816: --- Assignee: Peter Toth > Functions does not resolve Columns correctly >

[jira] [Resolved] (SPARK-25270) lint-python: Add flake8 to find syntax errors and undefined names

2018-10-21 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25270. - Resolution: Fixed Fix Version/s: 3.0.0 > lint-python: Add flake8 to find syntax errors and

[jira] [Commented] (SPARK-25783) Spark shell fails because of jline incompatibility

2018-10-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657368#comment-16657368 ] Xiao Li commented on SPARK-25783: - cc [~dbtsai] Could you take a look? > Spark shell fails because of

[jira] [Commented] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655776#comment-16655776 ] Xiao Li commented on SPARK-24499: - Feel free to create the extra tasks to improve the documentation. I

[jira] [Resolved] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24499. - Resolution: Fixed Assignee: Yuanjian Li Fix Version/s: 2.4.0 Target

[jira] [Updated] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24499: Summary: Split the page of sql-programming-guide.html to multiple separate pages (was: Documentation

[jira] [Updated] (SPARK-24499) Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24499: Component/s: (was: Spark Core) > Split the page of sql-programming-guide.html to multiple separate

[jira] [Commented] (SPARK-23390) Flaky test: FileBasedDataSourceSuite

2018-10-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654417#comment-16654417 ] Xiao Li commented on SPARK-23390: - Thanks! > Flaky test: FileBasedDataSourceSuite >

[jira] [Assigned] (SPARK-23390) Flaky test: FileBasedDataSourceSuite

2018-10-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-23390: --- Assignee: Dongjoon Hyun (was: Wenchen Fan) > Flaky test: FileBasedDataSourceSuite >

[jira] [Updated] (SPARK-24424) Support ANSI-SQL compliant syntax for GROUPING SET

2018-10-17 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24424: Description: Currently, our Group By clause follows Hive

[jira] [Updated] (SPARK-24433) R Bindings for K8S

2018-10-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24433: Summary: R Bindings for K8S (was: Add Spark R support) > R Bindings for K8S > -- > >

[jira] [Updated] (SPARK-24215) Implement eager evaluation for DataFrame APIs

2018-10-16 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-24215: Summary: Implement eager evaluation for DataFrame APIs (was: Implement __repr__ and _repr_html_ for

[jira] [Resolved] (SPARK-25716) Project and Aggregate generate valid constraints with unnecessary operation

2018-10-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25716. - Resolution: Fixed Assignee: SongYadong Fix Version/s: 3.0.0 > Project and Aggregate

[jira] [Updated] (SPARK-25547) Pluggable jdbc connection factory

2018-10-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25547: Target Version/s: 3.0.0 > Pluggable jdbc connection factory > - > >

[jira] [Updated] (SPARK-25727) makeCopy failed in InMemoryRelation

2018-10-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25727: Description: {code} val data = Seq(100).toDF("count").cache()

[jira] [Created] (SPARK-25727) makeCopy failed in InMemoryRelation

2018-10-13 Thread Xiao Li (JIRA)
Xiao Li created SPARK-25727: --- Summary: makeCopy failed in InMemoryRelation Key: SPARK-25727 URL: https://issues.apache.org/jira/browse/SPARK-25727 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-25372) Deprecate Yarn-specific configs in regards to keytab login for SparkSubmit

2018-10-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25372: Labels: release-notes (was: ) > Deprecate Yarn-specific configs in regards to keytab login for

[jira] [Updated] (SPARK-25714) Null Handling in the Optimizer rule BooleanSimplification

2018-10-13 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25714: Fix Version/s: (was: 2.3.3) > Null Handling in the Optimizer rule BooleanSimplification >

[jira] [Resolved] (SPARK-25714) Null Handling in the Optimizer rule BooleanSimplification

2018-10-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25714. - Resolution: Fixed Fix Version/s: 2.4.0 2.3.3 Target Version/s:

[jira] [Resolved] (SPARK-25660) Impossible to use the backward slash as the CSV fields delimiter

2018-10-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25660. - Resolution: Fixed Assignee: Maxim Gekk Fix Version/s: 2.4.0 > Impossible to use the

[jira] [Resolved] (SPARK-25708) HAVING without GROUP BY means global aggregate

2018-10-12 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25708. - Resolution: Fixed Fix Version/s: 2.4.0 > HAVING without GROUP BY means global aggregate >

[jira] [Resolved] (SPARK-25690) Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely

2018-10-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25690. - Resolution: Fixed Assignee: Maryann Xue Fix Version/s: 2.4.0 > Analyzer rule

[jira] [Updated] (SPARK-25714) Null Handling in the Optimizer rule BooleanSimplification

2018-10-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25714: Target Version/s: 2.3.2, 2.2.2, 2.4.0 Priority: Blocker (was: Major) > Null Handling in the

[jira] [Created] (SPARK-25714) Null Handling in the Optimizer rule BooleanSimplification

2018-10-11 Thread Xiao Li (JIRA)
Xiao Li created SPARK-25714: --- Summary: Null Handling in the Optimizer rule BooleanSimplification Key: SPARK-25714 URL: https://issues.apache.org/jira/browse/SPARK-25714 Project: Spark Issue Type:

[jira] [Updated] (SPARK-25714) Null Handling in the Optimizer rule BooleanSimplification

2018-10-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25714: Labels: correctness (was: ) > Null Handling in the Optimizer rule BooleanSimplification >

[jira] [Updated] (SPARK-25708) HAVING without GROUP BY means global aggregate

2018-10-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25708: Target Version/s: 2.4.0 > HAVING without GROUP BY means global aggregate >

[jira] [Assigned] (SPARK-25708) HAVING without GROUP BY means global aggregate

2018-10-11 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-25708: --- Assignee: Wenchen Fan > HAVING without GROUP BY means global aggregate >

[jira] [Commented] (SPARK-24130) Data Source V2: Join Push Down

2018-10-10 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645565#comment-16645565 ] Xiao Li commented on SPARK-24130: - Any data source migration work is being blocked by

[jira] [Commented] (SPARK-22390) Aggregate push down

2018-10-10 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645567#comment-16645567 ] Xiao Li commented on SPARK-22390: - Any data source migration work is being blocked by

[jira] [Updated] (SPARK-25640) Clarify/Improve EvalType for grouped aggregate and window aggregate

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25640: Target Version/s: 3.0.0 > Clarify/Improve EvalType for grouped aggregate and window aggregate >

[jira] [Commented] (SPARK-25690) Analyzer rule "HandleNullInputsForUDF" does not stabilize and can be applied infinitely

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16644273#comment-16644273 ] Xiao Li commented on SPARK-25690: - The changes made in https://issues.apache.org/jira/browse/SPARK-25044

[jira] [Updated] (SPARK-25692) Flaky test: ChunkFetchIntegrationSuite

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25692: Affects Version/s: (was: 2.4.0) 3.0.0 > Flaky test: ChunkFetchIntegrationSuite

[jira] [Updated] (SPARK-25692) Flaky test: ChunkFetchIntegrationSuite

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25692: Priority: Blocker (was: Major) > Flaky test: ChunkFetchIntegrationSuite >

[jira] [Commented] (SPARK-25688) Potential resource leak in ORC

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643910#comment-16643910 ] Xiao Li commented on SPARK-25688: -

[jira] [Commented] (SPARK-25688) Potential resource leak in ORC

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643870#comment-16643870 ] Xiao Li commented on SPARK-25688: - It sounds like ORC still has a resource leak even after the latest

[jira] [Updated] (SPARK-25688) Potential resource leak in ORC

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25688: Summary: Potential resource leak in ORC (was: org.apache.spark.sql.FileBasedDataSourceSuite never pass)

[jira] [Updated] (SPARK-25688) org.apache.spark.sql.FileBasedDataSourceSuite never pass

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25688: Description:

[jira] [Updated] (SPARK-25688) org.apache.spark.sql.FileBasedDataSourceSuite never pass

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25688: Priority: Critical (was: Blocker) > org.apache.spark.sql.FileBasedDataSourceSuite never pass >

[jira] [Created] (SPARK-25688) org.apache.spark.sql.FileBasedDataSourceSuite never pass

2018-10-09 Thread Xiao Li (JIRA)
Xiao Li created SPARK-25688: --- Summary: org.apache.spark.sql.FileBasedDataSourceSuite never pass Key: SPARK-25688 URL: https://issues.apache.org/jira/browse/SPARK-25688 Project: Spark Issue Type:

[jira] [Updated] (SPARK-25688) org.apache.spark.sql.FileBasedDataSourceSuite never pass

2018-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25688: Priority: Blocker (was: Major) > org.apache.spark.sql.FileBasedDataSourceSuite never pass >

[jira] [Commented] (SPARK-25591) PySpark Accumulators with multiple PythonUDFs

2018-10-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642506#comment-16642506 ] Xiao Li commented on SPARK-25591: - RC3 is not out yet. Thus, RC3 will include the fix. > PySpark

[jira] [Updated] (SPARK-25591) PySpark Accumulators with multiple PythonUDFs

2018-10-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25591: Labels: correctness (was: data-loss) > PySpark Accumulators with multiple PythonUDFs >

[jira] [Updated] (SPARK-25591) PySpark Accumulators with multiple PythonUDFs

2018-10-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25591: Priority: Blocker (was: Critical) > PySpark Accumulators with multiple PythonUDFs >

[jira] [Updated] (SPARK-25591) PySpark Accumulators with multiple PythonUDFs

2018-10-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25591: Fix Version/s: (was: 2.4.1) 2.4.0 > PySpark Accumulators with multiple PythonUDFs

[jira] [Resolved] (SPARK-25630) HiveOrcHadoopFsRelationSuite: SPARK-8406: Avoids name collision while writing files 21 sec

2018-10-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25630. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 3.0.0 >

[jira] [Resolved] (SPARK-25671) Build external/spark-ganglia-lgpl in Jenkins Test

2018-10-06 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25671. - Resolution: Fixed Fix Version/s: 2.4.0 > Build external/spark-ganglia-lgpl in Jenkins Test >

[jira] [Created] (SPARK-25671) Build external/spark-ganglia-lgpl in Jenkins Test

2018-10-06 Thread Xiao Li (JIRA)
Xiao Li created SPARK-25671: --- Summary: Build external/spark-ganglia-lgpl in Jenkins Test Key: SPARK-25671 URL: https://issues.apache.org/jira/browse/SPARK-25671 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-25610) DatasetCacheSuite: cache UDF result correctly 25 seconds

2018-10-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25610. - Resolution: Fixed Fix Version/s: 3.0.0 > DatasetCacheSuite: cache UDF result correctly 25

[jira] [Assigned] (SPARK-25610) DatasetCacheSuite: cache UDF result correctly 25 seconds

2018-10-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-25610: --- Assignee: Dilip Biswal > DatasetCacheSuite: cache UDF result correctly 25 seconds >

[jira] [Resolved] (SPARK-20536) Extend ColumnName to create StructFields with explicit nullable

2018-10-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20536. - Resolution: Won't Fix > Extend ColumnName to create StructFields with explicit nullable >

[jira] [Resolved] (SPARK-25653) Add tag ExtendedHiveTest for HiveSparkSubmitSuite

2018-10-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25653. - Resolution: Fixed Assignee: Gengliang Wang Fix Version/s: 3.0.0 > Add tag

[jira] [Resolved] (SPARK-25635) Support selective direct encoding in native ORC write

2018-10-05 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25635. - Resolution: Fixed Fix Version/s: 3.0.0 > Support selective direct encoding in native ORC write >

<    5   6   7   8   9   10   11   12   13   14   >