[jira] [Commented] (SPARK-29206) Number of shuffle Netty server threads should be a multiple of number of chunk fetch handler threads

2019-09-22 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935556#comment-16935556 ] Min Shen commented on SPARK-29206: -- We initially tried an alternative approach to resolve this issue by

[jira] [Commented] (SPARK-29206) Number of shuffle Netty server threads should be a multiple of number of chunk fetch handler threads

2019-09-22 Thread Min Shen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935552#comment-16935552 ] Min Shen commented on SPARK-29206: -- [~redsanket], [~tgraves], Since you worked on committing the

[jira] [Created] (SPARK-29206) Number of shuffle Netty server threads should be a multiple of number of chunk fetch handler threads

2019-09-22 Thread Min Shen (Jira)
Min Shen created SPARK-29206: Summary: Number of shuffle Netty server threads should be a multiple of number of chunk fetch handler threads Key: SPARK-29206 URL: https://issues.apache.org/jira/browse/SPARK-29206

[jira] [Created] (SPARK-29205) Pyspark tests failed for suspected performance problem on ARM

2019-09-22 Thread zhao bo (Jira)
zhao bo created SPARK-29205: --- Summary: Pyspark tests failed for suspected performance problem on ARM Key: SPARK-29205 URL: https://issues.apache.org/jira/browse/SPARK-29205 Project: Spark Issue

[jira] [Updated] (SPARK-29016) Update LICENSE and NOTICE for Hive 2.3

2019-09-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29016: Summary: Update LICENSE and NOTICE for Hive 2.3 (was: Update, fix LICENSE and NOTICE for Hive

[jira] [Commented] (SPARK-29197) Remove saveModeForDSV2 in DataFrameWriter

2019-09-22 Thread Burak Yavuz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935498#comment-16935498 ] Burak Yavuz commented on SPARK-29197: - Hi Ido, Thanks for your interest. I already have a PR on it

[jira] [Commented] (SPARK-29204) Remove `Spark Release` Jenkins tab and its four jobs

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935433#comment-16935433 ] Dongjoon Hyun commented on SPARK-29204: --- Thanks. Specifically, it's about deleting the view and

[jira] [Commented] (SPARK-29204) Remove `Spark Release` Jenkins tab and its four jobs

2019-09-22 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935432#comment-16935432 ] Sean Owen commented on SPARK-29204: --- Is this just a matter of deleting the view? I have permissions to

[jira] [Resolved] (SPARK-29084) Check method bytecode size in BenchmarkQueryTest

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29084. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25788

[jira] [Assigned] (SPARK-29084) Check method bytecode size in BenchmarkQueryTest

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29084: - Assignee: Takeshi Yamamuro > Check method bytecode size in BenchmarkQueryTest >

[jira] [Commented] (SPARK-29204) Remove `Spark Release` Jenkins tab and its four jobs

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935423#comment-16935423 ] Dongjoon Hyun commented on SPARK-29204: --- cc [~shaneknapp] and [~srowen] > Remove `Spark Release`

[jira] [Updated] (SPARK-29204) Remove `Spark Release` Jenkins tab and its four jobs

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29204: -- Description: Since last two years, we didn't use `Spark Release` Jenkins jobs. Although we

[jira] [Updated] (SPARK-29204) Remove `Spark Release` Jenkins tab and its four jobs

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29204: -- Description: Since last two years, we didn't use `Spark Release` Jenkins jobs. Although we

[jira] [Created] (SPARK-29204) Remove `Spark Release` Jenkins tab and its four jobs

2019-09-22 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-29204: - Summary: Remove `Spark Release` Jenkins tab and its four jobs Key: SPARK-29204 URL: https://issues.apache.org/jira/browse/SPARK-29204 Project: Spark Issue

[jira] [Updated] (SPARK-29204) Remove `Spark Release` Jenkins tab and its four jobs

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29204: -- Attachment: Spark Release Jobs.png > Remove `Spark Release` Jenkins tab and its four jobs >

[jira] [Resolved] (SPARK-28599) Fix `Execution Time` and `Duration` column sorting for ThriftServerSessionPage

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-28599. --- Fix Version/s: 3.0.0 Assignee: Yuming Wang Resolution: Fixed This is

[jira] [Updated] (SPARK-28599) Fix wrong column sorting in ThriftServerSessionPage

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28599: -- Summary: Fix wrong column sorting in ThriftServerSessionPage (was: Support sorting for

[jira] [Updated] (SPARK-28599) Fix `Execution Time` and `Duration` column sorting for ThriftServerSessionPage

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28599: -- Summary: Fix `Execution Time` and `Duration` column sorting for ThriftServerSessionPage

[jira] [Updated] (SPARK-28599) Support sorting for ThriftServerSessionPage

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28599: -- Priority: Minor (was: Major) > Support sorting for ThriftServerSessionPage >

[jira] [Updated] (SPARK-28599) Support sorting for ThriftServerSessionPage

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28599: -- Issue Type: Bug (was: Improvement) > Support sorting for ThriftServerSessionPage >

[jira] [Assigned] (SPARK-29191) Add tag ExtendedSQLTest for SQLQueryTestSuite

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29191: - Assignee: Dongjoon Hyun > Add tag ExtendedSQLTest for SQLQueryTestSuite >

[jira] [Resolved] (SPARK-29191) Add tag ExtendedSQLTest for SQLQueryTestSuite

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29191. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25872

[jira] [Resolved] (SPARK-29162) Simplify NOT(isnull(x)) and NOT(isnotnull(x))

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29162. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25878

[jira] [Assigned] (SPARK-29162) Simplify NOT(isnull(x)) and NOT(isnotnull(x))

2019-09-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29162: - Assignee: angerszhu > Simplify NOT(isnull(x)) and NOT(isnotnull(x)) >

[jira] [Updated] (SPARK-29016) Update, fix LICENSE and NOTICE for Hive 2.3

2019-09-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29016: Description: Hive 2.3 newly added jars: {noformat}

[jira] [Commented] (SPARK-28001) Dataframe throws 'socket.timeout: timed out' exception

2019-09-22 Thread Ido Michael (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935321#comment-16935321 ] Ido Michael commented on SPARK-28001: - I can take a look Can you please also post the dataset?  

[jira] [Commented] (SPARK-29197) Remove saveModeForDSV2 in DataFrameWriter

2019-09-22 Thread Ido Michael (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935320#comment-16935320 ] Ido Michael commented on SPARK-29197: - I can take it,   I could find the final class 

[jira] [Commented] (SPARK-29157) DataSourceV2: Add DataFrameWriterV2 to Python API

2019-09-22 Thread Ido Michael (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935307#comment-16935307 ] Ido Michael commented on SPARK-29157: - It looks like

[jira] [Commented] (SPARK-29166) Add parameters to limit the number of dynamic partitions for data source table

2019-09-22 Thread Ido Michael (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935306#comment-16935306 ] Ido Michael commented on SPARK-29166: - I can take this if no one started to work on it? Ido > Add

[jira] [Updated] (SPARK-28599) Support sorting for ThriftServerSessionPage

2019-09-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28599: Description: Support sorting `Execution Time` and `Duration` columns for ThriftServerSessionPage

[jira] [Resolved] (SPARK-29201) Add Hadoop 2.6 combination to GitHub Action

2019-09-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29201. -- Fix Version/s: 2.4.5 Resolution: Fixed Issue resolved by pull request 25886

[jira] [Updated] (SPARK-28599) Support sorting for ThriftServerSessionPage

2019-09-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-28599: Summary: Support sorting for ThriftServerSessionPage (was: Support pagination and sorting for

[jira] [Updated] (SPARK-29203) Reduce shuffle partitions in SQLQueryTestSuite

2019-09-22 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29203: Description: spark.sql.shuffle.partitions=200(default): {noformat} [info] -

[jira] [Created] (SPARK-29203) Reduce shuffle partitions in SQLQueryTestSuite

2019-09-22 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-29203: --- Summary: Reduce shuffle partitions in SQLQueryTestSuite Key: SPARK-29203 URL: https://issues.apache.org/jira/browse/SPARK-29203 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-28141) Date type can not accept special values

2019-09-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-28141: Assignee: Maxim Gekk > Date type can not accept special values >

[jira] [Resolved] (SPARK-28141) Date type can not accept special values

2019-09-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28141. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25708

[jira] [Resolved] (SPARK-29200) Optimize `extract`/`date_part` for epoch

2019-09-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29200. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25881

[jira] [Assigned] (SPARK-29200) Optimize `extract`/`date_part` for epoch

2019-09-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-29200: Assignee: Maxim Gekk > Optimize `extract`/`date_part` for epoch >

[jira] [Updated] (SPARK-29202) --driver-java-options are not passed to driver process in yarn client mode

2019-09-22 Thread Sandeep Katta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandeep Katta updated SPARK-29202: -- Description: Run the below command  ./bin/spark-sql --master yarn

[jira] [Created] (SPARK-29202) --driver-java-options are not passed to driver process in yarn client mode

2019-09-22 Thread Sandeep Katta (Jira)
Sandeep Katta created SPARK-29202: - Summary: --driver-java-options are not passed to driver process in yarn client mode Key: SPARK-29202 URL: https://issues.apache.org/jira/browse/SPARK-29202