spark git commit: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow are installed or not in PySpark SQL tests

Repository: spark Updated Branches: refs/heads/master a24c03138 -> 8141c3e3d [SPARK-23300][TESTS] Prints out if Pandas and PyArrow are installed or not in PySpark SQL tests ## What changes were proposed in this pull request? This PR proposes to log if PyArrow and Pandas are installed or not

spark git commit: [SPARK-23290][SQL][PYTHON] Use datetime.date for date type when converting Spark DataFrame to Pandas DataFrame.

Repository: spark Updated Branches: refs/heads/master f3f1e14bb -> a24c03138 [SPARK-23290][SQL][PYTHON] Use datetime.date for date type when converting Spark DataFrame to Pandas DataFrame. ## What changes were proposed in this pull request? In #18664, there was a change in how `DateType` is

spark git commit: [SPARK-23326][WEBUI] schedulerDelay should return 0 when the task is running

Repository: spark Updated Branches: refs/heads/branch-2.3 4aa9aafcd -> 521494d7b [SPARK-23326][WEBUI] schedulerDelay should return 0 when the task is running ## What changes were proposed in this pull request? When a task is still running, metrics like executorRunTime are not available.

spark git commit: [SPARK-23326][WEBUI] schedulerDelay should return 0 when the task is running

Repository: spark Updated Branches: refs/heads/master c2766b07b -> f3f1e14bb [SPARK-23326][WEBUI] schedulerDelay should return 0 when the task is running ## What changes were proposed in this pull request? When a task is still running, metrics like executorRunTime are not available. Then

svn commit: r24719 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_05_18_01-4aa9aaf-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Tue Feb 6 02:15:28 2018 New Revision: 24719 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_05_18_01-4aa9aaf docs [This commit notification would consist of 1442 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24718 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_05_16_01-c2766b0-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Tue Feb 6 00:15:17 2018 New Revision: 24718 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_05_16_01-c2766b0 docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23330][WEBUI] Spark UI SQL executions page throws NPE

Repository: spark Updated Branches: refs/heads/branch-2.3 173449c2b -> 4aa9aafcd [SPARK-23330][WEBUI] Spark UI SQL executions page throws NPE ## What changes were proposed in this pull request? Spark SQL executions page throws the following error and the page crashes: ``` HTTP ERROR 500

spark git commit: [SPARK-23330][WEBUI] Spark UI SQL executions page throws NPE

Repository: spark Updated Branches: refs/heads/master 03b7e120d -> c2766b07b [SPARK-23330][WEBUI] Spark UI SQL executions page throws NPE ## What changes were proposed in this pull request? Spark SQL executions page throws the following error and the page crashes: ``` HTTP ERROR 500 Problem

svn commit: r24715 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_05_14_01-173449c-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Mon Feb 5 22:15:08 2018 New Revision: 24715 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_05_14_01-173449c docs [This commit notification would consist of 1442 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24710 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_05_12_01-03b7e12-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Mon Feb 5 20:15:39 2018 New Revision: 24710 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_05_12_01-03b7e12 docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23310][CORE] Turn off read ahead input stream for unshafe shuffle reader

Repository: spark Updated Branches: refs/heads/branch-2.3 e688ffee2 -> 173449c2b [SPARK-23310][CORE] Turn off read ahead input stream for unshafe shuffle reader To fix regression for TPC-DS queries Author: Sital Kedia Closes #20492 from sitalkedia/turn_off_async_inputstream.

spark git commit: [SPARK-23310][CORE] Turn off read ahead input stream for unshafe shuffle reader

Repository: spark Updated Branches: refs/heads/master a6bf3db20 -> 03b7e120d [SPARK-23310][CORE] Turn off read ahead input stream for unshafe shuffle reader To fix regression for TPC-DS queries Author: Sital Kedia Closes #20492 from sitalkedia/turn_off_async_inputstream.

svn commit: r24696 - in /dev/spark/2.3.1-SNAPSHOT-2018_02_05_06_02-e688ffe-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Mon Feb 5 14:16:32 2018 New Revision: 24696 Log: Apache Spark 2.3.1-SNAPSHOT-2018_02_05_06_02-e688ffe docs [This commit notification would consist of 1442 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24694 - in /dev/spark/2.4.0-SNAPSHOT-2018_02_05_04_01-a6bf3db-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Mon Feb 5 12:18:04 2018 New Revision: 24694 Log: Apache Spark 2.4.0-SNAPSHOT-2018_02_05_04_01-a6bf3db docs [This commit notification would consist of 1444 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23307][WEBUI] Sort jobs/stages/tasks/queries with the completed timestamp before cleaning up them

Repository: spark Updated Branches: refs/heads/branch-2.3 430025cba -> e688ffee2 [SPARK-23307][WEBUI] Sort jobs/stages/tasks/queries with the completed timestamp before cleaning up them ## What changes were proposed in this pull request? Sort jobs/stages/tasks/queries with the completed

spark git commit: [SPARK-23307][WEBUI] Sort jobs/stages/tasks/queries with the completed timestamp before cleaning up them

Repository: spark Updated Branches: refs/heads/master 6fb3fd153 -> a6bf3db20 [SPARK-23307][WEBUI] Sort jobs/stages/tasks/queries with the completed timestamp before cleaning up them ## What changes were proposed in this pull request? Sort jobs/stages/tasks/queries with the completed