[GitHub] [spark] HyukjinKwon opened a new pull request #32015: [SPARK-34821][INFRA] Set up a workflow for developers to run benchmark in their fork

[GitBox]

HyukjinKwon opened a new pull request #32015:
URL: https://github.com/apache/spark/pull/32015


   ### What changes were proposed in this pull request?
   
   This PR proposes to add a workflow that allows developers to run benchmarks 
and download the results files.  After this PR, developers can go to Actions in 
their fork.
   
   ### Why are the changes needed?
   
   1. Very easy to use.
   2. We can use the (almost) same environment to run the benchmarks. Given my 
few experiments and observation, the CPU, cores, and memory are same.
   3. Does not burden ASF's resource at GitHub Actions.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No, dev-only.
   
   ### How was this patch tested?
   
   Manually tested in https://github.com/HyukjinKwon/spark/pull/31.
   
   Entire benchmarks are being run as below:
   - [Run benchmarks: * (JDK 
11)](https://github.com/HyukjinKwon/spark/actions/runs/705052680)
   - [Run benchmarks: * (JDK 
8)](https://github.com/HyukjinKwon/spark/actions/runs/705052116)
   
   ### How do I use it?
   
   1. **Go to Actions in your fork, and click "Run benchmarks"**
   
       ![Screen Shot 2021-03-31 at 10 15 13 
PM](https://user-images.githubusercontent.com/6477701/113150018-99d71680-926e-11eb-8647-4ecf062c55f2.png)
   
   
   2. **Run the benchmarks with JDK 8 or 11 with benchmark classes to run. Glob 
pattern is supported just like `testOnly` in SBT**
   
       ![Screen Shot 2021-03-31 at 10 17 11 
PM](https://user-images.githubusercontent.com/6477701/113150315-e6baed00-926e-11eb-98b2-9cde1257a065.png)
   
   
   3. **After finishing the jobs, the benchmark results are available on the 
top in the underlying workflow:**
   
       ![Screen Shot 2021-03-31 at 10 17 21 
PM](https://user-images.githubusercontent.com/6477701/113150332-ede1fb00-926e-11eb-9c0e-97d195070508.png)
   
   4. **After downloading it, unzip and untar at Spark git root directory:**
   
       ```bash
       cd .../spark
       mv ~/Downloads/benchmark-results-8.zip .
       unzip benchmark-results-8.zip
       tar -xvf benchmark-results-8.tar
       ```
   
   5. **Check the results:**
   
       ```bash
       git status
       ```
   
       ```
       ...
           modified:   core/benchmarks/MapStatusesSerDeserBenchmark-results.txt
       ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org