[ https://issues.apache.org/jira/browse/SPARK-27365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16820549#comment-16820549 ]
Calvin Park edited comment on SPARK-27365 at 4/17/19 10:34 PM: --------------------------------------------------------------- Simple Jenkinsfile {code:java} pipeline { agent { dockerfile { label 'docker-gpu' args '--runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all' } } stages { stage('smi') { steps { sh 'nvidia-smi' } } } }{code} with Dockerfile {code:java} FROM ubuntu:xenial RUN apt-get update && \ DEBIAN_FRONTEND=noninteractive apt-get install -y \ curl \ flake8 \ git-core \ openjdk-8-jdk \ python2.7 \ python-pip \ wget RUN pip install \ requests \ numpy # Build script looks for javac in jre dir ENV JAVA_HOME "/usr/lib/jvm/java-8-openjdk-amd64" # http://spark.apache.org/docs/latest/building-spark.html#setting-up-mavens-memory-usage # We have a pretty beefy server ENV MAVEN_OPTS "-Xmx20g -XX:ReservedCodeCacheSize=2g" {code} was (Author: calvinatnvidia): Simple Jenkinsfile {code:java} pipeline { agent { dockerfile { label 'docker-gpu' args '--runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all' } } stages { stage('smi') { steps { sh 'nvidia-smi' } } } }{code} with Dockerfile {code:java} FROM ubuntu:xenial RUN apt-get update && \ DEBIAN_FRONTEND=noninteractive apt-get install -y \ curl \ flake8 \ git-core \ openjdk-8-jdk \ python2.7 \ python-pip \ wget RUN DEBIAN_FRONTEND=noninteractive pip install \ requests \ numpy # Build script looks for javac in jre dir ENV JAVA_HOME "/usr/lib/jvm/java-8-openjdk-amd64" # http://spark.apache.org/docs/latest/building-spark.html#setting-up-mavens-memory-usage # We have a pretty beefy server ENV MAVEN_OPTS "-Xmx20g -XX:ReservedCodeCacheSize=2g" {code} > Spark Jenkins supports testing GPU-aware scheduling features > ------------------------------------------------------------ > > Key: SPARK-27365 > URL: https://issues.apache.org/jira/browse/SPARK-27365 > Project: Spark > Issue Type: Story > Components: jenkins > Affects Versions: 3.0.0 > Reporter: Xiangrui Meng > Priority: Major > > Upgrade Spark Jenkins to install GPU cards and run GPU integration tests > triggered by "GPU" in PRs. > cc: [~afeng] [~shaneknapp] -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org