[
https://issues.apache.org/jira/browse/BEAM-14068?focusedWorklogId=777761&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-777761
]
ASF GitHub Bot logged work on BEAM-14068:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 02/Jun/22 17:47
Start Date: 02/Jun/22 17:47
Worklog Time Spent: 10m
Work Description: yeandy commented on code in PR #17462:
URL: https://github.com/apache/beam/pull/17462#discussion_r888218977
##########
build.gradle.kts:
##########
@@ -327,7 +327,7 @@ tasks.register("python39PostCommit") {
dependsOn(":sdks:python:test-suites:direct:py39:postCommitIT")
dependsOn(":sdks:python:test-suites:direct:py39:hdfsIntegrationTest")
dependsOn(":sdks:python:test-suites:portable:py39:postCommitPy39")
- dependsOn(":sdks:python:test-suites:dataflow:py39:torchTests")
+ dependsOn(":sdks:python:test-suites:direct:py39:inferencePostCommitIT")
Review Comment:
Do we also want to add
`dependsOn(":sdks:python:test-suites:direct:py38:inferencePostCommitIT")`
##########
sdks/python/test-suites/direct/common.gradle:
##########
@@ -185,3 +185,36 @@ tasks.register("hdfsIntegrationTest") {
}
}
}
+
+// Pytorch RunInference IT tests
+task torchTests {
+ dependsOn 'installGcpTest'
+ dependsOn ':sdks:python:sdist'
+ def requirementsFile =
"${rootDir}/sdks/python/apache_beam/ml/inference/torch_tests_requirements.txt"
+ doFirst {
+ exec {
+ executable 'sh'
+ args '-c', ". ${envdir}/bin/activate && pip install -r
$requirementsFile"
+ }
+ }
+ doLast {
+ def testOpts = basicTestOpts
+ def argMap = [
+ "test_opts": testOpts,
+ "suite": "postCommitIT-direct-py${pythonVersionSuffix}",
+ "collect": "uses_pytorch and it_postcommit",
+ "runner": "TestDirectRunner"
+ ]
+ def cmdArgs = mapToArgString(argMap)
+ exec {
+ executable 'sh'
+ args '-c', ". ${envdir}/bin/activate && export FORCE_TORCH_IT=1 &&
${runScriptsDir}/run_integration_test.sh $cmdArgs"
+ }
+ }
+}
+
+// Add all the RunInference framework IT tests to this gradle task that runs
on Direct Runner Post commit suite.
+// TODO(anandinguva): Add sklearn IT test here
Review Comment:
Do we need a JIRA for this?
Issue Time Tracking
-------------------
Worklog Id: (was: 777761)
Time Spent: 7h 50m (was: 7h 40m)
> RunInference Benchmarking tests
> -------------------------------
>
> Key: BEAM-14068
> URL: https://issues.apache.org/jira/browse/BEAM-14068
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Anand Inguva
> Assignee: Anand Inguva
> Priority: P2
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> RunInference benchmarks will evaluate performance of Pipelines, which
> represent common use cases of Beam + Dataflow in Pytorch, sklearn and
> possibly TFX. These benchmarks would be the integration tests that exercise
> several software components using Beam, PyTorch, Scikit learn and TensorFlow
> extended.
> we would use the datasets that's available publicly (Eg; Kaggle).
> Size: small / 10 GB / 1 TB etc
> The default execution runner would be Dataflow unless specified otherwise.
> These tests would be run very less frequently(every release cycle).
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
