This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new bd03c732f6d Fix produce/consume pydoc (#28051)
bd03c732f6d is described below
commit bd03c732f6d05db8cfda50b4ab1a0d13886fd6b8
Author: Danny McCormick <dannymccorm...@google.com>
AuthorDate: Thu Aug 17 17:16:50 2023 -0400
Fix produce/consume pydoc (#28051)
* Fix produce/consume pydoc
* remove produce/consume language from docs
* Update sdks/python/apache_beam/ml/transforms/base.py
Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>
---------
Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>
---
sdks/python/apache_beam/ml/transforms/base.py | 31 +++++++++++++--------------
1 file changed, 15 insertions(+), 16 deletions(-)
diff --git a/sdks/python/apache_beam/ml/transforms/base.py
b/sdks/python/apache_beam/ml/transforms/base.py
index 7e13fa39136..a45928f5c8b 100644
--- a/sdks/python/apache_beam/ml/transforms/base.py
+++ b/sdks/python/apache_beam/ml/transforms/base.py
@@ -122,12 +122,13 @@ class
MLTransform(beam.PTransform[beam.PCollection[ExampleT],
MLTransform is a Beam PTransform that can be used to apply
transformations to the data. MLTransform is used to wrap the
data processing transforms provided by Apache Beam. MLTransform
- works in two modes: produce and consume. In the produce mode,
+ works in two modes: write and read. In the write mode,
MLTransform will apply the transforms to the data and store the
- artifacts in the artifact_location. In the consume mode, MLTransform
- will read the artifacts from the artifact_location and apply the
- transforms to the data. The artifact_location should be a valid
- storage path where the artifacts can be written to or read from.
+ artifacts in the write_artifact_location. In the read mode,
+ MLTransform will read the artifacts from the
+ read_artifact_location and apply the transforms to the data. The
+ artifact location should be a valid storage path where the artifacts
+ can be written to or read from.
Note that when consuming artifacts, it is not necessary to pass the
transforms since they are inherently stored within the artifacts
@@ -137,17 +138,15 @@ class
MLTransform(beam.PTransform[beam.PCollection[ExampleT],
write_artifact_location: A storage location for artifacts resulting from
MLTransform. These artifacts include transformations applied to
the dataset and generated values like min, max from ScaleTo01,
- and mean, var from ScaleToZScore. Artifacts are produced and stored
- in this location when the `artifact_mode` is set to 'produce'.
- Conversely, when `artifact_mode` is set to 'consume', artifacts are
- retrieved from this location. Note that when consuming artifacts,
- it is not necessary to pass the transforms since they are inherently
- stored within the artifacts themselves. The value assigned to
- `write_artifact_location` should be a valid storage directory where the
- artifacts from this transform can be written to. If no directory exists
- at this location, one will be created. This will overwrite any
- artifacts already in this location, so distinct locations should be
- used for each instance of MLTransform. Only one of
+ and mean, var from ScaleToZScore. Artifacts are produced and written
+ to this location when using `write_artifact_mode`.
+ Later MLTransforms can reuse produced artifacts by setting
+ `read_artifact_mode` instead of `write_artifact_mode`. The value
+ assigned to `write_artifact_location` should be a valid storage
+ directory that the artifacts from this transform can be written to.
+ If no directory exists at this location, one will be created. This will
+ overwrite any artifacts already in this location, so distinct locations
+ should be used for each instance of MLTransform. Only one of
write_artifact_location and read_artifact_location should be specified.
read_artifact_location: A storage location to read artifacts resulting
froma previous MLTransform. These artifacts include transformations
