[PIO-151] Add S3 storage docs This closes #449
Project: http://git-wip-us.apache.org/repos/asf/predictionio/repo Commit: http://git-wip-us.apache.org/repos/asf/predictionio/commit/c1270e23 Tree: http://git-wip-us.apache.org/repos/asf/predictionio/tree/c1270e23 Diff: http://git-wip-us.apache.org/repos/asf/predictionio/diff/c1270e23 Branch: refs/heads/livedoc Commit: c1270e23b00ff459e76a4e9dde589bb2e1d0d0db Parents: e156cb4 Author: James Ward <ja...@jamesward.org> Authored: Thu Mar 1 13:26:35 2018 -0800 Committer: Chan Lee <chanlee...@gmail.com> Committed: Thu Mar 1 13:48:59 2018 -0800 ---------------------------------------------------------------------- .../source/system/anotherdatastore.html.md | 43 ++++++++++++++++++++ 1 file changed, 43 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/predictionio/blob/c1270e23/docs/manual/source/system/anotherdatastore.html.md ---------------------------------------------------------------------- diff --git a/docs/manual/source/system/anotherdatastore.html.md b/docs/manual/source/system/anotherdatastore.html.md index b3a3709..071a197 100644 --- a/docs/manual/source/system/anotherdatastore.html.md +++ b/docs/manual/source/system/anotherdatastore.html.md @@ -114,6 +114,10 @@ PredictionIO comes with the following sources: * Type name is **hdfs**. * Can be used for *Model Data* repository +- **S3**: + * Type name is **s3**. + * Can be used for *Model Data* repository + Each repository can be configured to use different sources as shown above. Each source has its own set of configuration parameters. Configuration variables will be explained in more details in later sections below (see Data Store Configuration). @@ -293,6 +297,45 @@ supported. `PIO_STORAGE_SOURCES_HDFS_PATH=/mymodels` +#### S3 Configuration + +Variable Format: `PIO_STORAGE_SOURCES_[NAME]_TYPE=s3` + +Supported Repositories: **model** + +To provide authentication information, you can set the `AWS_ACCESS_KEY_ID` +and `AWS_SECRET_ACCESS_KEY` environment variables or use one of the other +methods in the [AWS Setup Docs](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#config-settings-and-precedence) + +When `TYPE` is set to `s3`, the following configuration keys are +supported. + +- REGION (mandatory) + + AWS Region to use, e.g. + `PIO_STORAGE_SOURCES_S3_REGION=us-east-1` + +- BUCKET_NAME (mandatory) + + S3 Bucket where models are stored, e.g. + `PIO_STORAGE_SOURCES_S3_BUCKET_NAME=pio_bucket` + +- BASE_PATH (optional) + + S3 base path where models are stored, e.g. + `PIO_STORAGE_SOURCES_S3_BASE_PATH=pio_model` + +- DISABLE_CHUNKED_ENCODING (optional) + + Disable the use of Chunked Encoding when transferring files to/from S3, e.g. + `PIO_STORAGE_SOURCES_S3_DISABLE_CHUNKED_ENCODING=true` + +- ENDPOINT (optional) + + S3 Endpoint to use, e.g. + `PIO_STORAGE_SOURCES_S3_ENDPOINT=http://localstack:4572` + + ## Adding Support of Other Backends It is quite straightforward to implement support of other backends. A good