This is an automated email from the ASF dual-hosted git repository. mcvsubbu pushed a commit to branch 0.2.0 in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit b1b187851848081b194b1e1a09659c1509de66ca Author: Jennifer Dai <[email protected]> AuthorDate: Thu Sep 26 10:44:31 2019 -0700 Adding sample ADL settings to documentation (#4645) --- docs/pluggable_storage.rst | 42 ++++++++++++++++++++++++++++++++++++++++-- docs/segment_fetcher.rst | 2 ++ 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/docs/pluggable_storage.rst b/docs/pluggable_storage.rst index c0aef40..52b4569 100644 --- a/docs/pluggable_storage.rst +++ b/docs/pluggable_storage.rst @@ -57,13 +57,51 @@ AzurePinotFS requires the following configs according to your environment: adl.accountId, adl.authEndpoint, adl.clientId, adl.clientSecret +Sample Controller Config + +.. code-block:: none + + "pinot.controller.storage.factory.class.adl": "org.apache.pinot.filesystem.AzurePinotFS" + "pinot.controller.storage.factory.adl.accountId": "xxxx" + "pinot.controller.storage.factory.adl.authEndpoint": "xxxx" + "pinot.controller.storage.factory.adl.clientId": "xxxx" + "pinot.controller.storage.factory.adl.clientId": "xxxx" + "pinot.controller.segment.fetcher.protocols": "adl" + + +Sample Server Config + +.. code-block:: none + + "pinot.server.storage.factory.class.adl": "org.apache.pinot.filesystem.AzurePinotFS" + "pinot.server.storage.factory.adl.accountId": "xxxx" + "pinot.server.storage.factory.adl.authEndpoint": "xxxx" + "pinot.server.storage.factory.adl.clientId": "xxxx" + "pinot.server.storage.factory.adl.clientId": "xxxx" + "pinot.server.segment.fetcher.protocols": "adl" + + +You can find the parameters in your account as follows: +https://stackoverflow.com/questions/56349040/what-is-clientid-authtokenendpoint-clientkey-for-accessing-azure-data-lake + Please also make sure to set the following config with the value "adl" -segment.fetcher.protocols : adl +.. code-block:: none + + "segment.fetcher.protocols" : "adl" + + +To see how to upload segments to different storage systems, check +:file:`../segment_fetcher.rst`. HadoopPinotFS requires the following configs according to your environment: hadoop.kerberos.principle, hadoop.kerberos.keytab, hadoop.conf.path Please make sure to also set the following config with the value "hdfs" -segment.fetcher.protocols : hdfs + +.. code-block:: none + + "segment.fetcher.protocols" : "hdfs" + + diff --git a/docs/segment_fetcher.rst b/docs/segment_fetcher.rst index e2f51b6..625cb16 100644 --- a/docs/segment_fetcher.rst +++ b/docs/segment_fetcher.rst @@ -17,6 +17,8 @@ .. under the License. .. +.. _segment-fetcher: + Segment Fetchers ================ --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
