I asked this same question a few weeks ago, we needed a way to have our pipeline connect to an Azure instance (I think the ticket actually came from that discussion). We decided to put the credentials in Google Cloud Storage, It can be locked down with restrictive permissions and its easy to access from Dataflow. Its working fine for us until we can pass it in as a hidden pipeline option.
On Fri, Jul 7, 2017 at 1:39 PM, Lukasz Cwik <lc...@google.com> wrote: > I think your looking for this feature: > https://issues.apache.org/jira/browse/BEAM-2492 > > On Fri, Jul 7, 2017 at 11:40 AM, Randal Moore <rdmoor...@gmail.com> wrote: > >> Maybe this is more of a question for DataFlow - but I'm submitting a >> pipeline that needs to access a rest service running in a GKE kubernetes >> instance. I need to pass in creds. I started with pipeline-options which >> work but all options get exposed on the DataFlow web pages. >> >> Is there a way to pass in pipelien options that get hidden? Or another >> way to pass in creds (e.g., like the kubernetes creds map? >> >> rdm >> > >
