[GitHub] [beam] jaketf edited a comment on pull request #11959: refactor HCLS IO ITs to support stores in other projects

Thu, 02 Jul 2020 12:41:02 -0700 


jaketf edited a comment on pull request #11959:
URL: https://github.com/apache/beam/pull/11959#issuecomment-653188130


   I was able to reproduce this failure.
   It occurs when using the import method and [falling back to using the temp 
storage](https://github.com/apache/beam/blob/e68a38b1706225c3b95216471e331ae410fadc0f/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/FhirIO.java#L801)
   namely "`gs://temp-storage-for-end-to-end-tests`" rather than using the
   "`gs://temp-storage-for-healthcare-io-tests`" for which the appropriate 
permissions have been granted to the Healthcare Service Account. [Example 
failure 
log](https://pantheon.corp.google.com/logs/viewer?dateRangeEnd=2020-07-02T19%3A11%3A45.905Z&dateRangeStart=2020-07-02T19%3A07%3A31.464Z&expandAll=false&interval=CUSTOM&project=apache-beam-testing&resource=dataflow_step%2Fjob_id%2F2020-07-02_12_07_30-13967846924370704060&logName=projects%2Fapache-beam-testing%2Flogs%2Fdataflow.googleapis.com%252Fworker&minLogLevel=0&timestamp=2020-07-02T19%3A14%3A20.173000000Z&customFacets&limitCustomFacetWidth=true&scrollTimestamp=2020-07-02T19%3A09%3A11.194000000Z&advancedFilter=resource.type%3D%22dataflow_step%22%0Aresource.labels.project_id%3D%22apache-beam-testing%22%0Aresource.labels.job_name%3D%22testpipeline-jferriero-0702190719-41c340ed%22%0Aresource.labels.job_id%3D%222020-07-02_12_07_30-13967846924370704060%22%0Aresource.labels.step_id%3D%22FhirIO.Write%2FFhirIO.Import%2FImport%20Batches%22%0Aresource.labels.region%3D%22us-central1%22%0Atimestamp%3D%222020-07-02T19%3A09%3A11.194000000Z%22%0AinsertId%3D%227321060105109343840%3A29763%3A0%3A119671%22)
   
   I thought this should be handled by setting the gcsTempPath pipeline options 
[here](https://github.com/apache/beam/pull/11959/files#diff-24800457f8326467edabf56d21f334ffR79)
   
   I'll try to pin down if the option is not being properly set or if this is a 
Bug in how FhirIO reads this parameter.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to