Hi,

I am using Apache Beam python sdk (apache-beam==2.11.0) to run a dataflow
job with BigQuerySource. Even though I checked the code, BigQueryReader
will delete the temporary dataset after the query is done.
https://github.com/apache/beam/blob/1ad61fd384bcd1edd11086a3cf9d7dddb154d934/sdks/python/apache_beam/io/gcp/bigquery_tools.py#L900

But I still see some temporary dataset in my GCP console. Could you help me
look into it?

Another thing is is that possible to set expiration for the temporary
dataset? right now I see is never.

Thanks,
Tom

Reply via email to