[
https://issues.apache.org/jira/browse/BEAM-6711?focusedWorklogId=201660&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-201660
]
ASF GitHub Bot logged work on BEAM-6711:
----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Feb/19 22:42
Start Date: 20/Feb/19 22:42
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #7892: [BEAM-6711]
[BEAM-6553] A Python SDK sink that supports File Loads into BQ (#7655)
URL: https://github.com/apache/beam/pull/7892#discussion_r258695330
##########
File path: sdks/python/apache_beam/io/gcp/big_query_query_to_table_pipeline.py
##########
@@ -50,9 +50,10 @@ def run_bq_pipeline(argv=None):
help='Output BQ table to write results to.')
parser.add_argument('--kms_key', default=None,
help='Use this Cloud KMS key with BigQuery.')
- parser.add_argument('--gs_location',
+ parser.add_argument('--gcs_location',
default=None,
- help='GCS bucket location to use to store files.')
+ help=('GCS bucket location to use to store files for '
Review comment:
nit: We can s/GCS bucket location/GCS bucket/ in the help string to avoid
confusion with GCS's concept of bucket location:
https://cloud.google.com/storage/docs/locations.
Perhaps `bq_temp_location` would be a better name?
Also, there are other places in this pr where we still use 'gs_location' -
just making sure that's intentional.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 201660)
Time Spent: 3h 40m (was: 3.5h)
> Bigquery Tornadoes IT is broken in Python3 PostCommit test suite.
> ------------------------------------------------------------------
>
> Key: BEAM-6711
> URL: https://issues.apache.org/jira/browse/BEAM-6711
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Valentyn Tymofieiev
> Assignee: Pablo Estrada
> Priority: Major
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> First failure was observed in
> https://builds.apache.org/job/beam_PostCommit_Python3_Verify/54 , after
> https://github.com/apache/beam/commit/cdea885872b3be7de9ba22f22700be89f7d53766
> was merged.
> [~pabloem], could you please take a look? I suggest we do a rollback +
> rollforward with a fix.
> {noformat}
> root: ERROR: Exception at bundle
> <apache_beam.runners.direct.bundle_factory._Bundle object at 0x7f35e5331240>,
> due to an exception.
> Traceback (most recent call last):
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/runners/common.py",
> line 727, in process
> return self.do_fn_invoker.invoke_process(windowed_value)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/runners/common.py",
> line 556, in invoke_process
> windowed_value, additional_args, additional_kwargs, output_processor)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/runners/common.py",
> line 622, in _invoke_per_window
> self.process_method(*args_for_process, **kwargs_for_process))
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/runners/common.py",
> line 823, in process_outputs
> for result in results:
> File "/home/jenkins/jenkins-slave/works
> pace/beam_PostCommit_Python3_Verify/src/sdks/python/apache_beam/io/gcp/bigquery_file_loads.py",
> line 191, in process
> if destination in self._destination_to_file_writer:
> TypeError: unhashable type: 'TableReference'
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
