[
https://issues.apache.org/jira/browse/BEAM-7137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16827114#comment-16827114
]
Valentyn Tymofieiev commented on BEAM-7137:
-------------------------------------------
[~yoshiki.obata], thanks a lot! I think we can fix this before next release, so
I'm setting the fix version to be 2.13.0.
textio_test has unit tests, and they incorrectly use a bytes headers. We should
adjust the tests. Looks like there are no integration tests in textio_test.py.
Integration tests are annotated with " @attr('IT') ", see fr example a file_io
integration test:
[https://github.com/apache/beam/blob/d46da9217a8a56380841f6f5d2a1fa4663851971/sdks/python/apache_beam/io/fileio_test.py#L207.]
We run these tests usually as a part of postcommit test suites, using different
runners, see for example:
[https://github.com/apache/beam/blob/d46da9217a8a56380841f6f5d2a1fa4663851971/sdks/python/test-suites/dataflow/py3/build.gradle#L41.]
[~chamikara], does it make sense to add a separate integration test for text_io?
> TypeError caused by using str variable as header argument in
> apache_beam.io.textio.WriteToText
> ----------------------------------------------------------------------------------------------
>
> Key: BEAM-7137
> URL: https://issues.apache.org/jira/browse/BEAM-7137
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Affects Versions: 2.11.0
> Environment: Python 3.5.6
> macOS Mojave 10.14.4
> Reporter: yoshiki obata
> Assignee: yoshiki obata
> Priority: Major
>
> Using str header to apache_beam.io.textio.WriteToText as argument cause
> TypeError with Python 3.5.6 - or maybe higher - despite docstring says header
> is str.
> This error occurred by writing header to file without encoding to bytes at
> apache_beam.io.textio._TextSink.open.
>
> {code:java}
> Traceback (most recent call last):
> File "apache_beam/runners/common.py", line 727, in
> apache_beam.runners.common.DoFnRunner.process
> File "apache_beam/runners/common.py", line 555, in
> apache_beam.runners.common.PerWindowInvoker.invoke_process
> File "apache_beam/runners/common.py", line 625, in
> apache_beam.runners.common.PerWindowInvoker._invoke_per_window
> File
> "/Users/yob/.local/share/virtualenvs/test/lib/python3.5/site-packages/apache_beam/io/iobase.py",
> line 1033, in process
> self.writer = self.sink.open_writer(init_result, str(uuid.uuid4()))
> File
> "/Users/yob/.local/share/virtualenvs/test/lib/python3.5/site-packages/apache_beam/options/value_provider.py",
> line 137, in _f
> return fnc(self, *args, **kwargs)
> File
> "/Users/yob/.local/share/virtualenvs/test/lib/python3.5/site-packages/apache_beam/io/filebasedsink.py",
> line 185, in open_writer
> return FileBasedSinkWriter(self, os.path.join(init_result, uid) + suffix)
> File
> "/Users/yob/.local/share/virtualenvs/test/lib/python3.5/site-packages/apache_beam/io/filebasedsink.py",
> line 389, in __init__
> self.temp_handle = self.sink.open(temp_shard_path)
> File
> "/Users/yob/.local/share/virtualenvs/test/lib/python3.5/site-packages/apache_beam/io/textio.py",
> line 393, in open
> file_handle.write(self._header)
> TypeError: a bytes-like object is required, not 'str'
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
