tvalentyn commented on a change in pull request #12927:
URL: https://github.com/apache/beam/pull/12927#discussion_r497092701
##########
File path: sdks/python/apache_beam/io/avroio.py
##########
@@ -627,11 +627,19 @@ def write_record(self, writer, value):
writer.append(value)
+class _FastAvroWriter(Writer):
+ """An adapter class which exposes a file handle so that it can be closed
+ by the sink. """
+ def __init__(self, file_handle, schema, codec):
+ super(_FastAvroWriter, self).__init__(file_handle, schema, codec)
Review comment:
Actually, it is a change in 0.22.0:
https://github.com/fastavro/fastavro/pull/343/commits/6c55c318735278d097b05da7077b23fead50a039#diff-96a4a8649900b3ff9f22a57de25baca4R498
https://github.com/fastavro/fastavro/blob/f468264b0f86661f1893c8b32bc592fb44e39c75/ChangeLog#L91
Cython is an optional dependency in Beam, so we should not have to rely on
presence of cython.
It seems strange that fastavro public api is now different in cythonized and
non-cythonized version. Are you interested in reaching out to fastavro folks to
clarify whether this is something they want to fix?
We explicitly close the filehandle due to:
https://issues.apache.org/jira/browse/BEAM-7747. Perhaps there is a better way?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
