[
https://issues.apache.org/jira/browse/BEAM-7746?focusedWorklogId=397047&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-397047
]
ASF GitHub Bot logged work on BEAM-7746:
----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Mar/20 21:22
Start Date: 03/Mar/20 21:22
Worklog Time Spent: 10m
Work Description: chadrik commented on pull request #11022: [BEAM-7746]
Resolve typing issues in filesystem
URL: https://github.com/apache/beam/pull/11022#discussion_r387300879
##########
File path: sdks/python/apache_beam/io/filesystemio.py
##########
@@ -175,10 +190,31 @@ def readall(self):
res.append(data)
return b''.join(res)
+ @classmethod
+ def create_buffered(cls, # type: Type[DownloaderStreamT]
+ downloader, # type: Downloader
+ read_buffer_size=io.DEFAULT_BUFFER_SIZE, mode='rb'):
+ # type: (...) -> BinaryIO
+ # In the python3 typeshed, io.BufferedReader and io.BufferedWriter do not
+ # inherit from typing.BinaryIO, and BinaryIO is not a Protocol, so a class
+ # must inherit from it. We could roll our own BinaryIO Protocol, but the
+ # stubs for io.Buffered* do not have the required 'mode' or 'name' attrs to
+ # meet the protocol (the classes seem to expose 'mode' and 'name'
+ # conditionally, if their underlying io.RawIOBase possess the attributes).
+ # Thus, it is necessary to cast these types to BinaryIO either way.
+ return cast(
+ BinaryIO,
+ io.BufferedReader(
+ cls(downloader, read_buffer_size=read_buffer_size, mode=mode),
+ buffer_size=read_buffer_size))
Review comment:
should I create separate args for the buffer size of the `DownloaderStream`
and the `io.BufferedReader`? The treatment of these varies between
filesystems.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 397047)
Time Spent: 71h 10m (was: 71h)
> Add type hints to python code
> -----------------------------
>
> Key: BEAM-7746
> URL: https://issues.apache.org/jira/browse/BEAM-7746
> Project: Beam
> Issue Type: New Feature
> Components: sdk-py-core
> Reporter: Chad Dombrova
> Assignee: Chad Dombrova
> Priority: Major
> Time Spent: 71h 10m
> Remaining Estimate: 0h
>
> As a developer of the beam source code, I would like the code to use pep484
> type hints so that I can clearly see what types are required, get completion
> in my IDE, and enforce code correctness via a static analyzer like mypy.
> This may be considered a precursor to BEAM-7060
> Work has been started here: [https://github.com/apache/beam/pull/9056]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
