Fokko commented on code in PR #301:
URL: https://github.com/apache/iceberg-python/pull/301#discussion_r1466602467
##########
pyiceberg/io/pyarrow.py:
##########
@@ -288,6 +288,8 @@ def create(self, overwrite: bool = False) -> OutputStream:
try:
if not overwrite and self.exists() is True:
raise FileExistsError(f"Cannot create file, already exists:
{self.location}")
+ # Parent directories must be created first in certain file
systems, such as the LocalFileSystem.
+ self._filesystem.create_dir(os.path.dirname(self._path),
recursive=True)
Review Comment:
Thinking a bit more about it, maybe we should still use the ArrowFileSystem
in this new FileIO to make sure that we can read the table into a Arrow
`pa.Table`.
##########
pyiceberg/io/pyarrow.py:
##########
@@ -288,6 +288,8 @@ def create(self, overwrite: bool = False) -> OutputStream:
try:
if not overwrite and self.exists() is True:
raise FileExistsError(f"Cannot create file, already exists:
{self.location}")
+ # Parent directories must be created first in certain file
systems, such as the LocalFileSystem.
+ self._filesystem.create_dir(os.path.dirname(self._path),
recursive=True)
Review Comment:
Thinking a bit more about it, maybe we should still use the ArrowFileSystem
in this new FileIO to make sure that we can read the table into an Arrow
`pa.Table`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
