[
https://issues.apache.org/jira/browse/ARROW-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17055814#comment-17055814
]
Joris Van den Bossche edited comment on ARROW-7677 at 3/10/20, 10:56 AM:
-------------------------------------------------------------------------
It came up in a partitioned parquet dataset test:
https://github.com/apache/arrow/blob/aec37d768abbee3ea3ce4c32002462ff0e1c3674/python/pyarrow/tests/test_dataset.py#L627-L662
Quoting myself from the PR that added this skip
(https://github.com/apache/arrow/pull/6242):
> From the discussion I think we thought the issue is coming from passing a
> base path path with backslashes to GetTargetStats, which is then leading to
> backslashes being used in the crawled file paths.
> partition discovery is not working with windows path, presumably because the
> path splitting doesn't work
was (Author: jorisvandenbossche):
It came up in a partitioned parquet dataset test:
https://github.com/apache/arrow/blob/aec37d768abbee3ea3ce4c32002462ff0e1c3674/python/pyarrow/tests/test_dataset.py#L627-L662
Quoting myself from the PR:
> From the discussion I think we thought the issue is coming from passing a
> base path path with backslashes to GetTargetStats, which is then leading to
> backslashes being used in the crawled file paths.
> partition discovery is not working with windows path, presumably because the
> path splitting doesn't work
> [C++] Handle Windows file paths with backslashes in GetTargetStats
> ------------------------------------------------------------------
>
> Key: ARROW-7677
> URL: https://issues.apache.org/jira/browse/ARROW-7677
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Reporter: Joris Van den Bossche
> Priority: Major
> Fix For: 0.17.0
>
>
> Currently, if the base path passed to {{GetTargetStats}} has backslashes,
> the produces FileStats also include them, resulting in some other
> functionality (such as splitting the path) not working.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
