[ https://issues.apache.org/jira/browse/BEAM-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chamikara Jayalath reassigned BEAM-5417: ---------------------------------------- Assignee: Udi Meiri (was: Chamikara Jayalath) > FileSystems.match behaviour diff between GCS and local file system > ------------------------------------------------------------------ > > Key: BEAM-5417 > URL: https://issues.apache.org/jira/browse/BEAM-5417 > Project: Beam > Issue Type: Bug > Components: sdk-py-core > Affects Versions: 2.5.0, 2.6.0 > Reporter: Joar Wandborg > Assignee: Udi Meiri > Priority: Major > Time Spent: 7h > Remaining Estimate: 0h > > Given the directory structure: > > {noformat} > . > ├── filesystem-match-test > │ ├── a > │ │ └── file.txt > │ └── b > │ └── file.txt > └── filesystem-match-test.py > {noformat} > > Where {{filesystem-match-test.py}} contains: > {code:python} > from __future__ import print_function > import os > import posixpath > from apache_beam.io.filesystem import MatchResult > from apache_beam.io.filesystems import FileSystems > BASES = [ > os.path.join(os.path.dirname(__file__), "./"), > "gs://my-bucket/test/", > ] > pattern = "filesystem-match-test/*/file.txt" > for base_path in BASES: > full_pattern = posixpath.join(base_path, pattern) > print("full_pattern: {}".format(full_pattern)) > match_result = FileSystems.match([full_pattern])[0] # type: MatchResult > print("metadata list: {}".format(match_result.metadata_list)) > {code} > Running {{python filesystem-match-test.py}} does not match any files locally, > but does match files on GCS: > {noformat} > full_pattern: ./filesystem-match-test/*/file.txt > metadata list: [] > full_pattern: gs://my-bucket/test/filesystem-match-test/*/file.txt > metadata list: > [FileMetadata(gs://my-bucket/test/filesystem-match-test/a/file.txt, 6), > FileMetadata(gs://my-bucket/test/filesystem-match-test/b/file.txt, 6)] > {noformat} > The expected result is that a/file.txt and b/file.txt should be matched for > both patterns. -- This message was sent by Atlassian JIRA (v7.6.3#76005)