Also before the release you can work-around this issue by passing "validate=False" to the source and that would prevent the pipeline from hitting this bug.
-Sourabh On Tue, May 23, 2017 at 11:05 AM Sourabh Bajaj <[email protected]> wrote: > Hi Vilhelm, > > Thank you for catching the issue, this is actually a problem in how the > limit is implemented. I have created > https://github.com/apache/beam/pull/3210 addressing this issue so should > be fixed in the upcoming release. > > -Sourabh > > On Tue, May 23, 2017 at 6:34 AM Peter Mueller <[email protected]> wrote: > >> Vilhelm, >> Did you forget (as I did, and ran into same problem you describe... ) to >> add the required python package - pip install apache-beam[gcp] to access >> platform-specific Read.IOs? It looks like 'apache-beam' is going to be the >> 'core' and we'll need apache-beam[xx] to run on specific runners... >> >> Good luck... >> >> Peter Mueller >> >> >> >> On Tue, May 23, 2017 at 3:45 AM, Vilhelm von Ehrenheim < >> [email protected]> wrote: >> >>> It definitely matches files. I successfully ran the same pattern and >>> files in a batch job using v0.6.0. After digging deeper into this I created >>> a bug as it seems to be broken in 2.0: >>> https://issues.apache.org/jira/browse/BEAM-2338. >>> >>> On Mon, May 22, 2017 at 4:47 PM, Chamikara Jayalath < >>> [email protected]> wrote: >>> >>>> What is the glob pattern and the runner you are using ? Please note >>>> that FileBasedSource fails for empty glob-patterns. So make sore that your >>>> pattern matches to at least one file. >>>> >>>> Thanks, >>>> Cham >>>> >>>> >>>> On Mon, May 22, 2017 at 4:44 AM Vilhelm von Ehrenheim < >>>> [email protected]> wrote: >>>> >>>>> Hi! >>>>> Have anyone else had problems with glob patterns in 2.0? My pipelines >>>>> are failing w the `No files found based on the file pattern %s' % pattern` >>>>> error. Trying to run `FileSystems.match()` on patterns seem to only give >>>>> me >>>>> results is there is no star in the pattern. >>>>> >>>>> Am I missing something obvious here? Do I need to update some >>>>> dependency or something to get this working? >>>>> >>>>> Thanks, >>>>> Vilhelm von Ehrenheim >>>>> >>>> >>> >>
