Josh Wills created CRUNCH-658:
---------------------------------
Summary: Add a way to skip the getSize checks for Sources from
object stores
Key: CRUNCH-658
URL: https://issues.apache.org/jira/browse/CRUNCH-658
Project: Crunch
Issue Type: Bug
Components: Core
Affects Versions: 0.14.0
Reporter: Josh Wills
Assignee: Josh Wills
Ran into a problem when using Crunch to process a _lot_ of data from S3: the
getSize checks can be very slow to run and don't materially add much to the
overall processing of a pipeline when things like reducer counts are manually
specified. I'd like to add a way to disable the file size checks, either
globally or for specific input sources.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
