gianm commented on a change in pull request #7181: Reduce default max # of subTasks to 2 for native parallel task URL: https://github.com/apache/incubator-druid/pull/7181#discussion_r262281540
########## File path: docs/content/ingestion/native_tasks.md ########## @@ -181,7 +181,7 @@ The tuningConfig is optional and default parameters will be used if no tuningCon |reportParseExceptions|If true, exceptions encountered during parsing will be thrown and will halt ingestion; if false, unparseable rows and fields will be skipped.|false|no| |pushTimeout|Milliseconds to wait for pushing segments. It must be >= 0, where 0 means to wait forever.|0|no| |segmentWriteOutMediumFactory|Segment write-out medium to use when creating segments. See [SegmentWriteOutMediumFactory](#segmentWriteOutMediumFactory).|Not specified, the value from `druid.peon.defaultSegmentWriteOutMediumFactory.type` is used|no| -|maxNumSubTasks|Maximum number of tasks which can be run at the same time.|Integer.MAX_VALUE|no| +|maxNumSubTasks|Maximum number of tasks which can be run at the same time. The supervisor task would spawn sub tasks up to `maxNumSubTasks` regardless of the reamining task slots. If this value is set to too large, too many sub tasks can be created which might block other ingestion.|2|no| Review comment: > I thought about it, but set to 2 because I’m thinking to make the supervisor task do the ingestion job on its own instead of spawning sub tasks if this is 1. If this is the case, I think 1 is an even better default. > What do you think about more aggressive way like making this option mandatory and throwing error if it’s missing? I think there should be some kind of default (we want to minimize the # of mandatory parameters) and at least "1" + a warning is something where it's fairly clear what's going on if you forgot to set it. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
