[ https://issues.apache.org/jira/browse/FLINK-16903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073322#comment-17073322 ]
Jingsong Lee edited comment on FLINK-16903 at 4/2/20, 2:39 AM: --------------------------------------------------------------- There are these ways: # Planner can read {{sink.parallelism}} property from TableConfig, it will be global. # Planner can obtains table properties, so it can read this {{sink.parallelism}} property. # Have a special {{SupportsCustomParallelism}} ability, but also we have {{sink.parallelism}} property in connector to let users have the ability to specify. Maybe the option 3 is the best one * first, It looks more natural than reading a specific key. * second, it can support custom parallelism inference. The property can not do this. What do you think? was (Author: lzljs3620320): There are these ways: # Planner can read {{sink.parallelism}} property from TableConfig, it will be global. # Planner can obtains table properties, so it can read this {{sink.parallelism}} property. # Have a special {{SupportsCustomParallelism}} ability Maybe the option 3 is the best one * first, It looks more natural than reading a specific key. * second, it can support custom parallelism inference. The property can not do this. What do you think? > Add sink.parallelism for file system factory > -------------------------------------------- > > Key: FLINK-16903 > URL: https://issues.apache.org/jira/browse/FLINK-16903 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Runtime > Reporter: Jingsong Lee > Priority: Major > Fix For: 1.11.0 > > > A single task may be writing multiple files at the same time. If the > parallelism is too high, it may lead to a large number of small files. If the > parallelism is too small, the performance is not enough. This requires that > the user can specify parallelism. > * Default is the same as upstream transformation > * Users can specify parallelism too. > |‘connector.sink.parallelism’ = ...| -- This message was sent by Atlassian Jira (v8.3.4#803005)