ninsmiracle opened a new issue, #2183:
URL: https://github.com/apache/incubator-pegasus/issues/2183
## Feature Request
**Is your feature request related to a problem? Please describe:**
When I test the performance bottleneck of duplication, I feel strange about
the task priority of `load_from_private_log stage`.
`load_from_private_log` is the main time-consuming stage that generates IO,
and is handled by the THREAD_POOL_REPLICATION_LONG thread. The priority of the
dup-related tasks is also defined as LOW.
We can see all the RPC heandled by THREAD_POOL_REPLICATION_LONG:
```
// THREAD_POOL_REPLICATION_LONG
#define CURRENT_THREAD_POOL THREAD_POOL_REPLICATION_LONG
MAKE_EVENT_CODE(LPC_LEARN_REMOTE_DELTA_FILES, TASK_PRIORITY_COMMON)
MAKE_EVENT_CODE_AIO(LPC_REPLICATION_COPY_REMOTE_FILES, TASK_PRIORITY_COMMON)
MAKE_EVENT_CODE(LPC_GARBAGE_COLLECT_LOGS_AND_REPLICAS, TASK_PRIORITY_COMMON)
MAKE_EVENT_CODE(LPC_OPEN_REPLICA, TASK_PRIORITY_COMMON)
MAKE_EVENT_CODE(LPC_CLOSE_REPLICA, TASK_PRIORITY_COMMON)
MAKE_EVENT_CODE(LPC_CHECKPOINT_REPLICA, TASK_PRIORITY_COMMON)
MAKE_EVENT_CODE(LPC_CATCHUP_WITH_PRIVATE_LOGS, TASK_PRIORITY_COMMON)
MAKE_EVENT_CODE(LPC_DISK_STAT, TASK_PRIORITY_COMMON)
MAKE_EVENT_CODE(LPC_BACKGROUND_COLD_BACKUP, TASK_PRIORITY_COMMON)
MAKE_EVENT_CODE(LPC_PARTITION_SPLIT_ASYNC_LEARN, TASK_PRIORITY_COMMON)
MAKE_EVENT_CODE(LPC_REPLICATION_LONG_LOW, TASK_PRIORITY_LOW)
MAKE_EVENT_CODE(LPC_REPLICATION_LONG_COMMON, TASK_PRIORITY_COMMON)
MAKE_EVENT_CODE(LPC_REPLICATION_LONG_HIGH, TASK_PRIORITY_HIGH)
#undef CURRENT_THREAD_POOL
```
In fact, the business requires real-time dup. If the delay of the dup
process of a piece of data is too high, when the master cluster down, the
backup cluster will be unable to completely replace all the data of the master
cluster to provide services.
And I do not think `LPC_DISK_STAT`,
`LPC_GARBAGE_COLLECT_LOGS_AND_REPLICAS` task is importanter than duplicate.
**Describe the feature you'd like:**
I'd like to raise the priority of `load_from_private_log` when doing
duplication.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
