[
https://issues.apache.org/jira/browse/IMPALA-14845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18074142#comment-18074142
]
Joe McDonnell commented on IMPALA-14845:
----------------------------------------
After looking into this some more, it would be better for the receiver to
calculate this itself and provide it back to the sender. The receiver knows
when it takes over the early senders and handles them. For end of stream, that
is basically enough. For transmit data, we actually want to know when the
receiver first calls GetBatch() on this SenderQueue. Up until that point, there
is no thread available to consume the input. There can be a large gap between
setting up the receiver (and handling the early senders) vs actually fetching
the first batch. In particular, for mt_dop=0, the probe side won't be consumed
until the build finishes.
> RpcRecvrTime should keep initial receiver startup delay separate
> ----------------------------------------------------------------
>
> Key: IMPALA-14845
> URL: https://issues.apache.org/jira/browse/IMPALA-14845
> Project: IMPALA
> Issue Type: Task
> Components: Distributed Exec
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Priority: Major
>
> RpcReceiverTime/RpcNetworkTime are used to understand the performance of RPCs
> for KRPC data stream senders. One complication for RpcReceiverTime is that a
> slow startup of the receiver can result in a high value once without
> impacting subsequent values. For example, if we set
> --stress_datastream_recvr_delay_ms=5000 to force the receiver to be delayed
> by 5 seconds, the max value for RpcReceiverTime would be 5 seconds. It would
> be useful to make a distinction between the setup delay vs the maximum seen
> after that point.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
