[
https://issues.apache.org/jira/browse/HDDS-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Andika resolved HDDS-15711.
--------------------------------
Resolution: Duplicate
> Streaming Write Pipeline increment container blockCount twice
> -------------------------------------------------------------
>
> Key: HDDS-15711
> URL: https://issues.apache.org/jira/browse/HDDS-15711
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
>
> Streaming close sends two EOF {{{}PutBlock{}}}s for the same block, while
> normal async write sends one.
> Normal async path:
> * BlockOutputStream.executePutBlock only calls {{{}putBlockAsync(...){}}}.
> * Close with buffered data sends one EOF {{{}PutBlock{}}}.
> * Close after an earlier non-EOF putBlock sends one forced EOF
> {{{}PutBlock{}}}; the earlier non-EOF putBlock has already created the block
> / pending cache state, so EOF does not increment blockCount again.
> Streaming path:
> * BlockDataStreamOutput.executePutBlock (line 425)] first sends an EOF
> putBlock through {{{}executePutBlockClose(...){}}}.
> * Then the same method also calls {{putBlockAsync(xceiverClient, blockData,
> close, token)}}
> * On the server, the data-stream close request is dispatched via
> {{{}STREAM_LINK{}}}, and the async request is dispatched as a normal
> {{{}PutBlock{}}}; both reach, then {{{}persistPutBlock{}}}.
> So for a small streaming write that does not hit the intermediate data-stream
> flush boundary, close can race two EOF full-block commits for the same local
> ID. Both can see “block does not exist” before either batch commits, and both
> increment source {{{}blockCount{}}}.
> I agree the broad write lock in {{persistPutBlock}} is too expensive for
> this. Better fix direction: remove that lock and make streaming close not
> submit two successful EOF {{{}PutBlock{}}}s for the same block. The cleanest
> target is probably {{{}BlockDataStreamOutput.executePutBlock{}}}: when
> {{{}close == true{}}}, use the data-stream close putBlock as the
> authoritative EOF commit and avoid the extra {{{}putBlockAsync{}}}, except
> for the existing fallback path when {{executePutBlockClose}} fails.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
