Re: [I] MinimizeDataMovement does not work correctly for upsert tables [pinot]

[via GitHub]

dang-stripe commented on issue #11894:
URL: https://github.com/apache/pinot/issues/11894#issuecomment-2375854779

   We've done some testing w/ upsert + `minimizeDataMovement` and it does not 
seem like this issue is fixed. We have a script to compare the ideal state w/ 
the output of the dry-run rebalance and still a significant number of segments 
being added to existing servers.
   
   In this case, we have 2 pools with 4 servers and are adding 1 server to each 
pool.
   
   With `minimizeDataMovement` off:
   ![Screenshot 2024-09-25 at 9 29 31 
PM](https://github.com/user-attachments/assets/90aa688a-0089-4f1f-a18e-6f5409cda2ee)
   
   With `minimizeDataMovement` on, there's barely any change:
   ![Screenshot 2024-09-25 at 9 29 36 
PM](https://github.com/user-attachments/assets/3453ec6b-f499-4ce7-99bf-93cf48f24be7)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org