ankitsultana opened a new issue, #10494: URL: https://github.com/apache/pinot/issues/10494
For Full Upsert tables, if we get a new event for an existing primary-key, we simply use that new event as the new record. However for Partial Upsert tables we have to read the entire existing record. This means that we have to do row-based reads on the columnar data which can have a big overhead especially if the number of columns are higher. Not only does it increase the overall Disk IO Utilization, if there's even a modest spike in Disk IO Utilization the ingestion latency for partial upsert tables can be impacted. I don't think there's a way around this given how Partial Upsert tables are designed but interested to see what the community thinks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
