hansva opened a new issue, #9430:
URL: https://github.com/apache/hop/issues/9430

   ### What needs to happen?
   
   `RowMeta.compare(...)` and `RowMeta.equals(Object[], Object[], int[])` take 
the row metadata's read lock for every call. Sort rows calls `compare` about 20 
times per row (39 million calls for 2 million rows), and every transform that 
works on sorted input calls it once or more per row: Merge join, Group by, 
Unique rows, Merge rows (diff), Sorted merge.
   
   The lock is no longer needed there. Since #7764, `RowMeta` keeps an 
immutable snapshot array of its value metadata that is replaced as a whole 
under the write lock, and `size()` and `getValueMeta(int)` already read that 
snapshot without locking. #8968 did the same for most of `cloneRow`. The 
compare methods are what is left.
   
   ### Proposed change
   
   In `org.apache.hop.core.row.RowMeta`, read the snapshot once into a local 
variable and compare against it, without taking the read lock, in:
   
   - `compare(Object[] rowData1, Object[] rowData2, int[] fieldnrs)`
   - `compare(Object[] rowData1, Object[] rowData2, int[] fieldnrs1, int[] 
fieldnrs2)`
   - `compare(Object[] rowData1, IRowMeta rowMeta2, Object[] rowData2, int[] 
fieldnrs1, int[] fieldnrs2)`
   - `compare(Object[] rowData1, Object[] rowData2)`
   - `equals(Object[] rowData1, Object[] rowData2, int[] fieldnrs)`
   
   One snapshot per call gives a comparison the same guarantee the read lock 
gave: it sees the metadata from before a concurrent change or from after it, 
never a mix.
   
   
   ### Issue Priority
   
   Priority: 2
   
   ### Issue Component
   
   Component: API


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to