[ https://issues.apache.org/jira/browse/TORQUE-364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843515#comment-17843515 ]
Max Philipp Wriedt edited comment on TORQUE-364 at 5/4/24 10:23 PM: -------------------------------------------------------------------- The integrated code is correct as I can see for now. As the MappingStrategy is regenerated everytime a new row is processed, this fix is sadly not doing anything for the performance. However its working and we can use it for a future performance fix e.g. by making it reusable by saving it with the criteria or somewhere appropriate. [~refarb] [~mwriedt] was (Author: JIRAUSER299058): The integrated code is correct as I can see for now. As the MappingStrategy is regenerated everytime a new row is processed, this fix is sadly not doing anything for the performance. However its working and we can use it for a future performance fix e.g. by making it reusable by saving it with the criteria or somewhere appropriate. [~refarb] [~mwriedt] > RecordMapper very slow on many columns in table > ----------------------------------------------- > > Key: TORQUE-364 > URL: https://issues.apache.org/jira/browse/TORQUE-364 > Project: Torque > Issue Type: Improvement > Components: Runtime, Templates > Affects Versions: 5.1 > Reporter: Max Philipp Wriedt > Priority: Major > Labels: criteria_api, om, performance, recordmapper, templates > > When "doSelect()" a large quantity of columns in a table the default > RecordMappers generated by Om-Templates (processRow()) cause an > !https://wikimedia.org/api/rest_v1/media/math/render/svg/4441d9689c0e6b2c47994e2f587ac5378faeefba! > Problem. (technically O(rows * columns)) > Specifically, constantly generating the SQL expression of all possible > columns for every row in the result causes excessive use of StringBuilders > which slow the mapping process to a crawl. > I currently have two ideas on how best to tackle this problem: > # Either generate all SQL column expressions once when a (template > generated) RecordMapper is first created (using final static String fields) > thus reducing the cost for every row to generating all selected column SQL > expressions once(instead of every selected column times every available > column) > # Or (in case the first approach generates unacceptably excessive number of > fields for RecordMappers) adjust the RecordMapper API to allow a > "prepare(Criteria, int offset)" method to be called once before processing > any rows and implement it on generated RecordMappers to scan the Criteria and > build two lists: One containing references to the setXXX methods of the > mapper in the order they appear in the ResultSet (via the order in the > Criteria) and a second list containing the corresponding column offsets. This > would allow the processRow method to only iterated over both lists > simultaneously and call the referenced methods with the result set and offset > immediately. (Alternatively one list using lambdas could be used but I am > currently not sure about the stance or impact of these lambdas in the Torque > project.) > credits to [~refarb] -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: torque-dev-unsubscr...@db.apache.org For additional commands, e-mail: torque-dev-h...@db.apache.org