virajjasani commented on PR #1736: URL: https://github.com/apache/phoenix/pull/1736#issuecomment-1895124476
@gjacoby126 i agree with your concerns and i really wish there was some level of possible way to make the tests generic, but honestly its almost impossible to make tests generic by using parameterized argument because of several reasons: 1. We need to use `hbase.client.scanner.max.result.size` before setting up `PhoenixTestDriver` 2. Set `phoenix.scanning.result.post.dummy.process` to a class that can process region moves after dummy is received by client (depending on how aggressive tests are, we need custom logic here) 3. Set `phoenix.scanning.result.post.valid.process` for some tests 4. Set `phoenix.server.page.size.ms` to 0ms to generate frequent timeouts at server side 5. Depending on tests, we also need to move regions after specific num of `rs#next` calls 6. Region move logic has to be very specific: only move regions of tables that are really going to be used by queries e.g. data table and index table for the given query. If we move all regions (including hbase and phoenix system tables), tests take 5x more time than what they do with the current setup. In fact, initially i was able to reduce the patch size from ~19k to < 15k by adding `ParallelStatsDisabledWithRegionMovesIT`, which is now being extended by several tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
