sreejasahithi commented on PR #11509:
URL: https://github.com/apache/ozone/pull/11509#issuecomment-6221474226
> @sreejasahithi as @jojochuang noticed (see #11569), this significantly
increases wall-clock CI time (+30 minutes?), since the runs affected are on the
critical path. While we may drop v1 support in the future, I don't think such
change should be merged to `master`. We should do the parallel version in the
first commit, not as follow-up.
>
> We can decrease both wall-clock and total CI time by running v1 and v2
tests on different backend configurations, instead of a full matrix. For
example I would guess that HA or non-HA cluster behaves the same, since the
differences are probably at the S3 Gateway-side, not in OM/SCM. So we can run
e.g. v1 with non-HA and v2 with HA (to reflect the importance of v2). We have
similar optimization for filesystem tests vs. bucket types vs. HA/non-HA:
>
> ```
> hadoop-ozone/dist/src/main/compose/ozone/test.sh
> 55:execute_robot_test scm -v SCHEME:o3fs -v BUCKET_TYPE:bucket -N
ozonefs-o3fs-bucket ozonefs/ozonefs.robot
>
> hadoop-ozone/dist/src/main/compose/ozone-ha/test.sh
> 38:execute_robot_test ${SCM} -v SCHEME:ofs -v BUCKET_TYPE:link -N
ozonefs-ofs-link ozonefs/ozonefs.robot
>
> hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
> 44:execute_robot_test scm -v SCHEME:ofs -v BUCKET_TYPE:bucket -N
ozonefs-ofs-bucket ozonefs/ozonefs.robot
>
> hadoop-ozone/dist/src/main/compose/ozonesecure-ha/test.sh
> 45:execute_robot_test s3g -v SCHEME:o3fs -v BUCKET_TYPE:link -N
ozonefs-o3fs-link ozonefs/ozonefs.robot
> ```
>
> I have some work in progress to drop existing duplicates
([HDDS-16818](https://issues.apache.org/jira/browse/HDDS-16818)). Instead of
dropping all duplicate S3 tests, we may convert some of them to v2.
Thanks for the feedback, I understand the concern about both the wall clock
CI time and the overall CI resource usage. I will look into splitting the v1
and v2 tests across the existing backend configurations rather than running the
full matrix.
and address this in the current PR rather than deferring it to a follow up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
