https://gcc.gnu.org/bugzilla/show_bug.cgi?id=126789
--- Comment #3 from Richard Biener <rguenth at gcc dot gnu.org> --- With the fix the larger example works while the mixed SSE/AVX512 style masking one fails as we get t2.c:8:25: note: node 0x39a48888 (max_nunits=2, refcnt=2) vector(2) <signed-boolean:32> t2.c:8:25: note: op template: patt_20 = patt_19 & test2_16; t2.c:8:25: note: stmt 0 patt_20 = patt_19 & test2_16; t2.c:8:25: note: stmt 1 patt_23 = patt_22 & test3_17; t2.c:8:25: note: children 0x39a48940 0x39a48c20 t2.c:8:25: note: node 0x39a48940 (max_nunits=2, refcnt=2) vector(2) <signed-boolean:32> t2.c:8:25: note: op template: patt_19 = (<signed-boolean:1>) test0_12; t2.c:8:25: note: stmt 0 patt_19 = (<signed-boolean:1>) test0_12; t2.c:8:25: note: stmt 1 patt_22 = (<signed-boolean:1>) test1_13; t2.c:8:25: note: children 0x39a489f8 t2.c:8:25: note: node 0x39a489f8 (max_nunits=2, refcnt=2) vector(2) <signed-boolean:1> t2.c:8:25: note: op template: test0_12 = _1 < g_11(D); t2.c:8:25: note: stmt 0 test0_12 = _1 < g_11(D); t2.c:8:25: note: stmt 1 test1_13 = _2 < g_11(D); t2.c:8:25: note: children 0x39a48ab0 0x39a48b68 t2.c:8:25: note: node 0x39a48c20 (max_nunits=2, refcnt=2) vector(2) <signed-boolean:32> t2.c:8:25: note: op template: test2_16 = _3 < f_15(D); t2.c:8:25: note: stmt 0 test2_16 = _3 < f_15(D); t2.c:8:25: note: stmt 1 test3_17 = _4 < f_15(D); t2.c:8:25: note: children 0x39a48cd8 0x39a48d90 so the V2DF compare produces a QImode mask which we then attempt to convert to a V2SImode mask (the scalar pattern conversion stmt looks wrong, but...) I think the proper way to convert a QImode mask to a V2SImode mask is via a vcond_mask, a missed pattern handling. For the reverse the only way would be a vec_cmp, but this might need prior packing/unpacking if the canonical mask mode for the corresponding data mode is not an integer mode. Alternatively vectorizable_conversion could directly synthesize this alternative as it's difficult to decide during pattern recog.
