https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95888
--- Comment #8 from CVS Commits <cvs-commit at gcc dot gnu.org> --- The master branch has been updated by Marek Polacek <mpola...@gcc.gnu.org>: https://gcc.gnu.org/g:88cfd531c69b3c1fe7a3c183d83cfeacc8f69402 commit r11-7197-g88cfd531c69b3c1fe7a3c183d83cfeacc8f69402 Author: Marek Polacek <pola...@redhat.com> Date: Tue Feb 9 15:17:48 2021 -0500 c++: Endless loop with targ deduction in member tmpl [PR95888] My r10-7007 patch tweaked tsubst not to reduce the template level of template parameters when tf_partial. That caused infinite looping in is_specialization_of: we ended up with a class template specialization whose TREE_TYPE (CLASSTYPE_TI_TEMPLATE (t)) == t, so the second for loop in is_specialization_of never finished. There's a lot going on in this test, but essentially: the template fn here has two template parameters, we call it with one explicitly provided, the other one has to be deduced. So we'll find ourselves in fn_type_unification which uses tf_partial when tsubsting the *explicit* template arguments into the function type. That leads to tsubstituting the return type, C<T>. C is a member template; its most general template is template<class U> template<class V> struct B<U>::C we figure out (tsubst_template_args) that the template argument list is <int, int>. They come from different levels, one comes from B<int>, the other one from fn<int>. So now we lookup_template_class to see if we have C<int, int>. We do the /* This is a full instantiation of a member template. Find the partial instantiation of which this is an instance. */ TREE_VEC_LENGTH (arglist)--; // arglist is now <int>, not <int, int> found = tsubst (gen_tmpl, arglist, complain, NULL_TREE); TREE_VEC_LENGTH (arglist)++; magic which is looking for the partial instantiation, in this case, that would be template<class V> struct B<int>::C. Note we're still in a tf_partial context! So the tsubst_template_args in the tsubst (which tries to substitute <int> into <U, V>) returns <int, V>, but V's template level hasn't been reduced! After tsubst_template_args, tsubst_template_decl looks to see if we already have this specialization: // t = template_decl C // full_args = <int, V> spec = retrieve_specialization (t, full_args, hash); but doesn't find the one we created a while ago, when processing B<int> b; in the test, because V's levels don't match. Whereupon tsubst_template_decl creates a new TEMPLATE_DECL, one that leads to the infinite looping problem. Fixed by using tf_none when looking for an existing partial instantiation. It also occurred to me that I should be able to trigger a similar problem with 'auto', since r10-7007 removed an is_auto check. And lo, I constructed deduce10.C which exhibits the same issue with pre-r10-7007 compilers. This patch fixes that problem as well. I'm ecstatic. gcc/cp/ChangeLog: PR c++/95888 * pt.c (lookup_template_class_1): Pass tf_none to tsubst when looking for the partial instantiation. gcc/testsuite/ChangeLog: PR c++/95888 * g++.dg/template/deduce10.C: New test. * g++.dg/template/deduce9.C: New test.
