I found that this condition happened from the upstream kernel. I checked kernel 6.12.35 ~ 6.12.40, they all had the same issues.
Maybe we can only wait until upstream fixes or upgrade to kernel-6.15. Debian 13 is about to be released, shipping with the 6.12 kernel. Unfortunately, after installation, it was found that the Intel B580 graphics card is not supported, which is quite a pity.
