This PR prevents a segmentation fault during `Compile::process_inline_types` 
caused by a missing check for failure.

This was originally spotted on our CI when running 
`applications/ctw/modules/java_desktop.java` with `StressBailout`, and is 
reproducible with replay compilation.

Near the end of `Compile::Optimize`, we have:
https://github.com/openjdk/valhalla/blob/f29dba868499db3b1369efee8ea22365efa36849/src/hotspot/share/opto/compile.cpp#L3238-L3243

In `Compile::process_late_inline_calls_no_inline`, we have a `if (failing())  
return;` check. When using `StressBailout`, this might trigger a compilation 
bailout.

Before valhalla this was fine because there is a failure check a few lines 
after the call to `process_late_inline_calls_no_inline`. Now we also have 
`process_inline_types(igvn)`, and this can go very wrong if `C->root()` is 
`nullptr`. Therefore we need a new `if failing()` check before processing the 
inline types.

I have checked other usages of `process_inline_types` and it seems there is no 
cause for concern there.

I don't think it's worth it to try to produce a regression test here as it 
would rely on setting a stress seed and would therefore be very sensitive.

### Testing
- [ ] tier1-3, plus some internal testing

Thank you for reviewing!

- [x] I confirm that I make this contribution in accordance with the [OpenJDK 
Interim AI Policy](https://openjdk.org/legal/ai).

-------------

Commit messages:
 - Add missing failing() check

Changes: https://git.openjdk.org/valhalla/pull/2332/files
  Webrev: https://webrevs.openjdk.org/?repo=valhalla&pr=2332&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8381866
  Stats: 3 lines in 1 file changed: 3 ins; 0 del; 0 mod
  Patch: https://git.openjdk.org/valhalla/pull/2332.diff
  Fetch: git fetch https://git.openjdk.org/valhalla.git pull/2332/head:pull/2332

PR: https://git.openjdk.org/valhalla/pull/2332

Reply via email to