Log4J and Logback both capture a reference to System.out and/or System.err and then use that reference thereafter. This means that any manipulation of System.out or System.err -- using System.setOut(PrintStream) or System.setErr(PrintStream) -- probably isn't going to play nicely with logging in general.
Here's what that means for Geode tests... If you're writing a Unit Test, then that test will be executed in a long-lived JVM in which other Unit Tests are also going to be executed. The SystemOutRule from system-rules will work with code that uses System.out directly but will not work repeatedly with loggers. If you're writing an Integration Test or Distributed Test, then that test will be executed in a fresh JVM (fork every one is the config), so as long as you can get the SystemOutRule before to invoke before logging, the test will behave. In general that means you'll want to annotate it with @BeforeClass. Here are a couple examples in which log4j2 captures System.out to illustrate why SystemOutRule can't intercept the output after invoking System.setOut(PrintStream): log4j-core/src/main/java/org/apache/logging/log4j/core/config/status/StatusConfiguration.java:43: private static final PrintStream DEFAULT_STREAM = System.out; log4j-api/src/main/java/org/apache/logging/log4j/simple/SimpleLoggerContext.java:87: ps = System.out; If SystemOutRule executes its before -- System.setOut(PrintStream) -- after the above code, logging will be printing to a different PrintStream than the PrintStream that SystemOutRule inserted into System.