Siying Dong created SPARK-44683:
-----------------------------------
Summary: Logging level isn't passed to RocksDB state store
provider correctly
Key: SPARK-44683
URL: https://issues.apache.org/jira/browse/SPARK-44683
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 3.4.1
Reporter: Siying Dong
We pass log4j's log level to RocksDB so that RocksDB debug log can go to log4j.
However, we pass in log level after we create the logger. However, RocksDB only
takes log level when a logger is created, so it never changes. This has two
impacts: (1) setting DEBUG level don't make RocksDB generate DEBUG level logs;
(2) setting WARN or ERROR level does prevent INFO level logging, but RocksDB
still makes JNI calls to Scala, which is an unnecessary overhead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
