Re: how can I remove the warning message

Hi, This is a JVM warning, as Sean explained. You cannot control it via loggers. You can disable it by passing --illegal-access=permit to java. Read more about it at https://softwaregarden.dev/en/posts/new-java/illegal-access-in-java-16/ On Sun, Jan 30, 2022 at 4:32 PM Sean Owen wrote: > This

Re: how can I remove the warning message

This one you can ignore. It's from the JVM so you might be able to disable it by configuring the right JVM logger as well, but it also tells you right in the message how to turn it off! But this is saying that some reflective operations are discouraged in Java 9+. They still work and Spark needs

Re: how can I remove the warning message

Hi, I have often found that logging in the warnings is extremely useful, they are just logs, and provide a lot of insights during upgrades, external package loading, deprecation, debugging, etc. Do you have any particular reason to disable the warnings in a submitted job? I used to disable

Re: how can I remove the warning message

you can try spark.sparkContext.setLogLevel("ERROR") HTH view my Linkedin profile *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise

how can I remove the warning message

When I submitted the job from scala client, I got the warning messages: WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark/jars/spark-unsafe_2.12-3.2.0.jar) to constructor