This one you can ignore. It's from the JVM so you might be able to disable
it by configuring the right JVM logger as well, but it also tells you right
in the message how to turn it off!
But this is saying that some reflective operations are discouraged in Java
9+. They still work and Spark needs them, but they cause a warning now. You
can however ignore it.
On Sun, Jan 30, 2022 at 2:56 AM Gourav Sengupta <gourav.sengu...@gmail.com>
wrote:
> Hi,
>
> I have often found that logging in the warnings is extremely useful, they
> are just logs, and provide a lot of insights during upgrades, external
> package loading, deprecation, debugging, etc.
>
> Do you have any particular reason to disable the warnings in a submitted
> job?
>
> I used to disable warnings in spark-shell using the
> Logger.getLogger("akka").setLevel(Level.OFF) in case I have not completely
> forgotten. Other details are mentioned here:
> https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.SparkContext.setLogLevel.html
>
>
>
> Regards,
> Gourav Sengupta
>
> On Fri, Jan 28, 2022 at 11:14 AM <capitnfrak...@free.fr> wrote:
>
>> When I submitted the job from scala client, I got the warning messages:
>>
>> WARNING: An illegal reflective access operation has occurred
>> WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform
>> (file:/opt/spark/jars/spark-unsafe_2.12-3.2.0.jar) to constructor
>> java.nio.DirectByteBuffer(long,int)
>> WARNING: Please consider reporting this to the maintainers of
>> org.apache.spark.unsafe.Platform
>> WARNING: Use --illegal-access=warn to enable warnings of further illegal
>> reflective access operations
>> WARNING: All illegal access operations will be denied in a future
>> release
>>
>> How can I just remove those messages?
>>
>> spark: 3.2.0
>> scala: 2.13.7
>>
>> Thank you.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>>
>>
