I've just had a deeper look at the possible issue and here are my findings: * In 3.0.1 KafkaTokenUtil.needTokenUpdate has 3 params * In 3.1.1 KafkaTokenUtil.needTokenUpdate has 2 params * I've decompiled spark-token-provider-kafka-0-10_2.12-3.1.1.jar and KafkaTokenUtil.needTokenUpdate has 2 params in it's definition * I've decompiled spark-sql-kafka-0-10_2.12-3.1.1.jar and KafkaTokenUtil.needTokenUpdate is called only from getOrRetrieveConsumer with 2 params * The type and number of params are matching withing the 3.1.1 jars * We have a delegation token integration test which compiled and passed (just checked it)
If you think there is an issue in the code and/or packaging please open a jira with more details. BR, G On Tue, Apr 6, 2021 at 12:54 PM Gabor Somogyi <gabor.g.somo...@gmail.com> wrote: > Since you've not shared too much details I presume you've updated the > spark-sql-kafka > jar only. > KafkaTokenUtil is in the token provider jar. > > As a general note if I'm right, please update Spark as a whole on all > nodes and not just jars independently. > > BR, > G > > > On Tue, Apr 6, 2021 at 10:21 AM Mich Talebzadeh <mich.talebza...@gmail.com> > wrote: > >> >> Hi, >> >> >> Any chance of someone testing the latest spark-sql-kafka-0-10_2.12-3.1.1.jar >> for Spark. It throws >> >> >> java.lang.NoSuchMethodError: >> org.apache.spark.kafka010.KafkaTokenUtil$.needTokenUpdate(Ljava/util/Map;Lscala/Option;)Z >> >> >> However, the previous version spark-sql-kafka-0-10_2.12-3.0.1.jar works >> fine >> >> >> Thanks >> >> >> view my Linkedin profile >> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >> >> >> >> *Disclaimer:* Use it at your own risk. Any and all responsibility for >> any loss, damage or destruction of data or any other property which may >> arise from relying on this email's technical content is explicitly >> disclaimed. The author will in no case be liable for any monetary damages >> arising from such loss, damage or destruction. >> >> >> >
