It will work with spark-submit, if putting the configuration, which is addressed below, under the maven-shade-plugin.
<configuration> <relocations> <relocation> <pattern>net.minidev</pattern> <shadedPattern>shaded.net.minidev</shadedPattern> </relocation> </relocations> </configuration> Still, need a way to make it work with spark-shell for testing purpose. Any idea will be grate. Thank you. On Wed, Apr 5, 2017 at 6:52 PM, Mungeol Heo <mungeol....@gmail.com> wrote: > Hello, > > I am using "minidev" which is a JSON lib to remove duplicated keys in > JSON object. > > -------------------- > minidev > -------------------- > > <dependency> > <groupId>net.minidev</groupId> > <artifactId>json-smart</artifactId> > <version>2.3</version> > </dependency> > > -------------------- > Test Code > -------------------- > > import net.minidev.json.parser.JSONParser > val badJson = "{\"keyA\":\"valueA\",\"keyB\":\"valueB\",\"keyA\":\"valueA\"}" > val json = new > JSONParser(JSONParser.MODE_PERMISSIVE).parse(badJson.toLowerCase()) > println(json) > > -------------------- > > The source code placed above works at IDE like intellij. > But, it gives error at spark-shell > > -------------------- > Error > -------------------- > > net.minidev.json.parser.ParseException: Unexpected duplicate key:keya > at position 33. > > -------------------- > > BTW, both IDE and spark-shell using same version of scala which is 2.11.8. > And, of course, same version of "minidev" > > Any help will be great. > Thank you. --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org