Re: Path to resource added with SQL: ADD FILE

to add the jar file. So instead of './my > resource file' (which is a relative path; this depends on where you > started Spark), use something like this '/some/path/my resource file' or > use an URI. > > Kind regards, > > Herman van Hövell > > > 2016-02-03 19:17 GMT+01:00 An

Path to resource added with SQL: ADD FILE

Sorry if this is more appropriate for user list, I asked there on 12/17 and got the silence treatment. I am writing a UDF that needs some additional info to perform its task. This information is in a file that I reference in a SQL ADD FILE statement. I expect that file to be accessible in the

Re: groupByKey() and keys with many values

You may also consider selecting distinct keys and fetching from database first, then join on key with values. This in case Sean's approach is not viable -- in case you need to have the DB data before the first reduce call. By not revealing your problem, you are forcing us to make guesses, which

Re: groupByKey() and keys with many values

To expand on what Sean said, I would look into replacing groupByKey with reduceByKey. Also take a look at this doc . I happen to have designed a library that was subject to