Re: Path to resource added with SQL: ADD FILE

[Herman van Hövell tot Westerflier]

Hi Antonio,

I am not sure you got the silent treatment on the user list. Stackoverflow
is also a good place to ask questions.

Could you use an absolute path to add the jar file. So instead of './my
resource file' (which is a relative path; this depends on where you started
Spark), use something like this '/some/path/my resource file' or use an URI.

Kind regards,

Herman van Hövell


2016-02-03 19:17 GMT+01:00 Antonio Piccolboni :

> Sorry if this is more appropriate for user list, I asked there on 12/17
> and got the silence treatment. I am writing a UDF that needs some
> additional info to perform its task. This information is in a file that I
> reference in a SQL ADD FILE statement. I expect that file to be accessible
> in the working directory for the UDF, but it doesn't seem to work (aka,
> failure on open("./my resource file"). What is the correct way to access
> the added resource? Thanks
>
>
> Antonio
>

Re: Path to resource added with SQL: ADD FILE

[Antonio Piccolboni]

Hi Herman,
thanks for your reply, I used an absolute path to add the file. I use a
relative path only to access it in the UDF. I am not sure what absolute
path I should use in the UDF, if any. The documentation

I am referring to is Beeline for Hive's, I hope it's the relevant bit.
Therein I read

add FILE /tmp/tt.py;

select from networks a MAP a.networkid USING 'python tt.py' as nn
where a.ds = '2009-01-04' limit 10;

So it's just referring to the file by basename, which I think is
equivalent to "./tt.py". I tried to follow this example closely, just
using R instead of python and the name of the file is hard-coded, not
an argument. If what you mean is that I should provide the same
absolute path in the ADD FILE and in the SELECT, then I will try that.
Thanks


Antonio


On Thu, Feb 4, 2016 at 4:21 AM Herman van Hövell tot Westerflier <
hvanhov...@questtec.nl> wrote:

> Hi Antonio,
>
> I am not sure you got the silent treatment on the user list. Stackoverflow
> is also a good place to ask questions.
>
> Could you use an absolute path to add the jar file. So instead of './my
> resource file' (which is a relative path; this depends on where you
> started Spark), use something like this '/some/path/my resource file' or
> use an URI.
>
> Kind regards,
>
> Herman van Hövell
>
>
> 2016-02-03 19:17 GMT+01:00 Antonio Piccolboni :
>
>> Sorry if this is more appropriate for user list, I asked there on 12/17
>> and got the silence treatment. I am writing a UDF that needs some
>> additional info to perform its task. This information is in a file that I
>> reference in a SQL ADD FILE statement. I expect that file to be accessible
>> in the working directory for the UDF, but it doesn't seem to work (aka,
>> failure on open("./my resource file"). What is the correct way to access
>> the added resource? Thanks
>>
>>
>> Antonio
>>
>
>