Hello everyone.
It seems pyspark dataframe read is broken for reading multiple files.
sql.read.json( "file1,file2") fails with java.io.IOException: No input
paths specified in job.
This used to work in spark 1.4 and also still work with sc.textFile
Blaž
i ran into the same thing in scala api. we depend heavily on comma
separated paths, and it no longer works.
On Tue, Oct 6, 2015 at 3:02 AM, Blaž Šnuderl <snud...@gmail.com> wrote:
> Hello everyone.
>
> It seems pyspark dataframe read is broken for reading multiple files.
&g
t; On Tue, Oct 6, 2015 at 3:02 AM, Blaž Šnuderl <snud...@gmail.com
>> <javascript:_e(%7B%7D,'cvml','snud...@gmail.com');>> wrote:
>>
>>> Hello everyone.
>>>
>>> It seems pyspark dataframe read is broken for reading multiple files.
>>>
>>>
t;> separated paths, and it no longer works.
>>>
>>>
>>> On Tue, Oct 6, 2015 at 3:02 AM, Blaž Šnuderl <snud...@gmail.com> wrote:
>>>
>>>> Hello everyone.
>>>>
>>>> It seems pyspark dataframe read is broken for reading
t;
> On Tue, Oct 6, 2015 at 3:02 AM, Blaž Šnuderl <snud...@gmail.com> wrote:
>
>> Hello everyone.
>>
>> It seems pyspark dataframe read is broken for reading multiple files.
>>
>> sql.read.json( "file1,file2") fails with java.io.IOException: No