Each JSON object needs to be on a single line since this is the boundary
the TextFileInputFormat uses when splitting up large files.
On Wed, Dec 24, 2014 at 12:34 PM, elliott cordo <elliottco...@gmail.com>
wrote:
> I have generally been impressed with the way jsonFile "eats" just about
> any json data model.. but getting this error when i try to ingest this
> file: "Unexpected close marker ']': expected '}"
>
> Here are the commands from the pyspark shell:
>
> from pyspark.sql import HiveContext
> hiveContext = HiveContext(sc)
> f = hiveContext.jsonFile("sample.json")
>
> Here is some sample json:
> {"wf_session": [
>
> {"id":"6021fb91-c9ec-4019-9ab9-f628aee8d259","machine_id":"b45c8c4a-7e8e-442d-8d49-fb7c32e2d813","session_id":"d65ca338-c6b8-4bff-93b1-7f2364726fb7","event_at":"2014-12-19T15:55:31.373Z","screen":"x","type":"1","time_left_secs":1},
>
> {"id":"7e696c19-3ba4-4469-be28-5ef1f0c03d63","machine_id":"b45c8c4a-7e8e-442d-8d49-fb7c32e2d813","session_id":"d65ca338-c6b8-4bff-93b1-7f2364726fb7","event_at":"2014-12-19T15:55:32.385Z","screen":"x","type":"2","ad_unit_id":null,"spot_started_at":"2014-12-19T15:55:12.364Z","spot_ended_at":"2014-12-19T15:55:32.385Z","spot_duration_secs":20,"impression_count":0,"impressions":[],"engagement_count":0,"engagements":[]},
>
> {"id":"68a43006-09bc-4c18-af55-1ebdc0e041a3","machine_id":"b45c8c4a-7e8e-442d-8d49-fb7c32e2d813","session_id":"d65ca338-c6b8-4bff-93b1-7f2364726fb7","event_at":"2014-12-19T15:55:32.375Z","screen":"x","type":"3","duration_secs":"20","to_ad_unit_id":"developmentbea1f3a4-be08-4119-b9f4-7"}
> ] }
>
>
> Any help would be appreciated! :) Merry Xmas!
>
