Thanks Cornelis!
On Mon, Jul 1, 2019 at 7:09 PM Cornelis Poppema wrote:
> To be able to traverse the JSON structure you'd normally need the entire
> structure in memory. For this reason you can't (easily) apply suggestions
> to iterate over a file efficiently to a JSON file: you can perhaps read
generation.
Eric
From: django-users@googlegroups.com on behalf
of PASCUAL Eric
Sent: Monday, July 1, 2019 17:06
To: django-users@googlegroups.com
Subject: Re: Accessing data from a 30 GB file in json format
Hi,
To be able to traverse the JSON structure you&#
m on behalf
of Cornelis Poppema
Sent: Monday, July 1, 2019 15:38
To: Django users
Subject: Re: Accessing data from a 30 GB file in json format
To be able to traverse the JSON structure you'd normally need the entire
structure in memory. For this reason you can't (easily) apply suggesti
To be able to traverse the JSON structure you'd normally need the entire
structure in memory. For this reason you can't (easily) apply suggestions
to iterate over a file efficiently to a JSON file: you can perhaps read the
file efficiently, but the structure in memory will still grow in memory.
https://stackoverflow.com/questions/2396238/memory-error-due-to-the-huge-input-file-size
Just iterate the file do whatever operations you want on the file.
https://towardsdatascience.com/python-basics-iteration-and-looping-6ca63b30835c
On Monday, July 1, 2019 at 11:07:39 AM UTC+1, Nibil Ashraf wr
A screenshot may be better
On Mon, Jul 1, 2019, 10:10 AM John Bagiliko
wrote:
> What is the error message?
>
> On Mon, Jul 1, 2019, 10:07 AM Nibil Ashraf
> wrote:
>
>> Hey,
>>
>> I have a file with a size of around 30GB. The file is in json format. I
>> have to access the data and write that to
What is the error message?
On Mon, Jul 1, 2019, 10:07 AM Nibil Ashraf wrote:
> Hey,
>
> I have a file with a size of around 30GB. The file is in json format. I
> have to access the data and write that to a csv file. When I tried to do
> that with my laptop which has a a RAM of 4GB, I am getting
Hey,
I have a file with a size of around 30GB. The file is in json format. I
have to access the data and write that to a csv file. When I tried to do
that with my laptop which has a a RAM of 4GB, I am getting some error. I
tried to load the json file like this json_parsed = json.loads(json_data)
8 matches
Mail list logo
