Re: Can spark somehow help with this usecase?

2016-04-05 Thread Marco Mistroni
Tuesday, April 5, 2016 at 9:13 AM > To: "user @spark" <user@spark.apache.org> > Subject: Can spark somehow help with this usecase? > > Hi > I m currently using spark to process a file containing a million of > rows(edgar quarterly filings files) > Each row contai

Re: Can spark somehow help with this usecase?

2016-04-05 Thread Andy Davidson
, and cores? Andy From: Marco Mistroni <mmistr...@gmail.com> Date: Tuesday, April 5, 2016 at 9:13 AM To: "user @spark" <user@spark.apache.org> Subject: Can spark somehow help with this usecase? > > Hi > I m currently using spark to process a file conta

Can spark somehow help with this usecase?

2016-04-05 Thread Marco Mistroni
Hi I m currently using spark to process a file containing a million of rows(edgar quarterly filings files) Each row contains some infos plus a location of a remote file which I need to retrieve using FTP and then process it's content. I want to do all 3 operations ( process filing file, fetch