Re: hadoop input/output format advanced control

2015-03-25 Thread Koert Kuipers
Subject: hadoop input/output format advanced control To: u...@spark.apache.org u...@spark.apache.org currently its pretty hard to control the Hadoop Input/Output formats used in Spark. The conventions seems to be to add extra parameters to all methods and then somewhere

Re: hadoop input/output format advanced control

2015-03-25 Thread Patrick Wendell
, 2015 at 3:52 PM, Koert Kuipers ko...@tresata.com wrote: see email below. reynold suggested i send it to dev instead of user -- Forwarded message -- From: Koert Kuipers ko...@tresata.com Date: Mon, Mar 23, 2015 at 4:36 PM Subject: hadoop input/output

Re: hadoop input/output format advanced control

2015-03-25 Thread Sandy Ryza
: Koert Kuipers ko...@tresata.com Date: Mon, Mar 23, 2015 at 4:36 PM Subject: hadoop input/output format advanced control To: u...@spark.apache.org u...@spark.apache.org currently its pretty hard to control the Hadoop Input/Output formats used in Spark. The conventions

Re: hadoop input/output format advanced control

2015-03-25 Thread Patrick Wendell
...@tresata.com wrote: see email below. reynold suggested i send it to dev instead of user -- Forwarded message -- From: Koert Kuipers ko...@tresata.com Date: Mon, Mar 23, 2015 at 4:36 PM Subject: hadoop input/output format advanced control To: u

Re: hadoop input/output format advanced control

2015-03-25 Thread Aaron Davidson
: hadoop input/output format advanced control To: u...@spark.apache.org u...@spark.apache.org currently its pretty hard to control the Hadoop Input/Output formats used in Spark. The conventions seems to be to add extra parameters to all methods and then somewhere

Re: hadoop input/output format advanced control

2015-03-24 Thread Koert Kuipers
Subject: hadoop input/output format advanced control To: u...@spark.apache.org u...@spark.apache.org currently its pretty hard to control the Hadoop Input/Output formats used in Spark. The conventions seems to be to add extra parameters to all methods and then somewhere deep inside

Re: hadoop input/output format advanced control

2015-03-24 Thread Imran Rashid
email below. reynold suggested i send it to dev instead of user -- Forwarded message -- From: Koert Kuipers ko...@tresata.com Date: Mon, Mar 23, 2015 at 4:36 PM Subject: hadoop input/output format advanced control To: u...@spark.apache.org u...@spark.apache.org currently its

Re: hadoop input/output format advanced control

2015-03-24 Thread Nick Pentreath
at 3:52 PM, Koert Kuipers ko...@tresata.com wrote: see email below. reynold suggested i send it to dev instead of user -- Forwarded message -- From: Koert Kuipers ko...@tresata.com Date: Mon, Mar 23, 2015 at 4:36 PM Subject: hadoop input/output format advanced control

Re: hadoop input/output format advanced control

2015-03-24 Thread Patrick Wendell
, Koert Kuipers ko...@tresata.com wrote: see email below. reynold suggested i send it to dev instead of user -- Forwarded message -- From: Koert Kuipers ko...@tresata.com Date: Mon, Mar 23, 2015 at 4:36 PM Subject: hadoop input/output format advanced control To: u

Fwd: hadoop input/output format advanced control

2015-03-23 Thread Koert Kuipers
see email below. reynold suggested i send it to dev instead of user -- Forwarded message -- From: Koert Kuipers ko...@tresata.com Date: Mon, Mar 23, 2015 at 4:36 PM Subject: hadoop input/output format advanced control To: u...@spark.apache.org u...@spark.apache.org currently its