Good point. You are right, the flag was more kind of workaround waiting
for Beam IO improvement, but not really a solution. And it's not good
anyway as it can disturb user (if the runner doesn't have a native
connector).
So, as I said on another e-mail, the obvious conclusion is that we have
to provide new good IOs ;)
I like it !!
I will work on this !
Thanks !
Regards
JB
On 04/28/2016 07:19 PM, Raghu Angadi wrote:
It would also be good to see examples of native I/O being noticeably more
performant. The CassandraIO and PubsubIO are examples of corresponding Beam
sources missing rather than Beam sources being slow.
I think it is better to look into why Beam IO can not be performant. I
think in most cases it can be addressed, there will always be small number
of exceptions that work best in a runner.
If the performance gap is because of a specific runner design or
implementation, then it should not ideally influence Beam API too much.
I suspect 'useNative()' would not be that clean to implement. What a Beam
source need to provide to runner might vary a lot between the runners. Can
we see the API between the source and the runner to support useNative()?
On Thu, Apr 28, 2016 at 5:41 AM, Jean-Baptiste Onofré <j...@nanthrax.net>
wrote:
* Beam doesn't provide the IO yet (for instance, spark cassandra connector
is available whereas we don't have yet any CassandraIO (I'm working on it
anyway ;))
--
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
