Appreciate if somebody has experience on the above and respond to the same.
Logically it should work if we have a property set hdfs.url =
file:///home/user/data
On Wed, May 15, 2019 at 5:06 AM Vinay Jain wrote:
> Redirecting to a file will not work, we would not be able to create a
> different
Redirecting to a file will not work, we would not be able to create a
different filenames after some time or filesize. HDFS connect sink already
has those options , also i could use some minor transformations while using
the connect
On Tue, May 14, 2019 at 7:28 AM Hans Jespersen wrote:
> Can you
Can you just use kafka-console-consumer and just redirect the output into a
file?
-hans
On Mon, May 13, 2019 at 1:55 PM Vinay Jain wrote:
> Hi
>
> The data needs to be transferred to some other system in other network, and
> due to some security reasons, the other systems cannot be exposed . S
Hi
The data needs to be transferred to some other system in other network, and
due to some security reasons, the other systems cannot be exposed . So the
available mechanism is file based integration. Is there a production ready
Kafka connect adapter which can create files in local directory.
Reg
Can you explain more about why you're writing a file with the data?
Presumably, this is for another application to consume; could it not take
the data from Kafka directly, whether with a native client or over the REST
proxy?
Oftentimes local files are unnecessary 'duck tape' for integration that ca
Hi
I would like to consume a Topic and save the AVRO messages in local
directory in the AVRO file formats. As per Kafka Connect File Stream
documentation, it is not for production use.
Other Option I am thinking to use Kafka Connect - HDFS Sink but I am not
sure if it can also write to the Local
