Hi Diego, If you want the data of all streams to be written to the same files, you can also union the streams before sending them to the sink.
Best, Fabian 2016-11-29 15:50 GMT+01:00 Kostas Kloudas <k.klou...@data-artisans.com>: > Hi Diego, > > You cannot prefix each stream with a different > string so that the paths do not collide? > > If I understand your use-case correctly, this might work. > > Cheers, > Kostas > > On Nov 29, 2016, at 10:04 AM, Diego Fustes Villadóniga <dfus...@oesia.com> > wrote: > > Hi Kostas, > > Thanks for your reply. > > The problem is at the initialization of the job. The reason was that I > was using the same HDFS path as sink for 3 different streams, which is > something that I would like. I can fix it by using different paths > for each stream. > > Maybe there is a way to achieve this in a different manner by joining the > streams somehow before sinking… maybe through Kafka? > > Kind Regards, > > Diego > > > > *De:* Kostas Kloudas [mailto:k.klou...@data-artisans.com > <k.klou...@data-artisans.com>] > *Enviado el:* lunes, 28 de noviembre de 2016 19:13 > *Para:* user@flink.apache.org > *Asunto:* Re: Problems with RollingSink > > Hi Diego, > > The message shows that two tasks are trying to touch concurrently the same > file. > > This message is thrown upon recovery after a failure, or at the > initialization of the job? > Could you please check the logs for other exceptions before this? > > Can this be related to this issue? > https://www.mail-archive.com/issues@flink.apache.org/msg73871.html > > Thanks, > Kostas > > > On Nov 28, 2016, at 5:37 PM, Diego Fustes Villadóniga <dfus...@oesia.com> > wrote: > > Hi colleagues, > > I am experiencing problems when trying to write events from a stream to > HDFS. I get the following exception: > > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException): > failed to create file > /user/biguardian/events/2016-11-28--15/flinkpart-0-0.text > for DFSClient_NONMAPREDUCE_1634980080_43 for client 172.21.40.75 because > current leaseholder is trying to recreate file. > > My Flink version is 1.1.3 and I am running it directly from a JAR (not in > YARN) with java -jar. > > Do you know the reason of this error? > > Kind regards, > > Diego > > >
