Re: log transfering into hadoop/spark

2022-08-02 Thread Gourav Sengupta
hi, I do it with simple bash scripts to transfer to s3. Takes less than 1 minute to write it, and another 1 min to include it bootstrap scripts. Never saw the need for so much hype for such simple tasks. Regards, Gourav Sengupta On Tue, Aug 2, 2022 at 2:16 PM ayan guha wrote: > ELK or Splunk

Re: log transfering into hadoop/spark

2022-08-02 Thread ayan guha
ELK or Splunk agents typically. Or if you are in cloud then there are cloud native solutions which can forward logs to object store, which can then be read like hdfs. On Tue, 2 Aug 2022 at 4:43 pm, pengyh wrote: > since flume is not continued to develop. > what's the current opensource tool to

log transfering into hadoop/spark

2022-08-02 Thread pengyh
since flume is not continued to develop. what's the current opensource tool to transfer webserver logs into hdfs/spark? thank you. - To unsubscribe e-mail: user-unsubscr...@spark.apache.org