Any suggestions ? to get ride of classic data and save up some space. On Wed, Dec 26, 2018 at 8:09 PM Farrukh Naveed Anjum < anjum.farr...@gmail.com> wrote:
> Any exact command / example ? I am using BRO Tool for Log Collection > > On Wed, Dec 26, 2018 at 6:03 PM zeo...@gmail.com <zeo...@gmail.com> wrote: > >> Aren't the logs stored in date-based directories? I thought we used to >> deploy a daily cron with a hadoop fs -rmr based on today's date minus 30 >> days or so. >> >> Jon >> >> On Wed, Dec 26, 2018 at 6:20 AM Farrukh Naveed Anjum < >> anjum.farr...@gmail.com> wrote: >> >>> what about data in HDFS and Other Logs ? Any idea to purne them ? >>> >>> On Wed, Dec 26, 2018 at 3:51 PM zeo...@gmail.com <zeo...@gmail.com> >>> wrote: >>> >>>> There used to be something that was automatically deployed as a cron >>>> that did that. For ES I know it moved to curator >>>> https://github.com/apache/metron/blob/master/metron-platform/metron-data-management/README.md#pruning-data-from-elasticsearch >>>> >>>> I don't know if the HDFS job still exists or not, but it used to. >>>> >>>> Jon >>>> >>>> On Wed, Dec 26, 2018, 2:06 AM Farrukh Naveed Anjum < >>>> anjum.farr...@gmail.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> How can write a data retention script for metron. So that safe hard >>>>> disk space. Like I want to Store only last N Days (Say 30 days) data in >>>>> HDFS, Elasticssearch and Logs in Metron. >>>>> >>>>> Right now, Metron eats up my all hard drive space on server. >>>>> >>>>> Any Idea / Help will be appreciated. >>>>> >>>>> >>>>> -- >>>>> With Regards >>>>> Farrukh Naveed Anjum >>>>> >>>> -- >>>> >>>> Jon Zeolla >>>> >>> >>> >>> -- >>> With Regards >>> Farrukh Naveed Anjum >>> >> -- >> >> Jon Zeolla >> > > > -- > With Regards > Farrukh Naveed Anjum > -- With Regards Farrukh Naveed Anjum
