>From my response on the other thread, but applicable to the backend stuff:
"The PCAP Query seems more like PCAP Report to me. You are generating a report based on parameters. That report is something that takes some time and external process to generate… ie you have to wait for it. I can almost imagine a flow where you: * Are in the AlertUI * Ask to generate a PCAP report based on some selected alerts/meta-alert, possibly picking from on or more report ‘templates’ that have query options etc * The report request is ‘queued’, that is dispatched to be be executed/generated * You as a user have a ‘queue’ of your report results, and when the report is done it is queued there * We ‘monitor’ the report/queue press through the yarn rest ( report info/meta has the yarn details ) * You can select the report from your queue and view it either in a new UI or custom component * You can then apply a different ‘view’ to the report or work with the report data * You can print / save etc * You can associate the report with the alerts ( again in the report info ) with…. a ‘case’ or ‘ticket’ or investigation something or other We can introduce extensibility into the report templates, report views ( thinks that work with the json data of the report ) Something like that.” Maybe we can do : template -> query parameters -> script => yarn info yarn info + query info + alert context + yarn status => report info -> stored in a user’s ‘report queue’ report persistence added to report info metron-rest -> api to monitor the queue, read results ( page ), etc etc On May 4, 2018 at 09:23:39, Ryan Merriman (merrim...@gmail.com) wrote: I started a separate thread on Pcap UI considerations and user requirements at Otto's request. This should help us keep these two related but separate discussions focused. On Fri, May 4, 2018 at 7:19 AM, Michel Sumbul <michelsum...@gmail.com> wrote: > Hello, > > > > (Youhouuu my first reply on this kind of mail chain^^) > > > > If I may, I would like to share my view on the following 3 points. > > - Backend: > > The current metron-api is totally seperate, it will be logic for me to have > it at the same place as the others rest api. Especially when more security > will be added, it will not be needed to do the job twice. > The current implementation send back a pcap object which still need to be > decoded. In the opensoc, the decoding was done with tshard on the frontend. > It will be good to have this decoding happening directly on the backend to > not create a load on frontend. An option will be to install tshark on the > rest server and to use to convert the pcap to xml and then to a json that > will be send to the frontend. > > I tried to start directly the map/reduce job to search over all the pcap > data from the rest server and as Ryan mention it, we had trouble. I will > try to find back the error. > > Then in the POC, what we tried is to use the pcap_query script and this > work fine. I just modified it that he sends back directly the job_id of > yarn and not waiting that the job is finished. Then it will allow the UI > and the rest server to know what the status of the research by querying the > yarn rest api. This will allow the UI and the rest server to be async > without any blocking phase. What do you think about that? > > > > Having the job submitted directly from the code of the rest server will be > perfect, but it will need a lot of investigation I think (but I'm not the > expert so I might be completely wrong ^^). > > We know that the pcap_query scritp work fine so why not calling it? Is it > that bad? (maybe stupid question, but I really don’t see a lot of drawback) > > > > - Front end: > > Adding the the pcap search to the alert UI is, I think, the easiest way to > move forward. But indeed, it will then be the “Alert UI and pcapquery”. > Maybe the name of the UI should just change to something like “Monitoring & > Investigation UI” ? > > > > Is there any roadmap or plan for the different UI? I mean did you already > had discussion on how you see the ui evolving with the new feature that > will come in the future? > > > > - Microservices: > > > > What do you mean exactly by microservices? Is it to separate all the > features in different projects? Or something like having the different > components in container like kubernet? (again maybe stupid question, but I > don’t clearly understand what you mean J ) > > > > Michel >
