[ https://issues.apache.org/jira/browse/METRON-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
James Sirota updated METRON-195: -------------------------------- Labels: ForwardLookingEpic (was: ) > Next Generation Metron UI > ------------------------- > > Key: METRON-195 > URL: https://issues.apache.org/jira/browse/METRON-195 > Project: Metron > Issue Type: Wish > Reporter: George Vetticaden > Labels: ForwardLookingEpic > Attachments: Metron Next Gen UI.png > > > !Metron Next Gen UI.png|width=800, height=400! > Over the last few weeks, we have had the chance to interview over 20 > different SOC/Security personnel from various organizations. These folks > represented the following personas: > * Tier 1, 2 and 3 SOC Analyst > * SOC Manager > * Security Platform Architects > * SIEM Engineers > * Data Scientists > The interviews ranged from 10 minute to 2 hour conversations and some of the > interview questions can be found on the following wiki page: > https://cwiki.apache.org/confluence/display/METRON/SOC+Interview+Discussion+Guide > In addition to these questions, we had the opportunity to watch them use > their current security tooling and also watch them use the current Metron > based Kibana UI (for folks that had Metron deployed) > The user interviews are documented here: > https://cwiki.apache.org/confluence/display/METRON/User+Interviews > This provides us a valuable trove of requirements and insights about the > challenges faced by SOC personnel. > While the current Metron UI based on Kibana 3/4 has some advantages including > the ability to connect to different elastic indexes, dashboard and panel > support, and rich support for different panel types, it was clear the > current Metron UI lacks some key capabilities that most SOC personnel would > require. Some key limitations identified based on these interviews included: > * Clunky unintuitive user interface. Someone has to to teach how to use > Metron UI couple of times for a user to understand even how to start with it. > * All things around searching was very difficult to understand. This included > things like > ** Difference between search and filtering > ** How to create a pinned/named query > ** How to associate pinned queries to panels > ** Search DSL was difficult to use > * No Authentication or RBAC support > * Alert Triaging not supported > In addition to these challenges, the following were key requirements that the > SOC personnel deemed critical for Metron to have: > * Support for Alert Triaging. The ability for the user to group/search alerts > and their related telemetries and create a case/ticket from it. > * Case Workflow Management > * Alert Queues that SOC analysts can pick up alerts to work on > * Authentication & Authorization/RBAC Support > * Make Search easier for a novice easier > * Faceting Support > * More robust pivoting functionality. > * Help the advanced user write more complicated queries/searches > * More advanced visualization support including link analysis / Knowledge > Graphs > * Curated/Filtered views of Alerts for SOC 1 Analysts > * KPIs that provides accountability (e/g: average time spent on case, top 10 > critical alerts, SLA violations..) > * Better/ more intelligent grouping of alerts into a "Meta-Alert" > * Drill down of Meta-Alert into the set of alerts the meta consists of. > Drilling down on a raw alert to the the telemetry data sources > * Management UI Tooling around provisioning, monitoring and managing Metron. > Based on this data, some key design principles started to emerge that the > Metron UI should adhere to: > # A Single Pane of Glass - One interface to monitor and act Telemetry feeds, > Behavior anomalies, Alerts and Meta-Alerts, Metron system health, > Investigate, slice and dice, hunt and seek, search & filter, Collaborate and > share info and Identify and collect data > # Find Needles in the Haystack - Find that tiny detail that helps you create > a better defense. Meta-Alerts are doorways to the data details Meta-Alerts > can contain hundreds of smaller related alerts. Search and Facet to dive > deeper > # Same Data, Many Angles - One data set can be visualized in many ways. Not > only do we need to show many views of a data set out of the box. We must also > allow for customization. > # It’s All Collaborative - I can share anything I see. I can preserve it for > later. I can save it for myself or send to someone else. Once a collaboration > begins, we can annotate and categorize data for future use and investigation. > Oversight. All activity is audited. So what I do and the amount of time I > spend doing it can be backtracked and replayed somehow. > # It Evolves - Off the shelve, the system will be powerful. However, it can > grow and evolve with threat intelligence feeds, Behavior Modeling, App store > style add-ons, and Learning Models. Learning Models use machine learning to > predict anomalous behavior. It takes a pattern and alerts when similar > patterns emerge. Behavior Modeling observers the average entity behavior and > creates an expected range of behavior. The system learns from what happens > as well as what you feed it. > # A Tailored Fit - The system is personalized and customizable. Based on > role, a user may get a specific interface. Based on experience, a user can > modify interfaces to suit there personal preferences. > # I’m Never Lost - No matter how deep a user dives into the data, they always > know where they are. I can move back through my diving to a previous result. > The complexity of these systems can be immense. We need to design for > interruption and the resuming of their place by providing some way-finding > tools to guide users. > # It Teaches You - The system teaches novice users to become advanced users. > As you use the system if gives you all the hints you need to become an > expert. Complex things are made easy enough for anyone to be able to do. > # Think for Them - The system anticipates user needs and does the thinking > for them. We put the burden of complexity on the software so that we simplify > the user experience. Make the experience as human as possible, guide and help > users achieve outcomes. Shorten the number clicks by decreasing complexity. > Hence, based on these challenges and requirements discovered from these > interviews, this is a proposal to build a set of next generation user > interfaces for Apache Metron that caters to the different SOC personas that > will use Metron. Fore more details on SOC personnas see here: > https://cwiki.apache.org/confluence/display/METRON/Metron+User+Personas+And+Benefits > As the below above illustrates, the proposal is to move away from Kibana and > introduce a number of different views/UIs that caters to different types of > SOC users across three Phases: > * Phase 1 - This phase focuses on SOC analysts and Managers. They includes > the following different Views/UIs: > ** Metron Investigator UI - Ability for SOC analyst to do alert triaging, > threat hunting and searching. It should also provide the ability to export a > view based on the filters and seed a new Metron Case with that context. > ** Case Management Workflow - The ability to manage Metron cases. This > entails supporting different workflow statuses (new, pending, in-progress, > escalate, etc..) It also should provide Metron case queue management > functionality. > KPI and Situational Awareness Dashboards - Set of Dashboards that provides > situational awareness of the environment as well key performance indicators > for managers. > * Phase 2 - The primary focus is on the Platform Engineers / Dev Ops that > will manage the Metron application. This involves interfaces to manage > topologies, enrichments, threat intel data sources, configuration, ec.. > * Phase 3 - This phase is all about the data scientist and providing an > integrated analytical environment that uses powerful tools such as Zeppelin, > Jupiter, Python etc.. > > This initial high level proposal will need to be fleshed out as the community > provides their feedback. -- This message was sent by Atlassian JIRA (v6.3.4#6332)