Machine learning, or artificial intelligence (AI), is the most recent buzzword making its way around the cybersecurity community as a way to up-level and advance security capabilities. As threats become more sophisticated, machine learning IS needed; however, not every product touting itself as AI is just that. How can organizations determine if a product leverages machine learning, and well?
In this blog, I will share some perspectives on what machine learning can mean for enhanced security threat response.
What is Machine Learning?
Let’s start by defining machine learning. It is not another name for analysis or analytics. Machine learning may be used in the process of analysis but it’s not a pseudonym for machine learning or vice versa.
Below are definitions that help explain the difference:
- Analysis is…the detailed examination of the elements or structure of something, typically as a basis for discussion or
- Machine learning is…an application of AI providing systems the ability to automatically learn and improve from experience without being explicitly programmed. Ultimately, machine learning focuses on the development of computer programs that can access data and use it to learn for themselves.
These two disciplines are not unique to cybersecurity and have a rich history of research and capabilities that support many different problem domains. We can learn a lot from other domains leveraging these disciplines to inform how we can use them with cybersecurity solutions.
For example, advances in automated bot responses for support portals can teach us about cybersecurity responses. The variability of user expressions – the various motives for a person reaching out for support – and the different outcomes expected by the user requires both machine learning and natural language processing. In many respects such a mechanism is similar to how a cyber defense response must act when reacting to different threat behaviors, actor motives and defensive outcomes expected in a security world.
At LookingGlass, we leverage both analysis and machine learning disciplines to create a more sophisticated approach to threat response. We call this combination Intelligent Threat Response and it drives the set of threat mitigation responses beyond rudimentary security techniques.
Machine learning can be used to assist cybersecurity in many regards, but we will focus on 4 of the key areas:
- Learning what data matters
- Learning where anomalies lie
- Learning behavioral patterns
- Learning applied to threat responses
Each of these areas are interconnected. Information learned by one area can inform the other areas in a cyclical manner. How these connections and results are integrated between areas is as important to the overall solution as the individual area. (*See neural networks below)
Learning What Data Matters
There are many sources of information from security telemetry (flow, firewall logs, audit logs, host events, etc.), host information, vulnerability knowledge and threat intelligence (campaigns, attack patterns, etc.) to name a few key sources. The breadth and depth of information available for the purposes of threat detection, correlation, prioritization and response continues to increase across all the sources.
How do we make sense of what informational elements are relevant to our security and the threats that may attack us?
There are several dimensions to this question:
- Sources (flow vs firewall vs Active Directory (AD) vs Open-Source vs Internal vs External Partner, etc.)
- Information Type (malware report vs. intel report vs. indicator vs. sighting, etc.)
- Data Provided (detailed vs. abstract vs. fine-grained)
Below is a small snippet of a STIX2 malware report highlighting the problem faced when attempting to determine what fields/data is relevant in a malware report:
A few machine learning techniques to consider in determining data relevance are described below:
Classification
- Generally, this refers to the identification of data categories that are known bad or the opposite based on a training set of data.
- Classification may be helpful identifying sources or categories of those sources and their data.
- There are multiple methods available including popular approaches such as Bayesian Classification.
Clustering
- Generally, this refers to the segregation of data into groups that have similar traits for the purposes of deciding a set of associated actions related to that grouping.
- Clustering may be helpful in identifying security data from similar attack patterns by actors.
- There are multiple clustering models based on connectivity, centroid, distribution…. etc.
Neural Networks
- Generally, this refers to a statistical modelling approach based on biological neural networks that are capable of modelling and processing nonlinear relationships between inputs/outputs in parallel.
- Learning helps tune the pathways through the neural network based on observed security data that is considered more relevant over others.
- Without specific training data for the neural network connections that are determined, it is possible the neural network indicates non-malicious activities occurring in the network and systems that are feeding telemetry. Therefore, the security team should focus on continuous training of the network to improve its results and interpretation of the results to validate results before acting to mitigate any specific behaviors.
Learning Where Anomalies Lie
Anomaly detection is not pattern detection.
It is the process of identifying data (and associated behaviors) that represents outliers. In order to do this, the process must first learn what is not anomalous. This can be especially useful for identifying “unknown unknowns” such as zero-day threats.
Pattern detection is the process of identifying a data set that matches a specific pattern (either known bad, known suspicious or known good).
Why do anomalies matter in cybersecurity?
Anomalies can provide identification and insight on malicious activity that humans may not be able to identify.
Anomalies could occur in multiple regards to the analysis of security data and intelligence where it represents unexpected:
- frequency
- type
- combination
- value
How do these anomaly results impact security responses? It can assist by:
- Confirming significance
- When threats occur, the information on the anomaly can help prioritize and inform its relevance
- Providing further pointers for investigating an anomaly
- Help identify root-cause and inform analyst pivot decisions for further investigation
Learning Behavioral Patterns
As described above, pattern detection can assist in identifying specific data within security telemetry, intelligence, etc. that represents a pattern. Machine learning can assist in this regard by identifying patterns that were not previously defined nor known by the analysis (human or machine). When combined with human review the number of false positive patterns that are not malicious can be significantly reduced especially when starting with known malicious patterns that help training the learning algorithms.
Behavioral Detection and Responses Must Remain Adaptive
The behavioral pattern process should not just be limited to detection of a behavior but rather the full set of interactions between all systems involved including state transitions exercised either by a machine or a human actor. This means that capturing the behaviors and associated telemetry of an adversary as they execute a campaign against an organization is critical for behavioral pattern detection.
Adaptive Cyber Security
Behavioral Detection and Responses Must Remain Adaptive
Adversaries can easily change tactics used as part of an attack execution and therefore the technical indicators associated with those new tactics are easily missed in simple correlation techniques that have yet to be programmed with the new indicators. Having a mechanism to identify behaviors that match patterns or components of those patterns to then feed that into machine-driven prioritization system is important.
Using Machine Learning for Threat Intelligence Responses
The final area we are covering is how machine learning can be applied to how security defenders respond to threats in unique and specific ways. Leveraging the results from the combination of anomaly detection and behavioral pattern detection, threat response systems can provide recommendations with learned specific actions that can be taken in response to an adversary for a specific scenario based on policies defined by humans.
We call the identification of an anomaly or a pattern an intelligence source event.
If we understand when intelligence source events occur and understand what causes that event we can consider any number of responses:
- Block
- whether we need to respond to the source of the event
- e.g. should we block the source
- Deceive
- whether we need to respond in a manner that causes further action by the instigator that causes further events
- e.g. should we modify our responses to the source so that we can learn more about their behaviors
- No Action
- whether we consider no action at all
- e.g. is this event not relevant to our environment or we don’t want to tip our hand that we know the source has been detected
Machine learning can be a valuable multi-faceted discipline that enhances many aspects of an intelligence threat response system. When we apply specific responses to a threat, then we can learn the outcome of that response and continuously evolve our understanding of the associated anomalies and patterns. We are then able to improve our responses so that we can achieve our goals as a security defender – protecting our customers and networks and fending off our adversaries.
If you are considering machine learning products or techniques in your environment, we hope you found this background useful. LookingGlass is continuously looking to further implement machine learning technology to improve threat intelligence monitoring and cybersecurity. Our portfolio of machine learning products and techniques such as scoutTHREAT™ and our On-Demand services utilize further the boundaries of cybersecurity innovation.
To learn more about LookingGlass use of machine learning in our Intelligent Threat Response systems, please contact us.
Related Content
