Tasks and Simulation
MIMIK can be used to calculate the probability of a succcess and other metrics on the killweb. In order to do this, a task must be defined for each component in the killweb. This tutorial outlines the Task class in MIMIK and demonstrates the Monte Carlo simulation capability.
[1]:
import os
import json
%matplotlib widget
from mimik.killweb import Killweb
Custom Tasks
The task is an attribute of each node in the component graph. MIMIK provides an abstract Task class. Each task will need to be created by the user. The forward method in the Task class returns a probability of success. The user must create a function to produce the probability of success, which can be an estimated value (e.g., the probability of finding a target is 0.90 or 90%), a probability randomly sampled from a distribution such as a Normal distribution of past performance, or a custom function provided by the user. The Task classes are required to be in the tasks directory.
The tasks directory is used by MIMIK to create a TaskFactory which searches for classes inheriting the AbstractTask class of MIMIK. Example classes used in this tutorial are provided in the docs/tutorials/tasks directory. There are two key things to note about creating a custom task. The first is that the task class must inherit from AbstractClass as previously mentioned. The second is that the task must include a forward() function which returns the probability the task succeeds. Any number of helper functions, classes, or libraries can be used to calculate the probability of a task, and more advanced examples can be found in the examples directory.
The task associated with each component is defined in the JSON configs file as an attribute of the component. The attribute contains the name of each Task and the arguments for each task. The JSON for this tutorial is displayed below. The only task argument for this simple tutorial is the probability of success for each task.
[2]:
# config filename
config_filename = "killweb_w_tasks.json"
configs_pth = os.path.join('configs', config_filename)
with open(configs_pth, 'r') as f:
killweb_configs = json.load(f)
print(json.dumps(killweb_configs, indent=4))
{
"killweb_w_tasks": {
"Find1": {
"connected_components": [
"Fix1",
"Fix2"
],
"attributes": {
"x": 0,
"y": 0,
"task": "Find",
"task_arguments": {
"success_probability": 0.9
}
}
},
"Fix1": {
"connected_components": [
"Track1",
"Track2"
],
"attributes": {
"x": 1,
"y": 0,
"task": "Fix",
"task_arguments": {
"success_probability": 0.9
}
}
},
"Track1": {
"connected_components": [],
"attributes": {
"x": 2,
"y": 0,
"task": "Track",
"task_arguments": {
"success_probability": 0.9
}
}
},
"Find2": {
"connected_components": [
"Fix1",
"Fix2"
],
"attributes": {
"x": 0,
"y": 1,
"task": "Find",
"task_arguments": {
"success_probability": 0.9
}
}
},
"Fix2": {
"connected_components": [
"Track1",
"Track2"
],
"attributes": {
"x": 1,
"y": 1,
"task": "Fix",
"task_arguments": {
"success_probability": 0.9
}
}
},
"Track2": {
"connected_components": [],
"attributes": {
"x": 2,
"y": 1,
"task": "Track",
"task_arguments": {
"success_probability": 0.9
}
}
}
}
}
Monte Carlo Simulation
MIMIK can perform Monte Carlo simulation over the killweb using the defined tasks.
[3]:
killweb = Killweb(config_file=configs_pth, silent=True)
The monte_carlo_on_paths() method performs Monte Carlo simulation on each possible path through the killweb. The argument for this method is the number of Monte Carlo simulations to run on each path. For this example, we run Monte Carlo simulation 10 times on each path and print the results as a dictionary.
The first dictionary in the results displays the success of each event. A 1 indicates that the event was successful and the path continued. A 0 indicates that the event was not successful and the path failed. The second dictionary displays the probability of success for each event in that run of the simulation.
[4]:
killweb.monte_carlo_on_paths(10)
mc_results = killweb.get_monte_carlo_results()
mc_results
[4]:
({'Find1, Fix1, Track1': [[1, 1, 1],
[1, 1, 1],
[0, 0, 0],
[1, 1, 1],
[1, 1, 1],
[0, 0, 0],
[0, 0, 0],
[1, 0, 0],
[1, 1, 1],
[1, 1, 1]],
'Find1, Fix2, Track1': [[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[0, 0, 0],
[1, 1, 1],
[1, 1, 1],
[1, 0, 0],
[1, 1, 1]],
'Find1, Fix1, Track2': [[1, 1, 1],
[1, 1, 1],
[0, 0, 0],
[0, 0, 0],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[0, 0, 0]],
'Find1, Fix2, Track2': [[1, 1, 0],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1]],
'Find2, Fix1, Track1': [[1, 1, 1],
[1, 1, 1],
[1, 0, 0],
[1, 1, 1],
[1, 1, 1],
[1, 1, 0],
[1, 1, 1],
[0, 0, 0],
[1, 1, 1],
[1, 1, 0]],
'Find2, Fix2, Track1': [[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[0, 0, 0],
[1, 1, 1],
[1, 1, 1],
[1, 1, 1]],
'Find2, Fix1, Track2': [[1, 1, 1],
[1, 1, 1],
[1, 1, 1],
[1, 0, 0],
[1, 1, 1],
[1, 1, 1],
[1, 0, 0],
[1, 1, 1],
[1, 1, 1],
[1, 0, 0]],
'Find2, Fix2, Track2': [[1, 1, 0],
[1, 1, 1],
[1, 1, 1],
[0, 0, 0],
[1, 0, 0],
[1, 1, 1],
[1, 1, 0],
[0, 0, 0],
[1, 1, 1],
[1, 1, 1]]},
{'Find1, Fix1, Track1': [[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0, 0],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0, 0],
[0.9, 0, 0],
[0.9, 0.9, 0],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9]],
'Find1, Fix2, Track1': [[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0, 0],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0],
[0.9, 0.9, 0.9]],
'Find1, Fix1, Track2': [[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0, 0],
[0.9, 0, 0],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0, 0]],
'Find1, Fix2, Track2': [[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9]],
'Find2, Fix1, Track1': [[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0, 0],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9]],
'Find2, Fix2, Track1': [[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0, 0],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9]],
'Find2, Fix1, Track2': [[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0]],
'Find2, Fix2, Track2': [[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0, 0],
[0.9, 0.9, 0],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9],
[0.9, 0, 0],
[0.9, 0.9, 0.9],
[0.9, 0.9, 0.9]]})
The probability of success for each path can be printed using the print_probabilities_of_paths() method. This method also prints the average number of successful events over the simulation.
[5]:
killweb.print_probabilities_of_paths()
Path: Find2, Fix2, Track1
Probability of Success: 0.9
Average Number of Successful Events: 2.7
Path: Find1, Fix2, Track2
Probability of Success: 0.9
Average Number of Successful Events: 2.9
Path: Find1, Fix2, Track1
Probability of Success: 0.8
Average Number of Successful Events: 2.5
Path: Find2, Fix1, Track2
Probability of Success: 0.7
Average Number of Successful Events: 2.4
Path: Find1, Fix1, Track2
Probability of Success: 0.7
Average Number of Successful Events: 2.1
Path: Find2, Fix1, Track1
Probability of Success: 0.6
Average Number of Successful Events: 2.3
Path: Find1, Fix1, Track1
Probability of Success: 0.6
Average Number of Successful Events: 1.9
Path: Find2, Fix2, Track2
Probability of Success: 0.5
Average Number of Successful Events: 2.0
The probability of success and the average number of successful events for each path can be extracted from MIMIK for use in other analysis.
[6]:
probability_of_success = killweb.get_probabilities_of_paths()
probability_of_success
[6]:
{'Find2, Fix2, Track2': np.float64(0.5),
'Find1, Fix1, Track1': np.float64(0.6),
'Find2, Fix1, Track1': np.float64(0.6),
'Find1, Fix1, Track2': np.float64(0.7),
'Find2, Fix1, Track2': np.float64(0.7),
'Find1, Fix2, Track1': np.float64(0.8),
'Find1, Fix2, Track2': np.float64(0.9),
'Find2, Fix2, Track1': np.float64(0.9)}
[7]:
avg_num_success_events = killweb.get_avg_number_success_events()
avg_num_success_events
[7]:
{'Find1, Fix1, Track1': np.float64(1.9),
'Find1, Fix2, Track1': np.float64(2.5),
'Find1, Fix1, Track2': np.float64(2.1),
'Find1, Fix2, Track2': np.float64(2.9),
'Find2, Fix1, Track1': np.float64(2.3),
'Find2, Fix2, Track1': np.float64(2.7),
'Find2, Fix1, Track2': np.float64(2.4),
'Find2, Fix2, Track2': np.float64(2.0)}
Individual paths can be analyzed using MIMIK. In the cells below, the probability of success and the average number of successful events for the selected path are displayed.
[8]:
path = ["Find1", "Fix1", "Track1"]
[9]:
killweb.print_proportion_complete(path)
Proportion of complete kill chains: 0.6
[9]:
np.float64(0.6)
[10]:
killweb.print_average_number_of_successes(path)
Average number of success events: 1.9
[10]:
np.float64(1.9)
MIMIK can gernate figures for the results of the Monte Carlo simulation. The distribution of successful events and the probabilities for each event are displayed below for the selected path.
[11]:
killweb.plot_monte_carlo_distribution(path)