Do It Yourself Evaluations
Retesting is an integral part of using ATT&CK Evaluations content, whether the goal is to validate results, test different products or configurations, gauge performance in an operational setting, or test as part of the entire security stack. Solutions are always evolving to better defend against the latest threats. Organizational needs and environment details are unique and ever changing.
We released the ATT&CK Evaluations methodology to enable users to perform their own ATT&CK evaluations and create tailored results that would empower them to make more informed decisions on procurement and deployment. We originally released the Operational Flow and procedural details within the vendor’s results to give context to the detections we noted during our evaluations. But we also wanted users to help with the re-evaluation process. The learning curve to use this information was relatively steep, so we released the Do It Yourself version of the ATT&CK Evaluations, which utilize CALDERA to hopefully lower the barrier of entry.
CALDERA is an open-source automated, adversary emulation system that is currently an active research project at MITRE. The Do It Yourself evaluations maintain the spirit of the official ATT&CK Evaluation scenarios, while enabling organizations without a sophisticated red team to still execute the evaluation, or those with a red team to refocus their priorities. We note that the CALDERA implementations are not exact matches to the methodology used during the hands-on evaluations we perform, but they do offer a unique variant on the activity.
For more information on the ATT&CK Evaluations plugin for CALDERA, please visit our github site.