Home  >  Methodology  >  Round 1  >  Scope


For Round 1 evaluations, we tested 56 Enterprise ATT&CK techniques across 10 ATT&CK tactics. The Initial Access tactic was considered out of scope for Round 1 evaluations. The in-scope techniques are displayed below and are also highlighted in each vendor’s results page.

We divided the tested techniques into “Primary” techniques and “Enabling” techniques. Execution of many of the techniques required Command-Line Interface, Execution through API, and PowerShell. We considered these to be “Enabling” techniques for the evaluation, and we generally did not capture detections directly associated with their execution (except in cases where one of those techniques was executing the behavior under test, such as “RunAs”). Instead, we focused on the Primary technique that was performed, rather than the mechanism of execution (which was considered the Enabling technique). For example, if Process Discovery was performed via Command-Line Interface, we captured detections for Process Discovery but not Command-Line Interface.

As we explained in the previous section, we divided technique execution into sequences we refer to as “Steps” to help capture detections as we performed the evaluations. Each Step corresponded to an adversary’s intended goal during an operation. We performed 20 Steps in total across two scenarios: 10 Steps corresponded to our first scenario (which used Cobalt Strike), and 10 Steps corresponded to our second scenario (which used Empire). We further divided each Step into Sub-Steps that are denoted by letters (e.g. 1A, 1B, etc.). Those Steps, Sub-Steps, and the corresponding techniques are outlined below.

First Scenario

We used Cobalt Strike, a commercially available red team tool, to execute our emulation for the first scenario.

Step 1 - Initial Compromise

Step 2 - Initial Discovery

Step 3 - Privilege Escalation

Step 4 - Discovery for Lateral Movement

Step 5 - Credential Access

Step 6 - Lateral Movement

Step 7 - Persistence

Step 8 - Collection

Step 9 - Exfiltration

Step 10 - Execution of Persistence

Second Scenario

We used Empire, an open-source red team tool, to execute our emulation for the second scenario.

Step 11 - Initial Compromise

Step 12 - Initial Discovery

Step 13 - Discovery for Lateral Movement

Step 14 - Privilege Escalation

Step 15 - Credential Access

Step 16 - Lateral Movement

Step 17 - Persistence

Step 18 - Collection

Step 19 - Exfiltration

Step 20 - Execution of Persistence