Skip to main content
{ Security Operations }

Microsoft Sentinel Detection Tuning

Finds the Microsoft Sentinel analytics rules that produce the most noise, reads each one against the alerts it raised, and proposes a specific tuning change with the evidence and what it would have missed.

What this agent does

This read-only agent ranks the analytics rules in Microsoft Sentinel by the noise they produce. It measures each rule's alert volume, the share of its alerts in incidents closed as false positive or benign positive, the time to close, and the entities that repeat. It reads the rule and the alerts behind the noise, and it proposes one tuning change per rule. The change is a query filter, a threshold, a watchlist-based exclusion, a grouping change, or a suppression period. Each proposal shows the alerts it would have removed and the true positives it would have hidden. A person applies the change.

The challenge

A handful of analytics rules raise most of the alerts, and the incidents they land in are closed the same way every day. Analysts know which rules are noisy. A safe fix needs a KQL change tested against weeks of records, and no one owns that work. An exclusion written in a hurry hides a real detection along with the noise.

The solution

The agent finds the field values behind each noisy rule's alerts. It proposes the narrowest change that removes them. It runs the changed query over the window's records to show what the change would have hidden. It presents every proposal as a candidate and changes nothing itself. Provides a ranked tuning queue where each item is ready for a detection engineer to review and apply.

Workflow

  1. 01

    Measure noise

    For each analytics rule, count alerts in SecurityAlert in the window, the classification of the incident each landed in, time to close, and repeated entities.

  2. 02

    Rank

    Rank rules by false positive and benign count times median time to close, and take the top 5.

  3. 03

    Read the alerts

    Read the alerts and the records that triggered them for each rule, and find the field values that explain the noise.

  4. 04

    Propose

    Write the narrowest change that removes the pattern, and run it against the window's records to show what it would have removed and hidden.

  5. 05

    Report

    Publish the ranked queue with each proposal and its evidence.

Agent template 

# Microsoft Sentinel Detection Tuning

## Measurable outcomes

Every run produces a ranked list of noisy analytics rules, each with one specific proposal and the evidence for it. Track the alert volume and the false positive share of each rule on every run. On the next run, compare each tuned rule's volume and false positive share with the run before its last change.

## Procedure

Each run, review the last 30 days of alerts and incidents in the Sentinel workspace I set, unless I set another window. For each analytics rule, count the alerts it raised in SecurityAlert, and read the classification of the incident each alert landed in. Count the share of its alerts in incidents closed as false positive or benign positive, and the median time to close. Count the distinct and repeated accounts, hosts, and addresses in the alert entities. Report the share of closed alerts with no disposition, meaning their incident is classified as Undetermined or not classified. Rank on volume alone when that share is above half. Otherwise, rank rules by false positive and benign count times median time to close. Take the top 5, unless I set another number. Skip rules I mark as tuned recently. For each one, read the rule's query, frequency, lookback, threshold, grouping, entity mapping, and suppression settings. Read the watchlists it already uses, and read its alerts with the records that triggered them. Find the field values that explain the noise. Typical causes are a service account, a scanner's address range, a maintenance window, a hostname pattern, or a threshold the normal baseline crosses. State the pattern as fields, values, and the share of alerts they cover. Propose the narrowest change that removes that pattern, and show the exact query text or watchlist rows. The change can be a filter added to the query, a higher threshold, or a watchlist entry for an exclusion the rule already applies. It can also be a different grouping or a suppression period on the rule. Run the proposed query against the window's records. Report the alerts it would have removed and every true positive or open incident it would have hidden. For each proposal, name what an attacker could do inside the excluded scope without an alert, such as acting as the excluded service account. When no change removes the pattern without hiding a true positive, say that no safe change exists. Never propose a change that hides a true positive without saying so. Never propose disabling a rule. Propose an automation rule that closes incidents only when the matching alerts must stay on record, and say so. Present every proposal as a candidate, and never change a rule, a watchlist, or an automation rule. When a rule's noise comes from a data quality problem, say so and point at the connector or table instead of proposing a filter.

## Requirements

It needs read-only access to the Sentinel workspace to query the tables in scope and the SecurityAlert and SecurityIncident tables, and read-only access to analytics rules, automation rules, and watchlists, and nothing more. It never changes rules, watchlists, automation rules, or incidents. Reports quote only the field values a proposal excludes, with raw events, other users, hosts, and alert IDs left out.