Can Your Public DSA Data Support Your Systemic Risk Assessment?

The risk assessment and the moderation system should tell a compatible story.

Recent DSA enforcement has shown the importance of service-specific evidence.

The Commission has already imposed major fines for failures in systemic-risk assessment and mitigation. In recent cases, it has criticised risk assessments that did not adequately reflect the risks of the actual service and found mitigation measures insufficiently effective.

For DSA teams, that creates a practical question:

Does the risk picture described in the platform’s systemic-risk assessment match what can be observed in its moderation data?

The DSA does not formulate that as a specific test. But the connection is built into the framework.

Article 34 requires very large online platforms and search engines to assess systemic risks stemming from the design and functioning of their services and related systems, and from the way those services are used. In doing so, they must consider, among other factors, their content-moderation systems and the enforcement of their terms and conditions.

Article 35 then requires mitigation measures to be tailored to the specific systemic risks identified.

In other words, the risk assessment is not separate from the moderation system. The way the service detects, reviews and acts on harmful or illegal activity forms part of the underlying risk picture.

Public moderation data can reveal an operational footprint

That makes the platform’s public DSA data potentially useful evidence.

Suppose a platform identifies scams and fraudulent behaviour as a systemic risk and describes account-security checks and automated spam detection among the measures used to address it.

The public Article 17 population may then show how many moderation decisions were associated with deceptive behaviour, how often particular account-security controls appear in those decisions, which moderation pathways were used and whether those patterns changed over time.

That creates an observable footprint of both the risk and the controls described elsewhere. It can show that a described mitigation is generating moderation activity, at what scale and through which moderation pathways.

It cannot, by itself, show that the mitigation is effective.

Effectiveness requires evidence that Statements of Reasons do not provide: for example, detection coverage, false negatives, quality-assurance results, exposure data and evidence about the underlying prevalence of the risk.

That distinction matters.

Public moderation data is not a substitute for the platform’s internal effectiveness evidence. But it can provide an additional quantitative view of whether the risks, controls and moderation pathways described by the platform are also visible in its public operational footprint.

The comparison can also reveal the opposite.

A sizeable moderation population may relate to a specific risk that is barely visible in the risk assessment because the relevant decisions sit inside a broad DSA category or a generic policy label.

That does not mean the risk assessment is necessarily wrong. It creates a reasonable question:

Why is this signal prominent in the operational data but much less visible in the documented risk picture?

That is the kind of question a platform may want to understand before it arises during an audit or regulatory review.

The comparison becomes more useful over time

A risk assessment captures the platform’s risk picture at a particular point in time, while the moderation system continues producing new decisions afterwards.

That creates another useful comparison:

Does the operational picture remain consistent with the risk picture over time?

A particular risk signal may suddenly become much more prominent. A control may begin generating substantially more interventions. A moderation pathway may move from predominantly human to automated handling.

None of those changes automatically means that the underlying systemic risk has increased.

The explanation could be better detection, a policy change, a reporting change, a change in enforcement practice or a genuine shift in activity.

The value is not automatically interpreting the change as increased risk. The value is identifying that the change happened, locating where it happened and determining whether the existing risk assessment still explains the observable pattern.

That matters because a systemic-risk assessment is not intended to be a static exercise. Subsequent assessments are expected to reflect the development of the platform’s risk environment.

Public moderation data can provide one additional source of evidence for examining that development.

This is different from reading the transparency report

The harmonised transparency report provides useful aggregate information about a platform’s moderation activity. Alongside it, the public Statement of Reasons population provides decision-level data. Those two views are not the same.

Broad reporting categories can contain several different risks and controls. Automation can mean different things at different stages of the moderation process. Large control populations can disappear inside generic terms-and-conditions reporting. Specific risk signals can be distributed across several DSA categories.

Analysing the public Statement of Reasons population makes it possible to examine those underlying structures in considerably greater detail:

• A broad category can be separated into more specific moderation signals.
• A generic policy population can reveal distinct operational controls.
• Automation can be examined separately across detection and decision-making.
• Changes in individual risks, controls and moderation pathways can be followed over time rather than being visible only through aggregate reporting totals.

This does not transform public data into internal risk-management evidence. But it makes the public data useful for something beyond describing how many moderation decisions occurred.

For a DSA team, that means the public data can be used not only to report what happened, but also to test and support the risk picture described elsewhere.

What Civility Bureau provides

Civility Bureau gives platforms an external view of the evidence visible in their public DSA footprint.

We connect full-population Statement of Reasons analysis with the platform’s risk assessment, transparency reporting and audit findings to identify:

• where observable moderation evidence supports the existing risk assessment;
• where material signals are not clearly connected to it;
• where observable control activity can strengthen the quantitative evidence base;
• where changes in risks, controls or moderation pathways may warrant further investigation; and
• where additional internal evidence or explanation may be needed.

We do not use moderation volume as a measure of risk prevalence or mitigation effectiveness.

A high number of moderation decisions can have many explanations. It may reflect higher prevalence, better detection, broader enforcement, a change in policy, increased reporting or differences in the way a platform records its decisions.

Instead, we use the public data to help answer a simpler question:

Does the evidence visible in your public DSA footprint support the systemic-risk story you are telling?

The risk assessment describes the systemic risks the platform has identified. Its mitigation documentation describes the measures it has put in place to address them. Its transparency reports provide aggregate information about moderation activity. Its public Statements of Reasons provide another view: what the moderation system is actually producing at decision level.

Civility Bureau connects those sources.

We analyse the public Statement of Reasons population and compare the resulting risk signals, observable control activity and moderation patterns with the platform’s risk assessment, transparency reporting and audit findings.

For a platform, that can answer three practical questions:

• Are important risk signals visible in our moderation data but missing or understated in our risk assessment?
• Can our public moderation data provide quantitative support for controls and mitigations we already describe?
• Are there parts of our public DSA reporting that an auditor or regulator may reasonably ask us to explain?

That is the value of looking at Statements of Reasons as a source of systemic-risk evidence, rather than simply as a transparency obligation.