Decode Before You Sanitise: The XSS Fix That Deletes User Text

Most web applications clean user HTML before they store it. A common way to clean is to decode the text first, then run a sanitiser. That order looks careful. It is not. It deletes real characters that your users typed.

This article shows the failure with measured results. It also gives a rule that keeps the protection and stops the deletion.

The short version

A sanitiser should read the exact string that you store and send back.
If you decode first, typed text like ac becomes markup. The sanitiser then removes it.
The damage is silent. No error. No log line. No security alert.
Delete the decode only if your client sends raw HTML. If your editor sends encoded HTML, you still need one decode.
Decode at most once, and only when the payload holds no markup of its own.

First, what an entity is

HTML uses angle brackets for tags. A less-than sign starts a tag. So a typed angle bracket has to travel as an entity.

Typed text:    a < b
On the wire:   a &lt; b

An entity is text. A tag is structure. Mix them and the meaning changes.

The bug

Here is the code. It comes from a real .NET service that protects clinical notes.

var decoded = System.Net.WebUtility.HtmlDecode(content);
var clean = sanitiser.Sanitize(decoded);

The first line turns entities back into characters. The second line parses the result as HTML. Text becomes structure.

These results come from the real service, with its own configuration.

Input if ac then flag became if ac then flag. Two characters gone.
Input Tom &amp; Jerry became Tom & Jerry. One decode level too deep.
Input Temp < 38.5 survived without damage.
Input <script>alert(1)</script> was still removed. The protection held.

Why did < 38.5 survive? A less-than sign followed by a space cannot start a tag. So the parser left it alone. That is luck, not design. Write Anti-B>4 or ac and the parser reads a tag.

Why the decode got there

The editor in that application sends encoded HTML. Its configuration says so.

const editor = $(element).kendoEditor({
    encoded: true
});

With that setting the browser posts Hello. The server must decode it, or every note shows literal tag text to every user.

So the decode fixed a real problem. It also created a new one.

Why you cannot just delete the decode

If the client sends encoded HTML and you stop decoding, every note is stored as visible  text. All formatting disappears for all users. That is worse than the first bug.

Both failures come from the same place. The client encodes the whole payload, so a typed bracket and a real tag look the same on the wire. The server cannot tell them apart.

The rule

Parse what you store. Store what you parsed.

A sanitiser is a parser. Give it the same string that the browser will receive. Decode after the check and the browser sees text that the check never saw. Decode before the check and you destroy the author’s text.

The fix

Decode once. Decode only when the payload cannot be markup already. A payload with no less-than sign holds no markup. A payload with a less-than sign is markup, so parse it as it stands.

private string ResolvePayloadEncoding(string content)
{
    // Already markup: parse as-is, never decode.
    if (content.Contains('<'))
    {
        return content;
    }

    // No markup of its own: this is the editor's encoded payload.
    var decoded = WebUtility.HtmlDecode(content);

    return decoded.Contains('<') ? decoded : content;
}

Then hand the result to the sanitiser.

var candidate = ResolvePayloadEncoding(content);
var clean = sanitiser.Sanitize(candidate);

The same four inputs after the change.

if ac then flag stays if ac then flag. Nothing lost.
Tom &amp; Jerry stays Tom &amp; Jerry. No extra decode.
Hello world still becomes bold text. Formatting works.
<script> is still removed. Protection works.

Test with four cases

Two tests protect the text. Two tests protect the users. Keep all four.

Raw HTML with a typed bracket must keep the bracket.
Raw HTML with a double-encoded entity must keep it.
An encoded payload must decode once and keep its formatting.
A script tag must be removed, in both raw and encoded form.

One assert is enough to catch the regression.

[Fact]
public void RawHtmlKeepsTypedBrackets()
{
    var output = _sanitiser.SanitizeHtml(
        "<p>if a&lt;b&gt;c then flag</p>",
        XssContentType.ClinicalNotes);

    Assert.Equal(
        "<p>if a&lt;b&gt;c then flag</p>",
        output);
}

That test fails on the old code. It passes on the new code. Run it before you change anything else.

Report the near miss

One more step is worth the effort. If the stored form is safe, but a decode would make it dangerous, raise an alert. Do not change the text. Just tell somebody.

if (!ContainsDangerousContent(content) &&
    ContainsDangerousContent(HtmlDecode(content)))
{
    findings.Add(new("Encoded payload",
        "Encoded markup that becomes executable if decoded downstream",
        RequiresSecurityAlert: true));
}

Encoded markup is inert while it stays encoded. The risk starts when another system decodes it later: an export, a mail template, a report. An alert closes that blind spot. Entity-encoded attempts such as onload= never matched the plain patterns before.

What this fix does not solve

The client still encodes the whole payload. So a typed angle bracket and a real tag still look the same on the wire. A complete fix needs the client change as well: send raw HTML, and keep the sanitiser on the server.

Do that change in one step and test it in a browser. Paste a table. Paste text with a bracket. Save the note. Open the note again. Check each field. If any editor field breaks, you moved too fast.

And know the limit. Old records already lost their characters. This fix does not repair them. It stops new damage only.

Takeaways

Give the sanitiser the exact string that you store.
Decode at most once, and only when the payload has no markup.
Add a test for typed angle brackets. That case breaks first.
Add a test for encoded input. That case keeps formatting alive.
Alerts must cover the encoded form, or the quiet attacks stay quiet.
When an alert does not fire, check the reason code. Routine cleanup hides damage.

This bug belongs with the other quiet failures in the practical security checklist for websites and online services. The checklist asks what protects each layer. This article adds one question to that list: does the check see the same text that the user will see?