HTML API: Auto-escape JavaScript and JSON script tag contents when necessary

[sirreal]

The HTML API currently rejects script tag contents that may be dangerous. This is a proposal to detect JavaScript and JSON script tags and automatically escape contents when necessary.

• JSON and JavaScript script tags may be detected according to the HTML standard.
• Script tag contents are escaped only when <script or </script (case-insensitive) is found.

In JSON

< is replaced with \u003C. This eliminates the problematic strings and aligns with the approach described in #63851 and applied in r60681.

This is proposed as a simple character replacement with strtr. This should be highly performant. A less invasive replacement could be done to only replace < in <script or </script where it's really necessary. This would preserve more of the JSON string, but likely at the cost of performance. It would require either a regular expression with case-insensitive matching (see JavaScript example).

In JavaScript

<script and </script (followed by a necessary tag termination character \t\n\r\f/>) the s is replaced with its Unicode escape. This should remain valid in all contexts where the text can appear and maintain identical behavior in all except a few edge cases (see ticket or quoted section below for full explanation and caveats).

From the ticket:

The HTML API prevents setting SCRIPT tag that could modify the tree either by closing the SCRIPT element prematurely, or by preventing the SCRIPT element from closing at the expected close tag.

This is handled by rejecting any script tag contents that are potentially dangerous and is safe. There are some improvements that could be made.

If the contents are found to be unsafe and the type of the script tag is JSON or JavaScript (this is well specified in the HTML standard), it should be possible to apply a syntactic transformation to the contents in such a way that the script contents become safe without semantically altering the script.

If the HTML API can safely and automatically escape the majority of SCRIPT tag contents, it can then be used to for SCRIPT tag creation and has the potential to eliminate the class of problem from #40737, #62797, and #63851. It also has the potential to address part of #51159 where SCRIPT tag escaping becomes less of an issue.

JSON

In JSON SCRIPT tags, the transformation is a simple replacement of < with its Unicode escape sequence \u003C. This can be applied to the entire contents of the script or specifically in case-insensitive matches for <script and </script.

JavaScript

JavaScript SCRIPT tags are more difficult because the language has vastly more syntax. Fortunately, there is prior art described in this 2022 blog post (external) from React team member Sophie Alpert. It's the same the JavaScript SCRIPT tag contents escaping strategy that React continues to employ today. In summary, the problematic text <script and </script syntactically appear in places where Unicode escape sequences can be used in the script part (Strings, Identifiers, and RegExp literals). React takes the approach of replacing the s character, resulting in <\u0073cript or </\u0073cript, completely safe in a Script tag.

There are a few notable exceptions where the transformed JavaScript has observably different runtime behavior. These are the only examples I'm aware of. They're more esoteric parts of the language and the likelihood of them being used in inline JavaScript with the problematic text sequences seems an acceptable tradeoff to me to enable cheap, automatic JavaScript escaping.

String.raw does not process escape sequences.

'<script>' === '<\u0073cript>'; // true
String.raw`<script>` === String.raw`<\u0073cript>`; // false

Tagged templates can also access the raw strings, again a form without processing escape sequences.

function taggedCooked( strings ) {
    return strings[0];
}
taggedCooked`<script>` === taggedCooked`<\u0073cript>`; // true

function taggedRaw( strings ) {
    return strings.raw[0];
}
taggedRaw`<script>` === taggedRaw`<\u0073cript>`; // false

The source property of RegExp contains a string representation of the pattern. JavaScript RegExp support Unicode escape sequences, but the Unicode escape sequence is not transformed in the source.

const rPlain = /<script>/;
const rEscaped = /<\u0073cript>/

rPlain.test('<script>'); // true
rEscaped.test('<script>'); // true

rPlain.source === rEscaped.source; // false
rPlain.source; // '<script>'
rEscaped.source; // '<\\u0073cript>'

Any better JavaScript escaping would likely require a complete JavaScript parser and much more invasive changes. It would be much more costly to perform. Even then, I'm not sure that the escaping could be done faithfully.

String.raw() could be split and joined:

String.raw`<script>` === String.raw`<s` + String.raw`cript>`; true ✅

Tagged template raw and RegExp source seem much more challenging.

Trac ticket: https://core.trac.wordpress.org/ticket/64419

---

This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

[github-actions]

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

• The Plugin and Theme Directories cannot be accessed within Playground.
• All changes will be lost when closing a tab with a Playground instance.
• All changes will be lost when refreshing the page.
• A fresh instance is created each time the link below is clicked.
• Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
  it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

[dmsnell]

it seems like this has the potential to be clearer if we did one of two things:

• early-abort when both the type and language attributes are missing.
• null-coalesce to some value like '' which would be semantically the same in these checks as null but allow us to treat the values as strings.

or even something like this…

$type = $this->get_attribute( 'type' );
$type = is_string( $type ) ?: '';        // combine `true` and `null` into ''.

[sirreal]

Agreed, this can be simpler. The different cases seem clear so we can also add some unit tests.

[sirreal]

I've reviewed this and simplified or clarified it slightly, but I think it matches the specified behavior well and I'm not sure that further changes will improve things.

[westonruter]

PhpStorm complains about having a separate case here:

It suggests moving the module case up above with the others.

This is purely stylistic, I acknowledge.

[sirreal]

I prefer this as-is, it allows quotes referencing different specifications to remain separate.

[dmsnell]

we’re not checking for these state transitions. in the code following, we’re checking for the simplified transition, akin to the simplified diagram

I think it would be helpful to match the discussion of the approach with the comment. that is, I don’t know if it’s particularly helpful to spell out all of the characters when terser descriptions are warranted. e.g.

 * If the plaintext contains any sequences which would could be interpreted as
 * SCRIPT opening or closing tags, then it is sufficient to escape these. This
 * prevents getting into the dangerous double-escape state. Technically, what
 * matters is not the presence of a full or actual SCRIPT tag, but the start of
 * a tag containing the "SCRIPT" tag name.
 *
 * @see URL
 */
if ( false !== stripos( ... ) || false !== stripos( ... ) ) {

}

[sirreal]

Those lines capture all of the relevant state transitions, it's really about these minimal transitions that either close the script or move to double-escaped:

---
config:
---
stateDiagram-v2
  state "script data" as ScriptData
  state "escaped" as Escaped
  state "double escaped" as DoubleEscaped
  state "Close script" as CloseScript

  Escaped --> DoubleEscaped : #60;script[ \t\f\r\n/>]
  ScriptData --> CloseScript : #60;/script[ \t\f\r\n/>]
  Escaped --> CloseScript : #60;/script[ \t\f\r\n/>]

Loading

The idea being that if those transitions are prevented then the script contents cannot break the HTML structure.

---

I agree the documentation here needs to be reviewed broadly. It's important to document the escaping well for posterity to understand why this is implemented as it is.

[dmsnell]

are you happy with this comment? I think some things are very elaborate and technical, and worded in ways which could use some refinement.

[sirreal]

This comment was one of the last things I added and could certainly use revision.

[dmsnell]

“There are a few exceptions where the escaped form can be detected:” is passive and feels indirect. Something more punching could go further with less.

 * This escaping cannot be done everywhere in JavaScript:
 *
 *  - Comments are not interpreted, meaning the escape sequences are visible, but
 *    only when reviewing the source code itself.
 *  - `String.raw()` and tagged template literal strings work on unescaped values.
 *  - The `source` property of a RegExp object returns unescaped strings.
 *
 * To avoid escaping in these situations it’s necessary to avoid presenting the
 * text which appears like a SCRIPT tag, for example, by splitting it into two
 * pieces and combining them.
 *
 * Example:
 *
 *     console.log( String.raw`</\u0073cript>` );           // </\u0073cript>
 *     console.log( String.raw`</scr` + String.raw`ipt>` ); // </script>

[sirreal]

I've created #10668 to align STYLE tag handling with this approach.

[sirreal]

Linking to a comment.

if we think about changing the design of the is_javascript_script_tag() methods, we could expose something like a MIME type and allow custom filtering of escapes to allow plugins to provide their own escaping routines rather than reject the updates.

for now I see no reason to block progress for this, especially since we already rejected these contents before, but I think we can imagine a situation where someone can replace their SCRIPT contents and we can perform the <script and </script checks after the filter, and only then reject if nothing was able to escape

[sirreal]

Linking to conversation about this escaping choice:

this is certainly easy, but I wonder what the relative value is compared to performing the same escape as with the JavaScript elements.

my guess is that <script and </script occur far less frequently within JSON than < does, so if we perform the same escaping then we might expect considerably fewer modifications to the JSON content.

from a performance standpoint I expect the picture to be complicated and hard to quantify, as we’re comparing a relatively simple pair of string searches against string allocations and string building.

[sirreal]

I thought about this as well. Ultimately, this seemed very reasonable, strtr is supposed to have good performance characteristics and there are no cases to worry about with <. This is close to the behavior of json_encode() with JSON_HEX_TAG, although that also escapes >.

Some other considerations:

• We could JSON decode/encode with the appropriate flags. That's likely expensive and can modify some types of values, I think it's best avoided.
• We could use the same escaping as JavaScript. preg_replace_callback is likely more expensive, but I don't have data on that.
• I considered doing strtr with all the case variants, but the list of strtr case variants would require something like 128 different cases for case-insensitive <script and </script, and that seems excessive.

There is a middle ground that is worth considering. This note on strtr() is relevant (bold mine):

If given two arguments, the second should be an array in the form array('from' => 'to', ...)…

The keys and the values may have any length… However, this function will be the most efficient when all the keys have the same size.

Every character added to the replacement becomes more targeted. We could use additional characters in the strtr pattern with something like the following:

function escape_json_script_contents_2( string $text ): string {
	return strtr(
		$text,
		array(
            '<s' => '\\u003Cs',
            '<S' => '\\u003CS',
            '</' => '\\u003C/',
	    )
	);
}

function escape_json_script_contents_3( string $text ): string {
	return strtr(
		$text,
		array(
            '<sc' => '\\u003Csc',
            '<Sc' => '\\u003CSc',
            '<sC' => '\\u003CsC',
            '<SC' => '\\u003CSC',
            '</s' => '\\u003C/s',
            '</S' => '\\u003C/S',
	    )
	);
}

[dmsnell]

these seem better than how it stands now as I think the occurrence rate of <s, <S, and </ will also be much lower than of < by itself.

I wouldn’t want us to get caught up in the weeds of performance without any metrics to guide us. We can keep in mind that these operations are going to happen relatively infrequently during a page render. Maybe they happen ten times, vs. something like tag parsing which might happen thousands of times.

So the preg_replace_callback() seems just fine to me, even though personally I think both of these methods are probably non-ideal. I still prefer inching forward on stripos() followed by strspn().

The thing is, without data we just don’t know and so who cares if we are potentially slower on a minor part of the page render. If our intention is clear we can follow-up with improvements where we can make them soundly.

Ultimately here my intuition says that the difference between the basic strtr() and the stripos()+strspn() replacements is going to fall down to the relative improvement strtr() brings against the higher rate of modifying the string. I would expect < to appear in a reasonably high fraction of strings, whereas <script and variants are unlikely to regularly appear. Both approaches are scanning through the string to perform a search, and while strtr() is surely efficient, not allocating and not performing replacement is surely more efficient.

I would caution here against piling up verbose source code to eke out minor performance improvements when we don’t know if they matter and when they obscure intent. None of your suggestions obscure intent, but I think they may also not be essential.

[sirreal]

Needs to be updated.

[sirreal]

We'll need to document that this is suitable for JavaScript and JSON escaping.

[westonruter]

Also good to include here initially:

Suggested change

	case 'application/json':
	case 'application/json':
	case 'application/ld+json':
	case 'application/manifest+json':

These and other such MIME types would be handled by the parser.