Repository navigation
Description
Description
Two problems in the v2 grammar patcher (src/util/grammar-patch.js, reached via Language.resolvedGrammar, src/core/classes/language.js:239) when the grammar contains values that are perfectly legal according to types.d.ts but are not plain objects.
1. Deep $insertBefore/$insertAfter keys crash when the path crosses a string inside
// src/util/grammar-patch.js:17-28 ($insertBefore; same shape at :42-52 for $insertAfter)
if (key?.includes('/')) {
let path = key.split('/');
const lastKey = path.pop();
path = path.flatMap(key => [key, 'inside']);
const obj = path.reduce((acc, key) => acc?.[key], grammar);
if (obj) {
insertBefore(obj, lastKey, tokens);
}
}The walk's guard is if (obj) β a plain truthiness check. But at grammarPatch time the grammar is still raw β token insides are commonly stored as language-id strings (inside?: string | Grammar | null, types.d.ts:219; used throughout src/languages/, e.g. actionscript.js:25, apex.js:44). If a deep key walks into such a token, obj is a string:
const grammar = {
'code': { pattern: /`[^`]*`/, inside: 'markup' }, // type-legal GrammarRef
$insertBefore: {
'code/comment': { 'my-token': { pattern: /Y/ } },
},
};
grammarPatch(grammar);
// TypeError: Cannot use 'in' operator to search for 'comment' in markupThe TypeError comes from insert() (src/util/insert.js:57: if (!(atKey in grammar))) being handed the string 'markup' as its "grammar". Since grammarPatch is what resolvedGrammar runs, this exception escapes to whoever touches the language β the language simply fails to load, with an error message that doesn't point anywhere near the user's $insertBefore key.
Two neighbouring shapes misbehave too:
- intermediate
insideis a bare RegExp token (RegExpLike = RegExp, types.d.ts:183): no TypeError, but a misleadingError: "comment" has to be a key of grammar.β thrown about a RegExp object the user never named. - intermediate
insideis a function (GrammarRefexplicitly allows "a function returning one"):'x' in fnis legal, and iflastKeymatches an own property of the function (name,length,prototype),insert()happily deletes and redefines the function's own properties and glues the new tokens onto the function object.
Repro output (running the real grammarPatch under node, current master):
--- Case 1: intermediate inside is a string (language id) ---
THROWS: TypeError: Cannot use 'in' operator to search for 'comment' in markup
--- Case 2: intermediate inside is a bare RegExp token ---
THROWS: Error: "comment" has to be a key of grammar.
--- Case 3: control, plain-object inside (works) ---
code.inside keys now: [ 'my-token', 'comment' ]
Expected: the walk should either treat non-object insides as "not patchable" (skip, like if (obj) does for undefined/null), or throw a descriptive error mentioning the $insertBefore/$insertAfter key. Ideally string insides would be resolved (like resolve() in src/core/tokenize/util.js does at tokenize time) so 'code/comment' can patch into the referenced language β but at minimum it must not crash with an internal TypeError on type-legal input.
2. $merge silently does nothing when the target token is a bare RegExp
// src/util/grammar-patch.js:76-81
if (grammar[key]) {
deepMerge(grammar[key], tokens);
}
else {
grammar[key] = tokens;
}deepMerge (src/util/objects.js:104-125) only mutates when both sides are plain Object/EventTarget containers; otherwise it returns source ?? target β and the caller discards the return value. So if the existing token is a bare RegExp (type-legal: GrammarTokens = Partial<Record<TokenName, RegExpLike | GrammarToken | ...>>), the whole $merge entry is silently dropped:
const grammar = {
'string': /"[^"]*"/, // type-legal RegExpLike token
$merge: { 'string': { greedy: true } },
};
grammarPatch(grammar);
grammar['string']; // still /"[^"]*"/ β greedy lost, no error, no warning--- Case 4: $merge silently no-ops when target token is a bare RegExp ---
after $merge, string token = /"[^"]*"/ (greedy lost)
--- Case 5: control, $merge into object token (works) ---
after $merge, string token = { pattern: /"[^"]*"/, greedy: true }
Expected: merge into a RegExp token by replacing it with { ...tokens, pattern: <the RegExp> } (or at least overwrite the token), not silently ignore the patch.
Environment
Prism 2.0 (current master, src/util/grammar-patch.js at commit of today). Both cases reproduced by importing grammarPatch directly β no browser needed.
Activity