Documentation

Lean.Data.Markdown.Parser.Inline

Pass 2: Inline Parser #

The second pass parses the inline elements of a block's text. This requires a global view of the document because forward references to defined labels determine whether a piece of syntax is a link or just plain text.

Inline Tokenization #

The emphasis algorithm in §6.2 of CommonMark consumes a flat sequence of tokens. Each token is either a fully-finalized non-emphasis Inline (inline code, link, image, autolink, soft/hard break, escaped char) or a raw */_ delimiter sequence. tokenizeInlines produces this sequence. It reads the inline source, recognizes each non-emphasis inline kind in turn (backslash escape §2.4, character reference §2.5, inline code §6.1, link §6.3, image §6.4, autolink §6.5, raw HTML §6.6, hard line break §6.7, soft line break §6.8), and saves delimiter sequences for §6.2 to handle later.

Links and images follow the bracket stack procedure described in the specification's appendix. A [ or ![ is emitted as a text token and pushed on a stack of bracket openers. A ] pops the nearest opener and tries to parse a link target or reference after itself. On success, the tokens that were emitted since the opener become the link's text, with their emphasis processed, and the opener's text token is replaced by the link or image; every earlier [ is then deactivated, since a link may not contain a link. On failure, the brackets stay text. Each ] thus does work proportional to the tokens since its opener.

The tokenizer's mutable state (InlineTokenizerState below) is touched through a few operations, which are recognize for “matched an inline,” skipChar for “no match, this character is literal text,” skipPast for the one place where we leap forward past several literal chars at once, and openBracket/closeBracket for the bracket stack.

def Lean.Markdown.parseInlines {s : String.Slice} (refs : RefTable s) (src : InlineSource s) (extensions : Extensions := { }) :

Parses the inline content of src, typically a paragraph or a heading's content, into a flat array of inlines.

Equations
  • One or more equations did not get rendered due to their size.
Instances For