The arguments inside a bracketed list, each given either by position or by name.
Arguments are separated by top level commas, or, when there are none, by whitespace, so that "rgb(1, 0.5, 0)" and "rgb(1 0.5 0)" agree. An argument is named when a word is followed by a colon; a number before a colon is left alone, which is what keeps "13:45" a time rather than an argument called thirteen.
The shape of a bracket. Interval notation permits a pair of brackets with different shapes, i.e. "[5,8)", so shapes are recorded rather than enforced. It is up to interpreters to decide which combinations they accept.
Choices which affect how a string is split into Fragments. These have to be made up front, because they change what the characters mean rather than what is done with them afterwards.
A string split into categorised fragments, plus the views derived from it.
Scanning happens once, on creation, and never fails. Problems such as an unterminated literal or an unmatched bracket are recorded on the fragment they belong to, so that half typed input still yields a complete and navigable result. Call Fragments.Validate() when well formed input is required.
Every view is pinned to the same Fragments.Source string and every fragment reports absolute indices into it, so error positions remain valid no matter how deeply a view is derived.
A numeric literal. Leading signs and trailing unit suffixes are not part of a number; they are separate fragments, because neither can be classified without knowing the grammar.
A single character which belongs to no other category. Symbols are always one character long; multi character operators are matched by testing adjacency with Fragment.IsAdjacentTo(Fragment).
A run of letters, digits and underscores, starting with a letter or an underscore. Words are scanned greedily, so "point2" is a single word and the 'e' inside "Hello" cannot be mistaken for Euler's number.