ifhub/inform7
1
0
Fork 0
mirror of https://github.com/ganelson/inform.git synced 2024-07-05 08:34:22 +03:00
Code Issues Projects Releases Wiki Activity
inform7/docs/building-module/2-tkn.html

1513 lines
397 KiB
HTML
Raw Blame History

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Tokenisation</title>
<link href="../docs-assets/Breadcrumbs.css" rel="stylesheet" rev="stylesheet" type="text/css">
<meta name="viewport" content="width=device-width initial-scale=1">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta http-equiv="Content-Language" content="en-gb">
<link href="../docs-assets/Contents.css" rel="stylesheet" rev="stylesheet" type="text/css">
<link href="../docs-assets/Progress.css" rel="stylesheet" rev="stylesheet" type="text/css">
<link href="../docs-assets/Navigation.css" rel="stylesheet" rev="stylesheet" type="text/css">
<link href="../docs-assets/Fonts.css" rel="stylesheet" rev="stylesheet" type="text/css">
<link href="../docs-assets/Base.css" rel="stylesheet" rev="stylesheet" type="text/css">
<script>
function togglePopup(material_id) {
var popup = document.getElementById(material_id);
popup.classList.toggle("show");
}
</script>
<link href="../docs-assets/Popups.css" rel="stylesheet" rev="stylesheet" type="text/css">
<link href="../docs-assets/Colours.css" rel="stylesheet" rev="stylesheet" type="text/css">
</head>
<body class="commentary-font">
<nav role="navigation">
<h1><a href="../index.html">
<img src="../docs-assets/Inform.png" height=72">
</a></h1>
<ul><li><a href="../index.html">home</a></li>
</ul><h2>Compiler</h2><ul>
<li><a href="../structure.html">structure</a></li>
<li><a href="../inbuildn.html">inbuild</a></li>
<li><a href="../inform7n.html">inform7</a></li>
<li><a href="../intern.html">inter</a></li>
<li><a href="../services.html">services</a></li>
<li><a href="../secrets.html">secrets</a></li>
</ul><h2>Other Tools</h2><ul>
<li><a href="../inblorbn.html">inblorb</a></li>
<li><a href="../indocn.html">indoc</a></li>
<li><a href="../inform6.html">inform6</a></li>
<li><a href="../inpolicyn.html">inpolicy</a></li>
</ul><h2>Resources</h2><ul>
<li><a href="../extensions.html">extensions</a></li>
<li><a href="../kits.html">kits</a></li>
</ul><h2>Repository</h2><ul>
<li><a href="https://github.com/ganelson/inform"><img src="../docs-assets/github.png" height=18> github</a></li>
</ul><h2>Related Projects</h2><ul>
<li><a href="../../../inweb/index.html">inweb</a></li>
<li><a href="../../../intest/index.html">intest</a></li>
</ul>
</nav>
<main role="main">
<!--Weave of 'Tokenisation' generated by Inweb-->
<div class="breadcrumbs">
<ul class="crumbs"><li><a href="../index.html">Home</a></li><li><a href="../intern.html">Inter Modules</a></li><li><a href="index.html">building</a></li><li><a href="index.html#2">Chapter 2: Blueprints</a></li><li><b>Tokenisation</b></li></ul></div>
<p class="purpose">Turning textual code written in Inform 6 syntax into a linked list of tokens.</p>
<p class="commentary firstcommentary"><a id="SP1" class="paragraph-anchor"></a><b>&#167;1. </b>The following code was sketched out on a long night flight to Hong Kong, but
there is otherwise nothing exotic about it. In as simple a way as possible, we
take a text <span class="extract"><span class="extract-syntax">from</span></span> and break it into Inform 6 tokens. What we return is not
literally a linked list, but it amounts to the same thing: a single node
holding an unstructured run of tokens &mdash;
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> EXPRESSION_ISNT</span>
<span class="plain-syntax"> T1</span>
<span class="plain-syntax"> T2</span>
<span class="plain-syntax"> T3</span>
<span class="plain-syntax"> ...</span>
</pre>
<p class="commentary">We follow the syntax of Inform 6, except that we have to look for three extra
syntaxes: <span class="extract"><span class="extract-syntax">{-braced-commands}</span></span>, <span class="extract"><span class="extract-syntax">(+ Inform 7 interpolation +)</span></span>, and, if the
abbreviated syntax is allowed, also some cryptic notations such as <span class="extract"><span class="extract-syntax">*1</span></span>.
</p>
<p class="commentary">The following scanner is basically a finite state machine, and these are the
states:
</p>
<pre class="definitions code-font"><span class="definition-keyword">enum</span> <span class="constant-syntax">NO_TOKSTATE</span><span class="plain-syntax"> </span><span class="identifier-syntax">from</span><span class="plain-syntax"> </span><span class="constant-syntax">1</span>
<span class="definition-keyword">enum</span> <span class="constant-syntax">COMMENT_TOKSTATE</span><span class="plain-syntax"> </span><span class="comment-syntax"> currently scanning... an I6 comment </span><span class="extract"><span class="extract-syntax">! ...</span></span>
<span class="definition-keyword">enum</span> <span class="constant-syntax">DQUOTED_TOKSTATE</span><span class="plain-syntax"> </span><span class="comment-syntax"> ...double-quoted text</span>
<span class="definition-keyword">enum</span> <span class="constant-syntax">SQUOTED_TOKSTATE</span><span class="plain-syntax"> </span><span class="comment-syntax"> ...single-quoted text</span>
<span class="definition-keyword">enum</span> <span class="constant-syntax">WHITE_TOKSTATE</span><span class="plain-syntax"> </span><span class="comment-syntax"> ...whitespace</span>
<span class="definition-keyword">enum</span> <span class="constant-syntax">TOK_TOKSTATE</span><span class="plain-syntax"> </span><span class="comment-syntax"> ...an actual token</span>
</pre>
<pre class="displayed-code all-displayed-code code-font">
<span class="reserved-syntax">void</span><span class="plain-syntax"> </span><span class="function-syntax">Tokenisation::go</span><button class="popup" onclick="togglePopup('usagePopup1')"><span class="comment-syntax">?</span><span class="popuptext" id="usagePopup1">Usage of <span class="code-font"><span class="function-syntax">Tokenisation::go</span></span>:<br/>Parsing Inter Schemas - <a href="2-pis.html#SP5">&#167;5</a></span></button><span class="plain-syntax">(</span><span class="reserved-syntax">inter_schema</span><span class="plain-syntax"> *</span><span class="identifier-syntax">sch</span><span class="plain-syntax">, </span><span class="identifier-syntax">text_stream</span><span class="plain-syntax"> *</span><span class="identifier-syntax">from</span><span class="plain-syntax">, </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">pos</span><span class="plain-syntax">, </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">abbreviated</span><span class="plain-syntax">,</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">no_quoted_inames</span><span class="plain-syntax">, </span><span class="reserved-syntax">void</span><span class="plain-syntax"> **</span><span class="identifier-syntax">quoted_inames</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">inter_schema_token</span><span class="plain-syntax"> *</span><span class="identifier-syntax">preceding_token</span><span class="plain-syntax"> = </span><span class="identifier-syntax">NULL</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">definition_length</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">line_offset</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">text_stream</span><span class="plain-syntax"> *</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::new</span><span class="plain-syntax">();</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> = </span><span class="constant-syntax">NO_TOKSTATE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (; </span><span class="identifier-syntax">pos</span><span class="plain-syntax">&lt;</span><span class="identifier-syntax">definition_length</span><span class="plain-syntax">; </span><span class="identifier-syntax">pos</span><span class="plain-syntax">++) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, </span><span class="identifier-syntax">pos</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Characters::is_whitespace</span><span class="plain-syntax">(</span><span class="identifier-syntax">c</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'\n'</span><span class="plain-syntax">) </span><span class="identifier-syntax">line_offset</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> == </span><span class="constant-syntax">TOK_TOKSTATE</span><span class="plain-syntax">) || (</span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> == </span><span class="constant-syntax">NO_TOKSTATE</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_2" class="named-paragraph-link"><span class="named-paragraph">Absorb raw material, if any</span><span class="named-paragraph-number">1.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> = </span><span class="constant-syntax">WHITE_TOKSTATE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="character-syntax">' '</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> == </span><span class="constant-syntax">WHITE_TOKSTATE</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_2" class="named-paragraph-link"><span class="named-paragraph">Absorb raw material, if any</span><span class="named-paragraph-number">1.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> = </span><span class="constant-syntax">NO_TOKSTATE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="identifier-syntax">DQUOTED_TOKSTATE:</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'"'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3" class="named-paragraph-link"><span class="named-paragraph">Absorb raw material, for sure</span><span class="named-paragraph-number">1.3</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> = </span><span class="constant-syntax">NO_TOKSTATE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">c</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="identifier-syntax">SQUOTED_TOKSTATE:</span><span class="plain-syntax"> {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">ends_here</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'\''</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">ends_here</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">) == </span><span class="constant-syntax">0</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, </span><span class="identifier-syntax">pos</span><span class="plain-syntax">+1) == </span><span class="character-syntax">'\''</span><span class="plain-syntax">))</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">ends_here</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">ends_here</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3" class="named-paragraph-link"><span class="named-paragraph">Absorb raw material, for sure</span><span class="named-paragraph-number">1.3</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> = </span><span class="constant-syntax">NO_TOKSTATE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">c</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="identifier-syntax">COMMENT_TOKSTATE:</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'\n'</span><span class="plain-syntax">) </span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> = </span><span class="constant-syntax">NO_TOKSTATE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="identifier-syntax">WHITE_TOKSTATE:</span><span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'!'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_2" class="named-paragraph-link"><span class="named-paragraph">Absorb raw material, if any</span><span class="named-paragraph-number">1.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> = </span><span class="constant-syntax">COMMENT_TOKSTATE</span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'"'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_2" class="named-paragraph-link"><span class="named-paragraph">Absorb raw material, if any</span><span class="named-paragraph-number">1.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> = </span><span class="constant-syntax">DQUOTED_TOKSTATE</span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'\''</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_2" class="named-paragraph-link"><span class="named-paragraph">Absorb raw material, if any</span><span class="named-paragraph-number">1.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> = </span><span class="constant-syntax">SQUOTED_TOKSTATE</span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'{'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">abbreviated</span><span class="plain-syntax"> == </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">))</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_7" class="named-paragraph-link"><span class="named-paragraph">Look for a possible bracing</span><span class="named-paragraph-number">1.7</span></a></span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'*'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">abbreviated</span><span class="plain-syntax"> == </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">))</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_8" class="named-paragraph-link"><span class="named-paragraph">Look for a possible abbreviated command</span><span class="named-paragraph-number">1.8</span></a></span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'('</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, </span><span class="identifier-syntax">pos</span><span class="plain-syntax">+1) == </span><span class="character-syntax">'+'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">abbreviated</span><span class="plain-syntax"> == </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">))</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_6" class="named-paragraph-link"><span class="named-paragraph">Look for a possible Inform 7 fragment</span><span class="named-paragraph-number">1.6</span></a></span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_1" class="named-paragraph-link"><span class="named-paragraph">Absorb a raw character</span><span class="named-paragraph-number">1.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_2" class="named-paragraph-link"><span class="named-paragraph">Absorb raw material, if any</span><span class="named-paragraph-number">1.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax">}</span>
</pre>
<p class="commentary firstcommentary"><a id="SP1_1" class="paragraph-anchor"></a><b>&#167;1.1. </b><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Absorb a raw character</span><span class="named-paragraph-number">1.1</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> = </span><span class="constant-syntax">TOK_TOKSTATE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">c</span><span class="plain-syntax">);</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1">&#167;1</a>, <a href="2-tkn.html#SP1_6">&#167;1.6</a>, <a href="2-tkn.html#SP1_7">&#167;1.7</a>, <a href="2-tkn.html#SP1_8">&#167;1.8</a> (twice).</li></ul>
<p class="commentary firstcommentary"><a id="SP1_2" class="paragraph-anchor"></a><b>&#167;1.2. </b><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Absorb raw material, if any</span><span class="named-paragraph-number">1.2</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">)) </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3" class="named-paragraph-link"><span class="named-paragraph">Absorb raw material, for sure</span><span class="named-paragraph-number">1.3</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> = </span><span class="constant-syntax">NO_TOKSTATE</span><span class="plain-syntax">;</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1">&#167;1</a> (6 times), <a href="2-tkn.html#SP1_6">&#167;1.6</a>, <a href="2-tkn.html#SP1_7">&#167;1.7</a>, <a href="2-tkn.html#SP1_8">&#167;1.8</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_3" class="paragraph-anchor"></a><b>&#167;1.3. </b><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Absorb raw material, for sure</span><span class="named-paragraph-number">1.3</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="identifier-syntax">WHITE_TOKSTATE:</span>
<span class="plain-syntax"> </span><a href="2-is.html#SP14" class="function-link"><span class="function-syntax">InterSchemas::add_token</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">sch</span><span class="plain-syntax">,</span>
<span class="plain-syntax"> </span><a href="2-is.html#SP10" class="function-link"><span class="function-syntax">InterSchemas::new_token</span></a><span class="plain-syntax">(</span><span class="constant-syntax">WHITE_SPACE_ISTT</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">" "</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, -1, </span><span class="identifier-syntax">line_offset</span><span class="plain-syntax">));</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="identifier-syntax">DQUOTED_TOKSTATE:</span>
<span class="plain-syntax"> </span><a href="2-tkn.html#SP2" class="function-link"><span class="function-syntax">Tokenisation::de_escape_text</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><a href="2-is.html#SP14" class="function-link"><span class="function-syntax">InterSchemas::add_token</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">sch</span><span class="plain-syntax">,</span>
<span class="plain-syntax"> </span><a href="2-is.html#SP10" class="function-link"><span class="function-syntax">InterSchemas::new_token</span></a><span class="plain-syntax">(</span><span class="constant-syntax">DQUOTED_ISTT</span><span class="plain-syntax">, </span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, -1, </span><span class="identifier-syntax">line_offset</span><span class="plain-syntax">));</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="identifier-syntax">SQUOTED_TOKSTATE:</span>
<span class="plain-syntax"> </span><a href="2-tkn.html#SP4" class="function-link"><span class="function-syntax">Tokenisation::de_escape_sq_text</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><a href="2-is.html#SP14" class="function-link"><span class="function-syntax">InterSchemas::add_token</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">sch</span><span class="plain-syntax">,</span>
<span class="plain-syntax"> </span><a href="2-is.html#SP10" class="function-link"><span class="function-syntax">InterSchemas::new_token</span></a><span class="plain-syntax">(</span><span class="constant-syntax">SQUOTED_ISTT</span><span class="plain-syntax">, </span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, -1, </span><span class="identifier-syntax">line_offset</span><span class="plain-syntax">));</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1" class="named-paragraph-link"><span class="named-paragraph">Look for individual tokens</span><span class="named-paragraph-number">1.3.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">Str::clear</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">tokeniser_state</span><span class="plain-syntax"> = </span><span class="constant-syntax">NO_TOKSTATE</span><span class="plain-syntax">;</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1">&#167;1</a> (twice), <a href="2-tkn.html#SP1_2">&#167;1.2</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_4" class="paragraph-anchor"></a><b>&#167;1.4. </b><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Process any escape character notation in single quotes</span><span class="named-paragraph-number">1.4</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (</span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax">=0; </span><span class="identifier-syntax">i</span><span class="plain-syntax">&lt;</span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">); </span><span class="identifier-syntax">i</span><span class="plain-syntax">++) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">unescaped</span><span class="plain-syntax">, </span><span class="identifier-syntax">c</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is never used.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_5" class="paragraph-anchor"></a><b>&#167;1.5. </b><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Process any escape character notation in double quotes</span><span class="named-paragraph-number">1.5</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (</span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax">=0; </span><span class="identifier-syntax">i</span><span class="plain-syntax">&lt;</span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">); </span><span class="identifier-syntax">i</span><span class="plain-syntax">++) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">unescaped</span><span class="plain-syntax">, </span><span class="identifier-syntax">c</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is never used.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_6" class="paragraph-anchor"></a><b>&#167;1.6. </b>Material in <span class="extract"><span class="extract-syntax">(+ ... +)</span></span> notation is an interpolation of I7 source text.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Look for a possible Inform 7 fragment</span><span class="named-paragraph-number">1.6</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">save_pos</span><span class="plain-syntax"> = </span><span class="identifier-syntax">pos</span><span class="plain-syntax">, </span><span class="identifier-syntax">accept</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">TEMPORARY_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">source_text_fragment</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">pos</span><span class="plain-syntax"> += </span><span class="constant-syntax">2</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">while</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, </span><span class="identifier-syntax">pos</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, </span><span class="identifier-syntax">pos</span><span class="plain-syntax">-1) == </span><span class="character-syntax">'+'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, </span><span class="identifier-syntax">pos</span><span class="plain-syntax">) == </span><span class="character-syntax">')'</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">Str::delete_last_character</span><span class="plain-syntax">(</span><span class="identifier-syntax">source_text_fragment</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">accept</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">source_text_fragment</span><span class="plain-syntax">, </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, </span><span class="identifier-syntax">pos</span><span class="plain-syntax">++));</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">accept</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_2" class="named-paragraph-link"><span class="named-paragraph">Absorb raw material, if any</span><span class="named-paragraph-number">1.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_6_1" class="named-paragraph-link"><span class="named-paragraph">Expand a fragment of Inform 7 text</span><span class="named-paragraph-number">1.6.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> { </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="character-syntax">'('</span><span class="plain-syntax">; </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_1" class="named-paragraph-link"><span class="named-paragraph">Absorb a raw character</span><span class="named-paragraph-number">1.1</span></a></span><span class="plain-syntax">; </span><span class="identifier-syntax">pos</span><span class="plain-syntax"> = </span><span class="identifier-syntax">save_pos</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">DISCARD_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">source_text_fragment</span><span class="plain-syntax">)</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1">&#167;1</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_6_1" class="paragraph-anchor"></a><b>&#167;1.6.1. </b>Note that the empty I7 interpolation is legal, but produces no token.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Expand a fragment of Inform 7 text</span><span class="named-paragraph-number">1.6.1</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">source_text_fragment</span><span class="plain-syntax">) &gt; </span><span class="constant-syntax">0</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><a href="2-is.html#SP14" class="function-link"><span class="function-syntax">InterSchemas::add_token</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">sch</span><span class="plain-syntax">,</span>
<span class="plain-syntax"> </span><a href="2-is.html#SP10" class="function-link"><span class="function-syntax">InterSchemas::new_token</span></a><span class="plain-syntax">(</span><span class="constant-syntax">I7_ISTT</span><span class="plain-syntax">, </span><span class="identifier-syntax">source_text_fragment</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, -1, </span><span class="identifier-syntax">line_offset</span><span class="plain-syntax">));</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1_6">&#167;1.6</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_7" class="paragraph-anchor"></a><b>&#167;1.7. </b>Material in braces sometimes indicates an inline command, but not always,
because braces often occur innocently in I6 code. So we require the first
character after the open-brace not to be white-space, and also not to be
a pipe (though I've forgotten why). The text inside the braces is called
a "bracing".
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Look for a possible bracing</span><span class="named-paragraph-number">1.7</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">save_pos</span><span class="plain-syntax"> = </span><span class="identifier-syntax">pos</span><span class="plain-syntax">++, </span><span class="identifier-syntax">accept</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">TEMPORARY_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">bracing</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">while</span><span class="plain-syntax"> (</span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, </span><span class="identifier-syntax">pos</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="constant-syntax">0</span><span class="plain-syntax">) </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'}'</span><span class="plain-syntax">) { </span><span class="identifier-syntax">accept</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">bracing</span><span class="plain-syntax">, </span><span class="identifier-syntax">c</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">pos</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">first</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_first_char</span><span class="plain-syntax">(</span><span class="identifier-syntax">bracing</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">accept</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">first</span><span class="plain-syntax"> != </span><span class="character-syntax">' '</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">first</span><span class="plain-syntax"> != </span><span class="character-syntax">'\t'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">first</span><span class="plain-syntax"> != </span><span class="character-syntax">'\n'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">first</span><span class="plain-syntax"> != </span><span class="character-syntax">'|'</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_2" class="named-paragraph-link"><span class="named-paragraph">Absorb raw material, if any</span><span class="named-paragraph-number">1.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_7_1" class="named-paragraph-link"><span class="named-paragraph">Parse a bracing into an inline command</span><span class="named-paragraph-number">1.7.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> { </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="character-syntax">'{'</span><span class="plain-syntax">; </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_1" class="named-paragraph-link"><span class="named-paragraph">Absorb a raw character</span><span class="named-paragraph-number">1.1</span></a></span><span class="plain-syntax">; </span><span class="identifier-syntax">pos</span><span class="plain-syntax"> = </span><span class="identifier-syntax">save_pos</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">DISCARD_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">bracing</span><span class="plain-syntax">)</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1">&#167;1</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_7_1" class="paragraph-anchor"></a><b>&#167;1.7.1. </b>That's everything, then, except the one thing that counts: how to expand
a bracing.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Parse a bracing into an inline command</span><span class="named-paragraph-number">1.7.1</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">inter_schema_token</span><span class="plain-syntax"> *</span><span class="identifier-syntax">t</span><span class="plain-syntax"> = </span><a href="2-is.html#SP10" class="function-link"><span class="function-syntax">InterSchemas::new_token</span></a><span class="plain-syntax">(</span><span class="constant-syntax">INLINE_ISTT</span><span class="plain-syntax">, </span><span class="identifier-syntax">bracing</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, -1, </span><span class="identifier-syntax">line_offset</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">bracing</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::duplicate</span><span class="plain-syntax">(</span><span class="identifier-syntax">bracing</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::new</span><span class="plain-syntax">();</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::new</span><span class="plain-syntax">();</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand2</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::new</span><span class="plain-syntax">();</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_7_1_1" class="named-paragraph-link"><span class="named-paragraph">Decompose the bracing</span><span class="named-paragraph-number">1.7.1.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">) &gt; </span><span class="constant-syntax">0</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">unknown_ISINC</span><span class="plain-syntax">, </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">no_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"primitive-definition"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">primitive_definition_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"repeat-through"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">repeat_through_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"repeat-through-list"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">repeat_through_list_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"number-of"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">number_of_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"random-of"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">random_of_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"total-of"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">total_of_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"extremal"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">extremal_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"function-application"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">function_application_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"description-application"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">description_application_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"solve-equation"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">solve_equation_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"switch"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">switch_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"break"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">break_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"verbose-checking"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">verbose_checking_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"new"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">new_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"new-list-of"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">new_list_of_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"printing-routine"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">printing_routine_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"ranger-routine"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">ranger_routine_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"indexing-routine"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">indexing_routine_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"next-routine"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">next_routine_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"previous-routine"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">previous_routine_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"strong-kind"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">strong_kind_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"weak-kind"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">weak_kind_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"backspace"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">backspace_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"erase"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">erase_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"open-brace"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">open_brace_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"close-brace"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">close_brace_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"label"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">label_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"counter"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">counter_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"counter-storage"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">counter_storage_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"counter-up"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">counter_up_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"counter-down"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">counter_down_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"counter-makes-array"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">counter_makes_array_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"by-reference"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">by_reference_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"by-reference-blank-out"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">by_reference_blank_out_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"reference-exists"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">reference_exists_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"lvalue-by-reference"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">lvalue_by_reference_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"by-value"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">by_value_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"box-quotation-text"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">box_quotation_text_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"try-action"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">try_action_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"try-action-silently"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">try_action_silently_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"return-value"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">return_value_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"return-value-from-rule"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">return_value_from_rule_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"property-holds-block-value"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">property_holds_block_value_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"mark-event-used"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">mark_event_used_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"my"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">my_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"unprotect"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">unprotect_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"copy"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">copy_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"initialise"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">initialise_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"matches-description"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">matches_description_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"now-matches-description"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">now_matches_description_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"arithmetic-operation"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">arithmetic_operation_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"say"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">say_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"show-me"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">show_me_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"segment-count"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">segment_count_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"final-segment-marker"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">final_segment_marker_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"list-together"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">list_together_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"unarticled"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">unarticled_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"articled"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">sc</span><span class="plain-syntax"> = </span><span class="constant-syntax">articled_ISINSC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"rescale"</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="constant-syntax">rescale_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="identifier-syntax">inline_command</span><span class="plain-syntax"> = </span><span class="identifier-syntax">c</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="identifier-syntax">inline_subcommand</span><span class="plain-syntax"> = </span><span class="identifier-syntax">sc</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><a href="2-is.html#SP14" class="function-link"><span class="function-syntax">InterSchemas::add_token</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">sch</span><span class="plain-syntax">, </span><span class="identifier-syntax">t</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">preceding_token</span><span class="plain-syntax"> = </span><span class="identifier-syntax">t</span><span class="plain-syntax">;</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1_7">&#167;1.7</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_7_1_1" class="paragraph-anchor"></a><b>&#167;1.7.1.1. </b>A bracing can take any of the following forms:
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> {-command}</span>
<span class="plain-syntax"> {-command:operand}</span>
<span class="plain-syntax"> {-command:operand:operand2}</span>
<span class="plain-syntax"> {-command:operand&lt;property name}</span>
<span class="plain-syntax"> {-command:operand&gt;property name}</span>
<span class="plain-syntax"> {some text}</span>
<span class="plain-syntax"> {-annotation:some text}</span>
</pre>
<p class="commentary">We parse this with the command or annotation in <span class="extract"><span class="extract-syntax">command</span></span>, the "some text"
or operand in <span class="extract"><span class="extract-syntax">bracing</span></span>, the property name (if given) in <span class="extract"><span class="extract-syntax">extremal_property</span></span>,
the direction of the <span class="extract"><span class="extract-syntax">&lt;</span></span> or <span class="extract"><span class="extract-syntax">&gt;</span></span> in <span class="extract"><span class="extract-syntax">extremal_property_sign</span></span>, and the second,
optional, operand in <span class="extract"><span class="extract-syntax">operand2</span></span>.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Decompose the bracing</span><span class="named-paragraph-number">1.7.1.1</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="identifier-syntax">TEMPORARY_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">pname</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::get_first_char</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">bracing</span><span class="plain-syntax">) == </span><span class="character-syntax">'-'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">portion</span><span class="plain-syntax"> = </span><span class="constant-syntax">1</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (</span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax">=1, </span><span class="identifier-syntax">L</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">bracing</span><span class="plain-syntax">); </span><span class="identifier-syntax">i</span><span class="plain-syntax">&lt;</span><span class="identifier-syntax">L</span><span class="plain-syntax">; </span><span class="identifier-syntax">i</span><span class="plain-syntax">++) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">bracing</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax">(</span><span class="identifier-syntax">portion</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">1</span><span class="plain-syntax">:</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">':'</span><span class="plain-syntax">) </span><span class="identifier-syntax">portion</span><span class="plain-syntax"> = </span><span class="constant-syntax">2</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">command</span><span class="plain-syntax">, </span><span class="identifier-syntax">c</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">2</span><span class="plain-syntax">:</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">':'</span><span class="plain-syntax">) </span><span class="identifier-syntax">portion</span><span class="plain-syntax"> = </span><span class="constant-syntax">3</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> #</span><span class="identifier-syntax">ifdef</span><span class="plain-syntax"> </span><span class="identifier-syntax">CORE_MODULE</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'&lt;'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">extremal_property_sign</span><span class="plain-syntax"> = </span><span class="identifier-syntax">MEASURE_T_OR_LESS</span><span class="plain-syntax">; </span><span class="identifier-syntax">portion</span><span class="plain-syntax"> = </span><span class="constant-syntax">4</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'&gt;'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">extremal_property_sign</span><span class="plain-syntax"> = </span><span class="identifier-syntax">MEASURE_T_OR_MORE</span><span class="plain-syntax">; </span><span class="identifier-syntax">portion</span><span class="plain-syntax"> = </span><span class="constant-syntax">4</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> #</span><span class="identifier-syntax">endif</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand</span><span class="plain-syntax">, </span><span class="identifier-syntax">c</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">3</span><span class="plain-syntax">:</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">operand2</span><span class="plain-syntax">, </span><span class="identifier-syntax">c</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">4</span><span class="plain-syntax">:</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">pname</span><span class="plain-syntax">, </span><span class="identifier-syntax">c</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> #</span><span class="identifier-syntax">ifdef</span><span class="plain-syntax"> </span><span class="identifier-syntax">CORE_MODULE</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">extremal_property_sign</span><span class="plain-syntax"> != </span><span class="identifier-syntax">MEASURE_T_EXACTLY</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wording</span><span class="plain-syntax"> </span><span class="identifier-syntax">W</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Feeds::feed_text</span><span class="plain-syntax">(</span><span class="identifier-syntax">pname</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="function-syntax">&lt;property-name&gt;</span><span class="plain-syntax">(</span><span class="identifier-syntax">W</span><span class="plain-syntax">)) </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">extremal_property</span><span class="plain-syntax"> = </span><span class="function-syntax">&lt;&lt;rp&gt;&gt;</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> #</span><span class="identifier-syntax">endif</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">Str::copy</span><span class="plain-syntax">(</span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">bracing</span><span class="plain-syntax">, </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="identifier-syntax">operand</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">DISCARD_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">pname</span><span class="plain-syntax">)</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1_7_1">&#167;1.7.1</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_8" class="paragraph-anchor"></a><b>&#167;1.8. </b>In abbreviated prototypes, <span class="extract"><span class="extract-syntax">*1</span></span> and <span class="extract"><span class="extract-syntax">*2</span></span> are placeholders, but a number
of modifiers are allowed. See <a href="../calculus-module/3-cs.html" class="internal">Compilation Schemas (in calculus)</a>.
</p>
<pre class="definitions code-font"><span class="definition-keyword">define</span> <span class="constant-syntax">GIVE_KIND_ID_ISSBM</span><span class="plain-syntax"> </span><span class="constant-syntax">1</span>
<span class="definition-keyword">define</span> <span class="constant-syntax">GIVE_COMPARISON_ROUTINE_ISSBM</span><span class="plain-syntax"> </span><span class="constant-syntax">2</span>
<span class="definition-keyword">define</span> <span class="constant-syntax">DEREFERENCE_PROPERTY_ISSBM</span><span class="plain-syntax"> </span><span class="constant-syntax">4</span>
<span class="definition-keyword">define</span> <span class="constant-syntax">ADOPT_LOCAL_STACK_FRAME_ISSBM</span><span class="plain-syntax"> </span><span class="constant-syntax">8</span>
<span class="definition-keyword">define</span> <span class="constant-syntax">CAST_TO_KIND_OF_OTHER_TERM_ISSBM</span><span class="plain-syntax"> </span><span class="constant-syntax">16</span>
<span class="definition-keyword">define</span> <span class="constant-syntax">BY_REFERENCE_ISSBM</span><span class="plain-syntax"> </span><span class="constant-syntax">32</span>
<span class="definition-keyword">define</span> <span class="constant-syntax">LVALUE_CONTEXT_ISSBM</span><span class="plain-syntax"> </span><span class="constant-syntax">64</span>
<span class="definition-keyword">define</span> <span class="constant-syntax">STORAGE_AS_FUNCTION_ISSBM</span><span class="plain-syntax"> </span><span class="constant-syntax">128</span>
</pre>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Look for a possible abbreviated command</span><span class="named-paragraph-number">1.8</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">at</span><span class="plain-syntax"> = </span><span class="identifier-syntax">pos</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, ++</span><span class="identifier-syntax">at</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'!'</span><span class="plain-syntax">: </span><a href="2-i6se.html#SP4" class="function-link"><span class="function-syntax">I6Errors::issue_at_node</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">sch</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">node_tree</span><span class="plain-syntax">,</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">I</span><span class="string-syntax">"the '*!' schema notation has been abolished"</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'%'</span><span class="plain-syntax">: </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> = </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> | </span><span class="constant-syntax">LVALUE_CONTEXT_ISSBM</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, ++</span><span class="identifier-syntax">at</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'$'</span><span class="plain-syntax">: </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> = </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> | </span><span class="constant-syntax">STORAGE_AS_FUNCTION_ISSBM</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, ++</span><span class="identifier-syntax">at</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'#'</span><span class="plain-syntax">: </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> = </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> | </span><span class="constant-syntax">GIVE_KIND_ID_ISSBM</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, ++</span><span class="identifier-syntax">at</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'_'</span><span class="plain-syntax">: </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> = </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> | </span><span class="constant-syntax">GIVE_COMPARISON_ROUTINE_ISSBM</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, ++</span><span class="identifier-syntax">at</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'+'</span><span class="plain-syntax">: </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> = </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> | </span><span class="constant-syntax">DEREFERENCE_PROPERTY_ISSBM</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, ++</span><span class="identifier-syntax">at</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'|'</span><span class="plain-syntax">: </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> = </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> | (</span><span class="constant-syntax">DEREFERENCE_PROPERTY_ISSBM</span><span class="plain-syntax"> + </span><span class="constant-syntax">LVALUE_CONTEXT_ISSBM</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, ++</span><span class="identifier-syntax">at</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'?'</span><span class="plain-syntax">: </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> = </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> | </span><span class="constant-syntax">ADOPT_LOCAL_STACK_FRAME_ISSBM</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, ++</span><span class="identifier-syntax">at</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'&lt;'</span><span class="plain-syntax">: </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> = </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> | </span><span class="constant-syntax">CAST_TO_KIND_OF_OTHER_TERM_ISSBM</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, ++</span><span class="identifier-syntax">at</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'^'</span><span class="plain-syntax">: </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> = </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> | (</span><span class="constant-syntax">ADOPT_LOCAL_STACK_FRAME_ISSBM</span><span class="plain-syntax"> + </span><span class="constant-syntax">BY_REFERENCE_ISSBM</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, ++</span><span class="identifier-syntax">at</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'&gt;'</span><span class="plain-syntax">: </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> = </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax"> | </span><span class="constant-syntax">BY_REFERENCE_ISSBM</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, ++</span><span class="identifier-syntax">at</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Characters::isdigit</span><span class="plain-syntax">(</span><span class="identifier-syntax">c</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_2" class="named-paragraph-link"><span class="named-paragraph">Absorb raw material, if any</span><span class="named-paragraph-number">1.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">TEMPORARY_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (</span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax">=</span><span class="identifier-syntax">pos</span><span class="plain-syntax">; </span><span class="identifier-syntax">i</span><span class="plain-syntax">&lt;=</span><span class="identifier-syntax">at</span><span class="plain-syntax">; </span><span class="identifier-syntax">i</span><span class="plain-syntax">++) </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">from</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">));</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">inter_schema_token</span><span class="plain-syntax"> *</span><span class="identifier-syntax">t</span><span class="plain-syntax"> = </span><a href="2-is.html#SP10" class="function-link"><span class="function-syntax">InterSchemas::new_token</span></a><span class="plain-syntax">(</span><span class="constant-syntax">INLINE_ISTT</span><span class="plain-syntax">, </span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, -1, </span><span class="identifier-syntax">line_offset</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="identifier-syntax">bracing</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::duplicate</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="identifier-syntax">inline_command</span><span class="plain-syntax"> = </span><span class="constant-syntax">substitute_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="identifier-syntax">inline_modifiers</span><span class="plain-syntax"> = </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="identifier-syntax">constant_number</span><span class="plain-syntax"> = (</span><span class="reserved-syntax">int</span><span class="plain-syntax">) </span><span class="identifier-syntax">c</span><span class="plain-syntax"> - (</span><span class="reserved-syntax">int</span><span class="plain-syntax">) </span><span class="character-syntax">'1'</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><a href="2-is.html#SP14" class="function-link"><span class="function-syntax">InterSchemas::add_token</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">sch</span><span class="plain-syntax">, </span><span class="identifier-syntax">t</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">preceding_token</span><span class="plain-syntax"> = </span><span class="identifier-syntax">t</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">DISCARD_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">pos</span><span class="plain-syntax"> = </span><span class="identifier-syntax">at</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'&amp;'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">inter_schema_token</span><span class="plain-syntax"> *</span><span class="identifier-syntax">t</span><span class="plain-syntax"> = </span><a href="2-is.html#SP10" class="function-link"><span class="function-syntax">InterSchemas::new_token</span></a><span class="plain-syntax">(</span><span class="constant-syntax">INLINE_ISTT</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"*&amp;"</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">, -1, </span><span class="identifier-syntax">line_offset</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="identifier-syntax">bracing</span><span class="plain-syntax"> = </span><span class="identifier-syntax">I</span><span class="string-syntax">"*&amp;"</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="identifier-syntax">inline_command</span><span class="plain-syntax"> = </span><span class="constant-syntax">combine_ISINC</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">t</span><span class="plain-syntax">-&gt;</span><span class="identifier-syntax">inline_modifiers</span><span class="plain-syntax"> = </span><span class="identifier-syntax">iss_bitmap</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><a href="2-is.html#SP14" class="function-link"><span class="function-syntax">InterSchemas::add_token</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">sch</span><span class="plain-syntax">, </span><span class="identifier-syntax">t</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">preceding_token</span><span class="plain-syntax"> = </span><span class="identifier-syntax">t</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">pos</span><span class="plain-syntax"> = </span><span class="identifier-syntax">at</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'-'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><a href="2-i6se.html#SP4" class="function-link"><span class="function-syntax">I6Errors::issue_at_node</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">sch</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">node_tree</span><span class="plain-syntax">,</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">I</span><span class="string-syntax">"the '*-' schema notation has been abolished"</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'*'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="character-syntax">'*'</span><span class="plain-syntax">; </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_1" class="named-paragraph-link"><span class="named-paragraph">Absorb a raw character</span><span class="named-paragraph-number">1.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">pos</span><span class="plain-syntax"> = </span><span class="identifier-syntax">at</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="character-syntax">'{'</span><span class="plain-syntax">; </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_1" class="named-paragraph-link"><span class="named-paragraph">Absorb a raw character</span><span class="named-paragraph-number">1.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1">&#167;1</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_3_1" class="paragraph-anchor"></a><b>&#167;1.3.1. </b>That leaves us with just the main case to handle: raw I6 code which is
outside of quotation marks and commentary, and which doesn't include
bracings or I7 interpolations. That might look like, for instance,
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="identifier-syntax">Frog</span><span class="plain-syntax"> + </span><span class="constant-syntax">2</span><span class="plain-syntax">*</span><span class="identifier-syntax">Toad</span><span class="plain-syntax">(</span>
</pre>
<p class="commentary">(there is no reason to suppose that this stretch of code is complete or
matches parentheses); we must tokenise it into
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> Frog</span>
<span class="plain-syntax"> WHITE SPACE</span>
<span class="plain-syntax"> +</span>
<span class="plain-syntax"> WHITE SPACE</span>
<span class="plain-syntax"> 2</span>
<span class="plain-syntax"> *</span>
<span class="plain-syntax"> Toad</span>
<span class="plain-syntax"> (</span>
</pre>
<p class="commentary">We scan through the text until we reach the start of a new token, and then break
off what we scanned through since the last time.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Look for individual tokens</span><span class="named-paragraph-number">1.3.1</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">L</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">c_start</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">, </span><span class="identifier-syntax">escaped</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (</span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">p</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">; </span><span class="identifier-syntax">p</span><span class="plain-syntax"> &lt; </span><span class="identifier-syntax">L</span><span class="plain-syntax">; </span><span class="identifier-syntax">p</span><span class="plain-syntax">++) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c1</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">p</span><span class="plain-syntax">), </span><span class="identifier-syntax">c2</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">, </span><span class="identifier-syntax">c3</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">p</span><span class="plain-syntax"> &lt; </span><span class="identifier-syntax">L</span><span class="plain-syntax">-1) </span><span class="identifier-syntax">c2</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">p</span><span class="plain-syntax"> &lt; </span><span class="identifier-syntax">L</span><span class="plain-syntax">-2) </span><span class="identifier-syntax">c3</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">p</span><span class="plain-syntax">+2);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">escaped</span><span class="plain-syntax"> == </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'$'</span><span class="plain-syntax">) &amp;&amp;</span>
<span class="plain-syntax"> ((</span><span class="identifier-syntax">p</span><span class="plain-syntax"> == </span><span class="constant-syntax">0</span><span class="plain-syntax">) ||</span>
<span class="plain-syntax"> (</span><span class="identifier-syntax">Characters::isalpha</span><span class="plain-syntax">(</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">p</span><span class="plain-syntax">-1)) == </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">)))</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_1" class="named-paragraph-link"><span class="named-paragraph">Break off here for real, binary or hexadecimal notation</span><span class="named-paragraph-number">1.3.1.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'-'</span><span class="plain-syntax">) </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_2" class="named-paragraph-link"><span class="named-paragraph">Break off here for negative number</span><span class="named-paragraph-number">1.3.1.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_3" class="named-paragraph-link"><span class="named-paragraph">Break off here for operators</span><span class="named-paragraph-number">1.3.1.3</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="constant-syntax">0x00A7</span><span class="plain-syntax">) </span><span class="identifier-syntax">escaped</span><span class="plain-syntax"> = </span><span class="identifier-syntax">escaped</span><span class="plain-syntax">?</span><span class="identifier-syntax">FALSE:TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c_start</span><span class="plain-syntax"> &lt; </span><span class="identifier-syntax">L</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">c_start</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">L</span><span class="plain-syntax">-1;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1_3">&#167;1.3</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_3_1_1" class="paragraph-anchor"></a><b>&#167;1.3.1.1. </b>Recall that in I6 notation, a dollar introduces a non-decimal number, and
the character after the initial dollar determines which:
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> $+3.14159E2</span>
<span class="plain-syntax"> $$1001001</span>
<span class="plain-syntax"> $1FE6</span>
</pre>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Break off here for real, binary or hexadecimal notation</span><span class="named-paragraph-number">1.3.1.1</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">c_start</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">-1;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c2</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'+'</span><span class="plain-syntax">: </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'-'</span><span class="plain-syntax">:</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">; </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">while</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax">+1) == </span><span class="character-syntax">'.'</span><span class="plain-syntax">) ||</span>
<span class="plain-syntax"> (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax">+1) == </span><span class="character-syntax">'E'</span><span class="plain-syntax">) ||</span>
<span class="plain-syntax"> (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax">+1) == </span><span class="character-syntax">'e'</span><span class="plain-syntax">) ||</span>
<span class="plain-syntax"> (</span><span class="identifier-syntax">Characters::isdigit</span><span class="plain-syntax">(</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax">+1))))</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">y</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">p</span><span class="plain-syntax"> = </span><span class="identifier-syntax">y</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c_start</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">continue</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'$'</span><span class="plain-syntax">:</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">; </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">while</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax">+1) == </span><span class="character-syntax">'0'</span><span class="plain-syntax">) ||</span>
<span class="plain-syntax"> (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax">+1) == </span><span class="character-syntax">'1'</span><span class="plain-syntax">))</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">y</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">p</span><span class="plain-syntax"> = </span><span class="identifier-syntax">y</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c_start</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">continue</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">; </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">while</span><span class="plain-syntax"> (</span><a href="2-tkn.html#SP5" class="function-link"><span class="function-syntax">Tokenisation::identchar</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax">+1)))</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">y</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">p</span><span class="plain-syntax"> = </span><span class="identifier-syntax">y</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c_start</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">continue</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1_3_1">&#167;1.3.1</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_3_1_2" class="paragraph-anchor"></a><b>&#167;1.3.1.2. </b>A token beginning with a minus sign and continuing with digits may still
not be a negative number: it may be the binary subtraction operator.
For example, we need to tokenise <span class="extract"><span class="extract-syntax">x-1</span></span> as
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> x</span>
<span class="plain-syntax"> -</span>
<span class="plain-syntax"> 1</span>
</pre>
<p class="commentary">and not as
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> x</span>
<span class="plain-syntax"> -1</span>
</pre>
<p class="commentary">This requires context, that is, remembering what the previous token was.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Break off here for negative number</span><span class="named-paragraph-number">1.3.1.2</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (((</span><span class="identifier-syntax">preceding_token</span><span class="plain-syntax"> == </span><span class="identifier-syntax">NULL</span><span class="plain-syntax">) ||</span>
<span class="plain-syntax"> (</span><span class="identifier-syntax">preceding_token</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">ist_type</span><span class="plain-syntax"> == </span><span class="constant-syntax">OPEN_ROUND_ISTT</span><span class="plain-syntax">) ||</span>
<span class="plain-syntax"> (</span><span class="identifier-syntax">preceding_token</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">ist_type</span><span class="plain-syntax"> == </span><span class="constant-syntax">OPERATOR_ISTT</span><span class="plain-syntax">) ||</span>
<span class="plain-syntax"> (</span><span class="identifier-syntax">preceding_token</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">ist_type</span><span class="plain-syntax"> == </span><span class="constant-syntax">DIVIDER_ISTT</span><span class="plain-syntax">)) &amp;&amp;</span>
<span class="plain-syntax"> (</span><span class="identifier-syntax">c_start</span><span class="plain-syntax"> == </span><span class="identifier-syntax">p</span><span class="plain-syntax">) &amp;&amp;</span>
<span class="plain-syntax"> (!((</span><span class="identifier-syntax">abbreviated</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">preceding_token</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">ist_type</span><span class="plain-syntax"> == </span><span class="constant-syntax">INLINE_ISTT</span><span class="plain-syntax">)))) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">dc</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">while</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Characters::isdigit</span><span class="plain-syntax">(</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">dc</span><span class="plain-syntax">))) </span><span class="identifier-syntax">dc</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">dc</span><span class="plain-syntax"> &gt; </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">c_start</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">-1;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">; </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">dc</span><span class="plain-syntax"> - </span><span class="constant-syntax">1</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">p</span><span class="plain-syntax"> = </span><span class="identifier-syntax">y</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c_start</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">continue</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1_3_1">&#167;1.3.1</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_3_1_3" class="paragraph-anchor"></a><b>&#167;1.3.1.3. </b>In I6, operators made of non-alphanumeric characters can be up to three
characters long, and we take the longest match: thus <span class="extract"><span class="extract-syntax">--&gt;</span></span> is a trigraph,
not the monograph <span class="extract"><span class="extract-syntax">-</span></span> followed by the digraph <span class="extract"><span class="extract-syntax">-&gt;</span></span>.
</p>
<p class="commentary">We treat the <span class="extract"><span class="extract-syntax">@</span></span> sign as if it were alphanumeric for the sake of assembly
language opcodes such as <span class="extract"><span class="extract-syntax">@pull</span></span>.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Break off here for operators</span><span class="named-paragraph-number">1.3.1.3</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">monograph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">, </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">, </span><span class="identifier-syntax">trigraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><a href="2-tkn.html#SP5" class="function-link"><span class="function-syntax">Tokenisation::identchar</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">c1</span><span class="plain-syntax">)) || (</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'_'</span><span class="plain-syntax">) || (</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'$'</span><span class="plain-syntax">)) </span><span class="identifier-syntax">monograph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="constant-syntax">0x00A7</span><span class="plain-syntax">) </span><span class="identifier-syntax">monograph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'#'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">Characters::isalpha</span><span class="plain-syntax">(</span><span class="identifier-syntax">c2</span><span class="plain-syntax">))) </span><span class="identifier-syntax">monograph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'_'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">Characters::isalpha</span><span class="plain-syntax">(</span><span class="identifier-syntax">c2</span><span class="plain-syntax">))) </span><span class="identifier-syntax">monograph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'#'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'#'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">Characters::isalpha</span><span class="plain-syntax">(</span><span class="identifier-syntax">c3</span><span class="plain-syntax">))) </span><span class="identifier-syntax">monograph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'@'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">Characters::isalpha</span><span class="plain-syntax">(</span><span class="identifier-syntax">c2</span><span class="plain-syntax">))) </span><span class="identifier-syntax">monograph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'+'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'+'</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'-'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'-'</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'&gt;'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'='</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'&lt;'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'='</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'='</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'='</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'-'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'&gt;'</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'.'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'&amp;'</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'.'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'#'</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'~'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'~'</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'~'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'='</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'&amp;'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'&amp;'</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'|'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'|'</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'&gt;'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'&gt;'</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">':'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">':'</span><span class="plain-syntax">)) </span><span class="identifier-syntax">digraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c1</span><span class="plain-syntax"> == </span><span class="character-syntax">'-'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c2</span><span class="plain-syntax"> == </span><span class="character-syntax">'-'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c3</span><span class="plain-syntax"> == </span><span class="character-syntax">'&gt;'</span><span class="plain-syntax">)) </span><span class="identifier-syntax">trigraph</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">trigraph</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">c_start</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">-1;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">; </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">+2;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">p</span><span class="plain-syntax"> += </span><span class="constant-syntax">2</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c_start</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">continue</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">digraph</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">c_start</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">-1;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">; </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">p</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c_start</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">continue</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">monograph</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">c_start</span><span class="plain-syntax">, </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">-1;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">; </span><span class="identifier-syntax">y</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4" class="named-paragraph-link"><span class="named-paragraph">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">c_start</span><span class="plain-syntax"> = </span><span class="identifier-syntax">p</span><span class="plain-syntax">+1;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">continue</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1_3_1">&#167;1.3.1</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP1_3_1_4" class="paragraph-anchor"></a><b>&#167;1.3.1.4. </b>In this code, the new token is between character positions <span class="extract"><span class="extract-syntax">x</span></span> and <span class="extract"><span class="extract-syntax">y</span></span>
inclusive; we ignore an empty token.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Break off a token</span><span class="named-paragraph-number">1.3.1.4</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">y</span><span class="plain-syntax"> &gt;= </span><span class="identifier-syntax">x</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">TEMPORARY_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (</span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax"> = </span><span class="identifier-syntax">x</span><span class="plain-syntax">; </span><span class="identifier-syntax">i</span><span class="plain-syntax"> &lt;= </span><span class="identifier-syntax">y</span><span class="plain-syntax">; </span><span class="identifier-syntax">i</span><span class="plain-syntax">++) </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">current_raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">));</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RAW_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">inter_ti</span><span class="plain-syntax"> </span><span class="identifier-syntax">which</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">, </span><span class="identifier-syntax">which_number</span><span class="plain-syntax"> = -1, </span><span class="identifier-syntax">which_quote</span><span class="plain-syntax"> = -1;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP1_3_1_4_1" class="named-paragraph-link"><span class="named-paragraph">Identify this new token</span><span class="named-paragraph-number">1.3.1.4.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">inter_schema_token</span><span class="plain-syntax"> *</span><span class="identifier-syntax">n</span><span class="plain-syntax"> = </span><a href="2-is.html#SP10" class="function-link"><span class="function-syntax">InterSchemas::new_token</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">is</span><span class="plain-syntax">, </span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">which</span><span class="plain-syntax">, </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax">, </span><span class="identifier-syntax">which_number</span><span class="plain-syntax">, </span><span class="identifier-syntax">line_offset</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> #</span><span class="identifier-syntax">ifdef</span><span class="plain-syntax"> </span><span class="identifier-syntax">CORE_MODULE</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">which_quote</span><span class="plain-syntax"> &gt;= </span><span class="constant-syntax">0</span><span class="plain-syntax">) </span><span class="identifier-syntax">n</span><span class="plain-syntax">-&gt;</span><span class="identifier-syntax">as_quoted</span><span class="plain-syntax"> = </span><span class="identifier-syntax">quoted_inames</span><span class="plain-syntax">[</span><span class="identifier-syntax">which_quote</span><span class="plain-syntax">];</span>
<span class="plain-syntax"> #</span><span class="identifier-syntax">endif</span>
<span class="plain-syntax"> </span><a href="2-is.html#SP14" class="function-link"><span class="function-syntax">InterSchemas::add_token</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">sch</span><span class="plain-syntax">, </span><span class="identifier-syntax">n</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">n</span><span class="plain-syntax">-&gt;</span><span class="element-syntax">ist_type</span><span class="plain-syntax"> != </span><span class="constant-syntax">WHITE_SPACE_ISTT</span><span class="plain-syntax">) </span><span class="identifier-syntax">preceding_token</span><span class="plain-syntax"> = </span><span class="identifier-syntax">n</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">DISCARD_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1_3_1">&#167;1.3.1</a>, <a href="2-tkn.html#SP1_3_1_1">&#167;1.3.1.1</a> (four times), <a href="2-tkn.html#SP1_3_1_2">&#167;1.3.1.2</a> (twice), <a href="2-tkn.html#SP1_3_1_3">&#167;1.3.1.3</a> (6 times).</li></ul>
<p class="commentary firstcommentary"><a id="SP1_3_1_4_1" class="paragraph-anchor"></a><b>&#167;1.3.1.4.1. </b>Finally, we identify what sort of token we're looking at. It would be elegant
to reimplement this with a trie (e.g. using <a href="../../../inweb/foundation-module/4-taa.html" class="internal">Tries and Avinues (in foundation)</a>),
but speed is not quite important enough to make it worthwhile.
</p>
<pre class="definitions code-font"><span class="definition-keyword">define</span> <span class="constant-syntax">LOWEST_XBIP_VALUE</span><span class="plain-syntax"> </span><span class="constant-syntax">HAS_XBIP</span>
<span class="definition-keyword">enum</span> <span class="constant-syntax">HAS_XBIP</span><span class="plain-syntax"> </span><span class="identifier-syntax">from</span><span class="plain-syntax"> </span><span class="constant-syntax">10000</span>
<span class="definition-keyword">enum</span> <span class="constant-syntax">HASNT_XBIP</span>
<span class="definition-keyword">enum</span> <span class="constant-syntax">READ_XBIP</span>
<span class="definition-keyword">enum</span> <span class="constant-syntax">OWNERKIND_XBIP</span>
<span class="definition-keyword">define</span> <span class="constant-syntax">HIGHEST_XBIP_VALUE</span><span class="plain-syntax"> </span><span class="constant-syntax">OWNERKIND_XBIP</span>
</pre>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Identify this new token</span><span class="named-paragraph-number">1.3.1.4.1</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">) == </span><span class="character-syntax">'@'</span><span class="plain-syntax">) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">OPCODE_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">) == </span><span class="constant-syntax">0x00A7</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">IDENTIFIER_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">) == </span><span class="character-syntax">'#'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">1</span><span class="plain-syntax">) == </span><span class="character-syntax">'#'</span><span class="plain-syntax">) &amp;&amp;</span>
<span class="plain-syntax"> (</span><span class="identifier-syntax">Characters::isalpha</span><span class="plain-syntax">(</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">2</span><span class="plain-syntax">)))) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">IDENTIFIER_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">LOOP_THROUGH_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">P</span><span class="plain-syntax">, </span><span class="identifier-syntax">T</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get</span><span class="plain-syntax">(</span><span class="identifier-syntax">P</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> != </span><span class="character-syntax">'_'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> != </span><span class="character-syntax">'#'</span><span class="plain-syntax">) &amp;&amp; (!</span><a href="2-tkn.html#SP5" class="function-link"><span class="function-syntax">Tokenisation::identchar</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">c</span><span class="plain-syntax">)))</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RAW_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">) == </span><span class="character-syntax">'#'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">Characters::isalpha</span><span class="plain-syntax">(</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">1</span><span class="plain-syntax">)))) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">IDENTIFIER_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">LOOP_THROUGH_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">P</span><span class="plain-syntax">, </span><span class="identifier-syntax">T</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get</span><span class="plain-syntax">(</span><span class="identifier-syntax">P</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> != </span><span class="character-syntax">'_'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> != </span><span class="character-syntax">'#'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> != </span><span class="character-syntax">'$'</span><span class="plain-syntax">) &amp;&amp; (!</span><a href="2-tkn.html#SP5" class="function-link"><span class="function-syntax">Tokenisation::identchar</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">c</span><span class="plain-syntax">)))</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RAW_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">) == </span><span class="character-syntax">'_'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">Characters::isalpha</span><span class="plain-syntax">(</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">1</span><span class="plain-syntax">)))) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">IDENTIFIER_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">LOOP_THROUGH_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">P</span><span class="plain-syntax">, </span><span class="identifier-syntax">T</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get</span><span class="plain-syntax">(</span><span class="identifier-syntax">P</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> != </span><span class="character-syntax">'_'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> != </span><span class="character-syntax">'#'</span><span class="plain-syntax">) &amp;&amp; (!</span><a href="2-tkn.html#SP5" class="function-link"><span class="function-syntax">Tokenisation::identchar</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">c</span><span class="plain-syntax">)))</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RAW_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Characters::isalpha</span><span class="plain-syntax">(</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">))) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">IDENTIFIER_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">LOOP_THROUGH_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">P</span><span class="plain-syntax">, </span><span class="identifier-syntax">T</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get</span><span class="plain-syntax">(</span><span class="identifier-syntax">P</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> != </span><span class="character-syntax">'_'</span><span class="plain-syntax">) &amp;&amp; (!</span><a href="2-tkn.html#SP5" class="function-link"><span class="function-syntax">Tokenisation::identchar</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">c</span><span class="plain-syntax">)))</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RAW_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::begins_with_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"QUOTED_INAME_0_"</span><span class="plain-syntax">)) </span><span class="identifier-syntax">which_quote</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::begins_with_wide_string</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">L</span><span class="string-syntax">"QUOTED_INAME_1_"</span><span class="plain-syntax">)) </span><span class="identifier-syntax">which_quote</span><span class="plain-syntax"> = </span><span class="constant-syntax">1</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"I7_string"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">Str::clear</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">); </span><span class="identifier-syntax">WRITE_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="string-syntax">"I7_String"</span><span class="plain-syntax">); }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"COMMA_WORD"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">Str::clear</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">); </span><span class="identifier-syntax">WRITE_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="string-syntax">"comma_word"</span><span class="plain-syntax">); }</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Characters::isdigit</span><span class="plain-syntax">(</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">))) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">NUMBER_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">LOOP_THROUGH_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">P</span><span class="plain-syntax">, </span><span class="identifier-syntax">T</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get</span><span class="plain-syntax">(</span><span class="identifier-syntax">P</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (!</span><span class="identifier-syntax">Characters::isdigit</span><span class="plain-syntax">(</span><span class="identifier-syntax">c</span><span class="plain-syntax">))</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RAW_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">) == </span><span class="character-syntax">'$'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">HEX_NUMBER_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">1</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'$'</span><span class="plain-syntax">) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">BIN_NUMBER_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'+'</span><span class="plain-syntax">) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">REAL_NUMBER_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'-'</span><span class="plain-syntax">) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">REAL_NUMBER_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">) == </span><span class="character-syntax">'-'</span><span class="plain-syntax">) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">NUMBER_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"false"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">NUMBER_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_number</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"true"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">NUMBER_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_number</span><span class="plain-syntax"> = </span><span class="constant-syntax">1</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"nothing"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">NUMBER_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_number</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"if"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">IF_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"else"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">ELSE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"style"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">STYLE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"return"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">RETURN_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"rtrue"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">RTRUE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"rfalse"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">RFALSE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"for"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">FOR_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"objectloop"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">OBJECTLOOP_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"while"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">WHILE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"do"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">DO_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"until"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">UNTIL_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"print"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">PRINT_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"print_ret"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">PRINTRET_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"new_line"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">NEWLINE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"give"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">GIVE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"move"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">MOVE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"remove"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">REMOVE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"jump"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">JUMP_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"switch"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">SWITCH_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"default"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">DEFAULT_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"font"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">FONT_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"continue"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">CONTINUE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"break"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">BREAK_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"quit"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">QUIT_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"restore"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESTORE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"spaces"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">SPACES_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"read"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">READ_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"inversion"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">RESERVED_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">INVERSION_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_insensitive</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"#IFDEF"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">DIRECTIVE_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">IFDEF_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_insensitive</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"#IFNDEF"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">DIRECTIVE_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">IFNDEF_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_insensitive</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"#IFTRUE"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">DIRECTIVE_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">IFTRUE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_insensitive</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"#IFFALSE"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">DIRECTIVE_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">IFFALSE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_insensitive</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"#IFNOT"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">DIRECTIVE_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">IFNOT_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_insensitive</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"#ENDIF"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">DIRECTIVE_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">ENDIF_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq_insensitive</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"#ORIGSOURCE"</span><span class="plain-syntax">)) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">DIRECTIVE_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which_rw</span><span class="plain-syntax"> = </span><span class="constant-syntax">ORIGSOURCE_I6RW</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">","</span><span class="plain-syntax">)) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">COMMA_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">":"</span><span class="plain-syntax">)) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">COLON_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"("</span><span class="plain-syntax">)) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">OPEN_ROUND_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">")"</span><span class="plain-syntax">)) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">CLOSE_ROUND_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"{"</span><span class="plain-syntax">)) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">OPEN_BRACE_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"}"</span><span class="plain-syntax">)) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">CLOSE_BRACE_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">";"</span><span class="plain-syntax">)) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">DIVIDER_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::eq</span><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"::"</span><span class="plain-syntax">)) </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">DCOLON_ISTT</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">inter_ti</span><span class="plain-syntax"> </span><span class="identifier-syntax">x</span><span class="plain-syntax"> = </span><a href="2-i6o.html#SP6" class="function-link"><span class="function-syntax">I6Operators::notation_to_BIP</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">T</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">x</span><span class="plain-syntax"> &gt; </span><span class="constant-syntax">0</span><span class="plain-syntax">) { </span><span class="identifier-syntax">is</span><span class="plain-syntax"> = </span><span class="constant-syntax">OPERATOR_ISTT</span><span class="plain-syntax">; </span><span class="identifier-syntax">which</span><span class="plain-syntax"> = </span><span class="identifier-syntax">x</span><span class="plain-syntax">; }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP1_3_1_4">&#167;1.3.1.4</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP2" class="paragraph-anchor"></a><b>&#167;2. </b>Inform 6 has a baroque set of not very self-consistent escape characters in
its double-quoted text syntax: here we take a deep breath, and plunge in. The
following converts <span class="extract"><span class="extract-syntax">text</span></span> from I6 notation to a (composed) Unicode-encoded
string, in which every character has its literal meaning.
</p>
<p class="commentary">Note that the test case <span class="extract"><span class="extract-syntax">schemas</span></span> of the <a href="../building-test/index.html" class="internal">building-test</a> module exercises
the following function.
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="reserved-syntax">void</span><span class="plain-syntax"> </span><span class="function-syntax">Tokenisation::de_escape_text</span><button class="popup" onclick="togglePopup('usagePopup2')"><span class="comment-syntax">?</span><span class="popuptext" id="usagePopup2">Usage of <span class="code-font"><span class="function-syntax">Tokenisation::de_escape_text</span></span>:<br/><a href="2-tkn.html#SP1_3">&#167;1.3</a></span></button><span class="plain-syntax">(</span><span class="identifier-syntax">text_stream</span><span class="plain-syntax"> *</span><span class="identifier-syntax">text</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">TEMPORARY_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">WRITE_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="string-syntax">"%S"</span><span class="plain-syntax">, </span><span class="identifier-syntax">text</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">Str::clear</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_1" class="named-paragraph-link"><span class="named-paragraph">Normalise the white space</span><span class="named-paragraph-number">2.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2" class="named-paragraph-link"><span class="named-paragraph">De-escape raw into text</span><span class="named-paragraph-number">2.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">DISCARD_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">)</span>
<span class="plain-syntax">}</span>
</pre>
<p class="commentary firstcommentary"><a id="SP2_1" class="paragraph-anchor"></a><b>&#167;2.1. </b>Where a newline occurs inside double-quoted text, all whitespace either side
of it is deleted, and the newline replaced by a single space.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Normalise the white space</span><span class="named-paragraph-number">2.1</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">run_start</span><span class="plain-syntax"> = -1, </span><span class="identifier-syntax">run_len</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">, </span><span class="identifier-syntax">run_includes</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (</span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax">=0; </span><span class="identifier-syntax">i</span><span class="plain-syntax">&lt;</span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">); </span><span class="identifier-syntax">i</span><span class="plain-syntax">++) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">' '</span><span class="plain-syntax">) || (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'\t'</span><span class="plain-syntax">) || (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'\n'</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">run_start</span><span class="plain-syntax"> == -1) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">run_start</span><span class="plain-syntax"> = </span><span class="identifier-syntax">i</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">run_len</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">run_includes</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">run_len</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'\n'</span><span class="plain-syntax">) </span><span class="identifier-syntax">run_includes</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">run_start</span><span class="plain-syntax"> &gt;= </span><span class="constant-syntax">0</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">run_includes</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">Str::put_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">run_start</span><span class="plain-syntax">, </span><span class="character-syntax">' '</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (</span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">j</span><span class="plain-syntax">=0; </span><span class="identifier-syntax">j</span><span class="plain-syntax">&lt;</span><span class="identifier-syntax">run_len</span><span class="plain-syntax">-1; </span><span class="identifier-syntax">j</span><span class="plain-syntax">++)</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">Str::delete_nth_character</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">run_start</span><span class="plain-syntax">+1);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax"> = </span><span class="identifier-syntax">run_start</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">run_start</span><span class="plain-syntax"> = -1;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP2">&#167;2</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP2_2" class="paragraph-anchor"></a><b>&#167;2.2. </b>I6 does not follow the C-like language convention of using backslash for
string escapes. Instead <span class="extract"><span class="extract-syntax">^</span></span> marks a forced newline and <span class="extract"><span class="extract-syntax">~</span></span> marks a double-quotation
mark. All other string escapes begin with <span class="extract"><span class="extract-syntax">@</span></span>.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">De-escape raw into text</span><span class="named-paragraph-number">2.2</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (</span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax">=0; </span><span class="identifier-syntax">i</span><span class="plain-syntax">&lt;</span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">); </span><span class="identifier-syntax">i</span><span class="plain-syntax">++) {</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_1" class="named-paragraph-link"><span class="named-paragraph">De-escape the Inform 7 unicode escape</span><span class="named-paragraph-number">2.2.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'^'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="character-syntax">'\n'</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'~'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="character-syntax">'\"'</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'@'</span><span class="plain-syntax">: {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">TEMPORARY_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">skip</span><span class="plain-syntax"> = </span><span class="constant-syntax">1</span><span class="plain-syntax">, </span><span class="identifier-syntax">decimal</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">, </span><span class="identifier-syntax">hexadecimal</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_2" class="named-paragraph-link"><span class="named-paragraph">Extract the escape token</span><span class="named-paragraph-number">2.2.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax"> += </span><span class="identifier-syntax">skip</span><span class="plain-syntax">-1;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">hexadecimal</span><span class="plain-syntax">) </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_3" class="named-paragraph-link"><span class="named-paragraph">Expand hexadecimal Unicode value</span><span class="named-paragraph-number">2.2.3</span></a></span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">decimal</span><span class="plain-syntax">) </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4" class="named-paragraph-link"><span class="named-paragraph">Expand decimal ZSCII value</span><span class="named-paragraph-number">2.2.4</span></a></span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_5" class="named-paragraph-link"><span class="named-paragraph">Expand TeX-style digraph</span><span class="named-paragraph-number">2.2.5</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">DISCARD_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="identifier-syntax">c</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP2">&#167;2</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP2_2_1" class="paragraph-anchor"></a><b>&#167;2.2.1. </b>This is not an I6 notation at all. If a character outside the range allowed
by I6 in string literals is present in an I7 source text file &mdash; for example,
a capital Cyrillic ef &mdash; then it is converted internally by the compiler to
something like <span class="extract"><span class="extract-syntax">[unicode 1060]</span></span>, with 1060 here being the decimal code point
for the character.
</p>
<p class="commentary">We will recognise this notation and translate it back into Unicode. The reason
for doing this, even though the stand-alone I6 compiler would not, is that
it means I6 source fed into this tokeniser will be treated the same whether
it comes from an Include directive in I7 source text, or whether it comes from
a kit source file.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">De-escape the Inform 7 unicode escape</span><span class="named-paragraph-number">2.2.1</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::includes_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">, </span><span class="identifier-syntax">I</span><span class="string-syntax">"[unicode "</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">unicode_point</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (</span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">j</span><span class="plain-syntax">=</span><span class="identifier-syntax">i</span><span class="plain-syntax">+9; </span><span class="identifier-syntax">j</span><span class="plain-syntax">&lt;</span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">); </span><span class="identifier-syntax">j</span><span class="plain-syntax">++) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">j</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">']'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">unicode_point</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::atoi</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">+9);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax"> = </span><span class="identifier-syntax">j</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Characters::isdigit</span><span class="plain-syntax">(</span><span class="identifier-syntax">c</span><span class="plain-syntax">) == </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">) </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">unicode_point</span><span class="plain-syntax"> &gt; </span><span class="constant-syntax">0</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="identifier-syntax">unicode_point</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">continue</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP2_2">&#167;2.2</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP2_2_2" class="paragraph-anchor"></a><b>&#167;2.2.2. </b>There are three different forms for an <span class="extract"><span class="extract-syntax">@</span></span>-escape. First, <span class="extract"><span class="extract-syntax">@{....}</span></span> with
hexadecimal digits inside the braces; then <span class="extract"><span class="extract-syntax">@@...</span></span> with decimal digits; and
otherwise <span class="extract"><span class="extract-syntax">@..</span></span> for any of the set of legal digraphs listed below. The
content represented by dots in these syntaxes we will store in <span class="extract"><span class="extract-syntax">token</span></span>,
and <span class="extract"><span class="extract-syntax">skip</span></span> will count the total length of the escape, in raw characters.
Thus for <span class="extract"><span class="extract-syntax">@{2af4}</span></span> the <span class="extract"><span class="extract-syntax">skip</span></span> count would be 7.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Extract the escape token</span><span class="named-paragraph-number">2.2.2</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">d</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">+1);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax"> == </span><span class="character-syntax">'{'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">skip</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">while</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">+</span><span class="identifier-syntax">skip</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">e</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">+</span><span class="identifier-syntax">skip</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">skip</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">e</span><span class="plain-syntax"> == </span><span class="character-syntax">'}'</span><span class="plain-syntax">) </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">, </span><span class="identifier-syntax">e</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">hexadecimal</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax"> == </span><span class="character-syntax">'@'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">skip</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">while</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Characters::isdigit</span><span class="plain-syntax">(</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">+</span><span class="identifier-syntax">skip</span><span class="plain-syntax">))) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">e</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">+</span><span class="identifier-syntax">skip</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">skip</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">, </span><span class="identifier-syntax">e</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">decimal</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">, </span><span class="identifier-syntax">d</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">, </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">+2));</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">skip</span><span class="plain-syntax"> += </span><span class="constant-syntax">2</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP2_2">&#167;2.2</a>, <a href="2-tkn.html#SP4_1">&#167;4.1</a>, <a href="2-tkn.html#SP4_2">&#167;4.2</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP2_2_3" class="paragraph-anchor"></a><b>&#167;2.2.3. </b>The hex notation refers directly to Unicode code points, so all we need to
do is convert the token from a string to hex and then put it as a character.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Expand hexadecimal Unicode value</span><span class="named-paragraph-number">2.2.3</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">N</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">LOOP_THROUGH_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">pos</span><span class="plain-syntax">, </span><span class="identifier-syntax">token</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get</span><span class="plain-syntax">(</span><span class="identifier-syntax">pos</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">D</span><span class="plain-syntax"> = </span><a href="2-tkn.html#SP3" class="function-link"><span class="function-syntax">Tokenisation::hex_val</span></a><span class="plain-syntax">(</span><span class="identifier-syntax">c</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">D</span><span class="plain-syntax"> == -1) { </span><span class="identifier-syntax">N</span><span class="plain-syntax"> = -1; </span><span class="reserved-syntax">break</span><span class="plain-syntax">; }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">N</span><span class="plain-syntax"> = </span><span class="constant-syntax">16</span><span class="plain-syntax">*</span><span class="identifier-syntax">N</span><span class="plain-syntax"> + </span><span class="identifier-syntax">D</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">N</span><span class="plain-syntax"> == -1) </span><span class="identifier-syntax">WRITE_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="string-syntax">"?ERROR&lt;%S&gt;"</span><span class="plain-syntax">, </span><span class="identifier-syntax">token</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="identifier-syntax">N</span><span class="plain-syntax">);</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP2_2">&#167;2.2</a>, <a href="2-tkn.html#SP4_2">&#167;4.2</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP2_2_4" class="paragraph-anchor"></a><b>&#167;2.2.4. </b>Decimal notation is substantially more annoying, because it uses the ZSCII
character set, not Unicode. ZSCII is (for our purposes at least) the same as
ASCII in the range 0 to 127, but is then very unlike ISO Latin-1 (and thus
Unicode) in the range 128 to 255. (Which is as far as it goes.) The following
therefore converts ZSCII to Unicode code points. Note that ZSCII cannot be
mapped faithfully into ISO Latin-1 alone: it contains the OE ligature, which
is in a different Unicode page. See "Table 2B: Higher ZSCII Character Set"
in the DM4.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Expand decimal ZSCII value</span><span class="named-paragraph-number">2.2.4</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">N</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::atoi</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">N</span><span class="plain-syntax">&lt;128) </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="identifier-syntax">N</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">N</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">155</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE4</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> a-diarhesis</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">156</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF6</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> o-diarhesis</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">157</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xFC</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> u-diarhesis</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">158</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC4</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> A-diarhesis</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">159</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD6</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> O-diarhesis</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">160</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xDC</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> U-diarhesis</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">161</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xDF</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> sharp s</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">162</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xBB</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> close double-angle quotation mark</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">163</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xAB</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> open double-angle quotation mark</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">164</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xEB</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> e-diarhesis</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">165</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xEF</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> i-diarhesis</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">166</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xFF</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> y-diarhesis</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">167</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCB</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> E-diarhesis</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">168</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCF</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> I-diarhesis</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">169</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE1</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> a-acute</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">170</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE9</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> e-acute</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">171</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xED</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> i-acute</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">172</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF3</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> o-acute</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">173</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xFA</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> u-acute</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">174</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xFD</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> y-acute</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">175</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC1</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> A-acute</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">176</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC9</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> E-acute</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">177</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCD</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> I-acute</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">178</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD3</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> O-acute</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">179</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xDA</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> U-acute</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">180</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xDD</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> Y-acute</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">181</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE0</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> a-grave</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">182</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE8</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> e-grave</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">183</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xEC</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> i-grave</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">184</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF2</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> o-grave</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">185</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF9</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> u-grave</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">186</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC0</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> A-grave</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">187</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC8</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> E-grave</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">188</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCC</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> I-grave</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">189</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD2</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> O-grave</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">190</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD9</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> U-grave</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">191</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE2</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> a-circumflex</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">192</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xEA</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> e-circumflex</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">193</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xEE</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> i-circumflex</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">194</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF4</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> o-circumflex</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">195</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xFB</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> u-circumflex</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">196</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC2</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> A-circumflex</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">197</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCA</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> E-circumflex</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">198</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCE</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> I-circumflex</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">199</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD4</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> O-circumflex</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">200</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xDB</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> U-circumflex</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">201</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE6</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> a-ring</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">202</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC6</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> A-ring</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">203</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF8</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> o-stroke</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">204</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD8</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> O-stroke</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">205</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE3</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> a-tilde</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">206</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF1</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> n-tilde</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">207</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF5</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> o-tilde</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">208</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC3</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> A-tilde</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">209</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD1</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> N-tilde</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">210</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD5</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> O-tilde</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">211</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE6</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> ae</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">212</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC6</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> AE</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">213</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE7</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> c-cedilla</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">214</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC7</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> C-cedilla</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">215</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xFE</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> thorn</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">216</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF0</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> eth</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">217</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xDE</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> Thorn</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">218</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD0</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> Eth</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">219</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xA3</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> pound sterling sign</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">220</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0x153</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> oe</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">221</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0x152</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> OE</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">222</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xA1</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> inverted !</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="constant-syntax">223</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xBF</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">; </span><span class="comment-syntax"> inverted ?</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP2_2">&#167;2.2</a>, <a href="2-tkn.html#SP4_2">&#167;4.2</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP2_2_5" class="paragraph-anchor"></a><b>&#167;2.2.5. </b>Now for the digraphs. For example, <span class="extract"><span class="extract-syntax">@'a</span></span> is an a-acute, while <span class="extract"><span class="extract-syntax">@ss</span></span> is a
German sharp s. Again, see the DM4 for the specification of these. A misprint
in the DM4 means that one part of that manual says that <span class="extract"><span class="extract-syntax">@cc</span></span> is the syntax
for c-cedilla, and another says it is <span class="extract"><span class="extract-syntax">@,c</span></span>. To be on the safe side, we
recognise both. For similar reasons, we recognise both <span class="extract"><span class="extract-syntax">@/o</span></span> and <span class="extract"><span class="extract-syntax">@\o</span></span> as
a Scandinavian o-stroke.
</p>
<p class="commentary"><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Expand TeX-style digraph</span><span class="named-paragraph-number">2.2.5</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">, </span><span class="constant-syntax">0</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">d</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">, </span><span class="constant-syntax">1</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'\''</span><span class="plain-syntax">: </span><span class="comment-syntax"> these are acute accents</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'a'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE1</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'e'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE9</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'i'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xED</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'o'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF3</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'u'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xFA</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'y'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xFD</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'A'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC1</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'E'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC9</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'I'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCD</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'O'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD3</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'U'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xDA</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'Y'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xDD</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'`'</span><span class="plain-syntax">: </span><span class="comment-syntax"> these are grave accents</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'a'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE0</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'e'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE8</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'i'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xEC</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'o'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF2</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'u'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF9</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'A'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC0</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'E'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC8</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'I'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCC</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'O'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD2</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'U'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD9</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'^'</span><span class="plain-syntax">: </span><span class="comment-syntax"> these are circumflex accents</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'a'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE2</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'e'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xEA</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'i'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xEE</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'o'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF4</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'u'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xFB</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'A'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC2</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'E'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCA</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'I'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCE</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'O'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD4</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'U'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xDB</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">':'</span><span class="plain-syntax">: </span><span class="comment-syntax"> these are diarhesis accents, that is, umlauts</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'a'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE4</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'e'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xEB</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'i'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xEF</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'o'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF6</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'u'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xFC</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'y'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xFF</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'A'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC4</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'E'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCB</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'I'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCF</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'O'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD6</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'U'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xDC</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'Y'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0x0178</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'~'</span><span class="plain-syntax">: </span><span class="comment-syntax"> these are tilde accents</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'a'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE3</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'n'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF1</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'o'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF5</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'A'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC3</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'N'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD1</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'O'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD5</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">','</span><span class="plain-syntax">: </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'c'</span><span class="plain-syntax">: </span><span class="comment-syntax"> cedilla (a misprint in the DM4 means both are said to work)</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'c'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE7</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'C'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC7</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'\\'</span><span class="plain-syntax">: </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'/'</span><span class="plain-syntax">: </span><span class="comment-syntax"> the Scandinavian slash thing</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'o'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF8</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'O'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD8</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'a'</span><span class="plain-syntax">: </span><span class="comment-syntax"> joined ae</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'e'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE6</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'A'</span><span class="plain-syntax">: </span><span class="comment-syntax"> joined AE</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'E'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC6</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'e'</span><span class="plain-syntax">: </span><span class="comment-syntax"> lower-case Icelandic eth</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'t'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xF0</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'E'</span><span class="plain-syntax">: </span><span class="comment-syntax"> capital Icelandic eth</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'t'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xD0</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'t'</span><span class="plain-syntax">: </span><span class="comment-syntax"> lower-case thorn</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'h'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xFE</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'T'</span><span class="plain-syntax">: </span><span class="comment-syntax"> capital thorn</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'h'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xCE</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'L'</span><span class="plain-syntax">: </span><span class="comment-syntax"> pound sign</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'L'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xA3</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'!'</span><span class="plain-syntax">: </span><span class="comment-syntax"> inverted Spanish exclamation mark</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax"> == </span><span class="character-syntax">'!'</span><span class="plain-syntax">) </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xA1</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'?'</span><span class="plain-syntax">: </span><span class="comment-syntax"> inverted Spanish question mark</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax"> == </span><span class="character-syntax">'?'</span><span class="plain-syntax">) </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xBF</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'&lt;'</span><span class="plain-syntax">: </span><span class="comment-syntax"> Double-angle open quotation mark</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax"> == </span><span class="character-syntax">'&lt;'</span><span class="plain-syntax">) </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xAB</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'&gt;'</span><span class="plain-syntax">: </span><span class="comment-syntax"> Double-angle close quotation mark</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax"> == </span><span class="character-syntax">'&gt;'</span><span class="plain-syntax">) </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xBB</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'s'</span><span class="plain-syntax">: </span><span class="comment-syntax"> German sharp s</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax"> == </span><span class="character-syntax">'s'</span><span class="plain-syntax">) </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xDF</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'o'</span><span class="plain-syntax">: </span><span class="comment-syntax"> joined oe and ring accent A</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'a'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xE5</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'A'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0xC5</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'e'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0x153</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'O'</span><span class="plain-syntax">: </span><span class="comment-syntax"> joined OE</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">switch</span><span class="plain-syntax"> (</span><span class="identifier-syntax">d</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">case</span><span class="plain-syntax"> </span><span class="character-syntax">'E'</span><span class="plain-syntax">: </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="constant-syntax">0x152</span><span class="plain-syntax">); </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4_1" class="named-paragraph-link"><span class="named-paragraph">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></a></span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">default:</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">WRITE_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="string-syntax">"TOKEN&lt;%S&gt;"</span><span class="plain-syntax">, </span><span class="identifier-syntax">token</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP2_2">&#167;2.2</a>, <a href="2-tkn.html#SP4_2">&#167;4.2</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP2_2_4_1" class="paragraph-anchor"></a><b>&#167;2.2.4.1. </b><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Unknown string token</span><span class="named-paragraph-number">2.2.4.1</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="identifier-syntax">WRITE_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="string-syntax">"@%S"</span><span class="plain-syntax">, </span><span class="identifier-syntax">token</span><span class="plain-syntax">);</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP2_2_4">&#167;2.2.4</a>, <a href="2-tkn.html#SP2_2_5">&#167;2.2.5</a> (21 times).</li></ul>
<p class="commentary firstcommentary"><a id="SP3" class="paragraph-anchor"></a><b>&#167;3. </b></p>
<pre class="displayed-code all-displayed-code code-font">
<span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="function-syntax">Tokenisation::hex_val</span><button class="popup" onclick="togglePopup('usagePopup3')"><span class="comment-syntax">?</span><span class="popuptext" id="usagePopup3">Usage of <span class="code-font"><span class="function-syntax">Tokenisation::hex_val</span></span>:<br/><a href="2-tkn.html#SP2_2_3">&#167;2.2.3</a></span></button><span class="plain-syntax">(</span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> &gt;= </span><span class="character-syntax">'0'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> &lt;= </span><span class="character-syntax">'9'</span><span class="plain-syntax">)) </span><span class="reserved-syntax">return</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> - </span><span class="character-syntax">'0'</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> &gt;= </span><span class="character-syntax">'a'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> &lt;= </span><span class="character-syntax">'f'</span><span class="plain-syntax">)) </span><span class="reserved-syntax">return</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> - </span><span class="character-syntax">'a'</span><span class="plain-syntax"> + </span><span class="constant-syntax">10</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> &gt;= </span><span class="character-syntax">'A'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> &lt;= </span><span class="character-syntax">'F'</span><span class="plain-syntax">)) </span><span class="reserved-syntax">return</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> - </span><span class="character-syntax">'A'</span><span class="plain-syntax"> + </span><span class="constant-syntax">10</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">return</span><span class="plain-syntax"> -1;</span>
<span class="plain-syntax">}</span>
</pre>
<p class="commentary firstcommentary"><a id="SP4" class="paragraph-anchor"></a><b>&#167;4. </b>And similarly for single-quoted text notation, which shares some of the same
conventions. In fact I6 for some reason does not support the <span class="extract"><span class="extract-syntax">@@...</span></span> decimal
notation within character or dictionary literals, throwing an error if it
is used; but we'll recognise it anyway, for the sake of using the same code as
is given above.
</p>
<p class="commentary">The tricky thing here is that single-quoted literals are characters if they
contain one character and do not have a <span class="extract"><span class="extract-syntax">//</span></span> marker, but dictionary literals
otherwise. We need to know which because <span class="extract"><span class="extract-syntax">^</span></span> is an escape character for a
single quotation mark in a dictionary literal, but not a character literal.
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="reserved-syntax">void</span><span class="plain-syntax"> </span><span class="function-syntax">Tokenisation::de_escape_sq_text</span><button class="popup" onclick="togglePopup('usagePopup4')"><span class="comment-syntax">?</span><span class="popuptext" id="usagePopup4">Usage of <span class="code-font"><span class="function-syntax">Tokenisation::de_escape_sq_text</span></span>:<br/><a href="2-tkn.html#SP1_3">&#167;1.3</a></span></button><span class="plain-syntax">(</span><span class="identifier-syntax">text_stream</span><span class="plain-syntax"> *</span><span class="identifier-syntax">text</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">TEMPORARY_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">WRITE_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="string-syntax">"%S"</span><span class="plain-syntax">, </span><span class="identifier-syntax">text</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">Str::clear</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">is_dictionary_word</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP4_1" class="named-paragraph-link"><span class="named-paragraph">Determine if this is a character or dictionary literal</span><span class="named-paragraph-number">4.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP4_2" class="named-paragraph-link"><span class="named-paragraph">Expand the literal text</span><span class="named-paragraph-number">4.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">DISCARD_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">)</span>
<span class="plain-syntax">}</span>
</pre>
<p class="commentary firstcommentary"><a id="SP4_1" class="paragraph-anchor"></a><b>&#167;4.1. </b><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Determine if this is a character or dictionary literal</span><span class="named-paragraph-number">4.1</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">char_count</span><span class="plain-syntax"> = </span><span class="constant-syntax">0</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (</span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax">=0; </span><span class="identifier-syntax">i</span><span class="plain-syntax">&lt;</span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">); </span><span class="identifier-syntax">i</span><span class="plain-syntax">++) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">) == </span><span class="character-syntax">'/'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">+1) == </span><span class="character-syntax">'/'</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">is_dictionary_word</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">; </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">char_count</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">) == </span><span class="character-syntax">'@'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">TEMPORARY_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">skip</span><span class="plain-syntax"> = </span><span class="constant-syntax">1</span><span class="plain-syntax">, </span><span class="identifier-syntax">decimal</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">, </span><span class="identifier-syntax">hexadecimal</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_2" class="named-paragraph-link"><span class="named-paragraph">Extract the escape token</span><span class="named-paragraph-number">2.2.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax"> += </span><span class="identifier-syntax">skip</span><span class="plain-syntax">-1;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">DISCARD_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">char_count</span><span class="plain-syntax"> &gt; </span><span class="constant-syntax">1</span><span class="plain-syntax">) </span><span class="identifier-syntax">is_dictionary_word</span><span class="plain-syntax"> = </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP4">&#167;4</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP4_2" class="paragraph-anchor"></a><b>&#167;4.2. </b><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Expand the literal text</span><span class="named-paragraph-number">4.2</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">for</span><span class="plain-syntax"> (</span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax">=0; </span><span class="identifier-syntax">i</span><span class="plain-syntax">&lt;</span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">); </span><span class="identifier-syntax">i</span><span class="plain-syntax">++) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax"> = </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'/'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">+1) == </span><span class="character-syntax">'/'</span><span class="plain-syntax">))</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP4_2_1" class="named-paragraph-link"><span class="named-paragraph">Past this point escape characters do not apply</span><span class="named-paragraph-number">4.2.1</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'@'</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">TEMPORARY_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="identifier-syntax">skip</span><span class="plain-syntax"> = </span><span class="constant-syntax">1</span><span class="plain-syntax">, </span><span class="identifier-syntax">decimal</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">, </span><span class="identifier-syntax">hexadecimal</span><span class="plain-syntax"> = </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_2" class="named-paragraph-link"><span class="named-paragraph">Extract the escape token</span><span class="named-paragraph-number">2.2.2</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">hexadecimal</span><span class="plain-syntax">) </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_3" class="named-paragraph-link"><span class="named-paragraph">Expand hexadecimal Unicode value</span><span class="named-paragraph-number">2.2.3</span></a></span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> (</span><span class="identifier-syntax">decimal</span><span class="plain-syntax">) </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_4" class="named-paragraph-link"><span class="named-paragraph">Expand decimal ZSCII value</span><span class="named-paragraph-number">2.2.4</span></a></span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="named-paragraph-container code-font"><a href="2-tkn.html#SP2_2_5" class="named-paragraph-link"><span class="named-paragraph">Expand TeX-style digraph</span><span class="named-paragraph-number">2.2.5</span></a></span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax"> += </span><span class="identifier-syntax">skip</span><span class="plain-syntax">-1;</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">DISCARD_TEXT</span><span class="plain-syntax">(</span><span class="identifier-syntax">token</span><span class="plain-syntax">)</span>
<span class="plain-syntax"> } </span><span class="reserved-syntax">else</span><span class="plain-syntax"> {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'^'</span><span class="plain-syntax">) &amp;&amp; (</span><span class="identifier-syntax">is_dictionary_word</span><span class="plain-syntax">)) </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="character-syntax">'\''</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">else</span><span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="identifier-syntax">c</span><span class="plain-syntax">);</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> }</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP4">&#167;4</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP4_2_1" class="paragraph-anchor"></a><b>&#167;4.2.1. </b><span class="named-paragraph-container code-font"><span class="named-paragraph-defn">Past this point escape characters do not apply</span><span class="named-paragraph-number">4.2.1</span></span><span class="comment-syntax"> =</span>
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="plain-syntax"> </span><span class="reserved-syntax">while</span><span class="plain-syntax"> (</span><span class="identifier-syntax">i</span><span class="plain-syntax"> &lt; </span><span class="identifier-syntax">Str::len</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">)) {</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">PUT_TO</span><span class="plain-syntax">(</span><span class="identifier-syntax">text</span><span class="plain-syntax">, </span><span class="identifier-syntax">Str::get_at</span><span class="plain-syntax">(</span><span class="identifier-syntax">raw</span><span class="plain-syntax">, </span><span class="identifier-syntax">i</span><span class="plain-syntax">));</span>
<span class="plain-syntax"> </span><span class="identifier-syntax">i</span><span class="plain-syntax">++;</span>
<span class="plain-syntax"> }</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">break</span><span class="plain-syntax">;</span>
</pre>
<ul class="endnotetexts"><li>This code is used in <a href="2-tkn.html#SP4_2">&#167;4.2</a>.</li></ul>
<p class="commentary firstcommentary"><a id="SP5" class="paragraph-anchor"></a><b>&#167;5. </b>Lastly, this defines valid identifier characters:
</p>
<pre class="displayed-code all-displayed-code code-font">
<span class="reserved-syntax">int</span><span class="plain-syntax"> </span><span class="function-syntax">Tokenisation::identchar</span><button class="popup" onclick="togglePopup('usagePopup5')"><span class="comment-syntax">?</span><span class="popuptext" id="usagePopup5">Usage of <span class="code-font"><span class="function-syntax">Tokenisation::identchar</span></span>:<br/><a href="2-tkn.html#SP1_3_1_1">&#167;1.3.1.1</a>, <a href="2-tkn.html#SP1_3_1_3">&#167;1.3.1.3</a>, <a href="2-tkn.html#SP1_3_1_4_1">&#167;1.3.1.4.1</a></span></button><span class="plain-syntax">(</span><span class="identifier-syntax">wchar_t</span><span class="plain-syntax"> </span><span class="identifier-syntax">c</span><span class="plain-syntax">) {</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">if</span><span class="plain-syntax"> ((</span><span class="identifier-syntax">Characters::isalnum</span><span class="plain-syntax">(</span><span class="identifier-syntax">c</span><span class="plain-syntax">)) || (</span><span class="identifier-syntax">c</span><span class="plain-syntax"> == </span><span class="character-syntax">'`'</span><span class="plain-syntax">)) </span><span class="reserved-syntax">return</span><span class="plain-syntax"> </span><span class="identifier-syntax">TRUE</span><span class="plain-syntax">;</span>
<span class="plain-syntax"> </span><span class="reserved-syntax">return</span><span class="plain-syntax"> </span><span class="identifier-syntax">FALSE</span><span class="plain-syntax">;</span>
<span class="plain-syntax">}</span>
</pre>
<nav role="progress"><div class="progresscontainer">
<ul class="progressbar"><li class="progressprev"><a href="2-pis.html">&#10094;</a></li><li class="progresschapter"><a href="P-wtmd.html">P</a></li><li class="progresschapter"><a href="1-bm.html">1</a></li><li class="progresscurrentchapter">2</li><li class="progresssection"><a href="2-is.html">is</a></li><li class="progresssection"><a href="2-pis.html">pis</a></li><li class="progresscurrent">tkn</li><li class="progresssection"><a href="2-rmf.html">rmf</a></li><li class="progresssection"><a href="2-i6o.html">i6o</a></li><li class="progresssection"><a href="2-eis.html">eis</a></li><li class="progresssection"><a href="2-if.html">if</a></li><li class="progresssection"><a href="2-i6a.html">i6a</a></li><li class="progresssection"><a href="2-i6se.html">i6se</a></li><li class="progresschapter"><a href="3-prd.html">3</a></li><li class="progressnext"><a href="2-rmf.html">&#10095;</a></li></ul></div>
</nav><!--End of weave-->
</main>
</body>
</html>
Reference in a new issue View git blame Copy permalink