mirror of
https://github.com/ganelson/inform.git
synced 2024-07-05 16:44:21 +03:00
2956 lines
392 KiB
HTML
2956 lines
392 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
|
|
<html>
|
|
<head>
|
|
<title>4/nw</title>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
|
|
<meta http-equiv="Content-Language" content="en-gb">
|
|
<link href="inweb.css" rel="stylesheet" rev="stylesheet" type="text/css">
|
|
</head>
|
|
<body>
|
|
|
|
<!--Weave of '4/prf' generated by 7-->
|
|
<ul class="crumbs"><li><a href="../webs.html">★</a></li><li><a href="index.html">words</a></li><li><a href="index.html#4">Chapter 4: Parsing</a></li><li><b>Preform</b></li></ul><p class="purpose">To read in structural definitions of natural language written in a meta-language called Preform.</p>
|
|
|
|
<ul class="toc"><li><a href="#SP1">§1. Definitions</a></li><li><a href="#SP2">§2. Introduction</a></li><li><a href="#SP8">§8. Implementation</a></li><li><a href="#SP18">§18. Logging</a></li><li><a href="#SP24">§24. Building grammar</a></li><li><a href="#SP32">§32. Optimisation calculations</a></li><li><a href="#SP50">§50. Parsing</a></li><li><a href="#SP54">§54. Reading Preform syntax from a file</a></li></ul><hr class="tocbar">
|
|
|
|
<p class="inwebparagraph"><a id="SP1"></a><b>§1. Definitions. </b></p>
|
|
|
|
<p class="inwebparagraph"><a id="SP2"></a><b>§2. Introduction. </b></p>
|
|
|
|
<p class="inwebparagraph"><a id="SP3"></a><b>§3. </b>The parser reads source text against a specific language only, if
|
|
<code class="display"><span class="extract">language_of_source_text</span></code> is set; or, if it isn't, from any language.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="identifier">PREFORM_LANGUAGE_TYPE</span><span class="plain"> *</span><span class="identifier">language_of_source_text</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="identifier">PREFORM_LANGUAGE_TYPE</span><span class="plain"> *</span><span class="identifier">language_being_read_by_Preform</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="inwebparagraph"><a id="SP4"></a><b>§4. </b>Preform is parsed with the regular lexer, using the following set of
|
|
characters as word-breaking punctuation marks:
|
|
</p>
|
|
|
|
|
|
<pre class="definitions">
|
|
<span class="definitionkeyword">define</span> <span class="constant">PREFORM_PUNCTUATION_MARKS</span><span class="plain"> </span><span class="identifier">L</span><span class="string">"{}[]_^?&\</span><span class="plain">\</span><span class="string">"</span>
|
|
</pre>
|
|
<p class="inwebparagraph"><a id="SP5"></a><b>§5. </b>That's what it would look like in the Preform file, but here is how it's
|
|
typed in the Inform source code. Definitions like this one are scattered all
|
|
across the Inform web, in order to keep them close to the code which relates to
|
|
them. The <code class="display"><span class="extract">inweb</span></code> tangler compiles them in two halves: the instructions right
|
|
of the <code class="display"><span class="extract">==></span></code> arrows are extracted and compiled into a C routine called the
|
|
"compositor" for the nonterminal (see below), while the actual grammar is
|
|
extracted and placed into Inform's "Preform.txt" file.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">In the document of Preform grammar extracted from Inform's source code to
|
|
lay the language out for translators, the <code class="display"><span class="extract">==></span></code> arrows and formulae to the
|
|
right of them are omitted — those represent semantics, not syntax.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="plain"><competitor> ::=</span>
|
|
<span class="plain"><ordinal-number> runner | ==> TRUE</span>
|
|
<span class="plain">runner no <cardinal-number> ==> FALSE</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="inwebparagraph"><a id="SP6"></a><b>§6. </b>Each nonterminal, when successfully matched, can provide both or more usually
|
|
just one of two results: an integer, to be stored in <code class="display"><span class="extract">*X</span></code>, and a void pointer,
|
|
to be stored in <code class="display"><span class="extract">*XP</span></code>. For example, <k-kind> matches if and only if the
|
|
text declares a legal kind, such as "number"; its pointer result is to the
|
|
kind found, such as <code class="display"><span class="extract">K_number</span></code>. But <competitor> only results in an integer.
|
|
The <code class="display"><span class="extract">==></span></code> arrow is optional, but if present, it says what the result is if
|
|
the given production is matched; the <code class="display"><span class="extract">inweb</span></code> tangler, if it sees an expression
|
|
on the right of the arrow, assigns that value to the integer result. So,
|
|
for example, "runner bean" or "beetroot" would not match <competitor>;
|
|
"4th runner" would match with integer result <code class="display"><span class="extract">TRUE</span></code>; "runner no 17" would
|
|
match with integer result <code class="display"><span class="extract">FALSE</span></code>.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">Usually, though, the result(s) of a nonterminal depend on the result(s) of
|
|
other nonterminals used to make the match. In the compositing expression,
|
|
so called because it composes together the various intermediate results into
|
|
one final result, <code class="display"><span class="extract">R[1]</span></code> is the integer result of the first nonterminal in
|
|
the production, <code class="display"><span class="extract">R[2]</span></code> the second, and so on; <code class="display"><span class="extract">RP[1]</span></code> and so on hold the
|
|
pointer results. Here, on both productions, there's just one nonterminal
|
|
in the line, <ordinal-number> in the first case, <cardinal-number> in
|
|
the second. So the following refinement of <competitor> means that "4th
|
|
runner" matches with integer result 4, because <ordinal-number> matches
|
|
"4th" with integer result 4, and that goes into <code class="display"><span class="extract">R[1]</span></code>. Similarly,
|
|
"runner no 17" ends up with integer result 17. "The pacemaker" matches
|
|
with integer result 1; here there are no intermediate results to make use
|
|
of, so <code class="display"><span class="extract">R[...]</span></code> can't be used.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="plain"><competitor> ::=</span>
|
|
<span class="plain">the pacemaker | ==> 1</span>
|
|
<span class="plain"><ordinal-number> runner | ==> R[1]</span>
|
|
<span class="plain">runner no <cardinal-number> ==> R[1]</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="inwebparagraph"><a id="SP7"></a><b>§7. </b>The arrows and expressions are optional, and if they are omitted, then the
|
|
result integer is set to the production number, counting up from 0. For
|
|
example, given the following, "polkadot" matches with result 1, and "green"
|
|
with result 2.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="plain"><race-jersey> ::=</span>
|
|
<span class="plain">yellow | polkadot | green | white</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="inwebparagraph"><a id="SP8"></a><b>§8. Implementation. </b>We read the Preform file for English early in Inform's run, and since it
|
|
goes through the standard lexer, it makes words. The following holds the
|
|
word number of the last of these words. (The same is also true for documentation
|
|
cross-references, which are not really anything to do with Preform.)
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">language_definition_top</span><span class="plain"> = -1;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">doc_references_top</span><span class="plain"> = -1;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="inwebparagraph"><a id="SP9"></a><b>§9. </b>Now for nonterminals. We must first clarify how word ranges, once matched in
|
|
the parser, will be stored. Within each production, word ranges are numbered
|
|
upwards from 1. Thus:
|
|
</p>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="plain">man with ... on his ...</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph">would, if it matched successfully, generate two word ranges, numbered 1 and 2.
|
|
These are stored in memory belonging to the nonterminal; they are usually, but
|
|
not always, then retrieved by whatever part of Inform requested the parse,
|
|
using the <code class="display"><span class="extract">GET_RW</span></code> macro rather than a function call for speed. It's rare,
|
|
but a few internal nonterminals also generate word ranges: they use the
|
|
corresponding <code class="display"><span class="extract">PUT_RW</span></code> macro to do so. Lastly, we can pass word ranges up
|
|
from one nonterminal to another, with <code class="display"><span class="extract">INHERIT_RANGES</span></code>.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">This form of storage incurs very little time or space overhead, and is possible
|
|
only because the parser never backtracks. But it also follows that word ranges
|
|
are overwritten if a nonterminal calls itself directly or indirectly: that is,
|
|
the inner one's results are wiped out by the outer one. But this is no problem,
|
|
since we never extract word-ranges from grammar which is recursive.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">Word range 0 is reserved in case we ever need it for the entire text matched
|
|
by the nonterminal, but at present we don't need that.
|
|
</p>
|
|
|
|
|
|
<pre class="definitions">
|
|
<span class="definitionkeyword">define</span> <span class="constant">MAX_RANGES_PER_PRODUCTION</span><span class="plain"> 5 </span> <span class="comment">in fact, one less than this, since range 0 is reserved</span>
|
|
<span class="definitionkeyword">define</span> <span class="identifier">GET_RW</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">, </span><span class="identifier">N</span><span class="plain">) (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>range_result</span><span class="plain">[</span><span class="identifier">N</span><span class="plain">])</span>
|
|
<span class="definitionkeyword">define</span> <span class="identifier">PUT_RW</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">, </span><span class="identifier">N</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">) { </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>range_result</span><span class="plain">[</span><span class="identifier">N</span><span class="plain">] = </span><span class="identifier">W</span><span class="plain">; }</span>
|
|
<span class="definitionkeyword">define</span> <span class="identifier">INHERIT_RANGES</span><span class="plain">(</span><span class="identifier">from</span><span class="plain">, </span><span class="identifier">to</span><span class="plain">) {</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">i</span><span class="plain">=1; </span><span class="identifier">i</span><span class="plain"><</span><span class="constant">MAX_RANGES_PER_PRODUCTION</span><span class="plain">; </span><span class="identifier">i</span><span class="plain">++) </span> <span class="comment">not copying range 0</span>
|
|
<span class="identifier">to</span><span class="plain">-</span><span class="element">>range_result</span><span class="plain">[</span><span class="identifier">i</span><span class="plain">] = </span><span class="identifier">from</span><span class="plain">-</span><span class="element">>range_result</span><span class="plain">[</span><span class="identifier">i</span><span class="plain">];</span>
|
|
<span class="plain">}</span>
|
|
<span class="definitionkeyword">define</span> <span class="identifier">CLEAR_RW</span><span class="plain">(</span><span class="identifier">from</span><span class="plain">) {</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">i</span><span class="plain">=0; </span><span class="identifier">i</span><span class="plain"><</span><span class="constant">MAX_RANGES_PER_PRODUCTION</span><span class="plain">; </span><span class="identifier">i</span><span class="plain">++) </span> <span class="comment">including range 0</span>
|
|
<span class="identifier">from</span><span class="plain">-</span><span class="element">>range_result</span><span class="plain">[</span><span class="identifier">i</span><span class="plain">] = </span><span class="constant">EMPTY_WORDING</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
<p class="inwebparagraph"><a id="SP10"></a><b>§10. </b>So here's the nonterminal structure. There are a few further complications
|
|
for speed reasons:
|
|
</p>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<ul class="items"><li>(a) The minimum and maximum number of words which could ever be a match are
|
|
precalculated. For example, if Preform can tell that N will only a run of
|
|
between 3 and 7 words inclusive, then it can quickly reject any run of words
|
|
outside that range. <code class="display"><span class="extract">INFINITE_WORD_COUNT</span></code> is taken as the maximum if N
|
|
could in principle match text of any length. (However: note that a maximum of
|
|
0 means that the maximum and minimum word counts are disregarded.)
|
|
</li></ul>
|
|
<ul class="items"><li>(b) A few internal nonterminals are "voracious". These are given the entire
|
|
word range for their productions to eat, and encouraged to eat as much as
|
|
they like, returning a word number to show how far they got. While this
|
|
effect could be duplicated with suitable grammar and non-voracious nonterminals,
|
|
it would be quite a bit slower, since it would have to test every possible
|
|
word range.
|
|
</li></ul>
|
|
|
|
<pre class="definitions">
|
|
<span class="definitionkeyword">define</span> <span class="constant">MAX_RESULTS_PER_PRODUCTION</span><span class="plain"> 10</span>
|
|
<span class="definitionkeyword">define</span> <span class="constant">INFINITE_WORD_COUNT</span><span class="plain"> 1000000000</span>
|
|
</pre>
|
|
|
|
<pre class="display">
|
|
<span class="reserved">typedef</span><span class="plain"> </span><span class="reserved">struct</span><span class="plain"> </span><span class="reserved">nonterminal</span><span class="plain"> {</span>
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">nonterminal_id</span><span class="plain">; </span> <span class="comment">e.g. <code class="display"><span class="extract">"<cardinal-number>"</span></code></span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">voracious</span><span class="plain">; </span> <span class="comment">if true, scans whole rest of word range</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">multiplicitous</span><span class="plain">;</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">marked_internal</span><span class="plain">; </span> <span class="comment">has, or will be given, an internal definition...</span>
|
|
<span class="reserved">int</span><span class="plain"> (*</span><span class="identifier">internal_definition</span><span class="plain">)(</span><span class="reserved">wording</span><span class="plain"> </span><span class="identifier">W</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> *</span><span class="identifier">result</span><span class="plain">, </span><span class="reserved">void</span><span class="plain"> **</span><span class="identifier">result_p</span><span class="plain">); </span> <span class="comment">...this one</span>
|
|
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">production_list</span><span class="plain"> *</span><span class="identifier">first_production_list</span><span class="plain">; </span> <span class="comment">if not internal, this defines it</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> (*</span><span class="identifier">result_compositor</span><span class="plain">)(</span><span class="reserved">int</span><span class="plain"> *</span><span class="identifier">r</span><span class="plain">, </span><span class="reserved">void</span><span class="plain"> **</span><span class="identifier">rp</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> *</span><span class="identifier">inters</span><span class="plain">, </span><span class="reserved">void</span><span class="plain"> **</span><span class="identifier">inter_ps</span><span class="plain">, </span><span class="reserved">wording</span><span class="plain"> *</span><span class="identifier">interW</span><span class="plain">, </span><span class="reserved">wording</span><span class="plain"> </span><span class="identifier">W</span><span class="plain">);</span>
|
|
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">wording</span><span class="plain"> </span><span class="identifier">range_result</span><span class="plain">[</span><span class="constant">MAX_RANGES_PER_PRODUCTION</span><span class="plain">]; </span> <span class="comment">storage for word ranges matched</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">optimised_in_this_pass</span><span class="plain">; </span> <span class="comment">have the following been worked out yet?</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">min_nt_words</span><span class="plain">, </span><span class="identifier">max_nt_words</span><span class="plain">; </span> <span class="comment">for speed</span>
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">range_requirement</span><span class="plain"> </span><span class="identifier">nonterminal_req</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">nt_req_bit</span><span class="plain">; </span> <span class="comment">which hashing category the words belong to, or -1 if none</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">number_words_by_production</span><span class="plain">;</span>
|
|
<span class="reserved">unsigned</span><span class="plain"> </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">flag_words_in_production</span><span class="plain">;</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">watched</span><span class="plain">; </span> <span class="comment">watch goings-on to the debugging log</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">nonterminal_tries</span><span class="plain">; </span> <span class="comment">used only in instrumented mode</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">nonterminal_matches</span><span class="plain">; </span> <span class="comment">ditto</span>
|
|
<span class="identifier">MEMORY_MANAGEMENT</span>
|
|
<span class="plain">} </span><span class="reserved">nonterminal</span><span class="plain">;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The structure nonterminal is private to this section.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP11"></a><b>§11. </b>Each (external) nonterminal is then defined by lists of productions:
|
|
potentially one for each language, though only English is required to define
|
|
all of them, and English will always be the first in the list of lists.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">typedef</span><span class="plain"> </span><span class="reserved">struct</span><span class="plain"> </span><span class="reserved">production_list</span><span class="plain"> {</span>
|
|
<span class="identifier">PREFORM_LANGUAGE_TYPE</span><span class="plain"> *</span><span class="identifier">definition_language</span><span class="plain">;</span>
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">production</span><span class="plain"> *</span><span class="identifier">first_production</span><span class="plain">;</span>
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">production_list</span><span class="plain"> *</span><span class="identifier">next_production_list</span><span class="plain">;</span>
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="identifier">match_avinue</span><span class="plain"> *</span><span class="identifier">as_avinue</span><span class="plain">; </span> <span class="comment">when compiled to a trie rather than for Preform</span>
|
|
<span class="identifier">MEMORY_MANAGEMENT</span>
|
|
<span class="plain">} </span><span class="reserved">production_list</span><span class="plain">;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The structure production_list is private to this section.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP12"></a><b>§12. </b>So now we reach the production, which encodes a typical "row" of grammar;
|
|
see the examples above. A production is another list, of "ptokens" (the
|
|
"p" is silent). For example, the production
|
|
</p>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="plain">runner no <cardinal-number></span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph">contains three ptokens. (Note that the stroke sign and the defined-by sign are
|
|
not ptokens; they divide up productions, but aren't part of them.)
|
|
</p>
|
|
|
|
<p class="inwebparagraph">Like nonterminals, productions also count the minimum and maximum words
|
|
matched: in the above example, both are 3.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">There's a new idea here as well, though: struts. A "strut" is a run of
|
|
ptokens in the interior of the production whose position relative to the
|
|
ends is not known. For example, if we match:
|
|
</p>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="plain">frogs like ... but not ... to eat</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph">then we know that in a successful match, "frogs" and "like" must be the
|
|
first two words in the text matched, and "eat" and "to" the last two.
|
|
They are said to have positions 1, 2, -1 and -2 respectively: a positive
|
|
number is relative to the start of the range, a negative relative to the end,
|
|
so that position 1 is always the first word and position -1 is the last.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">But we don't know where "but not" will occur; it could be anywhere in the
|
|
middle of the text. So the ptokens for these words have position 0. A run of
|
|
such ptokens, not counting wildcards like <code class="display"><span class="extract">...</span></code>, is called a strut. We can
|
|
think of it as a partition which can slide backwards and forwards. Many
|
|
productions have no struts at all; the above example has just one. It has
|
|
length 2, not because it contains two ptokens, but because it is always
|
|
two words wide.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">Finding struts when Preform grammar is read in means that we don't have to
|
|
do so much work devising search patterns at parsing time, when speed is
|
|
critical.
|
|
</p>
|
|
|
|
|
|
<pre class="definitions">
|
|
<span class="definitionkeyword">define</span> <span class="constant">MAX_STRUTS_PER_PRODUCTION</span><span class="plain"> 10</span>
|
|
<span class="definitionkeyword">define</span> <span class="constant">MAX_PTOKENS_PER_PRODUCTION</span><span class="plain"> 16</span>
|
|
</pre>
|
|
|
|
<pre class="display">
|
|
<span class="reserved">typedef</span><span class="plain"> </span><span class="reserved">struct</span><span class="plain"> </span><span class="reserved">production</span><span class="plain"> {</span>
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">first_ptoken</span><span class="plain">; </span> <span class="comment">the linked list of ptokens</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">match_number</span><span class="plain">; </span> <span class="comment">0 for <code class="display"><span class="extract">/a/</span></code>, 1 for <code class="display"><span class="extract">/b/</span></code> and so on</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">no_ranges</span><span class="plain">; </span> <span class="comment">actually one more, since range 0 is reserved (see above)</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">min_pr_words</span><span class="plain">, </span><span class="identifier">max_pr_words</span><span class="plain">; </span> <span class="comment">for speed</span>
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">range_requirement</span><span class="plain"> </span><span class="identifier">production_req</span><span class="plain">;</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">no_struts</span><span class="plain">; </span> <span class="comment">the actual number, this time</span>
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">struts</span><span class="plain">[</span><span class="constant">MAX_STRUTS_PER_PRODUCTION</span><span class="plain">]; </span> <span class="comment">first ptoken in strut</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">strut_lengths</span><span class="plain">[</span><span class="constant">MAX_STRUTS_PER_PRODUCTION</span><span class="plain">]; </span> <span class="comment">length of the strut in words</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">production_tries</span><span class="plain">; </span> <span class="comment">used only in instrumented mode</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">production_matches</span><span class="plain">; </span> <span class="comment">ditto</span>
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">wording</span><span class="plain"> </span><span class="identifier">sample_text</span><span class="plain">; </span> <span class="comment">ditto</span>
|
|
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">production</span><span class="plain"> *</span><span class="identifier">next_production</span><span class="plain">; </span> <span class="comment">within its production list</span>
|
|
<span class="identifier">MEMORY_MANAGEMENT</span>
|
|
<span class="plain">} </span><span class="reserved">production</span><span class="plain">;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The structure production is private to this section.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP13"></a><b>§13. </b>And at the bottom of the chain, the lowly ptoken. Even this can spawn another
|
|
list, though: the token <code class="display"><span class="extract">fried/green/tomatoes</span></code> is a list of three ptokens joined
|
|
by the <code class="display"><span class="extract">alternative_ptoken</span></code> links.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">There are two modifiers left to represent: the effects of <code class="display"><span class="extract">^</span></code> (negation) and
|
|
<code class="display"><span class="extract">_</span></code> (casing), and they each have flags. If the ptoken is at the head of a list
|
|
of alternatives, they apply to all of the alternatives, even though set only
|
|
for the headword.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">Each ptoken has a <code class="display"><span class="extract">range_starts</span></code> and <code class="display"><span class="extract">range_ends</span></code> number. This is either -1,
|
|
or marks that the ptoken occurs as the first or last in a range (or both). For
|
|
example, in the production
|
|
</p>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="plain">make ... from {rice ... onions} and peppers</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph">the first <code class="display"><span class="extract">...</span></code> ptoken has start and end set to 1; <code class="display"><span class="extract">rice</span></code> has start 2; <code class="display"><span class="extract">onions</span></code>
|
|
has end 2. Note that the second <code class="display"><span class="extract">...</span></code>, inside the braces, doesn't start or
|
|
end anything; it normally would, but the wider range consumes it.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">There are really only three kinds of ptoken, wildcards, fixed words, and
|
|
nonterminals, but it's fractionally quicker to differentiate the sorts of
|
|
wildcard here, so we'll actually divide them into five. The remaining wildcard,
|
|
the <code class="display"><span class="extract">......</span></code> form of <code class="display"><span class="extract">...</span></code>, is represented as <code class="display"><span class="extract">MULTIPLE_WILDCARD_PTC</span></code> but with
|
|
the <code class="display"><span class="extract">balanced_wildcard</span></code> flag set.
|
|
</p>
|
|
|
|
|
|
<pre class="definitions">
|
|
<span class="definitionkeyword">define</span> <span class="constant">SINGLE_WILDCARD_PTC</span><span class="plain"> 1</span>
|
|
<span class="definitionkeyword">define</span> <span class="constant">MULTIPLE_WILDCARD_PTC</span><span class="plain"> 2</span>
|
|
<span class="definitionkeyword">define</span> <span class="constant">POSSIBLY_EMPTY_WILDCARD_PTC</span><span class="plain"> 3</span>
|
|
<span class="definitionkeyword">define</span> <span class="constant">FIXED_WORD_PTC</span><span class="plain"> 4</span>
|
|
<span class="definitionkeyword">define</span> <span class="constant">NONTERMINAL_PTC</span><span class="plain"> 5</span>
|
|
</pre>
|
|
|
|
<pre class="display">
|
|
<span class="reserved">typedef</span><span class="plain"> </span><span class="reserved">struct</span><span class="plain"> </span><span class="reserved">ptoken</span><span class="plain"> {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">ptoken_category</span><span class="plain">; </span> <span class="comment">one of the <code class="display"><span class="extract">*_PTC</span></code> values</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">negated_ptoken</span><span class="plain">; </span> <span class="comment">the <code class="display"><span class="extract">^</span></code> modifier applies</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">disallow_unexpected_upper</span><span class="plain">; </span> <span class="comment">the <code class="display"><span class="extract">_</span></code> modifier applies</span>
|
|
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt_pt</span><span class="plain">; </span> <span class="comment">for <code class="display"><span class="extract">NONTERMINAL_PTC</span></code> ptokens</span>
|
|
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">ve_pt</span><span class="plain">; </span> <span class="comment">for <code class="display"><span class="extract">FIXED_WORD_PTC</span></code> ptokens</span>
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">alternative_ptoken</span><span class="plain">; </span> <span class="comment">linked list of other vocabulary ptokens</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">balanced_wildcard</span><span class="plain">; </span> <span class="comment">for <code class="display"><span class="extract">MULTIPLE_WILDCARD_PTC</span></code> ptokens: brackets balanced?</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">result_index</span><span class="plain">; </span> <span class="comment">for <code class="display"><span class="extract">NONTERMINAL_PTC</span></code> ptokens: what result number, counting from 1?</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">range_starts</span><span class="plain">; </span> <span class="comment">1, 2, 3, ... if word range 1, 2, 3, ... starts with this</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">range_ends</span><span class="plain">; </span> <span class="comment">1, 2, 3, ... if word range 1, 2, 3, ... ends with this</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">ptoken_position</span><span class="plain">; </span> <span class="comment">fixed position in range: 1, 2, ... for left, -1, -2, ... for right</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">strut_number</span><span class="plain">; </span> <span class="comment">if this is part of a strut, what number? or -1 if not</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">ptoken_is_fast</span><span class="plain">; </span> <span class="comment">can be checked in the fast pass of the parser</span>
|
|
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">range_requirement</span><span class="plain"> </span><span class="identifier">token_req</span><span class="plain">;</span>
|
|
|
|
<span class="reserved">struct</span><span class="plain"> </span><span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">next_ptoken</span><span class="plain">; </span> <span class="comment">within its production list</span>
|
|
<span class="identifier">MEMORY_MANAGEMENT</span>
|
|
<span class="plain">} </span><span class="reserved">ptoken</span><span class="plain">;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The structure ptoken is private to this section.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP14"></a><b>§14. </b>The parser records the result of the most recently matched nonterminal in the
|
|
following global variables:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">most_recent_result</span><span class="plain"> = 0; </span> <span class="comment">this is the variable which <code class="display"><span class="extract">inweb</span></code> writes <code class="display"><span class="extract"><<r>></span></code></span>
|
|
<span class="reserved">void</span><span class="plain"> *</span><span class="identifier">most_recent_result_p</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">; </span> <span class="comment">this is the variable which <code class="display"><span class="extract">inweb</span></code> writes <code class="display"><span class="extract"><<rp>></span></code></span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="inwebparagraph"><a id="SP15"></a><b>§15. </b>Preform's aim is to purge the Inform source code of all English vocabulary,
|
|
but we do still the letters "K" and "L", to define the wording of kind constructors.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">CAPITAL_K_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">CAPITAL_L_V</span><span class="plain">;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="inwebparagraph"><a id="SP16"></a><b>§16. </b>Preform can run in an instrumented mode, which collects statistics on the
|
|
usage of syntax it sees, but there's a performance hit for this. So it's
|
|
enabled only if the constant <code class="display"><span class="extract">INSTRUMENTED_PREFORM</span></code> defined to <code class="display"><span class="extract">TRUE</span></code>: here's
|
|
where to do it.
|
|
</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP17"></a><b>§17. </b></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">typedef</span><span class="plain"> </span><span class="reserved">struct</span><span class="plain"> </span><span class="reserved">range_requirement</span><span class="plain"> {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">no_requirements</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">ditto_flag</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">DW_req</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">DS_req</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">CW_req</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">CS_req</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">FW_req</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">FS_req</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">range_requirement</span><span class="plain">;</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">no_req_bits</span><span class="plain"> = 0;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The structure range_requirement is private to this section.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP18"></a><b>§18. Logging. </b>Descending these wheels within wheels:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::log_language</span><span class="plain">(</span><span class="reserved">void</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">detailed</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">;</span>
|
|
<span class="identifier">LOOP_OVER</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain">) {</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">INSTRUMENTED_PREFORM</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"%d/%d: "</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_matches</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_tries</span><span class="plain">);</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"%V: "</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_id</span><span class="plain">);</span>
|
|
<span class="functiontext">Preform::log_range_requirement</span><span class="plain">(&(</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req</span><span class="plain">));</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"\</span><span class="plain">n</span><span class="string">"</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>internal_definition</span><span class="plain">) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">" (internal)\</span><span class="plain">n</span><span class="string">"</span><span class="plain">);</span>
|
|
<span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="reserved">production_list</span><span class="plain"> *</span><span class="identifier">pl</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>next_production_list</span><span class="plain">) {</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">" $J:\</span><span class="plain">n</span><span class="string">"</span><span class="plain">, </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>definition_language</span><span class="plain">);</span>
|
|
<span class="reserved">production</span><span class="plain"> *</span><span class="identifier">pr</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>first_production</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>next_production</span><span class="plain">) {</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">" "</span><span class="plain">); </span><span class="functiontext">Preform::log_production</span><span class="plain">(</span><span class="identifier">pr</span><span class="plain">, </span><span class="identifier">detailed</span><span class="plain">);</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">INSTRUMENTED_PREFORM</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">" %d/%d: "</span><span class="plain">, </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_matches</span><span class="plain">, </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_tries</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Wordings::nonempty</span><span class="plain">(</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>sample_text</span><span class="plain">)) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"<%W>"</span><span class="plain">, </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>sample_text</span><span class="plain">);</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">" ==> "</span><span class="plain">);</span>
|
|
<span class="functiontext">Preform::log_range_requirement</span><span class="plain">(&(</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_req</span><span class="plain">));</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"\</span><span class="plain">n</span><span class="string">"</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">" min %d, max %d\</span><span class="plain">n</span><span class="string">\</span><span class="plain">n</span><span class="string">"</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>min_nt_words</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"%d req bits.\</span><span class="plain">n</span><span class="string">"</span><span class="plain">, </span><span class="identifier">no_req_bits</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::log_language appears nowhere else.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP19"></a><b>§19. </b></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::log_production</span><span class="plain">(</span><span class="reserved">production</span><span class="plain"> *</span><span class="identifier">pr</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">detailed</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>first_ptoken</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"<empty-production>"</span><span class="plain">);</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>first_ptoken</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">) {</span>
|
|
<span class="functiontext">Preform::log_ptoken</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">, </span><span class="identifier">detailed</span><span class="plain">);</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">" "</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::log_production is used in <a href="#SP18">§18</a>, <a href="#SP51_2">§51.2</a>, <a href="#SP51_2_1">§51.2.1</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP20"></a><b>§20. </b></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::log_ptoken</span><span class="plain">(</span><span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">detailed</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">detailed</span><span class="plain">) && (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_position</span><span class="plain"> != 0)) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"(@%d)"</span><span class="plain">, </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_position</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">detailed</span><span class="plain">) && (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>strut_number</span><span class="plain"> >= 0)) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"(S%d)"</span><span class="plain">, </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>strut_number</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>disallow_unexpected_upper</span><span class="plain">) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"_"</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain">) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"^"</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_starts</span><span class="plain"> >= 0) { </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"{"</span><span class="plain">); </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">detailed</span><span class="plain">) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"%d:"</span><span class="plain">, </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_starts</span><span class="plain">); }</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">alt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">alt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">alt</span><span class="plain">; </span><span class="identifier">alt</span><span class="plain"> = </span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>alternative_ptoken</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">) {</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"%V"</span><span class="plain">, </span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>nonterminal_id</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">detailed</span><span class="plain">) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"=%d"</span><span class="plain">, </span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>result_index</span><span class="plain">);</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"%V"</span><span class="plain">, </span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>ve_pt</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>alternative_ptoken</span><span class="plain">) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"/"</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_ends</span><span class="plain"> >= 0) { </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">detailed</span><span class="plain">) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">":%d"</span><span class="plain">, </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_ends</span><span class="plain">); </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"}"</span><span class="plain">); }</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::log_ptoken is used in <a href="#SP19">§19</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP21"></a><b>§21. </b>A less detailed form used in linguistic problem messages:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::write_ptoken</span><span class="plain">(</span><span class="identifier">OUTPUT_STREAM</span><span class="plain">, </span><span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>disallow_unexpected_upper</span><span class="plain">) </span><span class="identifier">WRITE</span><span class="plain">(</span><span class="string">"_"</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain">) </span><span class="identifier">WRITE</span><span class="plain">(</span><span class="string">"^"</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_starts</span><span class="plain"> >= 0) </span><span class="identifier">WRITE</span><span class="plain">(</span><span class="string">"{"</span><span class="plain">);</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">alt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">alt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">alt</span><span class="plain">; </span><span class="identifier">alt</span><span class="plain"> = </span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>alternative_ptoken</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">) {</span>
|
|
<span class="identifier">WRITE</span><span class="plain">(</span><span class="string">"%V"</span><span class="plain">, </span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>nonterminal_id</span><span class="plain">);</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="identifier">WRITE</span><span class="plain">(</span><span class="string">"%V"</span><span class="plain">, </span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>ve_pt</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>alternative_ptoken</span><span class="plain">) </span><span class="identifier">WRITE</span><span class="plain">(</span><span class="string">"/"</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_ends</span><span class="plain"> >= 0) </span><span class="identifier">WRITE</span><span class="plain">(</span><span class="string">"}"</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::write_ptoken appears nowhere else.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP22"></a><b>§22. </b>This is a typical internal nonterminal being defined. It's used only to parse
|
|
inclusion requests for the debugging log. Note that we use the "1" to signal
|
|
that a correct match must have exactly one word.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="plain"><</span><span class="identifier">preform</span><span class="plain">-</span><span class="reserved">nonterminal</span><span class="plain">> </span><span class="identifier">internal</span><span class="plain"> 1 {</span>
|
|
<span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain"> = </span><span class="functiontext">Preform::detect_nonterminal</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)));</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">) { *</span><span class="identifier">XP</span><span class="plain"> = </span><span class="identifier">nt</span><span class="plain">; </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">; }</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="inwebparagraph"><a id="SP23"></a><b>§23. </b>To use which, the debugging log code needs:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::watch</span><span class="plain">(</span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">state</span><span class="plain">) {</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>watched</span><span class="plain"> = </span><span class="identifier">state</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::watch appears nowhere else.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP24"></a><b>§24. Building grammar. </b>So, to begin. Since we can't use Preform to parse Preform, we have to define
|
|
its syntactic tokens by hand:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">AMPERSAND_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">BACKSLASH_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">CARET_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">COLONCOLONEQUALS_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">QUESTIONMARK_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">QUOTEQUOTE_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">SIXDOTS_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">THREEASTERISKS_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">THREEDOTS_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">THREEHASHES_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">UNDERSCORE_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">language_V</span><span class="plain">;</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">internal_V</span><span class="plain">;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="inwebparagraph"><a id="SP25"></a><b>§25. </b>And off we go.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::begin</span><span class="plain">(</span><span class="reserved">void</span><span class="plain">) {</span>
|
|
<span class="identifier">CAPITAL_K_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"k"</span><span class="plain">);</span>
|
|
<span class="identifier">CAPITAL_L_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"l"</span><span class="plain">);</span>
|
|
|
|
<<span class="cwebmacro">Register the internal and source-code-referred-to nonterminals</span> <span class="cwebmacronumber">25.1</span>><span class="plain">;</span>
|
|
|
|
<span class="identifier">AMPERSAND_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"&"</span><span class="plain">);</span>
|
|
<span class="identifier">BACKSLASH_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"\</span><span class="plain">\</span><span class="string">"</span><span class="plain">);</span>
|
|
<span class="identifier">CARET_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"^"</span><span class="plain">);</span>
|
|
<span class="identifier">COLONCOLONEQUALS_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">":"</span><span class="plain"> </span><span class="string">":="</span><span class="plain">);</span>
|
|
<span class="identifier">QUESTIONMARK_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"?"</span><span class="plain">);</span>
|
|
<span class="identifier">QUOTEQUOTE_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"\</span><span class="plain">"</span><span class="string">\</span><span class="plain">"</span><span class="string">"</span><span class="plain">);</span>
|
|
<span class="identifier">SIXDOTS_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"......"</span><span class="plain">);</span>
|
|
<span class="identifier">THREEASTERISKS_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"***"</span><span class="plain">);</span>
|
|
<span class="identifier">THREEDOTS_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"..."</span><span class="plain">);</span>
|
|
<span class="identifier">THREEHASHES_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"###"</span><span class="plain">);</span>
|
|
<span class="identifier">UNDERSCORE_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"_"</span><span class="plain">);</span>
|
|
<span class="identifier">language_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"language"</span><span class="plain">);</span>
|
|
<span class="identifier">internal_V</span><span class="plain"> = </span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">L</span><span class="string">"internal"</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::begin is used in 1/wm (<a href="1-wm.html#SP3">§3</a>).</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP25_1"></a><b>§25.1. </b>The tangler of <code class="display"><span class="extract">inweb</span></code> replaces the <code class="display"><span class="extract">[[nonterminals]]</span></code> below with
|
|
invocations of the <code class="display"><span class="extract">REGISTER_NONTERMINAL</span></code> and <code class="display"><span class="extract">INTERNAL_NONTERMINAL</span></code> macros.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Register the internal and source-code-referred-to nonterminals</span> <span class="cwebmacronumber">25.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="plain">[[</span><span class="identifier">nonterminals</span><span class="plain">]];</span>
|
|
<span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">;</span>
|
|
<span class="identifier">LOOP_OVER</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain">)</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>marked_internal</span><span class="plain">) && (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>internal_definition</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">))</span>
|
|
<span class="identifier">internal_error</span><span class="plain">(</span><span class="string">"internal undefined"</span><span class="plain">);</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP25">§25</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP26"></a><b>§26. </b>These macros connect nonterminals with their mentions in the Inform source
|
|
code, and with the compositor routines compiled for them by <code class="display"><span class="extract">inweb</span></code>. It invokes
|
|
<code class="display"><span class="extract">REGISTER_NONTERMINAL</span></code> if it has compiled Preform productions for a nonterminal,
|
|
and compiled a compositor routine; the name of which is the nonterminal's name
|
|
with a <code class="display"><span class="extract">C</span></code> suffix. If it found an internal nonterminal, it invokes
|
|
<code class="display"><span class="extract">INTERNAL_NONTERMINAL</span></code>, and compiles a routine whose name has the suffix <code class="display"><span class="extract">R</span></code>
|
|
as the definition.
|
|
</p>
|
|
|
|
|
|
<pre class="definitions">
|
|
<span class="definitionkeyword">define</span> <span class="identifier">REGISTER_NONTERMINAL</span><span class="plain">(</span><span class="identifier">quotedname</span><span class="plain">, </span><span class="identifier">identifier</span><span class="plain">)</span>
|
|
<span class="identifier">identifier</span><span class="plain"> = </span><span class="functiontext">Preform::find_nonterminal</span><span class="plain">(</span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">quotedname</span><span class="plain">));</span>
|
|
<span class="identifier">identifier</span><span class="plain">-</span><span class="element">>result_compositor</span><span class="plain"> = </span><span class="identifier">identifier</span><span class="plain">##</span><span class="identifier">C</span><span class="plain">;</span>
|
|
<span class="definitionkeyword">define</span> <span class="identifier">INTERNAL_NONTERMINAL</span><span class="plain">(</span><span class="identifier">quotedname</span><span class="plain">, </span><span class="identifier">identifier</span><span class="plain">, </span><span class="identifier">min</span><span class="plain">, </span><span class="identifier">max</span><span class="plain">)</span>
|
|
<span class="identifier">identifier</span><span class="plain"> = </span><span class="functiontext">Preform::find_nonterminal</span><span class="plain">(</span><span class="functiontext">Vocabulary::entry_for_text</span><span class="plain">(</span><span class="identifier">quotedname</span><span class="plain">));</span>
|
|
<span class="identifier">identifier</span><span class="plain">-</span><span class="element">>min_nt_words</span><span class="plain"> = </span><span class="identifier">min</span><span class="plain">; </span><span class="identifier">identifier</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain"> = </span><span class="identifier">max</span><span class="plain">;</span>
|
|
<span class="identifier">identifier</span><span class="plain">-</span><span class="element">>internal_definition</span><span class="plain"> = </span><span class="identifier">identifier</span><span class="plain">##</span><span class="identifier">R</span><span class="plain">;</span>
|
|
<span class="identifier">identifier</span><span class="plain">-</span><span class="element">>marked_internal</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
</pre>
|
|
<p class="inwebparagraph"><a id="SP27"></a><b>§27. </b>Parsing Preform is exactly what Preform would do elegantly, but of course,
|
|
for chicken-and-egg reasons, we need to do the job by hand. Fortunately the
|
|
syntax is very simple.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::parse_preform</span><span class="plain">(</span><span class="reserved">wording</span><span class="plain"> </span><span class="identifier">W</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">break_first</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">break_first</span><span class="plain">) {</span>
|
|
<span class="identifier">TEMPORARY_TEXT</span><span class="plain">(</span><span class="identifier">wd</span><span class="plain">);</span>
|
|
<span class="identifier">WRITE_TO</span><span class="plain">(</span><span class="identifier">wd</span><span class="plain">, </span><span class="string">"%+W"</span><span class="plain">, </span><span class="functiontext">Wordings::one_word</span><span class="plain">(</span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)));</span>
|
|
<span class="identifier">W</span><span class="plain"> = </span><span class="functiontext">Feeds::feed_stream_punctuated</span><span class="plain">(</span><span class="identifier">wd</span><span class="plain">, </span><span class="constant">PREFORM_PUNCTUATION_MARKS</span><span class="plain">);</span>
|
|
<span class="identifier">DISCARD_TEXT</span><span class="plain">(</span><span class="identifier">wd</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">nonterminals_declared</span><span class="plain"> = 0;</span>
|
|
<span class="identifier">LOOP_THROUGH_WORDING</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">) == </span><span class="identifier">PARBREAK_V</span><span class="plain">) </span><span class="reserved">continue</span><span class="plain">;</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">PREFORM_LANGUAGE_FROM_NAME</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">) >= </span><span class="identifier">wn</span><span class="plain">+1) && (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">) == </span><span class="identifier">language_V</span><span class="plain">)) {</span>
|
|
<<span class="cwebmacro">Parse a definition language switch</span> <span class="cwebmacronumber">27.1</span>><span class="plain">;</span>
|
|
<span class="reserved">continue</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">) >= </span><span class="identifier">wn</span><span class="plain">+1) && (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">+1) == </span><span class="identifier">internal_V</span><span class="plain">)) {</span>
|
|
<<span class="cwebmacro">Parse an internal nonterminal declaration</span> <span class="cwebmacronumber">27.2</span>><span class="plain">;</span>
|
|
<span class="identifier">nonterminals_declared</span><span class="plain">++;</span>
|
|
<span class="reserved">continue</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">) >= </span><span class="identifier">wn</span><span class="plain">+2) && (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">+1) == </span><span class="identifier">COLONCOLONEQUALS_V</span><span class="plain">)) {</span>
|
|
<<span class="cwebmacro">Parse an external nonterminal declaration</span> <span class="cwebmacronumber">27.3</span>><span class="plain">;</span>
|
|
<span class="identifier">nonterminals_declared</span><span class="plain">++;</span>
|
|
<span class="reserved">continue</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">internal_error</span><span class="plain">(</span><span class="string">"language definition failed"</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="functiontext">Preform::optimise_counts</span><span class="plain">();</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">nonterminals_declared</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::parse_preform appears nowhere else.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP27_1"></a><b>§27.1. </b>We either switch to an existing natural language, or create a new one.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Parse a definition language switch</span> <span class="cwebmacronumber">27.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="identifier">TEMPORARY_TEXT</span><span class="plain">(</span><span class="identifier">lname</span><span class="plain">);</span>
|
|
<span class="identifier">WRITE_TO</span><span class="plain">(</span><span class="identifier">lname</span><span class="plain">, </span><span class="string">"%W"</span><span class="plain">, </span><span class="functiontext">Wordings::one_word</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">+1));</span>
|
|
<span class="identifier">PREFORM_LANGUAGE_TYPE</span><span class="plain"> *</span><span class="identifier">nl</span><span class="plain"> = </span><span class="identifier">PREFORM_LANGUAGE_FROM_NAME</span><span class="plain">(</span><span class="identifier">lname</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nl</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) {</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"Missing: %S\</span><span class="plain">n</span><span class="string">"</span><span class="plain">, </span><span class="identifier">lname</span><span class="plain">);</span>
|
|
<span class="identifier">internal_error</span><span class="plain">(</span><span class="string">"tried to define for missing language"</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">DISCARD_TEXT</span><span class="plain">(</span><span class="identifier">lname</span><span class="plain">);</span>
|
|
<span class="identifier">language_being_read_by_Preform</span><span class="plain"> = </span><span class="identifier">nl</span><span class="plain">;</span>
|
|
<span class="identifier">wn</span><span class="plain">++;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP27">§27</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP27_2"></a><b>§27.2. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Parse an internal nonterminal declaration</span> <span class="cwebmacronumber">27.2</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain"> = </span><span class="functiontext">Preform::find_nonterminal</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">));</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain">) </span><span class="identifier">internal_error</span><span class="plain">(</span><span class="string">"internal is defined"</span><span class="plain">);</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>marked_internal</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="identifier">wn</span><span class="plain">++;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP27">§27</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP27_3"></a><b>§27.3. </b>The declaration continues until the end of the text, or until we reach a
|
|
paragraph break. Internally, it's a list of productions divided by stroke symbols.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Parse an external nonterminal declaration</span> <span class="cwebmacronumber">27.3</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain"> = </span><span class="functiontext">Preform::find_nonterminal</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">));</span>
|
|
<span class="reserved">production_list</span><span class="plain"> *</span><span class="identifier">pl</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Find or create the production list for this language</span> <span class="cwebmacronumber">27.3.1</span>><span class="plain">;</span>
|
|
<span class="identifier">wn</span><span class="plain"> += 2;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">pc</span><span class="plain"> = 0;</span>
|
|
<span class="reserved">while</span><span class="plain"> (</span><span class="identifier">TRUE</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">x</span><span class="plain"> = </span><span class="identifier">wn</span><span class="plain">;</span>
|
|
<span class="reserved">while</span><span class="plain"> ((</span><span class="identifier">x</span><span class="plain"> <= </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)) && (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">x</span><span class="plain">) != </span><span class="identifier">STROKE_V</span><span class="plain">) && (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">x</span><span class="plain">) != </span><span class="identifier">PARBREAK_V</span><span class="plain">)) </span><span class="identifier">x</span><span class="plain">++;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">wn</span><span class="plain"> < </span><span class="identifier">x</span><span class="plain">) {</span>
|
|
<span class="reserved">production</span><span class="plain"> *</span><span class="identifier">pr</span><span class="plain"> = </span><span class="functiontext">Preform::new_production</span><span class="plain">(</span><span class="functiontext">Wordings::new</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">, </span><span class="identifier">x</span><span class="plain">-1), </span><span class="identifier">nt</span><span class="plain">, </span><span class="identifier">pc</span><span class="plain">++);</span>
|
|
<span class="identifier">wn</span><span class="plain"> = </span><span class="identifier">x</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Place the new production within the production list</span> <span class="cwebmacronumber">27.3.2</span>><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">wn</span><span class="plain"> > </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)) || (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">x</span><span class="plain">) == </span><span class="identifier">PARBREAK_V</span><span class="plain">)) </span><span class="reserved">break</span><span class="plain">; </span> <span class="comment">reached end</span>
|
|
<span class="identifier">wn</span><span class="plain">++; </span> <span class="comment">advance past the stroke and continue</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">wn</span><span class="plain">--;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP27">§27</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP27_3_1"></a><b>§27.3.1. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Find or create the production list for this language</span> <span class="cwebmacronumber">27.3.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>next_production_list</span><span class="plain">)</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>definition_language</span><span class="plain"> == </span><span class="identifier">language_being_read_by_Preform</span><span class="plain">)</span>
|
|
<span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pl</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) {</span>
|
|
<span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">CREATE</span><span class="plain">(</span><span class="reserved">production_list</span><span class="plain">);</span>
|
|
<span class="identifier">pl</span><span class="plain">-</span><span class="element">>definition_language</span><span class="plain"> = </span><span class="identifier">language_being_read_by_Preform</span><span class="plain">;</span>
|
|
<span class="identifier">pl</span><span class="plain">-</span><span class="element">>first_production</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="identifier">pl</span><span class="plain">-</span><span class="element">>as_avinue</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Place the new production list within the nonterminal</span> <span class="cwebmacronumber">27.3.1.1</span>><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP27_3">§27.3</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP27_3_1_1"></a><b>§27.3.1.1. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Place the new production list within the nonterminal</span> <span class="cwebmacronumber">27.3.1.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">;</span>
|
|
<span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="reserved">production_list</span><span class="plain"> *</span><span class="identifier">p</span><span class="plain"> = </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain">;</span>
|
|
<span class="reserved">while</span><span class="plain"> ((</span><span class="identifier">p</span><span class="plain">) && (</span><span class="identifier">p</span><span class="plain">-</span><span class="element">>next_production_list</span><span class="plain">)) </span><span class="identifier">p</span><span class="plain"> = </span><span class="identifier">p</span><span class="plain">-</span><span class="element">>next_production_list</span><span class="plain">;</span>
|
|
<span class="identifier">p</span><span class="plain">-</span><span class="element">>next_production_list</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP27_3_1">§27.3.1</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP27_3_2"></a><b>§27.3.2. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Place the new production within the production list</span> <span class="cwebmacronumber">27.3.2</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>first_production</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>first_production</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">;</span>
|
|
<span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="reserved">production</span><span class="plain"> *</span><span class="identifier">p</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>first_production</span><span class="plain">;</span>
|
|
<span class="reserved">while</span><span class="plain"> ((</span><span class="identifier">p</span><span class="plain">) && (</span><span class="identifier">p</span><span class="plain">-</span><span class="element">>next_production</span><span class="plain">)) </span><span class="identifier">p</span><span class="plain"> = </span><span class="identifier">p</span><span class="plain">-</span><span class="element">>next_production</span><span class="plain">;</span>
|
|
<span class="identifier">p</span><span class="plain">-</span><span class="element">>next_production</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP27_3">§27.3</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP28"></a><b>§28. </b>Nonterminals are identified by their name-words:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">nonterminal</span><span class="plain"> *</span><span class="functiontext">Preform::detect_nonterminal</span><span class="plain">(</span><span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">ve</span><span class="plain">) {</span>
|
|
<span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">;</span>
|
|
<span class="identifier">LOOP_OVER</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain">)</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ve</span><span class="plain"> == </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_id</span><span class="plain">)</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">nt</span><span class="plain">;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">nonterminal</span><span class="plain"> *</span><span class="functiontext">Preform::find_nonterminal</span><span class="plain">(</span><span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">ve</span><span class="plain">) {</span>
|
|
<span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain"> = </span><span class="functiontext">Preform::detect_nonterminal</span><span class="plain">(</span><span class="identifier">ve</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">nt</span><span class="plain">;</span>
|
|
|
|
<span class="identifier">nt</span><span class="plain"> = </span><span class="identifier">CREATE</span><span class="plain">(</span><span class="reserved">nonterminal</span><span class="plain">);</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_id</span><span class="plain"> = </span><span class="identifier">ve</span><span class="plain">;</span>
|
|
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>voracious</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>multiplicitous</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>optimised_in_this_pass</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>min_nt_words</span><span class="plain"> = 1; </span><span class="identifier">nt</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain"> = </span><span class="constant">INFINITE_WORD_COUNT</span><span class="plain">;</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nt_req_bit</span><span class="plain"> = -1;</span>
|
|
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>marked_internal</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>internal_definition</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>result_compositor</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>number_words_by_production</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>flag_words_in_production</span><span class="plain"> = 0;</span>
|
|
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">i</span><span class="plain">=0; </span><span class="identifier">i</span><span class="plain"><</span><span class="constant">MAX_RANGES_PER_PRODUCTION</span><span class="plain">; </span><span class="identifier">i</span><span class="plain">++)</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>range_result</span><span class="plain">[</span><span class="identifier">i</span><span class="plain">] = </span><span class="constant">EMPTY_WORDING</span><span class="plain">;</span>
|
|
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>watched</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_tries</span><span class="plain"> = 0; </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_matches</span><span class="plain"> = 0;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">nt</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::detect_nonterminal is used in <a href="#SP22">§22</a>.</p>
|
|
|
|
<p class="endnote">The function Preform::find_nonterminal is used in <a href="#SP26">§26</a>, <a href="#SP27_2">§27.2</a>, <a href="#SP27_3">§27.3</a>, <a href="#SP31">§31</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP29"></a><b>§29. </b>We now descend to the creation of productions for (external) nonterminals.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">production</span><span class="plain"> *</span><span class="functiontext">Preform::new_production</span><span class="plain">(</span><span class="reserved">wording</span><span class="plain"> </span><span class="identifier">W</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">pc</span><span class="plain">) {</span>
|
|
<span class="reserved">production</span><span class="plain"> *</span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">CREATE</span><span class="plain">(</span><span class="reserved">production</span><span class="plain">);</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>match_number</span><span class="plain"> = </span><span class="identifier">pc</span><span class="plain">;</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>next_production</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_ranges</span><span class="plain"> = 1; </span> <span class="comment">so that they count from 1; range 0 is unused</span>
|
|
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_struts</span><span class="plain"> = 0; </span> <span class="comment">they will be detected later</span>
|
|
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>min_pr_words</span><span class="plain"> = 1; </span><span class="identifier">pr</span><span class="plain">-></span><span class="identifier">max_pr_words</span><span class="plain"> = </span><span class="constant">INFINITE_WORD_COUNT</span><span class="plain">;</span>
|
|
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_tries</span><span class="plain"> = 0; </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_matches</span><span class="plain"> = 0;</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>sample_text</span><span class="plain"> = </span><span class="constant">EMPTY_WORDING</span><span class="plain">;</span>
|
|
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">head</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">, *</span><span class="identifier">tail</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Parse the row of production tokens into a linked list of ptokens</span> <span class="cwebmacronumber">29.1</span>><span class="plain">;</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>first_ptoken</span><span class="plain"> = </span><span class="identifier">head</span><span class="plain">;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">pr</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::new_production is used in <a href="#SP27_3">§27.3</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP29_1"></a><b>§29.1. </b></p>
|
|
|
|
|
|
<pre class="definitions">
|
|
<span class="definitionkeyword">define</span> <span class="constant">OUTSIDE_PTBRACE</span><span class="plain"> 0</span>
|
|
<span class="definitionkeyword">define</span> <span class="constant">ABOUT_TO_OPEN_PTBRACE</span><span class="plain"> 1</span>
|
|
<span class="definitionkeyword">define</span> <span class="constant">INSIDE_PTBRACE</span><span class="plain"> 2</span>
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Parse the row of production tokens into a linked list of ptokens</span> <span class="cwebmacronumber">29.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">result_count</span><span class="plain"> = 1;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">negation_modifier</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">, </span><span class="identifier">lower_case_modifier</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">unescaped</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">bracing_mode</span><span class="plain"> = </span><span class="constant">OUTSIDE_PTBRACE</span><span class="plain">;</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">bracing_begins_at</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">tc</span><span class="plain"> = 0;</span>
|
|
<span class="identifier">LOOP_THROUGH_WORDING</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">unescaped</span><span class="plain">) </span><<span class="cwebmacro">Parse the token modifier symbols</span> <span class="cwebmacronumber">29.1.1</span>><span class="plain">;</span>
|
|
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain"> = </span><span class="functiontext">Preform::parse_slashed_chain</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">, </span><span class="identifier">pr</span><span class="plain">, </span><span class="identifier">i</span><span class="plain">, </span><span class="identifier">unescaped</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) </span><span class="reserved">continue</span><span class="plain">; </span> <span class="comment">we have set the production match number instead</span>
|
|
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> == </span><span class="constant">NONTERMINAL_PTC</span><span class="plain">) </span><<span class="cwebmacro">Assign the ptoken a result number</span> <span class="cwebmacronumber">29.1.3</span>><span class="plain">;</span>
|
|
|
|
<<span class="cwebmacro">Modify the new token according to the current token modifier settings</span> <span class="cwebmacronumber">29.1.2</span>><span class="plain">;</span>
|
|
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">tc</span><span class="plain">++ < </span><span class="constant">MAX_PTOKENS_PER_PRODUCTION</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">head</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) </span><span class="identifier">head</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">; </span><span class="reserved">else</span><span class="plain"> </span><span class="identifier">tail</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="identifier">tail</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP29">§29</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP29_1_1"></a><b>§29.1.1. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Parse the token modifier symbols</span> <span class="cwebmacronumber">29.1.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">) == </span><span class="identifier">CARET_V</span><span class="plain">) { </span><span class="identifier">negation_modifier</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">; </span><span class="reserved">continue</span><span class="plain">; }</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">) == </span><span class="identifier">UNDERSCORE_V</span><span class="plain">) { </span><span class="identifier">lower_case_modifier</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">; </span><span class="reserved">continue</span><span class="plain">; }</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">) == </span><span class="identifier">BACKSLASH_V</span><span class="plain">) { </span><span class="identifier">unescaped</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">; </span><span class="reserved">continue</span><span class="plain">; }</span>
|
|
<span class="reserved">switch</span><span class="plain"> (</span><span class="identifier">bracing_mode</span><span class="plain">) {</span>
|
|
<span class="reserved">case</span><span class="plain"> </span><span class="constant">OUTSIDE_PTBRACE</span><span class="plain">:</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">) == </span><span class="identifier">OPENBRACE_V</span><span class="plain">) {</span>
|
|
<span class="identifier">bracing_mode</span><span class="plain"> = </span><span class="constant">ABOUT_TO_OPEN_PTBRACE</span><span class="plain">; </span><span class="reserved">continue</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="reserved">case</span><span class="plain"> </span><span class="constant">INSIDE_PTBRACE</span><span class="plain">:</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">) == </span><span class="identifier">CLOSEBRACE_V</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">bracing_begins_at</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">rnum</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_ranges</span><span class="plain">++;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">i</span><span class="plain">+2 <= </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)) && (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">+1) == </span><span class="identifier">QUESTIONMARK_V</span><span class="plain">) &&</span>
|
|
<span class="plain">(</span><span class="functiontext">Vocabulary::test_flags</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">+2, </span><span class="constant">NUMBER_MC</span><span class="plain">))) {</span>
|
|
<span class="identifier">rnum</span><span class="plain"> = </span><span class="functiontext">Vocabulary::get_literal_number_value</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">+2));</span>
|
|
<span class="identifier">i</span><span class="plain"> += 2;</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">bracing_begins_at</span><span class="plain">-</span><span class="element">>range_starts</span><span class="plain"> = </span><span class="identifier">rnum</span><span class="plain">;</span>
|
|
<span class="identifier">tail</span><span class="plain">-</span><span class="element">>range_ends</span><span class="plain"> = </span><span class="identifier">rnum</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">bracing_mode</span><span class="plain"> = </span><span class="constant">OUTSIDE_PTBRACE</span><span class="plain">; </span><span class="identifier">bracing_begins_at</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">; </span><span class="reserved">continue</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP29_1">§29.1</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP29_1_2"></a><b>§29.1.2. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Modify the new token according to the current token modifier settings</span> <span class="cwebmacronumber">29.1.2</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">negation_modifier</span><span class="plain">) </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">lower_case_modifier</span><span class="plain">) </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>disallow_unexpected_upper</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
|
|
<span class="identifier">unescaped</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="identifier">negation_modifier</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="identifier">lower_case_modifier</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
|
|
<span class="reserved">switch</span><span class="plain"> (</span><span class="identifier">bracing_mode</span><span class="plain">) {</span>
|
|
<span class="reserved">case</span><span class="plain"> </span><span class="constant">OUTSIDE_PTBRACE</span><span class="plain">:</span>
|
|
<span class="reserved">if</span><span class="plain"> (((</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> == </span><span class="constant">SINGLE_WILDCARD_PTC</span><span class="plain">) ||</span>
|
|
<span class="plain">(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> == </span><span class="constant">MULTIPLE_WILDCARD_PTC</span><span class="plain">) ||</span>
|
|
<span class="plain">(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> == </span><span class="constant">POSSIBLY_EMPTY_WILDCARD_PTC</span><span class="plain">))</span>
|
|
<span class="plain">&& (</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_ranges</span><span class="plain"> < </span><span class="constant">MAX_RANGES_PER_PRODUCTION</span><span class="plain">)) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">rnum</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_ranges</span><span class="plain">++;</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_starts</span><span class="plain"> = </span><span class="identifier">rnum</span><span class="plain">;</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_ends</span><span class="plain"> = </span><span class="identifier">rnum</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="reserved">case</span><span class="plain"> </span><span class="constant">ABOUT_TO_OPEN_PTBRACE</span><span class="plain">:</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_ranges</span><span class="plain"> < </span><span class="constant">MAX_RANGES_PER_PRODUCTION</span><span class="plain">)</span>
|
|
<span class="identifier">bracing_begins_at</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="identifier">bracing_mode</span><span class="plain"> = </span><span class="constant">INSIDE_PTBRACE</span><span class="plain">;</span>
|
|
<span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP29_1">§29.1</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP29_1_3"></a><b>§29.1.3. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Assign the ptoken a result number</span> <span class="cwebmacronumber">29.1.3</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">result_count</span><span class="plain"> < </span><span class="constant">MAX_RESULTS_PER_PRODUCTION</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">i</span><span class="plain">+2 <= </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)) && (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">+1) == </span><span class="identifier">QUESTIONMARK_V</span><span class="plain">) &&</span>
|
|
<span class="plain">(</span><span class="functiontext">Vocabulary::test_flags</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">+2, </span><span class="constant">NUMBER_MC</span><span class="plain">))) {</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>result_index</span><span class="plain"> = </span><span class="functiontext">Vocabulary::get_literal_number_value</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">+2));</span>
|
|
<span class="identifier">i</span><span class="plain"> += 2;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>result_index</span><span class="plain"> = </span><span class="identifier">result_count</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">result_count</span><span class="plain">++;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP29_1">§29.1</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP30"></a><b>§30. </b></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="functiontext">Preform::parse_slashed_chain</span><span class="plain">(</span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">production</span><span class="plain"> *</span><span class="identifier">pr</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">wn</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">unescaped</span><span class="plain">) {</span>
|
|
<span class="reserved">wording</span><span class="plain"> </span><span class="identifier">AW</span><span class="plain"> = </span><span class="functiontext">Wordings::one_word</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">);</span>
|
|
<<span class="cwebmacro">Expand the word range if the token text is slashed</span> <span class="cwebmacronumber">30.1</span>><span class="plain">;</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Parse the word range into a linked list of alternative ptokens</span> <span class="cwebmacronumber">30.2</span>><span class="plain">;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::parse_slashed_chain is used in <a href="#SP29_1">§29.1</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP30_1"></a><b>§30.1. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Expand the word range if the token text is slashed</span> <span class="cwebmacronumber">30.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="identifier">wchar_t</span><span class="plain"> *</span><span class="identifier">p</span><span class="plain"> = </span><span class="functiontext">Lexer::word_raw_text</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">);</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">k</span><span class="plain">, </span><span class="identifier">breakme</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">unescaped</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">p</span><span class="plain">[0] == </span><span class="character">'/'</span><span class="plain">) && (</span><span class="identifier">islower</span><span class="plain">(</span><span class="identifier">p</span><span class="plain">[1])) && (</span><span class="identifier">p</span><span class="plain">[2] == </span><span class="character">'/'</span><span class="plain">) && (</span><span class="identifier">p</span><span class="plain">[3] == 0)) {</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>match_number</span><span class="plain"> = </span><span class="identifier">p</span><span class="plain">[1] - </span><span class="character">'a'</span><span class="plain">;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">p</span><span class="plain">[0] == </span><span class="character">'/'</span><span class="plain">) && (</span><span class="identifier">islower</span><span class="plain">(</span><span class="identifier">p</span><span class="plain">[1])) && (</span><span class="identifier">p</span><span class="plain">[2] == </span><span class="identifier">p</span><span class="plain">[1]) && (</span><span class="identifier">p</span><span class="plain">[3] == </span><span class="character">'/'</span><span class="plain">) && (</span><span class="identifier">p</span><span class="plain">[4] == 0)) {</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>match_number</span><span class="plain"> = </span><span class="identifier">p</span><span class="plain">[1] - </span><span class="character">'a'</span><span class="plain"> + 26;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">k</span><span class="plain">=0; (</span><span class="identifier">p</span><span class="plain">[</span><span class="identifier">k</span><span class="plain">]) && (</span><span class="identifier">p</span><span class="plain">[</span><span class="identifier">k</span><span class="plain">+1]); </span><span class="identifier">k</span><span class="plain">++)</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">k</span><span class="plain"> > 0) && (</span><span class="identifier">p</span><span class="plain">[</span><span class="identifier">k</span><span class="plain">] == </span><span class="character">'/'</span><span class="plain">))</span>
|
|
<span class="identifier">breakme</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">breakme</span><span class="plain">) </span><span class="identifier">AW</span><span class="plain"> = </span><span class="functiontext">Feeds::feed_text_full</span><span class="plain">(</span><span class="identifier">p</span><span class="plain">, </span><span class="identifier">FALSE</span><span class="plain">, </span><span class="identifier">L</span><span class="string">"/"</span><span class="plain">); </span> <span class="comment">break only at slashes</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP30">§30</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP30_2"></a><b>§30.2. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Parse the word range into a linked list of alternative ptokens</span> <span class="cwebmacronumber">30.2</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">alt</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (; </span><span class="functiontext">Wordings::nonempty</span><span class="plain">(</span><span class="identifier">AW</span><span class="plain">); </span><span class="identifier">AW</span><span class="plain"> = </span><span class="functiontext">Wordings::trim_first_word</span><span class="plain">(</span><span class="identifier">AW</span><span class="plain">))</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">AW</span><span class="plain">)) != </span><span class="identifier">FORWARDSLASH_V</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">mode</span><span class="plain"> = </span><span class="identifier">unescaped</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Wordings::length</span><span class="plain">(</span><span class="identifier">AW</span><span class="plain">) > 1) </span><span class="identifier">mode</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">latest</span><span class="plain"> = </span><span class="functiontext">Preform::new_ptoken</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">AW</span><span class="plain">)), </span><span class="identifier">mode</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">, </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>match_number</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">alt</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) </span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">latest</span><span class="plain">;</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>alternative_ptoken</span><span class="plain"> = </span><span class="identifier">latest</span><span class="plain">;</span>
|
|
<span class="identifier">alt</span><span class="plain"> = </span><span class="identifier">latest</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP30">§30</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP31"></a><b>§31. </b>So we come to the end of the trail: the code to create a single ptoken.
|
|
In "escaped" mode, where a backslash has made the text literal, it just
|
|
becomes a fixed word; otherwise it could be any of the five categories.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">If the text refers to a nonterminal which doesn't yet exist, then this
|
|
creates it; that's how we deal with forward references.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="functiontext">Preform::new_ptoken</span><span class="plain">(</span><span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">ve</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">unescaped</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">pc</span><span class="plain">) {</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">CREATE</span><span class="plain">(</span><span class="reserved">ptoken</span><span class="plain">);</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>alternative_ptoken</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>disallow_unexpected_upper</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>result_index</span><span class="plain"> = 1;</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_starts</span><span class="plain"> = -1; </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_ends</span><span class="plain"> = -1;</span>
|
|
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_position</span><span class="plain"> = 0;</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>strut_number</span><span class="plain"> = -1;</span>
|
|
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>ve_pt</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>balanced_wildcard</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_is_fast</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
|
|
<span class="identifier">wchar_t</span><span class="plain"> *</span><span class="identifier">p</span><span class="plain"> = </span><span class="functiontext">Vocabulary::get_exemplar</span><span class="plain">(</span><span class="identifier">ve</span><span class="plain">, </span><span class="identifier">FALSE</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">unescaped</span><span class="plain">) && (</span><span class="identifier">p</span><span class="plain">) && (</span><span class="identifier">p</span><span class="plain">[0] == </span><span class="character">'<'</span><span class="plain">) && (</span><span class="identifier">p</span><span class="plain">[</span><span class="identifier">Wide::len</span><span class="plain">(</span><span class="identifier">p</span><span class="plain">)-1] == </span><span class="character">'>'</span><span class="plain">)) {</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain"> = </span><span class="functiontext">Preform::find_nonterminal</span><span class="plain">(</span><span class="identifier">ve</span><span class="plain">);</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> = </span><span class="constant">NONTERMINAL_PTC</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>ve_pt</span><span class="plain"> = </span><span class="identifier">ve</span><span class="plain">;</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> = </span><span class="constant">FIXED_WORD_PTC</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">unescaped</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ve</span><span class="plain"> == </span><span class="identifier">SIXDOTS_V</span><span class="plain">) {</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> = </span><span class="constant">MULTIPLE_WILDCARD_PTC</span><span class="plain">;</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>balanced_wildcard</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ve</span><span class="plain"> == </span><span class="identifier">THREEDOTS_V</span><span class="plain">) </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> = </span><span class="constant">MULTIPLE_WILDCARD_PTC</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ve</span><span class="plain"> == </span><span class="identifier">THREEHASHES_V</span><span class="plain">) </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> = </span><span class="constant">SINGLE_WILDCARD_PTC</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ve</span><span class="plain"> == </span><span class="identifier">THREEASTERISKS_V</span><span class="plain">) </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> = </span><span class="constant">POSSIBLY_EMPTY_WILDCARD_PTC</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> == </span><span class="constant">FIXED_WORD_PTC</span><span class="plain">) {</span>
|
|
<span class="identifier">ve</span><span class="plain">-</span><span class="element">>flags</span><span class="plain"> |= (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>flag_words_in_production</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>number_words_by_production</span><span class="plain">) </span><span class="identifier">ve</span><span class="plain">-</span><span class="element">>literal_number_value</span><span class="plain"> = </span><span class="identifier">pc</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::new_ptoken is used in <a href="#SP30_2">§30.2</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP32"></a><b>§32. Optimisation calculations. </b>After each round of fresh Preform grammar, we need to recalculate the various
|
|
maximum and minimum lengths, struts, and so on, because those all depend on
|
|
knowing the length of text a token will match, and new grammar may have
|
|
changed that.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">first_round_of_nt_optimisation_made</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::optimise_counts</span><span class="plain">(</span><span class="reserved">void</span><span class="plain">) {</span>
|
|
<span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">;</span>
|
|
<span class="identifier">LOOP_OVER</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain">) {</span>
|
|
<span class="functiontext">Preform::clear_rreq</span><span class="plain">(&(</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req</span><span class="plain">));</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>marked_internal</span><span class="plain">) {</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>optimised_in_this_pass</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>optimised_in_this_pass</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>min_nt_words</span><span class="plain"> = 1; </span><span class="identifier">nt</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain"> = </span><span class="constant">INFINITE_WORD_COUNT</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">first_round_of_nt_optimisation_made</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">) {</span>
|
|
<span class="identifier">first_round_of_nt_optimisation_made</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">LINGUISTICS_MODULE</span>
|
|
<span class="identifier">LinguisticsModule::preform_optimiser</span><span class="plain">();</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">PREFORM_OPTIMISER</span>
|
|
<span class="identifier">PREFORM_OPTIMISER</span><span class="plain">();</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">LOOP_OVER</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain">) </span><span class="functiontext">Preform::optimise_nt</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">);</span>
|
|
<span class="identifier">LOOP_OVER</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain">) </span><span class="functiontext">Preform::optimise_nt_reqs</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::optimise_nt</span><span class="plain">(</span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>optimised_in_this_pass</span><span class="plain">) </span><span class="reserved">return</span><span class="plain">;</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>optimised_in_this_pass</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Compute the minimum and maximum match lengths</span> <span class="cwebmacronumber">32.1</span>><span class="plain">;</span>
|
|
|
|
<span class="reserved">production_list</span><span class="plain"> *</span><span class="identifier">pl</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>next_production_list</span><span class="plain">) {</span>
|
|
<span class="reserved">production</span><span class="plain"> *</span><span class="identifier">pr</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>first_production</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>next_production</span><span class="plain">) {</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">last</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">; </span> <span class="comment">this will point to the last ptoken in the production</span>
|
|
<<span class="cwebmacro">Compute front-end ptoken positions</span> <span class="cwebmacronumber">32.2</span>><span class="plain">;</span>
|
|
<<span class="cwebmacro">Compute back-end ptoken positions</span> <span class="cwebmacronumber">32.3</span>><span class="plain">;</span>
|
|
<<span class="cwebmacro">Compute struts within the production</span> <span class="cwebmacronumber">32.4</span>><span class="plain">;</span>
|
|
<<span class="cwebmacro">Work out which ptokens are fast</span> <span class="cwebmacronumber">32.5</span>><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<<span class="cwebmacro">Mark the vocabulary's incidence list with this nonterminal</span> <span class="cwebmacronumber">32.6</span>><span class="character">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::optimise_counts is used in <a href="#SP27">§27</a>.</p>
|
|
|
|
<p class="endnote">The function Preform::optimise_nt is used in <a href="#SP32_6">§32.6</a>, <a href="#SP49">§49</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP32_1"></a><b>§32.1. </b>The minimum matched text length for a nonterminal is the smallest of the
|
|
minima for its possible productions; for a production, it's the sum of the
|
|
minimum match lengths of its tokens.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Compute the minimum and maximum match lengths</span> <span class="cwebmacronumber">32.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">min</span><span class="plain"> = -1, </span><span class="identifier">max</span><span class="plain"> = -1;</span>
|
|
<span class="reserved">production_list</span><span class="plain"> *</span><span class="identifier">pl</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>next_production_list</span><span class="plain">) {</span>
|
|
<span class="reserved">production</span><span class="plain"> *</span><span class="identifier">pr</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>first_production</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>next_production</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">min_p</span><span class="plain"> = 0, </span><span class="identifier">max_p</span><span class="plain"> = 0;</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>first_ptoken</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">min_t</span><span class="plain">, </span><span class="identifier">max_t</span><span class="plain">;</span>
|
|
<span class="functiontext">Preform::ptoken_extrema</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">, &</span><span class="identifier">min_t</span><span class="plain">, &</span><span class="identifier">max_t</span><span class="plain">);</span>
|
|
<span class="identifier">min_p</span><span class="plain"> += </span><span class="identifier">min_t</span><span class="plain">; </span><span class="identifier">max_p</span><span class="plain"> += </span><span class="identifier">max_t</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">min_p</span><span class="plain"> > </span><span class="constant">INFINITE_WORD_COUNT</span><span class="plain">) </span><span class="identifier">min_p</span><span class="plain"> = </span><span class="constant">INFINITE_WORD_COUNT</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">max_p</span><span class="plain"> > </span><span class="constant">INFINITE_WORD_COUNT</span><span class="plain">) </span><span class="identifier">max_p</span><span class="plain"> = </span><span class="constant">INFINITE_WORD_COUNT</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>min_pr_words</span><span class="plain"> = </span><span class="identifier">min_p</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain">-></span><span class="identifier">max_pr_words</span><span class="plain"> = </span><span class="identifier">max_p</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">min</span><span class="plain"> == -1) && (</span><span class="identifier">max</span><span class="plain"> == -1)) { </span><span class="identifier">min</span><span class="plain"> = </span><span class="identifier">min_p</span><span class="plain">; </span><span class="identifier">max</span><span class="plain"> = </span><span class="identifier">max_p</span><span class="plain">; }</span>
|
|
<span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">min_p</span><span class="plain"> < </span><span class="identifier">min</span><span class="plain">) </span><span class="identifier">min</span><span class="plain"> = </span><span class="identifier">min_p</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">max_p</span><span class="plain"> > </span><span class="identifier">max</span><span class="plain">) </span><span class="identifier">max</span><span class="plain"> = </span><span class="identifier">max_p</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">min</span><span class="plain"> >= 1) {</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>min_nt_words</span><span class="plain"> = </span><span class="identifier">min</span><span class="plain">; </span><span class="identifier">nt</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain"> = </span><span class="identifier">max</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP32">§32</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP32_2"></a><b>§32.2. </b>A token is "elastic" if it can match text of differing lengths, and
|
|
"inelastic" otherwise. For example, in English, <indefinite-article> is
|
|
elastic (it always matches a single word). If the first ptoken is inelastic,
|
|
we know it must match words 1 to L_1 of whatever text is to be matched,
|
|
and we give it position 1; if the second is also inelastic, that will match
|
|
L_1+1 to L_2, and it gets position L_1+1; and so on. As soon as we
|
|
hit an elastic token — a wildcard like <code class="display"><span class="extract">...</span></code>, for example — this
|
|
predictability stops, and we can only assign position 0, which means that
|
|
we don't know.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">Note that we only assign a nonzero position if we know where the ptoken both
|
|
starts and finishes; it's not enough just to know where it starts.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Compute front-end ptoken positions</span> <span class="cwebmacronumber">32.2</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">posn</span><span class="plain"> = 1;</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>first_ptoken</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">) {</span>
|
|
<span class="identifier">last</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">L</span><span class="plain"> = </span><span class="functiontext">Preform::ptoken_width</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">posn</span><span class="plain"> != 0) && (</span><span class="identifier">L</span><span class="plain"> != </span><span class="constant">PTOKEN_ELASTIC</span><span class="plain">)) {</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_position</span><span class="plain"> = </span><span class="identifier">posn</span><span class="plain">;</span>
|
|
<span class="identifier">posn</span><span class="plain"> += </span><span class="identifier">L</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_position</span><span class="plain"> = 0; </span> <span class="comment">thus clearing any expired positions from earlier</span>
|
|
<span class="identifier">posn</span><span class="plain"> = 0;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP32">§32</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP32_3"></a><b>§32.3. </b>And similarly from the back end, if there are inelastic ptokens at the end
|
|
of the production (and which are separated from the front end by at least one
|
|
elastic one).
|
|
</p>
|
|
|
|
<p class="inwebparagraph">The following has quadratic running time in the number of tokens in the
|
|
production, but this is never larger than about 10.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Compute back-end ptoken positions</span> <span class="cwebmacronumber">32.3</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">posn</span><span class="plain"> = -1;</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">last</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain">; ) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_position</span><span class="plain"> != 0) </span><span class="reserved">break</span><span class="plain">; </span> <span class="comment">don't use a back-end position if there's a front one</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">L</span><span class="plain"> = </span><span class="functiontext">Preform::ptoken_width</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">posn</span><span class="plain"> != 0) && (</span><span class="identifier">L</span><span class="plain"> != </span><span class="constant">PTOKEN_ELASTIC</span><span class="plain">)) {</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_position</span><span class="plain"> = </span><span class="identifier">posn</span><span class="plain">;</span>
|
|
<span class="identifier">posn</span><span class="plain"> -= </span><span class="identifier">L</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> </span><span class="reserved">break</span><span class="plain">;</span>
|
|
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">prevt</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">prevt</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>first_ptoken</span><span class="plain">; </span><span class="identifier">prevt</span><span class="plain">; </span><span class="identifier">prevt</span><span class="plain"> = </span><span class="identifier">prevt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">)</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">prevt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain"> == </span><span class="identifier">pt</span><span class="plain">)</span>
|
|
<span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">prevt</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP32">§32</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP32_4"></a><b>§32.4. </b>By definition, a strut is a maximal sequence of one or more inelastic ptokens
|
|
each of which has no known position. (Clearly if one of them has a known
|
|
position then all of them have, but we're in no hurry so we don't exploit that.)
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Compute struts within the production</span> <span class="cwebmacronumber">32.4</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_struts</span><span class="plain"> = 0;</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>first_ptoken</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_position</span><span class="plain"> == 0) && (</span><span class="functiontext">Preform::ptoken_width</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">) != </span><span class="constant">PTOKEN_ELASTIC</span><span class="plain">)) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_struts</span><span class="plain"> >= </span><span class="constant">MAX_STRUTS_PER_PRODUCTION</span><span class="plain">) </span><span class="reserved">continue</span><span class="plain">;</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>struts</span><span class="plain">[</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_struts</span><span class="plain">] = </span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>strut_lengths</span><span class="plain">[</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_struts</span><span class="plain">] = 0;</span>
|
|
<span class="reserved">while</span><span class="plain"> ((</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_position</span><span class="plain"> == 0) && (</span><span class="functiontext">Preform::ptoken_width</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">) != </span><span class="constant">PTOKEN_ELASTIC</span><span class="plain">)) {</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>strut_number</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_struts</span><span class="plain">;</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>strut_lengths</span><span class="plain">[</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_struts</span><span class="plain">] += </span><span class="functiontext">Preform::ptoken_width</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) </span><span class="reserved">break</span><span class="plain">; </span> <span class="comment">should be impossible</span>
|
|
<span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_struts</span><span class="plain">++;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP32">§32</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP32_5"></a><b>§32.5. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Work out which ptokens are fast</span> <span class="cwebmacronumber">32.5</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>first_ptoken</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">)</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> == </span><span class="constant">FIXED_WORD_PTC</span><span class="plain">) && (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_position</span><span class="plain"> != 0)</span>
|
|
<span class="plain">&& (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_starts</span><span class="plain"> < 0) && (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_ends</span><span class="plain"> < 0))</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_is_fast</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP32">§32</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP32_6"></a><b>§32.6. </b>Weak requirement: one word in range must match one of these bits
|
|
Strong ": all bits in this range must be matched by one word
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Mark the vocabulary's incidence list with this nonterminal</span> <span class="cwebmacronumber">32.6</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">first_production</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="functiontext">Preform::clear_rreq</span><span class="plain">(&(</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req</span><span class="plain">));</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">PREFORM_CIRCULARITY_BREAKER</span>
|
|
<span class="identifier">PREFORM_CIRCULARITY_BREAKER</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">);</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
<span class="reserved">range_requirement</span><span class="plain"> </span><span class="identifier">nnt</span><span class="plain">;</span>
|
|
<span class="functiontext">Preform::clear_rreq</span><span class="plain">(&</span><span class="identifier">nnt</span><span class="plain">);</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>next_production_list</span><span class="plain">) {</span>
|
|
<span class="reserved">production</span><span class="plain"> *</span><span class="identifier">pr</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>first_production</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>next_production</span><span class="plain">) {</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>first_ptoken</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> == </span><span class="constant">FIXED_WORD_PTC</span><span class="plain">) && (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">)) {</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">alt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">alt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">alt</span><span class="plain">; </span><span class="identifier">alt</span><span class="plain"> = </span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>alternative_ptoken</span><span class="plain">)</span>
|
|
<span class="functiontext">Preform::set_nt_incidence</span><span class="plain">(</span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>ve_pt</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>next_production_list</span><span class="plain">) {</span>
|
|
<span class="reserved">production</span><span class="plain"> *</span><span class="identifier">pr</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>first_production</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>next_production</span><span class="plain">) {</span>
|
|
<span class="reserved">range_requirement</span><span class="plain"> </span><span class="identifier">prt</span><span class="plain">;</span>
|
|
<span class="functiontext">Preform::clear_rreq</span><span class="plain">(&</span><span class="identifier">prt</span><span class="plain">);</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">all</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">, </span><span class="identifier">first</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>first_ptoken</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">) {</span>
|
|
<span class="functiontext">Preform::clear_rreq</span><span class="plain">(&(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>token_req</span><span class="plain">));</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> == </span><span class="constant">FIXED_WORD_PTC</span><span class="plain">) && (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">)) {</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">alt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">alt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">alt</span><span class="plain">; </span><span class="identifier">alt</span><span class="plain"> = </span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>alternative_ptoken</span><span class="plain">)</span>
|
|
<span class="functiontext">Preform::set_nt_incidence</span><span class="plain">(</span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>ve_pt</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">);</span>
|
|
<span class="functiontext">Preform::atomic_rreq</span><span class="plain">(&(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>token_req</span><span class="plain">), </span><span class="identifier">nt</span><span class="plain">);</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> </span><span class="identifier">all</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">self_referential</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">, </span><span class="identifier">empty</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> == </span><span class="constant">NONTERMINAL_PTC</span><span class="plain">) &&</span>
|
|
<span class="plain">(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>min_nt_words</span><span class="plain"> == 0) && (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain"> == 0))</span>
|
|
<span class="identifier">empty</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">; </span> <span class="comment">even if negated, notice</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> == </span><span class="constant">NONTERMINAL_PTC</span><span class="plain">) && (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">)) {</span>
|
|
<span class="comment">if (pt->nt_pt == nt) self_referential = TRUE;</span>
|
|
<span class="functiontext">Preform::optimise_nt</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">);</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>token_req</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>nonterminal_req</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">self_referential</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">) && (</span><span class="identifier">empty</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">)) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">first</span><span class="plain">) {</span>
|
|
<span class="identifier">prt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>token_req</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="functiontext">Preform::concatenate_rreq</span><span class="plain">(&</span><span class="identifier">prt</span><span class="plain">, &(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>token_req</span><span class="plain">));</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">first</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">first_production</span><span class="plain">) {</span>
|
|
<span class="identifier">nnt</span><span class="plain"> = </span><span class="identifier">prt</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="functiontext">Preform::disjoin_rreq</span><span class="plain">(&</span><span class="identifier">nnt</span><span class="plain">, &</span><span class="identifier">prt</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">first_production</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_req</span><span class="plain"> = </span><span class="identifier">prt</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req</span><span class="plain"> = </span><span class="identifier">nnt</span><span class="plain">;</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">PREFORM_CIRCULARITY_BREAKER</span>
|
|
<span class="identifier">PREFORM_CIRCULARITY_BREAKER</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">);</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP32">§32</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP33"></a><b>§33. </b>The constant <code class="display"><span class="extract">AL_BITMAP</span></code> used in this code has a pleasingly Arabic sound to it
|
|
— a second-magnitude star, an idiotically tall hotel — but is in fact a
|
|
combination of the meaning codes found in an adjective list.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::optimise_nt_reqs</span><span class="plain">(</span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="reserved">production_list</span><span class="plain"> *</span><span class="identifier">pl</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>next_production_list</span><span class="plain">) {</span>
|
|
<span class="reserved">production</span><span class="plain"> *</span><span class="identifier">pr</span><span class="plain">;</span>
|
|
<span class="reserved">range_requirement</span><span class="plain"> *</span><span class="identifier">prev_req</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>first_production</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>next_production</span><span class="plain">) {</span>
|
|
<span class="functiontext">Preform::optimise_req</span><span class="plain">(&(</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_req</span><span class="plain">), </span><span class="identifier">prev_req</span><span class="plain">);</span>
|
|
<span class="identifier">prev_req</span><span class="plain"> = &(</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_req</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="functiontext">Preform::optimise_req</span><span class="plain">(&(</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req</span><span class="plain">), </span><span class="identifier">NULL</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::optimise_req</span><span class="plain">(</span><span class="reserved">range_requirement</span><span class="plain"> *</span><span class="identifier">req</span><span class="plain">, </span><span class="reserved">range_requirement</span><span class="plain"> *</span><span class="identifier">prev</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain"> & </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">) == </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">) </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain"> = 0;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain"> & </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">) == </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">) </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain"> = 0;</span>
|
|
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain"> & </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">) == </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">) </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain"> = 0;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain"> & </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">) == </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">) </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain"> = 0;</span>
|
|
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain"> & </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">) == </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">) </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain"> = 0;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain"> & </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">) == </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">) </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain"> = 0;</span>
|
|
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain"> & </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">) == </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">) </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain"> = 0;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain"> & </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">) == </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">) </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain"> = 0;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain"> & </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">) == </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">) </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain"> = 0;</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>no_requirements</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">) || (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">) || (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">) || (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">) || (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">) || (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">))</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>no_requirements</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>ditto_flag</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">prev</span><span class="plain">) &&</span>
|
|
<span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain"> == </span><span class="identifier">prev</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain"> == </span><span class="identifier">prev</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">) &&</span>
|
|
<span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain"> == </span><span class="identifier">prev</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain"> == </span><span class="identifier">prev</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">) &&</span>
|
|
<span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain"> == </span><span class="identifier">prev</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain"> == </span><span class="identifier">prev</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">))</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>ditto_flag</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::optimise_nt_reqs is used in <a href="#SP32">§32</a>.</p>
|
|
|
|
<p class="endnote">The function Preform::optimise_req appears nowhere else.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP34"></a><b>§34. </b></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::mark_nt_as_requiring_itself</span><span class="plain">(</span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.DS_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.DW_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::mark_nt_as_requiring_itself_first</span><span class="plain">(</span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.DS_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.DW_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.FS_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.FW_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::mark_nt_as_requiring_itself_conj</span><span class="plain">(</span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.DS_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.DW_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.CS_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.CW_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.FS_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.FW_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::mark_nt_as_requiring_itself_augmented</span><span class="plain">(</span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">x</span><span class="plain">) {</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.DS_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.DW_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.CW_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">) + </span><span class="identifier">x</span><span class="plain">);</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req.FW_req</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">) + </span><span class="identifier">x</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::set_nt_incidence</span><span class="plain">(</span><span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">ve</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">R</span><span class="plain"> = </span><span class="functiontext">Vocabulary::get_ntb</span><span class="plain">(</span><span class="identifier">ve</span><span class="plain">);</span>
|
|
<span class="identifier">R</span><span class="plain"> |= (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">));</span>
|
|
<span class="functiontext">Vocabulary::set_ntb</span><span class="plain">(</span><span class="identifier">ve</span><span class="plain">, </span><span class="identifier">R</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::test_nt_incidence</span><span class="plain">(</span><span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">ve</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">R</span><span class="plain"> = </span><span class="functiontext">Vocabulary::get_ntb</span><span class="plain">(</span><span class="identifier">ve</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">R</span><span class="plain"> & (</span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">))) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::mark_nt_as_requiring_itself appears nowhere else.</p>
|
|
|
|
<p class="endnote">The function Preform::mark_nt_as_requiring_itself_first appears nowhere else.</p>
|
|
|
|
<p class="endnote">The function Preform::mark_nt_as_requiring_itself_conj appears nowhere else.</p>
|
|
|
|
<p class="endnote">The function Preform::mark_nt_as_requiring_itself_augmented appears nowhere else.</p>
|
|
|
|
<p class="endnote">The function Preform::set_nt_incidence is used in <a href="#SP32_6">§32.6</a>, <a href="#SP35">§35</a>.</p>
|
|
|
|
<p class="endnote">The function Preform::test_nt_incidence appears nowhere else.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP35"></a><b>§35. </b></p>
|
|
|
|
|
|
<pre class="definitions">
|
|
<span class="definitionkeyword">define</span> <span class="constant">RESERVED_NT_BITS</span><span class="plain"> 6</span>
|
|
</pre>
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nt_req_bit</span><span class="plain"> == -1) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">b</span><span class="plain"> = </span><span class="constant">RESERVED_NT_BITS</span><span class="plain"> + ((</span><span class="identifier">no_req_bits</span><span class="plain">++)%(32-</span><span class="constant">RESERVED_NT_BITS</span><span class="plain">));</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nt_req_bit</span><span class="plain"> = (1 << </span><span class="identifier">b</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nt_req_bit</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::assign_bitmap_bit</span><span class="plain">(</span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">b</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) </span><span class="identifier">internal_error</span><span class="plain">(</span><span class="string">"null NT"</span><span class="plain">);</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nt_req_bit</span><span class="plain"> = (1 << </span><span class="identifier">b</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::test_word</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">wn</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">b</span><span class="plain"> = </span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="functiontext">Vocabulary::get_ntb</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">))) & </span><span class="identifier">b</span><span class="plain">) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::mark_word</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">wn</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="functiontext">Preform::set_nt_incidence</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">), </span><span class="identifier">nt</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::mark_vocabulary</span><span class="plain">(</span><span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">ve</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="functiontext">Preform::set_nt_incidence</span><span class="plain">(</span><span class="identifier">ve</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::test_vocabulary</span><span class="plain">(</span><span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">ve</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">b</span><span class="plain"> = </span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="functiontext">Vocabulary::get_ntb</span><span class="plain">(</span><span class="identifier">ve</span><span class="plain">)) & </span><span class="identifier">b</span><span class="plain">) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::get_range_disjunction</span><span class="plain">(</span><span class="reserved">wording</span><span class="plain"> </span><span class="identifier">W</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">R</span><span class="plain"> = 0;</span>
|
|
<span class="identifier">LOOP_THROUGH_WORDING</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">)</span>
|
|
<span class="identifier">R</span><span class="plain"> |= </span><span class="functiontext">Vocabulary::get_ntb</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">));</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">R</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::get_range_conjunction</span><span class="plain">(</span><span class="reserved">wording</span><span class="plain"> </span><span class="identifier">W</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">R</span><span class="plain"> = 0;</span>
|
|
<span class="identifier">LOOP_THROUGH_WORDING</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">i</span><span class="plain"> == </span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)) </span><span class="identifier">R</span><span class="plain"> = </span><span class="functiontext">Vocabulary::get_ntb</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">));</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="identifier">R</span><span class="plain"> &= </span><span class="functiontext">Vocabulary::get_ntb</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">));</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">R</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::nt_bitmap_bit is used in <a href="#SP34">§34</a>, <a href="#SP47">§47</a>.</p>
|
|
|
|
<p class="endnote">The function Preform::assign_bitmap_bit appears nowhere else.</p>
|
|
|
|
<p class="endnote">The function Preform::test_word appears nowhere else.</p>
|
|
|
|
<p class="endnote">The function Preform::mark_word appears nowhere else.</p>
|
|
|
|
<p class="endnote">The function Preform::mark_vocabulary appears nowhere else.</p>
|
|
|
|
<p class="endnote">The function Preform::test_vocabulary appears nowhere else.</p>
|
|
|
|
<p class="endnote">The function Preform::get_range_disjunction appears nowhere else.</p>
|
|
|
|
<p class="endnote">The function Preform::get_range_conjunction appears nowhere else.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP36"></a><b>§36. </b></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::nt_bitmap_violates</span><span class="plain">(</span><span class="reserved">wording</span><span class="plain"> </span><span class="identifier">W</span><span class="plain">, </span><span class="reserved">range_requirement</span><span class="plain"> *</span><span class="identifier">req</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>no_requirements</span><span class="plain">) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Wordings::length</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">) == 1) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">bm</span><span class="plain"> = </span><span class="functiontext">Vocabulary::get_ntb</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)));</span>
|
|
<span class="reserved">if</span><span class="plain"> (((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">)) != (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">)) == 0) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">)) != (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">)) == 0) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">)) != (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">)) == 0) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">C_set</span><span class="plain"> = ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">) | (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">));</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">D_set</span><span class="plain"> = ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">) | (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">));</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">F_set</span><span class="plain"> = ((</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">) | (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">));</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">C_set</span><span class="plain">) && (</span><span class="identifier">D_set</span><span class="plain">)) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">disj</span><span class="plain"> = 0;</span>
|
|
<span class="identifier">LOOP_THROUGH_WORDING</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">bm</span><span class="plain"> = </span><span class="functiontext">Vocabulary::get_ntb</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">));</span>
|
|
<span class="identifier">disj</span><span class="plain"> |= </span><span class="identifier">bm</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">)) != (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">)) == 0) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">i</span><span class="plain"> == </span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)) && (</span><span class="identifier">F_set</span><span class="plain">)) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">)) != (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">)) == 0) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (((</span><span class="identifier">disj</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">)) != (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((((</span><span class="identifier">disj</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">)) == 0) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">C_set</span><span class="plain">) {</span>
|
|
<span class="identifier">LOOP_THROUGH_WORDING</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">bm</span><span class="plain"> = </span><span class="functiontext">Vocabulary::get_ntb</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">));</span>
|
|
<span class="reserved">if</span><span class="plain"> (((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">)) != (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">)) == 0) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">i</span><span class="plain"> == </span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)) && (</span><span class="identifier">F_set</span><span class="plain">)) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">)) != (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">)) == 0) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">D_set</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">disj</span><span class="plain"> = 0;</span>
|
|
<span class="identifier">LOOP_THROUGH_WORDING</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">bm</span><span class="plain"> = </span><span class="functiontext">Vocabulary::get_ntb</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">));</span>
|
|
<span class="identifier">disj</span><span class="plain"> |= </span><span class="identifier">bm</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">i</span><span class="plain"> == </span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)) && (</span><span class="identifier">F_set</span><span class="plain">)) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">)) != (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">)) == 0) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (((</span><span class="identifier">disj</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">)) != (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((((</span><span class="identifier">disj</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">)) == 0) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">F_set</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">bm</span><span class="plain"> = </span><span class="functiontext">Vocabulary::get_ntb</span><span class="plain">(</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)));</span>
|
|
<span class="reserved">if</span><span class="plain"> (((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">)) != (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((((</span><span class="identifier">bm</span><span class="plain">) & (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">)) == 0) && (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">)) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::nt_bitmap_violates is used in <a href="#SP51_2">§51.2</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP37"></a><b>§37. </b>The first operation on RRs is concatenation. Suppose we are required to
|
|
match some words against X, then some more against Y.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::concatenate_rreq</span><span class="plain">(</span><span class="reserved">range_requirement</span><span class="plain"> *</span><span class="identifier">req</span><span class="plain">, </span><span class="reserved">range_requirement</span><span class="plain"> *</span><span class="identifier">with</span><span class="plain">) {</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain"> = </span><span class="functiontext">Preform::concatenate_ds</span><span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">, </span><span class="identifier">with</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">);</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain"> = </span><span class="functiontext">Preform::concatenate_dw</span><span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">, </span><span class="identifier">with</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">);</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain"> = </span><span class="functiontext">Preform::concatenate_cs</span><span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">, </span><span class="identifier">with</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">);</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain"> = </span><span class="functiontext">Preform::concatenate_cw</span><span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">, </span><span class="identifier">with</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">);</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain"> = </span><span class="functiontext">Preform::concatenate_fs</span><span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">, </span><span class="identifier">with</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">);</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain"> = </span><span class="functiontext">Preform::concatenate_fw</span><span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">, </span><span class="identifier">with</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::concatenate_rreq is used in <a href="#SP32_6">§32.6</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP38"></a><b>§38. </b>The strong requirements are well-defined. Suppose all of the bits of <code class="display"><span class="extract">m1</span></code>
|
|
are found in X, and all of the bits of <code class="display"><span class="extract">m2</span></code> are found in Y. Then clearly
|
|
all of the bits in the union of these two sets are found in XY, and that's
|
|
the strongest requirement we can make. So:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::concatenate_ds</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">) {</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain"> | </span><span class="identifier">m2</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::concatenate_ds is used in <a href="#SP37">§37</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP39"></a><b>§39. </b>Similarly, suppose all of the bits of <code class="display"><span class="extract">m1</span></code> are found in every word of X,
|
|
and all of those of <code class="display"><span class="extract">m2</span></code> are in every word of Y. The most which can be said
|
|
about every word of XY is to take the intersection, so:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::concatenate_cs</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">) {</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain"> & </span><span class="identifier">m2</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::concatenate_cs is used in <a href="#SP37">§37</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP40"></a><b>§40. </b>Now suppose that at least one bit of <code class="display"><span class="extract">m1</span></code> can be found in X, and one bit
|
|
of <code class="display"><span class="extract">m2</span></code> can be found in Y. This gives us two pieces of information about
|
|
XY, and we can freely choose which to go for: we may as well pick <code class="display"><span class="extract">m1</span></code> and
|
|
say that one bit of <code class="display"><span class="extract">m1</span></code> can be found in XY. In principle we ought to choose
|
|
the rarest for best effect, but that's too much work.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::concatenate_dw</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">m1</span><span class="plain"> == 0) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">; </span> <span class="comment">the case where we have no information about X</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">m2</span><span class="plain"> == 0) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">; </span> <span class="comment">and about Y</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">; </span> <span class="comment">the general case discussed above</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::concatenate_dw is used in <a href="#SP37">§37</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP41"></a><b>§41. </b>Now suppose that each word of X matches at least one bit of <code class="display"><span class="extract">m1</span></code>, and
|
|
similarly for Y and <code class="display"><span class="extract">m2</span></code>. Then each word of XY matches at least one bit of
|
|
the union, so:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::concatenate_cw</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">m1</span><span class="plain"> == 0) </span><span class="reserved">return</span><span class="plain"> 0; </span> <span class="comment">the case where we have no information about X</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">m2</span><span class="plain"> == 0) </span><span class="reserved">return</span><span class="plain"> 0; </span> <span class="comment">and about Y</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain"> | </span><span class="identifier">m2</span><span class="plain">; </span> <span class="comment">the general case discussed above</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::concatenate_cw is used in <a href="#SP37">§37</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP42"></a><b>§42. </b>The first word of XY is the first word of X, so:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::concatenate_fs</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">) {</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::concatenate_fw</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">) {</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::concatenate_fs is used in <a href="#SP37">§37</a>.</p>
|
|
|
|
<p class="endnote">The function Preform::concatenate_fw is used in <a href="#SP37">§37</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP43"></a><b>§43. </b>The second operation is disjunction: we'll write X/Y, meaning that the text
|
|
has to match either X or Y. This is easier, since it amounts to a disguised
|
|
form of de Morgan's laws.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::disjoin_rreq</span><span class="plain">(</span><span class="reserved">range_requirement</span><span class="plain"> *</span><span class="identifier">req</span><span class="plain">, </span><span class="reserved">range_requirement</span><span class="plain"> *</span><span class="identifier">with</span><span class="plain">) {</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain"> = </span><span class="functiontext">Preform::disjoin_ds</span><span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">, </span><span class="identifier">with</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">);</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain"> = </span><span class="functiontext">Preform::disjoin_dw</span><span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">, </span><span class="identifier">with</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">);</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain"> = </span><span class="functiontext">Preform::disjoin_cs</span><span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">, </span><span class="identifier">with</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">);</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain"> = </span><span class="functiontext">Preform::disjoin_cw</span><span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">, </span><span class="identifier">with</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">);</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain"> = </span><span class="functiontext">Preform::disjoin_fs</span><span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">, </span><span class="identifier">with</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">);</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain"> = </span><span class="functiontext">Preform::disjoin_fw</span><span class="plain">(</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">, </span><span class="identifier">with</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::disjoin_rreq is used in <a href="#SP32_6">§32.6</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP44"></a><b>§44. </b>Suppose all of the bits of <code class="display"><span class="extract">m1</span></code> are found in X, and all of the bits of <code class="display"><span class="extract">m2</span></code>
|
|
are found in Y. Then the best we can say is that all of the bits in the
|
|
intersection of these two sets are found in X/Y. (If they have no bits in
|
|
common, we can't say anything.)
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::disjoin_ds</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">) {</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain"> & </span><span class="identifier">m2</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::disjoin_ds is used in <a href="#SP43">§43</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP45"></a><b>§45. </b>Similarly, suppose all of the bits of <code class="display"><span class="extract">m1</span></code> are found in every word of X,
|
|
and all of those of <code class="display"><span class="extract">m2</span></code> are in every word of Y. The most which can be said
|
|
about every word of XY is to take the intersection, so:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::disjoin_cs</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">) {</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain"> & </span><span class="identifier">m2</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::disjoin_cs is used in <a href="#SP43">§43</a>, <a href="#SP47">§47</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP46"></a><b>§46. </b>Now suppose that at least one bit of <code class="display"><span class="extract">m1</span></code> can be found in X, and one bit
|
|
of <code class="display"><span class="extract">m2</span></code> can be found in Y. All we can say is that one of these various bits
|
|
must be found in X/Y, so:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::disjoin_dw</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">m1</span><span class="plain"> == 0) </span><span class="reserved">return</span><span class="plain"> 0; </span> <span class="comment">the case where we have no information about X</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">m2</span><span class="plain"> == 0) </span><span class="reserved">return</span><span class="plain"> 0; </span> <span class="comment">and about Y</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain"> | </span><span class="identifier">m2</span><span class="plain">; </span> <span class="comment">the general case discussed above</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::disjoin_dw is used in <a href="#SP43">§43</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP47"></a><b>§47. </b>And exactly the same is true for conjunctions:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::disjoin_cw</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">m1</span><span class="plain"> == 0) </span><span class="reserved">return</span><span class="plain"> 0; </span> <span class="comment">the case where we have no information about X</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">m2</span><span class="plain"> == 0) </span><span class="reserved">return</span><span class="plain"> 0; </span> <span class="comment">and about Y</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain"> | </span><span class="identifier">m2</span><span class="plain">; </span> <span class="comment">the general case discussed above</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::disjoin_fw</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">) {</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="functiontext">Preform::disjoin_cw</span><span class="plain">(</span><span class="identifier">m1</span><span class="plain">, </span><span class="identifier">m2</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::disjoin_fs</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m1</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">m2</span><span class="plain">) {</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="functiontext">Preform::disjoin_cs</span><span class="plain">(</span><span class="identifier">m1</span><span class="plain">, </span><span class="identifier">m2</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::clear_rreq</span><span class="plain">(</span><span class="reserved">range_requirement</span><span class="plain"> *</span><span class="identifier">req</span><span class="plain">) {</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain"> = 0; </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain"> = 0;</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain"> = 0; </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain"> = 0;</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain"> = 0; </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain"> = 0;</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::atomic_rreq</span><span class="plain">(</span><span class="reserved">range_requirement</span><span class="plain"> *</span><span class="identifier">req</span><span class="plain">, </span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">b</span><span class="plain"> = </span><span class="functiontext">Preform::nt_bitmap_bit</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">);</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain"> = </span><span class="identifier">b</span><span class="plain">; </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain"> = </span><span class="identifier">b</span><span class="plain">;</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain"> = </span><span class="identifier">b</span><span class="plain">; </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain"> = </span><span class="identifier">b</span><span class="plain">;</span>
|
|
<span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain"> = 0; </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain"> = 0;</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::log_range_requirement</span><span class="plain">(</span><span class="reserved">range_requirement</span><span class="plain"> *</span><span class="identifier">req</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">) { </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">" DW: %08x"</span><span class="plain">, </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DW_req</span><span class="plain">); }</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">) { </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">" DS: %08x"</span><span class="plain">, </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>DS_req</span><span class="plain">); }</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">) { </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">" CW: %08x"</span><span class="plain">, </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CW_req</span><span class="plain">); }</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">) { </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">" CS: %08x"</span><span class="plain">, </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>CS_req</span><span class="plain">); }</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">) { </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">" FW: %08x"</span><span class="plain">, </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FW_req</span><span class="plain">); }</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">) { </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">" FS: %08x"</span><span class="plain">, </span><span class="identifier">req</span><span class="plain">-</span><span class="element">>FS_req</span><span class="plain">); }</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::disjoin_cw is used in <a href="#SP43">§43</a>.</p>
|
|
|
|
<p class="endnote">The function Preform::disjoin_fw is used in <a href="#SP43">§43</a>.</p>
|
|
|
|
<p class="endnote">The function Preform::disjoin_fs is used in <a href="#SP43">§43</a>.</p>
|
|
|
|
<p class="endnote">The function Preform::clear_rreq is used in <a href="#SP32">§32</a>, <a href="#SP32_6">§32.6</a>.</p>
|
|
|
|
<p class="endnote">The function Preform::atomic_rreq is used in <a href="#SP32_6">§32.6</a>.</p>
|
|
|
|
<p class="endnote">The function Preform::log_range_requirement is used in <a href="#SP18">§18</a>, <a href="#SP51_2">§51.2</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP48"></a><b>§48. </b>Now to define elasticity:
|
|
</p>
|
|
|
|
|
|
<pre class="definitions">
|
|
<span class="definitionkeyword">define</span> <span class="constant">PTOKEN_ELASTIC</span><span class="plain"> -1</span>
|
|
</pre>
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::ptoken_width</span><span class="plain">(</span><span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">min</span><span class="plain">, </span><span class="identifier">max</span><span class="plain">;</span>
|
|
<span class="functiontext">Preform::ptoken_extrema</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">, &</span><span class="identifier">min</span><span class="plain">, &</span><span class="identifier">max</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">min</span><span class="plain"> != </span><span class="identifier">max</span><span class="plain">) </span><span class="reserved">return</span><span class="plain"> </span><span class="constant">PTOKEN_ELASTIC</span><span class="plain">;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">min</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::ptoken_width is used in <a href="#SP32_2">§32.2</a>, <a href="#SP32_3">§32.3</a>, <a href="#SP32_4">§32.4</a>, <a href="#SP51_2_1_2_3_3_3_1">§51.2.1.2.3.3.3.1</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP49"></a><b>§49. </b>An interesting point here is that the negation of a ptoken can in principle
|
|
have any length, except that we specified <code class="display"><span class="extract">^ example</span></code> to match only a single
|
|
word — any word other than "example". So the extrema for <code class="display"><span class="extract">^ example</span></code> are
|
|
1 and 1, whereas for <code class="display"><span class="extract">^ <sample-nonterminal></span></code> they would have to be 0 and
|
|
infinity.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::ptoken_extrema</span><span class="plain">(</span><span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> *</span><span class="identifier">min_t</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> *</span><span class="identifier">max_t</span><span class="plain">) {</span>
|
|
<span class="plain">*</span><span class="identifier">min_t</span><span class="plain"> = 1; *</span><span class="identifier">max_t</span><span class="plain"> = 1;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> != </span><span class="constant">FIXED_WORD_PTC</span><span class="plain">) { *</span><span class="identifier">min_t</span><span class="plain"> = 0; *</span><span class="identifier">max_t</span><span class="plain"> = </span><span class="constant">INFINITE_WORD_COUNT</span><span class="plain">; }</span>
|
|
<span class="reserved">return</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">switch</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain">) {</span>
|
|
<span class="reserved">case</span><span class="plain"> </span><span class="constant">NONTERMINAL_PTC</span><span class="plain">:</span>
|
|
<span class="functiontext">Preform::optimise_nt</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">); </span> <span class="comment">recurse as needed to find its extrema</span>
|
|
<span class="plain">*</span><span class="identifier">min_t</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>min_nt_words</span><span class="plain">;</span>
|
|
<span class="plain">*</span><span class="identifier">max_t</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain">;</span>
|
|
<span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="reserved">case</span><span class="plain"> </span><span class="constant">MULTIPLE_WILDCARD_PTC</span><span class="plain">:</span>
|
|
<span class="plain">*</span><span class="identifier">max_t</span><span class="plain"> = </span><span class="constant">INFINITE_WORD_COUNT</span><span class="plain">;</span>
|
|
<span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="reserved">case</span><span class="plain"> </span><span class="constant">POSSIBLY_EMPTY_WILDCARD_PTC</span><span class="plain">:</span>
|
|
<span class="plain">*</span><span class="identifier">min_t</span><span class="plain"> = 0;</span>
|
|
<span class="plain">*</span><span class="identifier">max_t</span><span class="plain"> = </span><span class="constant">INFINITE_WORD_COUNT</span><span class="plain">;</span>
|
|
<span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::ptoken_extrema is used in <a href="#SP32_1">§32.1</a>, <a href="#SP48">§48</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP50"></a><b>§50. Parsing. </b>Since I have found that well-known computer programmers look at me strangely
|
|
when I tell them that Inform doesn't use <code class="display"><span class="extract">yacc</span></code>, or <code class="display"><span class="extract">antlr</span></code>, or for that
|
|
matter any of the elegant theory of LALR parsers, perhaps an explanation
|
|
is called for.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">One reason is that I am sceptical that formal grammars specify natural language
|
|
terribly well — which is ironic, considering that the relevant computer
|
|
science, dating from the 1950s and 1960s, was strongly influenced by Noam
|
|
Chomsky's generative linguistics. Such formal descriptions tend to be too rigid
|
|
to be applied universally. The classical use case for <code class="display"><span class="extract">yacc</span></code> is to manage
|
|
hierarchies of associative operators on different levels: well, natural language
|
|
doesn't have those.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">Another reason is that <code class="display"><span class="extract">yacc</span></code>-style grammars tend to react badly to uncompliant
|
|
input: that is, they correctly reject it, but are bad at diagnosing the
|
|
problem, and at recovering their wits afterwards. For Inform purposes, this
|
|
would be too sloppy: the user more often miscompiles than compiles, and quality
|
|
lies in how good our problem messages are in reply.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">Lastly, there are two pragmatic reasons. In order to make Preform grammar
|
|
extensible, we couldn't use a parser-compiler like <code class="display"><span class="extract">yacc</span></code> anyway: we have to
|
|
interpret our grammar, not compile code to parse it. And we also want speed;
|
|
folk wisdom has it that <code class="display"><span class="extract">yacc</span></code> parsers are about half as fast as a shrewdly
|
|
hand-coded equivalent. (<code class="display"><span class="extract">gcc</span></code> abandoned the use of <code class="display"><span class="extract">bison</span></code> for exactly this
|
|
reason some years ago.) Until Preform's arrival in February 2011, Inform had a
|
|
hard-coded syntax analyser scattered throughout its code, which often made what
|
|
were provably the minimum possible number of comparisons. Even Preform's
|
|
parser is intentionally lean.
|
|
</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51"></a><b>§51. </b>Make of that apologia what you will. Speed is important in the following
|
|
code, but not critical: I optimised it until profiling showed that Inform spent
|
|
only about 6\% of its time here.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">ptraci</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">; </span> <span class="comment">in this mode, we trace parsing to the debugging log</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">preform_lookahead_mode</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">; </span> <span class="comment">in this mode, we are looking ahead</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">fail_nonterminal_quantum</span><span class="plain"> = 0; </span> <span class="comment">jump forward by this many words in lookahead</span>
|
|
<span class="reserved">void</span><span class="plain"> *</span><span class="identifier">preform_backtrack</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">; </span> <span class="comment">position to backtrack from in voracious internal</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::parse_nt_against_word_range</span><span class="plain">(</span><span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">nt</span><span class="plain">, </span><span class="reserved">wording</span><span class="plain"> </span><span class="identifier">W</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> *</span><span class="identifier">result</span><span class="plain">,</span>
|
|
<span class="reserved">void</span><span class="plain"> **</span><span class="identifier">result_p</span><span class="plain">) {</span>
|
|
<span class="identifier">time_t</span><span class="plain"> </span><span class="identifier">start_of_nt</span><span class="plain"> = </span><span class="identifier">time</span><span class="plain">(0);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) </span><span class="identifier">internal_error</span><span class="plain">(</span><span class="string">"can't parse a null nonterminal"</span><span class="plain">);</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">INSTRUMENTED_PREFORM</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_tries</span><span class="plain">++;</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">success_rval</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">; </span> <span class="comment">what to return in the event of a successful match</span>
|
|
<span class="identifier">fail_nonterminal_quantum</span><span class="plain"> = 0;</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">teppic</span><span class="plain"> = </span><span class="identifier">ptraci</span><span class="plain">; </span> <span class="comment">Teppic saves Ptraci</span>
|
|
<span class="identifier">ptraci</span><span class="plain"> = </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>watched</span><span class="plain">;</span>
|
|
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">preform_lookahead_mode</span><span class="plain">) </span><span class="identifier">ptraci</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"%V: <%W>\</span><span class="plain">n</span><span class="string">"</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_id</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">input_length</span><span class="plain"> = </span><span class="functiontext">Wordings::length</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">nt</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain"> == 0) ||</span>
|
|
<span class="plain">((</span><span class="identifier">input_length</span><span class="plain"> >= </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>min_nt_words</span><span class="plain">) && (</span><span class="identifier">input_length</span><span class="plain"> <= </span><span class="identifier">nt</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain">))) {</span>
|
|
<<span class="cwebmacro">Try to match the input text to the nonterminal</span> <span class="cwebmacronumber">51.2</span>><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
|
|
<<span class="cwebmacro">The nonterminal has failed to parse</span> <span class="cwebmacronumber">51.1</span>><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::parse_nt_against_word_range is used in <a href="#SP51_2_1_2_3_3_5">§51.2.1.2.3.3.5</a>, <a href="#SP51_2_1_2_3_3_3_1">§51.2.1.2.3.3.3.1</a>, <a href="#SP52">§52</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_1"></a><b>§51.1. </b>The routine ends here...
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">The nonterminal has failed to parse</span> <span class="cwebmacronumber">51.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"Failed %V (time %d)\</span><span class="plain">n</span><span class="string">"</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_id</span><span class="plain">, </span><span class="identifier">time</span><span class="plain">(0)-</span><span class="identifier">start_of_nt</span><span class="plain">);</span>
|
|
<span class="identifier">ptraci</span><span class="plain"> = </span><span class="identifier">teppic</span><span class="plain">;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51">§51</a>, <a href="#SP51_2_1_2_1">§51.2.1.2.1</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP_1"></a><b>§.1. </b>...unless a match was made, in which case it ends here. At this point <code class="display"><span class="extract">Q</span></code>
|
|
and <code class="display"><span class="extract">QP</span></code> will hold the results of the match.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">The nonterminal has successfully parsed</span> <span class="cwebmacronumber">.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">result</span><span class="plain">) *</span><span class="identifier">result</span><span class="plain"> = </span><span class="identifier">Q</span><span class="plain">; </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">result_p</span><span class="plain">) *</span><span class="identifier">result_p</span><span class="plain"> = </span><span class="identifier">QP</span><span class="plain">;</span>
|
|
<span class="identifier">most_recent_result</span><span class="plain"> = </span><span class="identifier">Q</span><span class="plain">; </span><span class="identifier">most_recent_result_p</span><span class="plain"> = </span><span class="identifier">QP</span><span class="plain">;</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">INSTRUMENTED_PREFORM</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_matches</span><span class="plain">++;</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
<span class="identifier">ptraci</span><span class="plain"> = </span><span class="identifier">teppic</span><span class="plain">;</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="identifier">success_rval</span><span class="plain">;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2">§51.2</a> (twice), <a href="#SP51_2_1">§51.2.1</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2"></a><b>§51.2. </b>Here we see that a successful voracious NT returns the word number it got
|
|
to, rather than <code class="display"><span class="extract">TRUE</span></code>. Otherwise this is straightforward: we delegate to
|
|
an internal NT, or try all possible productions for an external one.
|
|
</p>
|
|
|
|
|
|
<pre class="definitions">
|
|
<span class="definitionkeyword">define</span> <span class="constant">RANGE_OPTIMISATION_LENGTH</span><span class="plain"> 10</span>
|
|
</pre>
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Try to match the input text to the nonterminal</span> <span class="cwebmacronumber">51.2</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">unoptimised</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="functiontext">Wordings::empty</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)) || (</span><span class="identifier">input_length</span><span class="plain"> >= </span><span class="constant">RANGE_OPTIMISATION_LENGTH</span><span class="plain">))</span>
|
|
<span class="identifier">unoptimised</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>internal_definition</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>voracious</span><span class="plain">) </span><span class="identifier">unoptimised</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">unoptimised</span><span class="plain">) || (</span><span class="functiontext">Preform::nt_bitmap_violates</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">, &(</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req</span><span class="plain">)) == </span><span class="identifier">FALSE</span><span class="plain">)) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">r</span><span class="plain">, </span><span class="identifier">Q</span><span class="plain">; </span><span class="reserved">void</span><span class="plain"> *</span><span class="identifier">QP</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">) >= 0) </span><span class="identifier">r</span><span class="plain"> = (*(</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>internal_definition</span><span class="plain">))(</span><span class="identifier">W</span><span class="plain">, &</span><span class="identifier">Q</span><span class="plain">, &</span><span class="identifier">QP</span><span class="plain">);</span>
|
|
<span class="reserved">else</span><span class="plain"> { </span><span class="identifier">r</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">; </span><span class="identifier">Q</span><span class="plain"> = 0; }</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">r</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>voracious</span><span class="plain">) </span><span class="identifier">success_rval</span><span class="plain"> = </span><span class="identifier">r</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"Succeeded %d\</span><span class="plain">n</span><span class="string">"</span><span class="plain">, </span><span class="identifier">time</span><span class="plain">(0)-</span><span class="identifier">start_of_nt</span><span class="plain">);</span>
|
|
<<span class="cwebmacro">The nonterminal has successfully parsed</span> <span class="cwebmacronumber">.1</span>><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">) {</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"%V: <%W> violates "</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_id</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">);</span>
|
|
<span class="functiontext">Preform::log_range_requirement</span><span class="plain">(&(</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req</span><span class="plain">));</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"\</span><span class="plain">n</span><span class="string">"</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">unoptimised</span><span class="plain">) || (</span><span class="functiontext">Preform::nt_bitmap_violates</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">, &(</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req</span><span class="plain">)) == </span><span class="identifier">FALSE</span><span class="plain">)) {</span>
|
|
<span class="reserved">void</span><span class="plain"> *</span><span class="identifier">acc_result</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="reserved">production_list</span><span class="plain"> *</span><span class="identifier">pl</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>first_production_list</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain">; </span><span class="identifier">pl</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>next_production_list</span><span class="plain">) {</span>
|
|
<span class="identifier">PREFORM_LANGUAGE_TYPE</span><span class="plain"> *</span><span class="identifier">nl</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>definition_language</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">language_of_source_text</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) || (</span><span class="identifier">language_of_source_text</span><span class="plain"> == </span><span class="identifier">nl</span><span class="plain">)) {</span>
|
|
<span class="reserved">production</span><span class="plain"> *</span><span class="identifier">pr</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">last_v</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pl</span><span class="plain">-</span><span class="element">>first_production</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain">; </span><span class="identifier">pr</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>next_production</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">violates</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">unoptimised</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_req.ditto_flag</span><span class="plain">) </span><span class="identifier">violates</span><span class="plain"> = </span><span class="identifier">last_v</span><span class="plain">;</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="identifier">violates</span><span class="plain"> = </span><span class="functiontext">Preform::nt_bitmap_violates</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">, &(</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_req</span><span class="plain">));</span>
|
|
<span class="identifier">last_v</span><span class="plain"> = </span><span class="identifier">violates</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">violates</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">) {</span>
|
|
<<span class="cwebmacro">Parse the given production</span> <span class="cwebmacronumber">51.2.1</span>><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">) {</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"production in %V: "</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_id</span><span class="plain">);</span>
|
|
<span class="functiontext">Preform::log_production</span><span class="plain">(</span><span class="identifier">pr</span><span class="plain">, </span><span class="identifier">FALSE</span><span class="plain">);</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">": <%W> violates "</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">);</span>
|
|
<span class="functiontext">Preform::log_range_requirement</span><span class="plain">(&(</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_req</span><span class="plain">));</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"\</span><span class="plain">n</span><span class="string">"</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>multiplicitous</span><span class="plain">) && (</span><span class="identifier">acc_result</span><span class="plain">)) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">Q</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">; </span><span class="reserved">void</span><span class="plain"> *</span><span class="identifier">QP</span><span class="plain"> = </span><span class="identifier">acc_result</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">The nonterminal has successfully parsed</span> <span class="cwebmacronumber">.1</span>><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">) {</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"%V: <%W> violates "</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_id</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">);</span>
|
|
<span class="functiontext">Preform::log_range_requirement</span><span class="plain">(&(</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>nonterminal_req</span><span class="plain">));</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"\</span><span class="plain">n</span><span class="string">"</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51">§51</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1"></a><b>§51.2.1. </b>So from here on we look only at the external case, where we're parsing the
|
|
text against a production.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Parse the given production</span> <span class="cwebmacronumber">51.2.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">) {</span>
|
|
<span class="identifier">LOG_INDENT</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Log the production match number</span> <span class="cwebmacronumber">51.2.1.1</span>><span class="plain">;</span>
|
|
<span class="functiontext">Preform::log_production</span><span class="plain">(</span><span class="identifier">pr</span><span class="plain">, </span><span class="identifier">FALSE</span><span class="plain">); </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"\</span><span class="plain">n</span><span class="string">"</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">INSTRUMENTED_PREFORM</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_tries</span><span class="plain">++;</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">slow_scan_needed</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">CORE_MODULE</span>
|
|
<span class="identifier">parse_node</span><span class="plain"> *</span><span class="identifier">added_to_result</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">input_length</span><span class="plain"> >= </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>min_pr_words</span><span class="plain">) && (</span><span class="identifier">input_length</span><span class="plain"> <= </span><span class="identifier">pr</span><span class="plain">-></span><span class="identifier">max_pr_words</span><span class="plain">)) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">Q</span><span class="plain">; </span><span class="reserved">void</span><span class="plain"> *</span><span class="identifier">QP</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Actually parse the given production, going to Fail if we can't</span> <span class="cwebmacronumber">51.2.1.2</span>><span class="character">;</span>
|
|
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">INSTRUMENTED_PREFORM</span><span class="plain"> </span> <span class="comment">record the sentence containing the longest example</span>
|
|
<span class="identifier">pr</span><span class="plain">-</span><span class="element">>production_matches</span><span class="plain">++;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Wordings::length</span><span class="plain">(</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>sample_text</span><span class="plain">) < </span><span class="functiontext">Wordings::length</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)) </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>sample_text</span><span class="plain"> = </span><span class="identifier">W</span><span class="plain">;</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">) {</span>
|
|
<<span class="cwebmacro">Log the production match number</span> <span class="cwebmacronumber">51.2.1.1</span>><span class="plain">;</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"succeeded (%s): "</span><span class="plain">, (</span><span class="identifier">slow_scan_needed</span><span class="plain">)?</span><span class="string">"slowly"</span><span class="plain">:</span><span class="string">"quickly"</span><span class="plain">);</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"result: %d\</span><span class="plain">n</span><span class="string">"</span><span class="plain">, </span><span class="identifier">Q</span><span class="plain">); </span><span class="identifier">LOG_OUTDENT</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<<span class="cwebmacro">The nonterminal has successfully parsed</span> <span class="cwebmacronumber">.1</span>><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
|
|
<span class="identifier">Fail</span><span class="plain">:</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">) {</span>
|
|
<<span class="cwebmacro">Log the production match number</span> <span class="cwebmacronumber">51.2.1.1</span>><span class="plain">;</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">CORE_MODULE</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">added_to_result</span><span class="plain">) </span><span class="identifier">LOG</span><span class="plain">(</span><span class="string">"added to result (%s): $P\</span><span class="plain">n</span><span class="string">"</span><span class="plain">,</span>
|
|
<span class="plain">(</span><span class="identifier">slow_scan_needed</span><span class="plain">)?</span><span class="string">"slowly"</span><span class="plain">:</span><span class="string">"quickly"</span><span class="plain">, </span><span class="identifier">added_to_result</span><span class="plain">);</span>
|
|
<span class="reserved">else</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"failed (%s)\</span><span class="plain">n</span><span class="string">"</span><span class="plain">, (</span><span class="identifier">slow_scan_needed</span><span class="plain">)?</span><span class="string">"slowly"</span><span class="plain">:</span><span class="string">"quickly"</span><span class="plain">);</span>
|
|
<span class="identifier">LOG_OUTDENT</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2">§51.2</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_1"></a><b>§51.2.1.1. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Log the production match number</span> <span class="cwebmacronumber">51.2.1.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>match_number</span><span class="plain"> >= 26) {</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"production /%c%c/: "</span><span class="plain">, </span><span class="character">'a'</span><span class="plain">+</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>match_number</span><span class="plain">-26, </span><span class="character">'a'</span><span class="plain">+</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>match_number</span><span class="plain">-26);</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"production /%c/: "</span><span class="plain">, </span><span class="character">'a'</span><span class="plain">+</span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>match_number</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1">§51.2.1</a> (three times).</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2"></a><b>§51.2.1.2. </b>Okay. So, the strategy is: a fast scan checking the easy things; if that's
|
|
not sufficient, a slow scan checking the rest; then making sure brackets
|
|
match, if there were any, and last composing the intermediate results into
|
|
the final ones. For example, if the production is
|
|
</p>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="plain">adjust the <achingly-slow> to the <exhaustive> at once</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph">then the fast scan verifies the presence of "adjust the" and "at once";
|
|
the slow scan next looks for all occurrences of "to the", the single strut
|
|
for this production; and only then does it test the two slow nonterminals
|
|
on the intervening words, if there are any.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Actually parse the given production, going to Fail if we can't</span> <span class="cwebmacronumber">51.2.1.2</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">checked</span><span class="plain">[</span><span class="constant">MAX_PTOKENS_PER_PRODUCTION</span><span class="plain">];</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">intermediates</span><span class="plain">[</span><span class="constant">MAX_RESULTS_PER_PRODUCTION</span><span class="plain">];</span>
|
|
<span class="reserved">void</span><span class="plain"> *</span><span class="identifier">intermediate_ps</span><span class="plain">[</span><span class="constant">MAX_RESULTS_PER_PRODUCTION</span><span class="plain">];</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">parsed_open_pos</span><span class="plain"> = -1, </span><span class="identifier">parsed_close_pos</span><span class="plain"> = -1;</span>
|
|
|
|
<<span class="cwebmacro">Try a fast scan through the production</span> <span class="cwebmacronumber">51.2.1.2.2</span>><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">slow_scan_needed</span><span class="plain">) </span><<span class="cwebmacro">Try a slow scan through the production</span> <span class="cwebmacronumber">51.2.1.2.3</span>><span class="plain">;</span>
|
|
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">parsed_open_pos</span><span class="plain"> >= 0) && (</span><span class="identifier">parsed_close_pos</span><span class="plain"> >= 0))</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Wordings::paired_brackets</span><span class="plain">(</span><span class="functiontext">Wordings::new</span><span class="plain">(</span><span class="identifier">parsed_open_pos</span><span class="plain">, </span><span class="identifier">parsed_close_pos</span><span class="plain">)) == </span><span class="identifier">FALSE</span><span class="plain">)</span>
|
|
<span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Fail</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Compose and store the result</span> <span class="cwebmacronumber">51.2.1.2.1</span>><span class="plain">;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1">§51.2.1</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2_1"></a><b>§51.2.1.2.1. </b>Once we have successfully matched the line, we need to compose the
|
|
intermediate results into a final result. If <code class="display"><span class="extract">inweb</span></code> has compiled a compositor
|
|
routine for the nonterminal, we call it: note that it can then return <code class="display"><span class="extract">FALSE</span></code>
|
|
to fail the production after all, and can even return <code class="display"><span class="extract">FAIL_NONTERMINAL</span></code> to
|
|
abandon not just this production, but all of the productions. (This is quite
|
|
useful as a way to put exceptional syntaxes into the grammar, since it can
|
|
make subsequent productions only available in some cases.)
|
|
</p>
|
|
|
|
<p class="inwebparagraph">If there's no compositor then the integer result is the production's number,
|
|
and the pointer result is null.
|
|
</p>
|
|
|
|
|
|
<pre class="definitions">
|
|
<span class="definitionkeyword">define</span> <span class="constant">FAIL_NONTERMINAL</span><span class="plain"> -100000</span>
|
|
<span class="definitionkeyword">define</span> <span class="constant">FAIL_NONTERMINAL_TO</span><span class="plain"> </span><span class="constant">FAIL_NONTERMINAL</span><span class="plain">+1000</span>
|
|
</pre>
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Compose and store the result</span> <span class="cwebmacronumber">51.2.1.2.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>result_compositor</span><span class="plain">) {</span>
|
|
<span class="identifier">intermediates</span><span class="plain">[0] = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>match_number</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">f</span><span class="plain"> = (*(</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>result_compositor</span><span class="plain">))(&</span><span class="identifier">Q</span><span class="plain">, &</span><span class="identifier">QP</span><span class="plain">, </span><span class="identifier">intermediates</span><span class="plain">, </span><span class="identifier">intermediate_ps</span><span class="plain">, </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>range_result</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">f</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Fail</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>multiplicitous</span><span class="plain">) {</span>
|
|
<span class="plain">#</span><span class="identifier">ifdef</span><span class="plain"> </span><span class="identifier">CORE_MODULE</span>
|
|
<span class="identifier">added_to_result</span><span class="plain"> = </span><span class="identifier">QP</span><span class="plain">;</span>
|
|
<span class="identifier">acc_result</span><span class="plain"> = (</span><span class="reserved">void</span><span class="plain"> *) </span><span class="identifier">ParseTree::add_possible_reading</span><span class="plain">((</span><span class="identifier">parse_node</span><span class="plain"> *) </span><span class="identifier">acc_result</span><span class="plain">, </span><span class="identifier">QP</span><span class="plain">, </span><span class="identifier">W</span><span class="plain">);</span>
|
|
<span class="plain">#</span><span class="identifier">endif</span>
|
|
<span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Fail</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">f</span><span class="plain"> >= </span><span class="constant">FAIL_NONTERMINAL</span><span class="plain">) && (</span><span class="identifier">f</span><span class="plain"> < </span><span class="constant">FAIL_NONTERMINAL_TO</span><span class="plain">)) {</span>
|
|
<span class="identifier">fail_nonterminal_quantum</span><span class="plain"> = </span><span class="identifier">f</span><span class="plain"> - </span><span class="constant">FAIL_NONTERMINAL</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">The nonterminal has failed to parse</span> <span class="cwebmacronumber">51.1</span>><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="identifier">Q</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>match_number</span><span class="plain">; </span><span class="identifier">QP</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1_2">§51.2.1.2</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2_2"></a><b>§51.2.1.2.2. </b>In the fast scan, we check that all fixed words with known positions
|
|
are in those positions.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Try a fast scan through the production</span> <span class="cwebmacronumber">51.2.1.2.2</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">wn</span><span class="plain"> = -1, </span><span class="identifier">tc</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>first_ptoken</span><span class="plain">, </span><span class="identifier">tc</span><span class="plain"> = 0; </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">, </span><span class="identifier">tc</span><span class="plain">++) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_is_fast</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">p</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_position</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">p</span><span class="plain"> > 0) </span><span class="identifier">wn</span><span class="plain"> = </span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)+</span><span class="identifier">p</span><span class="plain">-1;</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">p</span><span class="plain"> < 0) </span><span class="identifier">wn</span><span class="plain"> = </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)+</span><span class="identifier">p</span><span class="plain">+1;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Preform::parse_fixed_word_ptoken</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">, </span><span class="identifier">pt</span><span class="plain">) == </span><span class="identifier">FALSE</span><span class="plain">) {</span>
|
|
<span class="identifier">slow_scan_needed</span><span class="plain"> = </span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Fail</span><span class="plain">; </span> <span class="comment">the word should have been here, and it wasn't</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ve_pt</span><span class="plain"> == </span><span class="identifier">OPENBRACKET_V</span><span class="plain">) </span><span class="identifier">parsed_open_pos</span><span class="plain"> = </span><span class="identifier">wn</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ve_pt</span><span class="plain"> == </span><span class="identifier">CLOSEBRACKET_V</span><span class="plain">) </span><span class="identifier">parsed_close_pos</span><span class="plain"> = </span><span class="identifier">wn</span><span class="plain">;</span>
|
|
<span class="identifier">checked</span><span class="plain">[</span><span class="identifier">tc</span><span class="plain">] = </span><span class="identifier">wn</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="identifier">slow_scan_needed</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="identifier">checked</span><span class="plain">[</span><span class="identifier">tc</span><span class="plain">] = -1;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">slow_scan_needed</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">) && (</span><span class="identifier">wn</span><span class="plain"> != </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">))) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Fail</span><span class="plain">; </span> <span class="comment">input text goes on further</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1_2">§51.2.1.2</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2_3"></a><b>§51.2.1.2.3. </b>The slow scan is more challenging. We want to loop through all possible
|
|
strut positions, where by "possible" we mean that
|
|
s_i+l_i <= s_{i+1}, i = 0, 1, ..., s
|
|
and that for each i the i-th strut matches the text beginning at s_i.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Try a slow scan through the production</span> <span class="cwebmacronumber">51.2.1.2.3</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">spos</span><span class="plain">[</span><span class="constant">MAX_STRUTS_PER_PRODUCTION</span><span class="plain">]; </span> <span class="comment">word numbers for where we are trying the struts</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">NS</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>no_struts</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Start from the lexicographically earliest strut position</span> <span class="cwebmacronumber">51.2.1.2.3.1</span>><span class="plain">;</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">backtrack_token</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">backtrack_index</span><span class="plain"> = -1, </span><span class="identifier">backtrack_to</span><span class="plain"> = -1, </span><span class="identifier">backtrack_tc</span><span class="plain"> = -1;</span>
|
|
<span class="reserved">while</span><span class="plain"> (</span><span class="identifier">TRUE</span><span class="plain">) {</span>
|
|
<<span class="cwebmacro">Try a slow scan with the current strut positions</span> <span class="cwebmacronumber">51.2.1.2.3.3</span>><span class="plain">;</span>
|
|
<span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="identifier">FailThisStrutPosition</span><span class="plain">: ;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">backtrack_token</span><span class="plain">) </span><span class="reserved">continue</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Move on to the next strut position</span> <span class="cwebmacronumber">51.2.1.2.3.2</span>><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1_2">§51.2.1.2</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2_3_1"></a><b>§51.2.1.2.3.1. </b>We start by finding the lexicographically earliest, i.e., we find the earliest
|
|
possible position for s_0, then the earliest position from s_0+l_0 for
|
|
s_1, and so on. (Our wildcards are not greedy: we match with shortest possible
|
|
text rather than longest.)
|
|
</p>
|
|
|
|
<p class="inwebparagraph">In all of the code below, the general case with <code class="display"><span class="extract">NS</span></code> greater than 1 is actually
|
|
valid code for all cases, but experiment shows about a 5\% speed gain from
|
|
handling the popular case of one strut separately.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Start from the lexicographically earliest strut position</span> <span class="cwebmacronumber">51.2.1.2.3.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">NS</span><span class="plain"> == 1) {</span>
|
|
<span class="identifier">spos</span><span class="plain">[0] = </span><span class="functiontext">Preform::next_strut_posn_after</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">, </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>struts</span><span class="plain">[0], </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>strut_lengths</span><span class="plain">[0], </span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">));</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">spos</span><span class="plain">[0] == -1) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Fail</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">NS</span><span class="plain"> > 1) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">s</span><span class="plain">, </span><span class="identifier">from</span><span class="plain"> = </span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">);</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">s</span><span class="plain">=0; </span><span class="identifier">s</span><span class="plain"><</span><span class="identifier">NS</span><span class="plain">; </span><span class="identifier">s</span><span class="plain">++) {</span>
|
|
<span class="identifier">spos</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">] = </span><span class="functiontext">Preform::next_strut_posn_after</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">, </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>struts</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">], </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>strut_lengths</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">], </span><span class="identifier">from</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">spos</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">] == -1) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Fail</span><span class="plain">;</span>
|
|
<span class="identifier">from</span><span class="plain"> = </span><span class="identifier">spos</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">] + </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>strut_lengths</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">] + 1;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1_2_3">§51.2.1.2.3</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2_3_2"></a><b>§51.2.1.2.3.2. </b>In the general case, we move the final strut forward if we can; if we can't,
|
|
we move the penultimate one, then move the final one to the first subsequent
|
|
position valid for it; and so on. Ultimately this results in the first strut
|
|
being unable to move forwards, at which point, we've lost.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Move on to the next strut position</span> <span class="cwebmacronumber">51.2.1.2.3.2</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">NS</span><span class="plain"> == 0) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Fail</span><span class="plain">;</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">NS</span><span class="plain"> == 1) {</span>
|
|
<span class="identifier">spos</span><span class="plain">[0] = </span><span class="functiontext">Preform::next_strut_posn_after</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">, </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>struts</span><span class="plain">[0], </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>strut_lengths</span><span class="plain">[0], </span><span class="identifier">spos</span><span class="plain">[0]+1);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">spos</span><span class="plain">[0] == -1) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Fail</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">NS</span><span class="plain"> > 1) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">s</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">s</span><span class="plain">=</span><span class="identifier">NS</span><span class="plain">-1; </span><span class="identifier">s</span><span class="plain">>=0; </span><span class="identifier">s</span><span class="plain">--) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">n</span><span class="plain"> = </span><span class="functiontext">Preform::next_strut_posn_after</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">, </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>struts</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">], </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>strut_lengths</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">], </span><span class="identifier">spos</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">]+1);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">n</span><span class="plain"> != -1) { </span><span class="identifier">spos</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">] = </span><span class="identifier">n</span><span class="plain">; </span><span class="reserved">break</span><span class="plain">; }</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">s</span><span class="plain"> == -1) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Fail</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">from</span><span class="plain"> = </span><span class="identifier">spos</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">] + 1; </span><span class="identifier">s</span><span class="plain">++;</span>
|
|
<span class="reserved">for</span><span class="plain"> (; </span><span class="identifier">s</span><span class="plain"><</span><span class="identifier">NS</span><span class="plain">; </span><span class="identifier">s</span><span class="plain">++) {</span>
|
|
<span class="identifier">spos</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">] = </span><span class="functiontext">Preform::next_strut_posn_after</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">, </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>struts</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">], </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>strut_lengths</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">], </span><span class="identifier">from</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">spos</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">] == -1) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Fail</span><span class="plain">;</span>
|
|
<span class="identifier">from</span><span class="plain"> = </span><span class="identifier">spos</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">] + </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>strut_lengths</span><span class="plain">[</span><span class="identifier">s</span><span class="plain">] + 1;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1_2_3">§51.2.1.2.3</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2_3_3"></a><b>§51.2.1.2.3.3. </b>We can now forget about struts, thankfully, and check the remaining unchecked
|
|
ptokens.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Try a slow scan with the current strut positions</span> <span class="cwebmacronumber">51.2.1.2.3.3</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">wn</span><span class="plain"> = </span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">), </span><span class="identifier">tc</span><span class="plain">;</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">, *</span><span class="identifier">nextpt</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">backtrack_token</span><span class="plain">) {</span>
|
|
<span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">backtrack_token</span><span class="plain">; </span><span class="identifier">nextpt</span><span class="plain"> = </span><span class="identifier">backtrack_token</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">;</span>
|
|
<span class="identifier">tc</span><span class="plain"> = </span><span class="identifier">backtrack_tc</span><span class="plain">; </span><span class="identifier">wn</span><span class="plain"> = </span><span class="identifier">backtrack_to</span><span class="plain">;</span>
|
|
<span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Reenter</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pr</span><span class="plain">-</span><span class="element">>first_ptoken</span><span class="plain">, </span><span class="identifier">nextpt</span><span class="plain"> = (</span><span class="identifier">pt</span><span class="plain">)?(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">):</span><span class="identifier">NULL</span><span class="plain">, </span><span class="identifier">tc</span><span class="plain"> = 0;</span>
|
|
<span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">nextpt</span><span class="plain">, </span><span class="identifier">nextpt</span><span class="plain"> = (</span><span class="identifier">pt</span><span class="plain">)?(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">):</span><span class="identifier">NULL</span><span class="plain">, </span><span class="identifier">tc</span><span class="plain">++) {</span>
|
|
<span class="identifier">Reenter</span><span class="plain">: ;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">known_pos</span><span class="plain"> = </span><span class="identifier">checked</span><span class="plain">[</span><span class="identifier">tc</span><span class="plain">];</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">known_pos</span><span class="plain"> >= 0) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">wn</span><span class="plain"> > </span><span class="identifier">known_pos</span><span class="plain">) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">Fail</span><span class="plain">; </span> <span class="comment">a theoretical possibility if strut lookahead overreaches</span>
|
|
<span class="identifier">wn</span><span class="plain"> = </span><span class="identifier">known_pos</span><span class="plain">+1;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_starts</span><span class="plain"> >= 0) </span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>range_result</span><span class="plain">[</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_starts</span><span class="plain">] = </span><span class="functiontext">Wordings::one_word</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">);</span>
|
|
<span class="reserved">switch</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain">) {</span>
|
|
<span class="reserved">case</span><span class="plain"> </span><span class="constant">FIXED_WORD_PTC</span><span class="plain">: </span><<span class="cwebmacro">Match a fixed word ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.1</span>><span class="plain">; </span><span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="reserved">case</span><span class="plain"> </span><span class="constant">SINGLE_WILDCARD_PTC</span><span class="plain">: </span><<span class="cwebmacro">Match a single wildcard ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.2</span>><span class="plain">; </span><span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="reserved">case</span><span class="plain"> </span><span class="constant">MULTIPLE_WILDCARD_PTC</span><span class="plain">: </span><<span class="cwebmacro">Match a multiple wildcard ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.3</span>><span class="plain">; </span><span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="reserved">case</span><span class="plain"> </span><span class="constant">POSSIBLY_EMPTY_WILDCARD_PTC</span><span class="plain">: </span><<span class="cwebmacro">Match a possibly empty wildcard ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.4</span>><span class="plain">; </span><span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="reserved">case</span><span class="plain"> </span><span class="constant">NONTERMINAL_PTC</span><span class="plain">: </span><<span class="cwebmacro">Match a nonterminal ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.5</span>><span class="plain">; </span><span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_ends</span><span class="plain"> >= 0)</span>
|
|
<span class="identifier">nt</span><span class="plain">-</span><span class="element">>range_result</span><span class="plain">[</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_ends</span><span class="plain">] = </span><span class="functiontext">Wordings::up_to</span><span class="plain">(</span><span class="identifier">nt</span><span class="plain">-</span><span class="element">>range_result</span><span class="plain">[</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>range_ends</span><span class="plain">], </span><span class="identifier">wn</span><span class="plain">-1);</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">wn</span><span class="plain"> != </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)+1) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">FailThisStrutPosition</span><span class="plain">;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1_2_3">§51.2.1.2.3</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2_3_3_1"></a><b>§51.2.1.2.3.3.1. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Match a fixed word ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">q</span><span class="plain"> = </span><span class="functiontext">Preform::parse_fixed_word_ptoken</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">, </span><span class="identifier">pt</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">q</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">FailThisStrutPosition</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ve_pt</span><span class="plain"> == </span><span class="identifier">OPENBRACKET_V</span><span class="plain">) </span><span class="identifier">parsed_open_pos</span><span class="plain"> = </span><span class="identifier">wn</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ve_pt</span><span class="plain"> == </span><span class="identifier">CLOSEBRACKET_V</span><span class="plain">) </span><span class="identifier">parsed_close_pos</span><span class="plain"> = </span><span class="identifier">wn</span><span class="plain">;</span>
|
|
<span class="identifier">wn</span><span class="plain">++;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1_2_3_3">§51.2.1.2.3.3</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2_3_3_2"></a><b>§51.2.1.2.3.3.2. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Match a single wildcard ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.2</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="identifier">wn</span><span class="plain">++;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1_2_3_3">§51.2.1.2.3.3</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2_3_3_3"></a><b>§51.2.1.2.3.3.3. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Match a multiple wildcard ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.3</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">wn</span><span class="plain"> > </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">FailThisStrutPosition</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">wt</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Calculate how much to stretch this elastic ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.3.1</span>><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">wn</span><span class="plain"> > </span><span class="identifier">wt</span><span class="plain">) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">FailThisStrutPosition</span><span class="plain">; </span> <span class="comment">zero length</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>balanced_wildcard</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">i</span><span class="plain">, </span><span class="identifier">bl</span><span class="plain"> = 0;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">i</span><span class="plain">=</span><span class="identifier">wn</span><span class="plain">; </span><span class="identifier">i</span><span class="plain"><=</span><span class="identifier">wt</span><span class="plain">; </span><span class="identifier">i</span><span class="plain">++) {</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">) == </span><span class="identifier">OPENBRACKET_V</span><span class="plain">) || (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">) == </span><span class="identifier">OPENBRACE_V</span><span class="plain">)) </span><span class="identifier">bl</span><span class="plain">++;</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">) == </span><span class="identifier">CLOSEBRACKET_V</span><span class="plain">) || (</span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">i</span><span class="plain">) == </span><span class="identifier">CLOSEBRACE_V</span><span class="plain">)) {</span>
|
|
<span class="identifier">bl</span><span class="plain">--;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">bl</span><span class="plain"> < 0) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">FailThisStrutPosition</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">bl</span><span class="plain"> != 0) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">FailThisStrutPosition</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">wn</span><span class="plain"> = </span><span class="identifier">wt</span><span class="plain">+1;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1_2_3_3">§51.2.1.2.3.3</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2_3_3_4"></a><b>§51.2.1.2.3.3.4. </b><code class="display">
|
|
<<span class="cwebmacrodefn">Match a possibly empty wildcard ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.4</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">wt</span><span class="plain">;</span>
|
|
<<span class="cwebmacro">Calculate how much to stretch this elastic ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.3.1</span>><span class="plain">;</span>
|
|
<span class="identifier">wn</span><span class="plain"> = </span><span class="identifier">wt</span><span class="plain">+1;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1_2_3_3">§51.2.1.2.3.3</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2_3_3_5"></a><b>§51.2.1.2.3.3.5. </b>A voracious nonterminal is offered the entire rest of the word range, and
|
|
returns how much it ate. Otherwise, we offer the maximum amount of space
|
|
available: if, for word-count reasons, that's never going to match, then
|
|
we rely on the recursive call to <code class="display"><span class="extract">Preform::parse_nt_against_word_range</span></code> returning a
|
|
quick no.
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Match a nonterminal ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.5</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">wn</span><span class="plain"> > </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)) && (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>min_nt_words</span><span class="plain"> > 0)) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">FailThisStrutPosition</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">wt</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>voracious</span><span class="plain">) </span><span class="identifier">wt</span><span class="plain"> = </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">);</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>min_nt_words</span><span class="plain"> > 0) && (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>min_nt_words</span><span class="plain"> == </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain">))</span>
|
|
<span class="identifier">wt</span><span class="plain"> = </span><span class="identifier">wn</span><span class="plain"> + </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>min_nt_words</span><span class="plain"> - 1;</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><<span class="cwebmacro">Calculate how much to stretch this elastic ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.3.1</span>><span class="plain">;</span>
|
|
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> == </span><span class="identifier">backtrack_token</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">)</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"Reached backtrack position %V: <%W>\</span><span class="plain">n</span><span class="string">"</span><span class="plain">,</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>nonterminal_id</span><span class="plain">, </span><span class="functiontext">Wordings::new</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">, </span><span class="identifier">wt</span><span class="plain">));</span>
|
|
<span class="identifier">preform_backtrack</span><span class="plain"> = </span><span class="identifier">intermediate_ps</span><span class="plain">[</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>result_index</span><span class="plain">];</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">) </span><span class="identifier">LOG_INDENT</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">q</span><span class="plain"> = </span><span class="functiontext">Preform::parse_nt_against_word_range</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">, </span><span class="functiontext">Wordings::new</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">, </span><span class="identifier">wt</span><span class="plain">),</span>
|
|
<span class="plain">&(</span><span class="identifier">intermediates</span><span class="plain">[</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>result_index</span><span class="plain">]), &(</span><span class="identifier">intermediate_ps</span><span class="plain">[</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>result_index</span><span class="plain">]));</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">) </span><span class="identifier">LOG_OUTDENT</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> == </span><span class="identifier">backtrack_token</span><span class="plain">) { </span><span class="identifier">preform_backtrack</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">; </span><span class="identifier">backtrack_token</span><span class="plain"> = </span><span class="identifier">NULL</span><span class="plain">; }</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>voracious</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">q</span><span class="plain"> > 0) { </span><span class="identifier">wt</span><span class="plain"> = </span><span class="identifier">q</span><span class="plain">; </span><span class="identifier">q</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">; }</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">q</span><span class="plain"> < 0) { </span><span class="identifier">wt</span><span class="plain"> = -</span><span class="identifier">q</span><span class="plain">; </span><span class="identifier">q</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="identifier">backtrack_index</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>result_index</span><span class="plain">; </span><span class="identifier">backtrack_to</span><span class="plain"> = </span><span class="identifier">wn</span><span class="plain">;</span>
|
|
<span class="identifier">backtrack_token</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">backtrack_tc</span><span class="plain"> = </span><span class="identifier">tc</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">ptraci</span><span class="plain">)</span>
|
|
<span class="identifier">LOG</span><span class="plain">(</span><span class="string">"Set backtrack position %V: <%W>\</span><span class="plain">n</span><span class="string">"</span><span class="plain">,</span>
|
|
<span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-</span><span class="element">>nonterminal_id</span><span class="plain">, </span><span class="functiontext">Wordings::new</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">, </span><span class="identifier">wt</span><span class="plain">));</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> { </span><span class="identifier">wt</span><span class="plain"> = </span><span class="identifier">wn</span><span class="plain">; }</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain">) </span><span class="identifier">q</span><span class="plain"> = </span><span class="identifier">q</span><span class="plain">?</span><span class="identifier">FALSE</span><span class="plain">:</span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">q</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">FailThisStrutPosition</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain"> > 0) </span><span class="identifier">wn</span><span class="plain"> = </span><span class="identifier">wt</span><span class="plain">+1;</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1_2_3_3">§51.2.1.2.3.3</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP51_2_1_2_3_3_3_1"></a><b>§51.2.1.2.3.3.3.1. </b>How much text from the input should this ptoken match? We feed it as much
|
|
as possible, and to calculate that, we must either be at the end of the run,
|
|
or else know exactly where the next ptoken starts: because its position is
|
|
known, or because it's a strut.
|
|
</p>
|
|
|
|
<p class="inwebparagraph">This is why two elastic nonterminals in a row won't parse correctly:
|
|
</p>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="plain">frog <amphibian> <pond-preference> toad</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph">Preform is unable to work out where the central boundary will occur. In theory
|
|
it should try every possibility. But that's inefficient: in practice the
|
|
solution is to write the grammar to minimise these cases, and then to set up
|
|
<amphibian> as a voracious token, so that it decides the boundary position
|
|
for itself. (If <amphibian> is not voracious, the following calculation
|
|
probably gives the wrong answer.)
|
|
</p>
|
|
|
|
|
|
<p class="macrodefinition"><code class="display">
|
|
<<span class="cwebmacrodefn">Calculate how much to stretch this elastic ptoken</span> <span class="cwebmacronumber">51.2.1.2.3.3.3.1</span>> =
|
|
</code></p>
|
|
|
|
|
|
<pre class="displaydefn">
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">lookahead</span><span class="plain"> = </span><span class="identifier">nextpt</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">lookahead</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) </span><span class="identifier">wt</span><span class="plain"> = </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">);</span>
|
|
<span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">p</span><span class="plain"> = </span><span class="identifier">lookahead</span><span class="plain">-</span><span class="element">>ptoken_position</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">p</span><span class="plain"> > 0) </span><span class="identifier">wt</span><span class="plain"> = </span><span class="functiontext">Wordings::first_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)+</span><span class="identifier">p</span><span class="plain">-2;</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">p</span><span class="plain"> < 0) </span><span class="identifier">wt</span><span class="plain"> = </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)+</span><span class="identifier">p</span><span class="plain">;</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="reserved">if</span><span class="plain"> (</span><span class="identifier">lookahead</span><span class="plain">-</span><span class="element">>strut_number</span><span class="plain"> >= 0) </span><span class="identifier">wt</span><span class="plain"> = </span><span class="identifier">spos</span><span class="plain">[</span><span class="identifier">lookahead</span><span class="plain">-</span><span class="element">>strut_number</span><span class="plain">]-1;</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">lookahead</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">)</span>
|
|
<span class="plain">&& (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">)</span>
|
|
<span class="plain">&& (</span><span class="functiontext">Preform::ptoken_width</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">) == </span><span class="constant">PTOKEN_ELASTIC</span><span class="plain">)) {</span>
|
|
<span class="identifier">wt</span><span class="plain"> = -1;</span>
|
|
<span class="reserved">nonterminal</span><span class="plain"> *</span><span class="identifier">target</span><span class="plain"> = </span><span class="identifier">lookahead</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">save_preform_lookahead_mode</span><span class="plain"> = </span><span class="identifier">preform_lookahead_mode</span><span class="plain">;</span>
|
|
<span class="identifier">preform_lookahead_mode</span><span class="plain"> = </span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">j</span><span class="plain"> = </span><span class="identifier">wn</span><span class="plain">+1; </span><span class="identifier">j</span><span class="plain"> <= </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">); </span><span class="identifier">j</span><span class="plain">++) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Preform::parse_nt_against_word_range</span><span class="plain">(</span><span class="identifier">target</span><span class="plain">, </span><span class="functiontext">Wordings::new</span><span class="plain">(</span><span class="identifier">j</span><span class="plain">, </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">)), </span><span class="identifier">NULL</span><span class="plain">, </span><span class="identifier">NULL</span><span class="plain">)) {</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain"> == </span><span class="identifier">NULL</span><span class="plain">) ||</span>
|
|
<span class="plain">(</span><span class="functiontext">Preform::parse_nt_against_word_range</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">, </span><span class="functiontext">Wordings::new</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">, </span><span class="identifier">j</span><span class="plain">-1), </span><span class="identifier">NULL</span><span class="plain">, </span><span class="identifier">NULL</span><span class="plain">))) {</span>
|
|
<span class="identifier">wt</span><span class="plain"> = </span><span class="identifier">j</span><span class="plain">-1; </span><span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">fail_nonterminal_quantum</span><span class="plain"> > 0) </span><span class="identifier">j</span><span class="plain"> += </span><span class="identifier">fail_nonterminal_quantum</span><span class="plain"> - 1;</span>
|
|
<span class="plain">}</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">preform_lookahead_mode</span><span class="plain"> = </span><span class="identifier">save_preform_lookahead_mode</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">wt</span><span class="plain"> < 0) </span><span class="reserved">goto</span><span class="plain"> </span><span class="identifier">FailThisStrutPosition</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> </span><span class="identifier">wt</span><span class="plain"> = </span><span class="identifier">wn</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">This code is used in <a href="#SP51_2_1_2_3_3_3">§51.2.1.2.3.3.3</a>, <a href="#SP51_2_1_2_3_3_4">§51.2.1.2.3.3.4</a>, <a href="#SP51_2_1_2_3_3_5">§51.2.1.2.3.3.5</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP52"></a><b>§52. </b>Here we find the next possible match position for the strut beginning <code class="display"><span class="extract">start</span></code>
|
|
and of width <code class="display"><span class="extract">len</span></code> in words, which begins at word <code class="display"><span class="extract">from</span></code> or after. Note that
|
|
the strut might run up right to the end of the input text: for example, in
|
|
</p>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="plain">neckties ... tied ***</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph">the word "tied" is a strut, because the <code class="display"><span class="extract">***</span></code> makes its position uncertain,
|
|
but since <code class="display"><span class="extract">***</span></code> might match the empty text, "tied" might legally be the
|
|
last word in the input text.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::next_strut_posn_after</span><span class="plain">(</span><span class="reserved">wording</span><span class="plain"> </span><span class="identifier">W</span><span class="plain">, </span><span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">start</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">len</span><span class="plain">, </span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">from</span><span class="plain">) {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">last_legal_position</span><span class="plain"> = </span><span class="functiontext">Wordings::last_wn</span><span class="plain">(</span><span class="identifier">W</span><span class="plain">) - </span><span class="identifier">len</span><span class="plain"> + 1;</span>
|
|
<span class="reserved">while</span><span class="plain"> (</span><span class="identifier">from</span><span class="plain"> <= </span><span class="identifier">last_legal_position</span><span class="plain">) {</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">;</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">pos</span><span class="plain"> = </span><span class="identifier">from</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">start</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">pt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>next_ptoken</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>ptoken_category</span><span class="plain"> == </span><span class="constant">FIXED_WORD_PTC</span><span class="plain">) {</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="functiontext">Preform::parse_fixed_word_ptoken</span><span class="plain">(</span><span class="identifier">pos</span><span class="plain">, </span><span class="identifier">pt</span><span class="plain">)) </span><span class="identifier">pos</span><span class="plain">++;</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="plain">} </span><span class="reserved">else</span><span class="plain"> {</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">q</span><span class="plain"> = </span><span class="functiontext">Preform::parse_nt_against_word_range</span><span class="plain">(</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">,</span>
|
|
<span class="functiontext">Wordings::new</span><span class="plain">(</span><span class="identifier">pos</span><span class="plain">, </span><span class="identifier">pos</span><span class="plain">+</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain">-1),</span>
|
|
<span class="identifier">NULL</span><span class="plain">, </span><span class="identifier">NULL</span><span class="plain">);</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain">) </span><span class="identifier">q</span><span class="plain"> = </span><span class="identifier">q</span><span class="plain">?</span><span class="identifier">FALSE</span><span class="plain">:</span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">q</span><span class="plain">) </span><span class="identifier">pos</span><span class="plain"> += </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>nt_pt</span><span class="plain">-></span><span class="identifier">max_nt_words</span><span class="plain">;</span>
|
|
<span class="reserved">else</span><span class="plain"> </span><span class="reserved">break</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">pos</span><span class="plain">-</span><span class="identifier">from</span><span class="plain"> >= </span><span class="identifier">len</span><span class="plain">) </span><span class="reserved">return</span><span class="plain"> </span><span class="identifier">from</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
<span class="identifier">from</span><span class="plain">++;</span>
|
|
<span class="plain">}</span>
|
|
<span class="reserved">return</span><span class="plain"> -1;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::next_strut_posn_after is used in <a href="#SP51_2_1_2_3_1">§51.2.1.2.3.1</a>, <a href="#SP51_2_1_2_3_2">§51.2.1.2.3.2</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP53"></a><b>§53. </b>Finally, a single fixed word, with its annotations and alternatives.
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="functiontext">Preform::parse_fixed_word_ptoken</span><span class="plain">(</span><span class="reserved">int</span><span class="plain"> </span><span class="identifier">wn</span><span class="plain">, </span><span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">pt</span><span class="plain">) {</span>
|
|
<span class="reserved">vocabulary_entry</span><span class="plain"> *</span><span class="identifier">ve</span><span class="plain"> = </span><span class="functiontext">Lexer::word</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">);</span>
|
|
<span class="reserved">int</span><span class="plain"> </span><span class="identifier">m</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>disallow_unexpected_upper</span><span class="plain">;</span>
|
|
<span class="reserved">ptoken</span><span class="plain"> *</span><span class="identifier">alt</span><span class="plain">;</span>
|
|
<span class="reserved">for</span><span class="plain"> (</span><span class="identifier">alt</span><span class="plain"> = </span><span class="identifier">pt</span><span class="plain">; </span><span class="identifier">alt</span><span class="plain">; </span><span class="identifier">alt</span><span class="plain"> = </span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>alternative_ptoken</span><span class="plain">)</span>
|
|
<span class="reserved">if</span><span class="plain"> ((</span><span class="identifier">ve</span><span class="plain"> == </span><span class="identifier">alt</span><span class="plain">-</span><span class="element">>ve_pt</span><span class="plain">) &&</span>
|
|
<span class="plain">((</span><span class="identifier">m</span><span class="plain"> == </span><span class="identifier">FALSE</span><span class="plain">) || (</span><span class="functiontext">Word::unexpectedly_upper_case</span><span class="plain">(</span><span class="identifier">wn</span><span class="plain">) == </span><span class="identifier">FALSE</span><span class="plain">)))</span>
|
|
<span class="reserved">return</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain">)?</span><span class="identifier">FALSE</span><span class="plain">:</span><span class="identifier">TRUE</span><span class="plain">;</span>
|
|
<span class="reserved">return</span><span class="plain"> (</span><span class="identifier">pt</span><span class="plain">-</span><span class="element">>negated_ptoken</span><span class="plain">)?</span><span class="identifier">TRUE</span><span class="plain">:</span><span class="identifier">FALSE</span><span class="plain">;</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::parse_fixed_word_ptoken is used in <a href="#SP51_2_1_2_2">§51.2.1.2.2</a>, <a href="#SP51_2_1_2_3_3_1">§51.2.1.2.3.3.1</a>, <a href="#SP52">§52</a>.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP54"></a><b>§54. Reading Preform syntax from a file. </b></p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">wording</span><span class="plain"> </span><span class="functiontext">Preform::load_from_file</span><span class="plain">(</span><span class="identifier">filename</span><span class="plain"> *</span><span class="identifier">F</span><span class="plain">) {</span>
|
|
<span class="constant">feed_t</span><span class="plain"> </span><span class="identifier">id</span><span class="plain"> = </span><span class="functiontext">Feeds::begin</span><span class="plain">();</span>
|
|
<span class="reserved">if</span><span class="plain"> (</span><span class="identifier">TextFiles::read</span><span class="plain">(</span><span class="identifier">F</span><span class="plain">, </span><span class="identifier">FALSE</span><span class="plain">,</span>
|
|
<span class="identifier">NULL</span><span class="plain">, </span><span class="identifier">FALSE</span><span class="plain">, </span><span class="functiontext">Preform::preform_helper</span><span class="plain">, </span><span class="identifier">NULL</span><span class="plain">, </span><span class="identifier">NULL</span><span class="plain">) == </span><span class="identifier">FALSE</span><span class="plain">)</span>
|
|
<span class="identifier">internal_error</span><span class="plain">(</span><span class="string">"Unable to open Preform definition"</span><span class="plain">);</span>
|
|
<span class="reserved">return</span><span class="plain"> </span><span class="functiontext">Feeds::end</span><span class="plain">(</span><span class="identifier">id</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::load_from_file appears nowhere else.</p>
|
|
|
|
<p class="inwebparagraph"><a id="SP55"></a><b>§55. </b>We simply feed the lines one at a time:
|
|
</p>
|
|
|
|
|
|
<pre class="display">
|
|
<span class="reserved">void</span><span class="plain"> </span><span class="functiontext">Preform::preform_helper</span><span class="plain">(</span><span class="identifier">text_stream</span><span class="plain"> *</span><span class="identifier">item_name</span><span class="plain">,</span>
|
|
<span class="identifier">text_file_position</span><span class="plain"> *</span><span class="identifier">tfp</span><span class="plain">, </span><span class="reserved">void</span><span class="plain"> *</span><span class="identifier">vnl</span><span class="plain">) {</span>
|
|
<span class="identifier">WRITE_TO</span><span class="plain">(</span><span class="identifier">item_name</span><span class="plain">, </span><span class="string">"\</span><span class="plain">n</span><span class="string">"</span><span class="plain">);</span>
|
|
<span class="functiontext">Feeds::feed_stream_punctuated</span><span class="plain">(</span><span class="identifier">item_name</span><span class="plain">, </span><span class="constant">PREFORM_PUNCTUATION_MARKS</span><span class="plain">);</span>
|
|
<span class="plain">}</span>
|
|
</pre>
|
|
|
|
<p class="inwebparagraph"></p>
|
|
|
|
<p class="endnote">The function Preform::preform_helper is used in <a href="#SP54">§54</a>.</p>
|
|
|
|
<hr class="tocbar">
|
|
<ul class="toc"><li><a href="4-nw.html">Back to 'Numbered Words'</a></li><li><a href="4-bn.html">Continue with 'Basic Nonterminals'</a></li></ul><hr class="tocbar">
|
|
<!--End of weave-->
|
|
</body>
|
|
</html>
|
|
|