<ulclass="crumbs"><li><ahref="../index.html">Home</a></li><li><ahref="../intern.html">Inter Modules</a></li><li><ahref="index.html">bytecode</a></li><li><ahref="index.html#P">Preliminaries</a></li><li><b>What This Module Does</b></li></ul></div>
<ulclass="toc"><li><ahref="P-wtmd.html#SP1">§1. Prerequisites</a></li><li><ahref="P-wtmd.html#SP2">§2. What is intermediate about inter</a></li><li><ahref="P-wtmd.html#SP3">§3. Textual, Binary, Memory</a></li><li><ahref="P-wtmd.html#SP4">§4. What textual Inter looks like</a></li><li><ahref="P-wtmd.html#SP5">§5. What binary Inter looks like</a></li><li><ahref="P-wtmd.html#SP6">§6. Data and types stored within bytecode</a></li><li><ahref="P-wtmd.html#SP8">§8. Symbols</a></li><li><ahref="P-wtmd.html#SP9">§9. What memory Inter looks like</a></li><li><ahref="P-wtmd.html#SP10">§10. Nodes and instructions</a></li><li><ahref="P-wtmd.html#SP12">§12. Wiring</a></li></ul><hrclass="tocbar">
<pclass="commentary firstcommentary"><aid="SP1"class="paragraph-anchor"></a><b>§1. Prerequisites. </b>The bytecode module is a part of the Inform compiler toolset. It is
full details, but essentially: it's C without predeclarations or header files,
and where functions have names like <spanclass="extract"><spanclass="extract-syntax">Tags::add_by_name</span></span> rather than just <spanclass="extract"><spanclass="extract-syntax">add_by_name</span></span>.
<pclass="commentary firstcommentary"><aid="SP2"class="paragraph-anchor"></a><b>§2. What is intermediate about inter. </b>This module is concerned with managing the <ahref="2-it.html#SP1"class="internal">inter_tree</a> data structure in
language by <ahref="../inform7/index.html"class="internal">inform7</a> and more conventional C-like code by <ahref="../inter/index.html"class="internal">inter</a>.
Having very different syntaxes, these have different ASTs:
<ulclass="items"><li>● For I7, a <spanclass="extract"><spanclass="extract-syntax">parse_node_tree</span></span>, managed by the <ahref="../syntax-module/index.html"class="internal">syntax</a> module.
</li><li>● For Inter, an <spanclass="extract"><spanclass="extract-syntax">inter_schema</span></span>, managed by the <ahref="../building-module/index.html"class="internal">building</a> module.
<ulclass="footnotetexts"><liclass="footnote"id="fn:1"><pclass="inwebfootnote"><supid="fnref:1"><ahref="#fn:1"rel="footnote">1</a></sup> In fact Inter schemas are so useful as a tool for generating short runs of
Inter that the main <ahref="../inform7/index.html"class="internal">inform7</a> compiler also uses them from time to time, but
<ahref="#fnref:1"title="return to text">↩</a></p></li><liclass="footnote"id="fn:2"><pclass="inwebfootnote"><supid="fnref:2"><ahref="#fn:2"rel="footnote">2</a></sup> Though IRs vary considerably. Microsoft's Common Intermediate Language (CIL),
used as a back-end by C#, has quite low-level bytecode but stores it in a
highly structured object-oriented way.
<ahref="#fnref:2"title="return to text">↩</a></p></li></ul>
<pclass="commentary firstcommentary"><aid="SP3"class="paragraph-anchor"></a><b>§3. Textual, Binary, Memory. </b>Inter code has three representations: as a binary file, as a textual file,
and in memory — a sort of cross-referenced form of binary. Binary or
textual inter files can be read in as memory inter, and memory inter can
be written out as either binary or textual files. Any inter program can
<pclass="commentary firstcommentary"><aid="SP4"class="paragraph-anchor"></a><b>§4. What textual Inter looks like. </b>There is a manual for writing <ahref="../inter/M-ti.html"class="internal">Textual Inter (in inter)</a>, and this may now be
worth skimming through. But here is a minimal example:
<pclass="commentary">This is because <spanclass="extract"><spanclass="extract-syntax">packagetype</span></span> and <spanclass="extract"><spanclass="extract-syntax">primitive</span></span> instructions are optional in textual
Inter. When we read <spanclass="extract"><spanclass="extract-syntax">package Main _code</span></span>, for example, we deduce that a <spanclass="extract"><spanclass="extract-syntax">_code</span></span>
package type is needed, and so we automatically declare it if it is not there
already; and similarly for any primitive like <spanclass="extract"><spanclass="extract-syntax">!enableprinting</span></span>, provided that
it is one of those in the standard set. (See <ahref="../building-module/1-ip.html"class="internal">Inter Primitives (in building)</a>.)
Nevertheless, those instructions are part of the program, which is why they
are printed out when we write it back as textual Inter.
</p>
<pclass="commentary"><spanclass="extract"><spanclass="extract-syntax">packagetype</span></span>, <spanclass="extract"><spanclass="extract-syntax">primitive</span></span>, <spanclass="extract"><spanclass="extract-syntax">package</span></span>, <spanclass="extract"><spanclass="extract-syntax">code</span></span>, <spanclass="extract"><spanclass="extract-syntax">inv</span></span> and so on are all examples
of <ahref="3-ic.html"class="internal">Inter Constructs</a>. Each has its own textual syntax. Most constructs give
rise to instructions — for example, every line using the <spanclass="extract"><spanclass="extract-syntax">val</span></span> construct
results in a single <spanclass="extract"><spanclass="extract-syntax">VAL_IST</span></span> instruction in the program — but just a few
"pseudo-constructs" such as <spanclass="extract"><spanclass="extract-syntax">version</span></span> specify something else.
</p>
<pclass="commentary">So it is not true that lines in textual Inter correspond exactly to the
instructions in a program, but it's very nearly true.
</p>
<pclass="commentary firstcommentary"><aid="SP5"class="paragraph-anchor"></a><b>§5. What binary Inter looks like. </b>The "hello world" program above would occupy a single <ahref="2-it.html#SP1"class="internal">inter_tree</a> once loaded
in to memory.
</p>
<pclass="commentary">The main organising idea of Inter trees is the <ahref="2-pck.html#SP2"class="internal">inter_package</a>. <ahref="2-pck.html"class="internal">Packages</a> are
like nested boxes: each one can hold either more packages, or Inter instructions
providing code or data, or both. In the case of "hello world":
<pclass="commentary">Each package has a name, and its location can be identified by a "URL". For
example, <spanclass="extract"><spanclass="extract-syntax">/main/BasicInformKit/properties</span></span> means "the package <spanclass="extract"><spanclass="extract-syntax">properties</span></span>
inside the package <spanclass="extract"><spanclass="extract-syntax">BasicInformKit</span></span> inside the package <spanclass="extract"><spanclass="extract-syntax">main</span></span>". Every package
also as a "package type". (This is not the same thing as a data type.) <spanclass="extract"><spanclass="extract-syntax">main</span></span>
always has type <spanclass="extract"><spanclass="extract-syntax">_plain</span></span>; any package holding a function body has type <spanclass="extract"><spanclass="extract-syntax">_code</span></span>.
All package types begin with an underscore <spanclass="extract"><spanclass="extract-syntax">_</span></span>.
<pclass="commentary firstcommentary"><aid="SP6"class="paragraph-anchor"></a><b>§6. Data and types stored within bytecode. </b>Each instruction occupies a sequence of words called bytecode,<supid="fnref:3"><ahref="#fn:3"rel="footnote">3</a></sup> called its
"frame": see <ahref="2-in.html"class="internal">Inter Nodes</a>. The opening word identifies which construct is
used: for example, if this is <spanclass="extract"><spanclass="extract-syntax">PACKAGE_IST</span></span> then the instruction is a <spanclass="extract"><spanclass="extract-syntax">package</span></span>.
What the remaining words mean depends on the construct, but here are some
typical ingredients:
</p>
<ulclass="items"><li>● Many constructs —<spanclass="extract"><spanclass="extract-syntax">constant</span></span>, for example — define a new symbol.
If so, the symbol ID — or SID — will be stored in one of the words;
this is the ID of the symbol in the <ahref="2-st.html#SP1"class="internal">inter_symbols_table</a> belonging
to the package containing the instruction. Some constructs also contain
SIDs for other reasons: for example, <spanclass="extract"><spanclass="extract-syntax">propertyvalue</span></span> needs to store the
SID of the property whose value is being recorded.
</li><li>● Values in Inter occupy two consecutive words of bytecode, and these
are called "pairs": see <ahref="3-ivp.html"class="internal">Inter Value Pairs</a>.
</li><li>● Some constructs also need to store a type ID, or TID. See <ahref="3-idt.html"class="internal">Inter Data Types</a>.
</li></ul>
<pclass="commentary">With both values and types, we need to be able to express an enormous range
of possibilities. This seems impossible. For example, how can we fit the list
<spanclass="extract"><spanclass="extract-syntax">{2, 3, 5, 7, 11, 13, 17, 19}</span></span> in two words, or the type <spanclass="extract"><spanclass="extract-syntax">function int32 int2 -> void</span></span>
in just one?
</p>
<pclass="commentary">In both cases the solution is the same: to use <spanclass="extract"><spanclass="extract-syntax">constant</span></span> or <spanclass="extract"><spanclass="extract-syntax">typename</span></span> to
assign a symbol to anything complicated, and then refer to that symbol. For
<ulclass="footnotetexts"><liclass="footnote"id="fn:3"><pclass="inwebfootnote"><supid="fnref:3"><ahref="#fn:3"rel="footnote">3</a></sup> The term "bytecode" is a misnomer, since this is word-based, not byte-based.
But it is traditional and seems to have been used as far back as the mid-1960s.
<ahref="#fnref:3"title="return to text">↩</a></p></li></ul>
<pclass="commentary firstcommentary"><aid="SP7"class="paragraph-anchor"></a><b>§7. </b>Constants are useful also for providing metadata about the program. This
is not simply commentary: what makes it "meta" is that it does not literally
<pclass="commentary">Here <spanclass="extract"><spanclass="extract-syntax">lucky_number</span></span> can be used in the program whenever a value is needed. But
<spanclass="extract"><spanclass="extract-syntax">^special_constant</span></span>, whose name begins with the magic metadata caret <spanclass="extract"><spanclass="extract-syntax">^</span></span>, cannot
be used as a value. Instead, the idea is that it communicates something to the
code-generation code in <ahref="../pipeline-module/index.html"class="internal">pipeline</a> and <ahref="../final-module/index.html"class="internal">final</a>— indicating the significance,
purpose or origins of something in the program. (<ahref="../inform7/index.html"class="internal">inform7</a> produces a lot
of metadata like this.)
</p>
<pclass="commentary">See <ahref="3-mtd.html"class="internal">Metadata</a> for functions to access this metadata.
</p>
<pclass="commentary firstcommentary"><aid="SP8"class="paragraph-anchor"></a><b>§8. Symbols. </b>Names of constants, packages, primitives and so on are all examples of "symbols".
</p>
<pclass="commentary">Packages provide <ahref="2-st.html"class="internal">Symbols Tables</a>: in fact, each package has its own symbols
table, recording symbols and their meanings within that package. For example,
if a package <spanclass="extract"><spanclass="extract-syntax">X</span></span> contains a definition of a constant called <spanclass="extract"><spanclass="extract-syntax">pi</span></span>, then the
definition will occupy an Inter instruction inside the package, and the
identifier name <spanclass="extract"><spanclass="extract-syntax">pi</span></span> will be an <ahref="2-sym.html#SP1"class="internal">inter_symbol</a> recorded in its <ahref="2-st.html#SP1"class="internal">inter_symbols_table</a>.
<pclass="commentary firstcommentary"><aid="SP9"class="paragraph-anchor"></a><b>§9. What memory Inter looks like. </b>Inter code stored in memory is not simply a binary copy of what the same thing
would be if stored in a binary Inter file: it is very heavily cross-referenced
for rapid access, editing and rearrangement.
</p>
<pclass="commentary">There is a lot of memory to be managed here: Inter trees can be huge, though
there are never more than one or two in memory at once.
</p>
<pclass="commentary">In particular, each <ahref="2-it.html#SP1"class="internal">inter_tree</a> structure contains two pools of data
besides the actual tree:<supid="fnref:4"><ahref="#fn:4"rel="footnote">4</a></sup>
</p>
<ulclass="items"><li>(a) A "building site", which contains workspace data needed by the <ahref="../building-module/index.html"class="internal">building</a>
module. <ahref="../building-module/index.html"class="internal">building</a> is essentially a piece of middleware sitting on top of
this one, and making it easier for the compilers to use our facilities. We
will ignore the building site completely here: it's not our problem.
</li><li>(b) A "warehouse", which very much is our problem: see <ahref="2-tw.html"class="internal">The Warehouse</a>.
This provides storage for strings, symbols tables and the like, assigning each
one an ID number. Resource number 178, for example, might be a <spanclass="extract"><spanclass="extract-syntax">text_stream</span></span>
which is the content of some text literal in a function, while 179 might be
an <ahref="2-st.html#SP1"class="internal">inter_symbols_table</a> belonging to some package.
</li></ul>
<ulclass="footnotetexts"><liclass="footnote"id="fn:4"><pclass="inwebfootnote"><supid="fnref:4"><ahref="#fn:4"rel="footnote">4</a></sup> In real-life botany, trees do not have building sites or warehouses, but
mixing some metaphors cannot really be helped. Trees in nature do not grow
the way they do in computer science.
<ahref="#fnref:4"title="return to text">↩</a></p></li></ul>
<pclass="commentary firstcommentary"><aid="SP10"class="paragraph-anchor"></a><b>§10. Nodes and instructions. </b>Each node in an Inter tree represents a single Inter instruction,<supid="fnref:5"><ahref="#fn:5"rel="footnote">5</a></sup> details of
which are stored as a stretch of bytecode in memory.
</p>
<pclass="commentary">This use of both a tree and also a mass of binary bytecode is an attempt to
have our cake and eat it. The tree structure makes it quick and easy to splice,
cut and reorder code; the binary bytecode storage is quick to load from a file.
Still, the result is an unusual hybrid of a data structure.
</p>
<pclass="commentary">For example, the tree might start out like this:
<pclass="commentary">Here <spanclass="extract"><spanclass="extract-syntax">node1</span></span> represents an instruction, with the details stored at bytecode
locations 103 to 105; <spanclass="extract"><spanclass="extract-syntax">node2</span></span> points to bytecode at 106 to 107, and so on.
But then we could decide, when optimising code, that we want instructions
<spanclass="extract"><spanclass="extract-syntax">node2</span></span> and <spanclass="extract"><spanclass="extract-syntax">node3</span></span> performed the other way round. Simple amendments to
the tree structure achieve this without needing to edit the bytecode:
<pclass="commentary">Indeed, we could decide that the instruction at <spanclass="extract"><spanclass="extract-syntax">node2</span></span> is redundant and cut it:
<pclass="commentary">It doesn't matter that the resulting bytecode storage is all mixed up in
sequencing; the tree is what gives us the sequence of instructions, and the
order of words in bytecode memory is only significant within a single
instruction.
</p>
<ulclass="footnotetexts"><liclass="footnote"id="fn:5"><pclass="inwebfootnote"><supid="fnref:5"><ahref="#fn:5"rel="footnote">5</a></sup> Well, except for the root node, which has no real meaning. But there is
only one of those.
<ahref="#fnref:5"title="return to text">↩</a></p></li></ul>
<pclass="commentary firstcommentary"><aid="SP11"class="paragraph-anchor"></a><b>§11. </b>As these diagrams suggest, we can generate Inter instructions quite flexibly,
and are under no obligation to do so in sequence or all at once. (Indeed, we
can add entirely new instructions in the linking process or when optimising
code.)
</p>
<pclass="commentary">So it is very useful to have a way to keep "bookmarks" in the tree, as positions
where we are currently writing code, and might want to return to. For this
purpose, we have the <ahref="2-bkm.html#SP1"class="internal">inter_bookmark</a> type, which can represent any feasible
write position in the tree. (This is not the same thing as representing any
existing node in the tree: see <ahref="2-bkm.html"class="internal">Bookmarks</a> for more.)
</p>
<pclass="commentary">And this in turn allows for a simple API for <ahref="2-np.html"class="internal">Node Placement</a>, allowing us
to move or remove nodes in the tree, and to keep track of cursor-like moving
bookmark positions when we generate a stream of new nodes and place them one
after another.
</p>
<pclass="commentary firstcommentary"><aid="SP12"class="paragraph-anchor"></a><b>§12. Wiring. </b>The bytecode in a package can only refer to resources using symbols in that
in another:<supid="fnref:6"><ahref="#fn:6"rel="footnote">6</a></sup> see <ahref="2-tw2.html"class="internal">The Wiring</a>. We write <spanclass="extract"><spanclass="extract-syntax">S ~~> T</span></span> if the symbol <spanclass="extract"><spanclass="extract-syntax">S</span></span> is "wired to"
<spanclass="extract"><spanclass="extract-syntax">T</span></span>, and we understand this as meaning that <spanclass="extract"><spanclass="extract-syntax">S</span></span> means whatever <spanclass="extract"><spanclass="extract-syntax">T</span></span> does.
<pclass="commentary">In this example, the symbol <spanclass="extract"><spanclass="extract-syntax">earth</span></span> in package <spanclass="extract"><spanclass="extract-syntax">X</span></span> is undefined. Instead it is
wired to a different symbol of the same name in package <spanclass="extract"><spanclass="extract-syntax">Y</span></span>, which is defined
as the name of a variable declared in that package. (The names do not have to
be the same, but they often are.)
</p>
<pclass="commentary">Wiring is directional: <spanclass="extract"><spanclass="extract-syntax">S ~~> T</span></span> very definitely does not mean that <spanclass="extract"><spanclass="extract-syntax">T ~~> S</span></span>,
and indeed circuits are forbidden, because <spanclass="extract"><spanclass="extract-syntax">S1 ~~> S2 ~~> ... ~~> S1</span></span> would
create a circular definition. To change metaphor for a moment, it's as if, on
looking up <spanclass="extract"><spanclass="extract-syntax">S</span></span> in the index of a book, we found the entry "<spanclass="extract"><spanclass="extract-syntax">S</span></span>, see <spanclass="extract"><spanclass="extract-syntax">T</span></span>": we
then have to look up <spanclass="extract"><spanclass="extract-syntax">T</span></span> to find, say, "<spanclass="extract"><spanclass="extract-syntax">T</span></span>, 125", and turn to page 125. It
would be no good to find instead "<spanclass="extract"><spanclass="extract-syntax">T</span></span>, see <spanclass="extract"><spanclass="extract-syntax">S</span></span>".
<ulclass="footnotetexts"><liclass="footnote"id="fn:6"><pclass="inwebfootnote"><supid="fnref:6"><ahref="#fn:6"rel="footnote">6</a></sup> There are fleeting exceptional cases when a symbol can be wired to another
<ahref="#fnref:6"title="return to text">↩</a></p></li></ul>
<pclass="commentary firstcommentary"><aid="SP13"class="paragraph-anchor"></a><b>§13. </b>Special symbols called plugs and sockets are used to import or export meanings
from one tree of Inter code to a potential other tree, which will be "linked"
into it later on.
</p>
<pclass="commentary">For example, <ahref="../inform7/index.html"class="internal">inform7</a> compiles a tree of Inter, but then <ahref="../inter/index.html"class="internal">inter</a> links
this with a separately compiled Inter tree from <ahref="../BasicInformKit/index.html"class="internal">BasicInformKit</a>. Each both
<pclass="commentary">It would be chaotic<supid="fnref:7"><ahref="#fn:7"rel="footnote">7</a></sup> to allow random symbols in packages all over each tree
to be wired directly to symbols in the other. Instead, every tree has a sort
of embassy package <spanclass="extract"><spanclass="extract-syntax">/main/connectors</span></span> (a package called <spanclass="extract"><spanclass="extract-syntax">connectors</span></span> which is
a subpackage of <spanclass="extract"><spanclass="extract-syntax">main</span></span>) which acts as an intermediary.
<ulclass="footnotetexts"><liclass="footnote"id="fn:7"><pclass="inwebfootnote"><supid="fnref:7"><ahref="#fn:7"rel="footnote">7</a></sup> And also slow, and prone to namespace collisions.
<ahref="#fnref:7"title="return to text">↩</a></p></li></ul>
<pclass="commentary firstcommentary"><aid="SP14"class="paragraph-anchor"></a><b>§14. </b>It is not literally the case that plugs in one tree are wired to sockets in
another, as the diagram above suggests. The actual wiring-together occurs only
when (part of) one tree is merged into another, in what is called <ahref="2-trn.html"class="internal">Transmigration</a>.
</p>
<pclass="commentary">Transmigration is by definition the process of moving a package from one tree
to another. Almost the whole design of Inter is motivated by the need to make this
fast — the hierarchies of packages, the use of wiring, and the existence of sockets
and plugs all came about working backwards from the goal of implementing
transmigration efficiently.
</p>
<pclass="commentary">Transmigration is how the <ahref="../pipeline-module/index.html"class="internal">pipeline</a> for processing Inter links a tree
produced by <ahref="../inform7/index.html"class="internal">inform7</a> to trees from kits produced by <ahref="../inter/index.html"class="internal">inter</a>. This
diagram is also a little simplified, but the idea is right. We start with:
<pclass="commentary">The original BasicInformKit tree is reduced to a husk and can be discarded.
</p>
<pclass="commentary">Plugs and sockets are important here because when BasicInformKit moves to the
main tree, its plugs looking for meanings in that tree can now be connected
to sockets in it; and conversely, plugs in the main tree hoping to connect
to meanings in BasicInformKit can now connect to the relevant sockets.
</p>
<pclass="commentary">There are conventions on what goes in the <spanclass="extract"><spanclass="extract-syntax">main</span></span> package of each tree: see
<ahref="../building-module/1-ls.html"class="internal">Large-Scale Structure (in building)</a> for more on that. (The <spanclass="extract"><spanclass="extract-syntax">architectural</span></span>
package in each tree just makes some definitions establishing the size of
integers, and so on, and for these two trees whose definitions will just be