Bytecode

The bytecode is Visual FoxPro's p-code analogue: the serialisable output of the compiler and the only input of the VM. It is a stack machine with a deliberately fat instruction set. Where a small machine would spell REPLACE out as twenty loads and stores, this one has a ReplaceField instruction, so the interpreter loop is one match, a statement boundary is an explicit instruction, and a disassembly reads like the program it came from.

This page is the format at version 9 and a tour of what the compiler makes of ordinary programs. Every listing on it is the output of foxvm disasm. The instruction reference lists all 199 instructions with their operands and stack effects, generated from the VM's source.

From source to module

The compiler produces a Module. In the IDE it is handed straight to load_module; Build App encodes it to bytes and stores them base64 in the .fxa bundle, and the player decodes them again. Nothing about a module depends on which road it took.

The container

A module is a byte string:

"FXVM"            4 bytes, the magic
version           u16, little-endian; 9 today
body              the Module, encoded with postcard

A module with the wrong magic is refused as not a FoxVM module. A module with another version is refused with "Module was built with bytecode version N; this runtime expects 9. Rebuild the project." - the format is not versioned for compatibility, it is versioned so that a stale .fxa says so plainly. An .fxa bundle also carries the crate version that built it, for the same reason.

postcard is the serde format for the body: compact, schemaless, and the crate the rest of the runtime already depends on rather than a wire format of our own. Two consequences follow that matter to anyone reading the format. An enum variant is encoded by its position in the enum, so an instruction can be added at the end but never moved; that is why a few instructions sit away from their family in the source and the reference shows them under the family instead. And integers are varints, so a small constant index or slot number costs one byte.

The module

The four kinds say what funcs[0] is:

Kind	What it holds
Program	A .prg: funcs[0] is the implicit main, the rest are its PROCEDUREs and FUNCTIONs.
Form	One function per method with source, listed in methods under "PGFMAIN.PAGE1.LBLGREETING.CLICK" or ".INIT" for the form itself. No main.
Snippet	A menu command, an EXECSCRIPT() body, a Command Window line: funcs[0] is the body.
Expression	EVALUATE(), &macro, a SKIP FOR condition: funcs[0] pushes one value and returns it.

Constants

Num(f64, chars, decimals)   a number and the width it was written in
Money(i64)                  $12.34 as written, in ten-thousandths
Str(String)
Date(Option<i32>)           days since 1970-01-01; None is the empty date
DateTime(Option<f64>)       seconds since 1970-01-01; None is the empty datetime
Bool(bool)                  only class property values need these; code pushes True/False
Null
Array(Vec<Constant>)        DIMENSION aRGB[3] in a class body

A numeric constant remembers how it was written because Visual FoxPro does: ? 001 prints three characters, and the width has to travel with the value for that to be true. In the listings below, Num(10.0, 2, 0) is the 10 of FOR i = 1 TO 10: two characters, none of them past the point.

Function prototypes

FuncProto
  name          upper-cased; MAIN for a program body, OBJPATH.EVENT for a method
  display_name  as error messages and PROGRAM() show it
  nparams       declared parameters, which occupy local slots 0..nparams
  locals        slot -> upper-cased name, so runtime-compiled code can find a local by name
  def_line      the source line LINENO(1) counts from
  captures      for a lambda: (from, to) slot pairs; empty for everything else
  code          the instructions

Classes

DEFINE CLASS ... ENDDEFINE becomes a ClassProto: the name and parent as written, the property values constant-folded, the ADD OBJECT members, and the methods as ("Init", func) or ("image1.Click", func). Inheritance is not flattened: a proto carries only what its own declaration says, and the host walks the chain when it instantiates.

A frame

Every function call pushes a frame, and the frame is where the instructions' operands point.

A LOCAL is a slot and is never visible to a callee. A PRIVATE or an undeclared name goes in the frame's privates map and is found by walking the callers, which is what dynamic scope has always meant here. #FOR_END1 and #FOR_STEP2 in the listings below are the compiler's own slots: a FOR loop's bounds are evaluated once, as Visual FoxPro evaluates them, and have to be kept somewhere a program cannot name.

What programs compile to

Each listing is foxvm disasm on the program above it. Read [before] -> [after] with the top of the stack on the right.

A statement, a local, a print

LOCAL n
n = 3
? n * 2

func 0 MAIN (demo) nparams=0 locals=["N"]
     0  Stmt(1)
     1  DeclLocal(0)
     2  Stmt(2)
     3  Const(0)                              ; Num(3.0, 1, 0)
     4  StoreLocal(0)
     5  Stmt(3)
     6  Print { newline: true, argc: 0 }
     7  LoadLocal(0)
     8  Const(1)                              ; Num(2.0, 1, 0)
     9  Mul
    10  Print { newline: false, argc: 1 }
    11  True
    12  EndOfCode

Every statement begins with Stmt(line): it records the line for error messages and LINENO(), and it is where a breakpoint hooks. ? is two prints, the newline it starts with and then the item, because ?? is the same instruction without the first. True then EndOfCode is the .T. a routine answers when it runs off the end, which is what Visual FoxPro answers, measured. EndOfCode is not Return: a written RETURN also has to leave a line a macro put together, and the routine that line belongs to.

IF, and a short-circuit AND

LOCAL a, b
a = 3
b = 4
IF a > 2 AND b < 5
    ? "both"
ELSE
    ? "not both"
ENDIF

    10  LoadLocal(0)
    11  Const(2)
    12  Gt
    13  JumpIfFalseKeep(18)
    14  LoadLocal(1)
    15  Const(3)
    16  Lt
    17  And
    18  JumpIfFalse(24)
    19  Stmt(5)
    20  Print { newline: true, argc: 0 }
    21  Const(4)                              ; Str("both")
    22  Print { newline: false, argc: 1 }
    23  Jump(28)
    24  Stmt(7)
    25  Print { newline: true, argc: 0 }
    26  Const(5)                              ; Str("not both")
    27  Print { newline: false, argc: 1 }
    28  True

a AND b compiles to a; JumpIfFalseKeep(end); b; And; end:. JumpIfFalseKeep peeks rather than pops, so when the left side is .F. it stays on the stack as the answer and the right side is never evaluated; when it is .T. the right side is evaluated and And combines the two, three-valued, because either may be .NULL.. Jump targets are absolute program counters.

FOR, with its bounds evaluated once

LOCAL i, total
total = 0
FOR i = 1 TO 10 STEP 2
    total = total + i
ENDFOR
? total

func 0 MAIN (loop) nparams=0 locals=["I", "TOTAL", "#FOR_END1", "#FOR_STEP2"]
     7  Const(1)                              ; Num(1.0, 1, 0)
     8  StoreLocal(0)
     9  Const(2)                              ; Num(10.0, 2, 0)
    10  StoreLocal(2)
    11  Const(3)                              ; Num(2.0, 1, 0)
    12  StoreLocal(3)
    13  LoadLocal(0)
    14  LoadLocal(2)
    15  LoadLocal(3)
    16  ForTest(27)
    17  Stmt(4)
    18  LoadLocal(1)
    19  LoadLocal(0)
    20  Add
    21  StoreLocal(1)
    22  LoadLocal(0)
    23  LoadLocal(3)
    24  Add
    25  StoreLocal(0)
    26  Jump(13)
    27  Stmt(6)

The end and the step go into two slots the program cannot name, so changing total inside the body cannot change how many times it runs. ForTest takes [var, end, step] and jumps out when the loop is finished, in either direction, which is why the step is on the stack too.

DO WITH a by-reference argument, and a FUNCTION call

LOCAL n
n = 5
DO double WITH n
? n
? twice(4)

PROCEDURE double
    LPARAMETERS pn
    pn = pn * 2
ENDPROC

FUNCTION twice(x)
    RETURN x * 2
ENDFUNC

func 0 MAIN (call) nparams=0 locals=["N"]
     6  Ref(Local(0))
     7  Do { name: 0, argc: 1, in_prog: false }     ; name 0: DOUBLE
    ...
    14  Const(1)                                    ; Num(4.0, 1, 0)
    15  IndexOrCall { name: 1, argc: 1 }            ; name 1: TWICE
func 1 DOUBLE (double) nparams=1 locals=["PN"]
     0  Stmt(9)
     1  LoadLocal(0)
     2  Const(2)
     3  Mul
     4  StoreLocal(0)
     5  True
     6  EndOfCode
func 2 TWICE (twice) nparams=1 locals=["X"]
     0  Stmt(13)
     1  LoadLocal(0)
     2  Const(2)
     3  Mul
     4  Return

DO ... WITH passes by reference, so the argument is Ref(Local(0)): a cell over the caller's slot that the callee's StoreLocal(0) writes through, which is why ? n prints 10. A function call in an expression is IndexOrCall, the one question asked of a name: an array of that name is subscripted, a function is called, and anything else looks for a program of that name. A parameter is slot 0 of the callee; nparams=1 says how many slots the arguments fill.

TRY, CATCH TO, FINALLY

TRY
    x = 1 / 0
CATCH TO oErr
    ? oErr.Message
FINALLY
    ? "done"
ENDTRY

     1  TryPush { catch: 9, finally: 17 }
     2  Stmt(2)
     3  Const(0)
     4  Const(1)
     5  Div
     6  StoreName(0)                          ; name 0: X
     7  TryPop
     8  Jump(17)
     9  CatchObject
    10  StoreName(1)                          ; name 1: OERR
    11  Stmt(4)
    12  Print { newline: true, argc: 0 }
    13  LoadField { area: Some(1), field: 0 } ; oErr.Message
    14  Print { newline: false, argc: 1 }
    15  TryPop
    16  Jump(17)
    17  Stmt(6)
    18  Print { newline: true, argc: 0 }
    19  Const(2)                              ; Str("done")
    20  Print { newline: false, argc: 1 }
    21  EndFinally

TryPush installs a handler naming both targets. When Div raises error 1307, the VM pops any frames above the one that installed the handler, cuts the value stack back to where it was at TryPush, and jumps to catch. CatchObject yields CreateException so the host can build the Exception object, and StoreName puts it in oErr. Both paths end at finally, and EndFinally re-raises whatever was still propagating when the block was entered - nothing, on the paths shown. oErr.Message is LoadField and not GetMember, because a bare name followed by a dot is only known at run time to be an object variable, an m. prefix, or a table alias.

A lambda and its capture

LOCAL n, f
n = 1
f = LAMBDA(x)
    RETURN x + n
ENDLAMBDA
? f(10)

func 0 MAIN (lambda) nparams=0 locals=["N", "F"]
     4  Const(0)                              ; Num(1.0, 1, 0)
     5  StoreLocal(0)
     6  Stmt(3)
     7  MakeLambda(1)
     8  StoreLocal(1)
     9  Stmt(6)
    10  Print { newline: true, argc: 0 }
    11  LoadLocal(1)
    12  Const(1)                              ; Num(10.0, 2, 0)
    13  IndexOrCallValue(1)
    14  Print { newline: false, argc: 1 }
func 1 #LAMBDA1 (lambda LAMBDA at line 3) nparams=1 locals=["X", "N", "F"]
     0  Stmt(4)
     1  LoadLocal(0)
     2  LoadLocal(1)
     3  Add
     4  Return

The lambda is an ordinary function of the module, #LAMBDA1, with a parameter in slot 0 and the enclosing routine's two locals in slots 1 and 2; its proto's captures say [(0, 1), (1, 2)]. MakeLambda(1) copies slot 0 and slot 1 of the current frame into a new function value and pushes it. IndexOrCallValue(1) asks of a value what IndexOrCall asks of a name, and when it is a function the VM pushes a frame for #LAMBDA1, writes the argument into slot 0 and the captured values into slots 1 and 2, and carries on round the loop. Nothing recurses.

SCAN FOR

USE customer
SCAN FOR country = "UK"
    ? company
ENDSCAN
USE

     1  Const(0)                              ; Str("customer")
     2  Use { alias: None, named_alias: false, exclusive: false, online: false, in_area: false }
     3  OpenIndex
     4  Stmt(2)
     5  Go(Top)
     6  CallBuiltin { id: 145, argc: 0 }      ; EOF()
     7  JumpIfTrue(19)
     8  LoadName(0)                           ; name 0: COUNTRY
     9  Const(1)                              ; Str("UK")
    10  Eq
    11  JumpIfFalse(16)
    12  Stmt(3)
    13  Print { newline: true, argc: 0 }
    14  LoadName(1)                           ; name 1: COMPANY
    15  Print { newline: false, argc: 1 }
    16  Const(2)                              ; Num(1.0, 1, 0)
    17  Skip
    18  Jump(6)
    19  Stmt(5)
    20  Const(3)                              ; Str("")
    21  Use { alias: None, ... }

A SCAN is nothing special: Go(Top), then a loop of EOF(), the FOR condition, the body, Skip 1. Use yields a request to open the file and OpenIndex reads the structural .cdx beside it if the header says one is there. A field is read by LoadName, the same instruction a variable uses, because Visual FoxPro looks a bare name up the same way: a variable first, then a field of the selected area. USE with an empty path closes the area.

SELECT-SQL

SELECT custno, SUM(amount) AS total ;
    FROM orders ;
    GROUP BY custno ;
    INTO CURSOR c_top

     1  SqlOpen { table: 0, alias: 0, named: false }
     2  SqlBegin(0)
     3  SelectSource(0)
     4  Go(Top)
     5  SelectSource(0)
     6  CallBuiltin { id: 145, argc: 0 }      ; EOF()
     7  JumpIfTrue(16)
     8  LoadName(1)                           ; CUSTNO, the group key
     9  LoadName(2)                           ; AMOUNT, for SUM
    10  LoadName(1)                           ; CUSTNO, the select list
    11  SqlRow(3)
    12  SelectSource(0)
    13  Const(1)
    14  Skip
    15  Jump(5)
    16  SqlEnd

A query is compiled to the same loop a hand-written SCAN uses: Go(Top), EOF(), Skip. What holds it together is Module::queries[0], a QueryPlan that says what the three values SqlRow(3) collects per record are for - a group key, an aggregate input, a column - and SqlEnd finishes the grouping, applies HAVING and ORDER BY, and writes the cursor. Several sources are nested loops with SelectSource(n) choosing which is current, and an outer join's miss is JoinMiss. That a query is bytecode over the same primitives is why it can read a table larger than memory the way a SCAN does.

Objects and WITH

oForm.Caption = "Hi"
WITH oForm
    .Left = 10
    .Show()
ENDWITH

     1  Const(0)                              ; Str("Hi")
     2  LoadName(0)                           ; OFORM
     3  SetMember(0)                          ; member 0: Caption
     4  Stmt(2)
     5  LoadName(0)
     6  PushWith
     7  Stmt(3)
     8  Const(1)
     9  LoadWith
    10  SetMember(1)                          ; member 1: Left
    11  Stmt(4)
    12  LoadWith
    13  CallMethod { name: 2, argc: 0 }       ; member 2: Show
    14  Pop
    15  PopWith

SetMember takes [value, obj] and yields SetProp to the host, which owns the object; the fiber is resumed once the property is written and any ProgrammaticChange it fired has run. CallMethod yields CallMethod and its result is pushed, so a call whose value is not wanted is followed by Pop. WITH is a stack in the frame: PushWith saves the object, LoadWith reads it for each .member, PopWith lets it go. Member names are stored as written and the host compares them case-insensitively; variable names are upper-cased in the module because the language is.

Which instructions leave the machine

About a third of the instructions can yield a host request; the rest complete inside the VM. The ones that yield are the ones whose effect is outside the machine's own memory: every object write and method call, every file and table operation, every dialog, DoForm, DoMenu, CreateObject, the DLL and library calls. An instruction that yields is resumed with a value and either pushes it, discards it, or - for the data instructions, which can take several round trips to read a record - rewinds the program counter and runs again with the reply in hand. The virtual machine page is about that loop; the instruction reference says which request each one yields.