This module contains the type definitions for the new evaluation engine. An instruction is 1-3 int32s in memory, it is a register based VM.
Types
Atom {.union.} = object ptrVal*: pointer ## akPtr strVal*: VmString ## akString seqVal*: VmSeq ## akSeq refVal*: HeapSlotHandle ## akRef callableVal*: VmFunctionPtr ## akCallable nodeVal*: PNode ## akPNode
- Convenience type to make working with atomic locations easier. Since the union stores gc'ed memory (PNode), it must never be used on either side of an assignment and also never used as a variable's type Source Edit
AtomKind = enum akInt, akFloat, akPtr, akSet, akString, akSeq, akRef, akCallable, akDiscriminator, akPNode, akObject, akArray
- Source Edit
BranchListEntry = object case kind*: BranchListEntryKind of blekStart: field*: FieldIndex ## The index of the corresponding discriminator numItems*: uint32 ## The number of items following this entry that are ## part of this record-case. Used for the fast skipping ## of inactive branches defaultBranch*: uint16 ## The index of the default branch numBranches*: uint16 ## The number of branches this record-case has. ## Does not include branches of sub record-cases of blekBranch, blekEnd: fieldRange*: Slice[FieldIndex] ## ## For 'branch' entries, the range of field indices the branch spans. ## For `end` entries, the range of fields past the corresponding ## record-case that are still part of the surrounding branch. ## ## In both cases, the slice may be empty, with `fieldRange.a` always ## being >= the previous entry's `fieldRange.a`. ## For the dedicated 'end' entry, `fieldRange.b` is always 0
-
An entry in a walk-list, describing an object variant in a flat manner. A record-case maps to the following entries:
- a 'start' entry (represents the discriminator)
- one or more 'branch' entries (one for each of-branch)
- an 'end' entry
If a branch contains a sub record-case, the record-case's entries follow the entry of the surrounding branch (i.e. the layout is stored depth-first). The first entry in the list is always a 'branch' (called the master branch) enclosing all fields in the object. The last entry is always a dedicated 'end' entry.
The idea behind this way of representing the layout of a variant object, is to allow for fast linear traversal of active fields without the need for recursion or auxiliary stack-like data structures
Source Edit BranchListEntryKind = enum blekStart, blekBranch, blekEnd
- Source Edit
CallableKind = enum ckDefault, ## A normal function ckCallback ## A VmCallback
- Source Edit
CellPtr = distinct ptr UncheckedArray[byte]
- A pointer that either is nil or points to the start of a VM memory cell Source Edit
CodeGenFlag = enum cgfAllowMeta ## If not present, type or other meta expressions are ## disallowed in imperative contexts and code-gen for meta ## function arguments (e.g. `typedesc`) is suppressed
- Source Edit
ConstantId = int
- The ID of a VmConstant. Currently just an index into TCtx.constants Source Edit
ConstantKind = enum cnstInt, cnstFloat, cnstNode, ## AST, type literals cnstSliceListInt, cnstSliceListFloat
- Source Edit
EhInstr = tuple[opcode: EhOpcode, a: uint16, ## meaning depends on the opcode b: uint32]
- Exception handling instruction. 8-byte in size. Source Edit
EhOpcode = enum ehoExcept, ## unconditional exception handler ehoExceptWithFilter, ## conditionl exception handler. If the exception is a subtype or equal ## to the specified type, the handler is entered ehoFinally, ## enter the ``finally`` handler ehoNext, ## relative jump to another instruction ehoLeave, ## abort the parent thread ehoEnd ## ends the thread without treating the exception as handled
- Source Edit
FieldIndex = distinct uint32
- Source Edit
FieldPosition = distinct uint32
- Source Edit
FuncTableEntry = object sym*: PSym retValDesc*: PVmType ## the return value type (may be empty) isClosure*: bool ## whether the closure calling convention is used sig*: RoutineSigId case kind*: CallableKind of ckDefault: regCount*: uint16 start*: int ## the code position where the function starts of ckCallback: cbOffset*: int ## the index into the callback list
- A function table entry. Stores all information necessary to call the corresponding procedure Source Edit
FunctionIndex = distinct int
- Source Edit
HandlerTableEntry = tuple[offset: uint32, ## instruction offset instr: uint32]
- Source Edit
HeapSlotHandle = int
- Source Edit
LinkIndex = uint32
- Identifies a linker-relevant entity. There are three namespaces, one for procedures, one for globals, and one for constants -- which namespace an index is part of is stored separately. Source Edit
LocHandle = object cell*: CellId ## the ID of the cell that owns the location p*: VmMemPointer ## the memory address of the reference location, in host address space typ* {.cursor.}: PVmType ## the type that the handles views the location as
- A handle to a VM memory location. Since looking up the cell a location is part of is a very common operation (due to access checks), the cell's ID is also stored here. Source Edit
NumericConvKind = enum nckFToI, nckFToU, nckIToF, nckUToF, nckFToF, ## float-to-float nckToB ## float or int to bool
- Identifies the numeric conversion kind. I = signed; U = unsigned; F = float; Source Edit
PrgCtr = int
- Program Counter, aliased as it's self-documenting and supports changing the type in the future (distinct/width) Source Edit
ProfileInfo = object time*: float ## the time spent on executing instructions (inclusive) count*: int ## the number of instructions executed (exclusive)
- Profiler data for a single procedure. Source Edit
Profiler = object enabled*: bool ## whether profiling is enabled tEnter*: float ## the point-in-time when the active measurment started data*: Table[PSym, ProfileInfo] ## maps the symbol of a procedure to the associated data gathered by the ## profiler
- Source Edit
RoutineSig = distinct PType
- Source Edit
RoutineSigId = distinct int
- Routine signature ID. Each different routine (proc, func, etc.) signature gets mapped to a unique ID. Source Edit
StackFrameIndex = int
- Source Edit
TCtx = object code*: seq[TInstr] debug*: seq[TLineInfo] ehTable*: seq[HandlerTableEntry] ## stores the instruction-to-EH mappings. Used to look up the EH ## instruction to start exception handling with in case of a normal ## instruction raising ehCode*: seq[EhInstr] ## stores the instructions for the exception handling (EH) mechanism globals*: seq[LocHandle] ## global slots constants*: seq[VmConstant] ## constant data complexConsts*: seq[LocHandle] ## complex constants (i.e. everything that ## is not a int/float/string literal) typeInfoCache*: TypeInfoCache ## manages the types used by the VM rtti*: seq[VmTypeInfo] ## run-time-type-information as needed by ## conversion and `repr` functions*: seq[FuncTableEntry] ## the function table. Contains an entry ## for each procedure known to the VM. Indexed by `FunctionIndex` memory*: VmMemoryManager flags*: set[CodeGenFlag] ## flags that alter the behaviour of the code ## generator. Initialized by the VM's callsite and queried by the JIT. callbackKeys*: Patterns module*: PSym callsite*: PNode mode*: TEvalMode features*: TSandboxFlags traceActive*: bool comesFromHeuristic*: TLineInfo callbacks*: seq[VmCallback] cache*: IdentCache config*: ConfigRef graph*: ModuleGraph idgen*: IdGenerator profiler*: Profiler templInstCounter*: ref int vmstateDiff*: seq[(PSym, PNode)] vmTraceHandler*: TraceHandler ## handle trace output from an executing vm
- Source Edit
TEvalMode = enum emRepl, ## evaluate because in REPL mode emConst, ## evaluate for 'const' according to spec emOptimize, ## evaluate for optimization purposes (same as ## emConst?) emStaticExpr, ## evaluate for enforced compile time eval ## ('static' context) emStaticStmt, ## 'static' as an expression emStandalone ## standalone execution separate from ## code-generation
- reason for evaluation Source Edit
TFullReg = object case kind*: TRegisterKind of rkNone: nil of rkInt: intVal*: BiggestInt of rkFloat: floatVal*: BiggestFloat of rkAddress: addrVal*: pointer addrTyp*: PVmType of rkLocation, rkHandle: handle*: LocHandle of rkNimNode: nimNode*: PNode
- Source Edit
TInstr = distinct TInstrType
- Source Edit
TInstrType = uint64
- Source Edit
TraceHandler = proc (c: TCtx; t: VmExecTrace): void
- Source Edit
TSandboxFlag = enum allowCast, ## allow unsafe language feature: 'cast' allowInfiniteLoops ## allow endless loops
- what the evaluation engine should allow Source Edit
TSandboxFlags = set[TSandboxFlag]
- Source Edit
TStackFrame = object prc*: PSym start*: int ## position in the thread's register sequence where the registers for ## the frame start eh*: HOslice[int] ## points to the active list of instruction-to-EH mappings baseOffset*: PrgCtr ## the instruction that all offsets in the instruction-to-EH list are ## relative to. Only valid when `eh` is not empty comesFrom*: int
- Source Edit
TypeInfoCache = object lut*: Table[ItemId, PVmType] ## `PType`-id -> `PVmType` mappings structs*: TypeTable ## All structural types created by ``vmtypegen`` signatures*: Table[RoutineSig, RoutineSigId] nextSigId*: RoutineSigId ## The ID to use for a new `signatures` entry types*: seq[PVmType] ## all generated types (including those created ## during VM setup) staticInfo*: array[AtomKind, tuple[size, align: uint8]] ## size and alignment information for atoms where this information is ## the same for every instance boolType*: PVmType charType*: PVmType stringType*: PVmType pointerType*: PVmType nodeType*: PVmType emptyType*: PVmType intTypes*: array[tyInt .. tyInt64, PVmType] uintTypes*: array[tyUInt .. tyUInt64, PVmType] floatTypes*: array[tyFloat .. tyFloat64, PVmType] rootRef*: PVmType ## the VM type corresponding to ``ref RootObj``.
- An append-only cache for everything type related Source Edit
TypeTable = object data*: seq[TypeTableEntry] counter*: int
- A partial hash-table overlay for TypeInfoCache.types. Not all types present in the latter need to be present in the table Source Edit
TypeTableEntry = tuple[hcode: int, typ: VmTypeId]
- Source Edit
VmAllocator = object cells*: seq[VmCell] ## the underlying storage for the cell slots. Both stack and ## (non-refcounted) heap slots are included here freeHead*, freeTail*: int byteType*: PVmType ## The VM type for `byte`
- Source Edit
VmArgs = object ra*, rb*, rc*: Natural slots*: ptr UncheckedArray[TFullReg] currentExceptionPtr*: ptr HeapSlotHandle currentLineInfo*: TLineInfo typeCache*: ptr TypeInfoCache mem*: ptr VmMemoryManager heap*: ptr VmHeap graph*: ModuleGraph config*: ConfigRef currentModule*: PSym ## module currently being compiled cache*: IdentCache idgen*: IdGenerator
- Source Edit
VmCallback = proc (args: VmArgs) {.closure.}
- Source Edit
VmCell = object count*: int ## stores the number of locations for cells that are in-use; for free ## cells this is the next pointer sizeInBytes*: int ## the total number of *guest-accesible* bytes the cell occupies p*: ptr UncheckedArray[byte] typ* {.cursor.}: PVmType
- Stores information about a memory cell allocated by the VM's allocator Source Edit
VmConstant = object case kind*: ConstantKind of cnstInt: intVal*: BiggestInt of cnstFloat: floatVal*: BiggestFloat of cnstNode: node*: PNode of cnstSliceListInt: intSlices*: seq[Slice[BiggestInt]] of cnstSliceListFloat: floatSlices*: seq[Slice[BiggestFloat]]
- VmConstants are used for passing constant data from vmgen to the execution engine. This includes constants for both internal use as well as user-defined literal values (e.g. string literals). Source Edit
VmEvent = object instLoc*: InstantiationInfo ## instantiation in VM's source case kind*: VmEventKind of vmEvtUserError: errLoc*: TLineInfo errMsg*: string of vmEvtArgNodeNotASymbol: callName*: string argAst*: PNode argPos*: int of vmEvtCannotCast, vmEvtIllegalConvFromXToY: typeMismatch*: VmTypeMismatch of vmEvtIndexError: indexSpec*: tuple[usedIdx, minIdx, maxIdx: Int128] of vmEvtErrInternal, vmEvtNilAccess, vmEvtIllegalConv, vmEvtFieldUnavailable, vmEvtFieldNotFound, vmEvtCacheKeyAlreadyExists, vmEvtMissingCacheKey, vmEvtCannotCreateNode: msg*: string of vmEvtCannotSetChild, vmEvtCannotAddChild, vmEvtCannotGetChild, vmEvtNoType, vmEvtNodeNotASymbol: ast*: PNode of vmEvtUnhandledException: exc*: PNode trace*: VmRawStackTrace of vmEvtNotAField: sym*: PSym else: nil
- Event data from a VM instance, mostly errors Source Edit
VmEventKind = enum vmEvtOpcParseExpectedExpression, vmEvtUserError, vmEvtUnhandledException, vmEvtCannotCast, vmEvtCannotModifyTypechecked, vmEvtNilAccess, vmEvtAccessOutOfBounds, vmEvtAccessTypeMismatch, vmEvtAccessNoLocation, vmEvtErrInternal, vmEvtIndexError, vmEvtOutOfRange, vmEvtOverOrUnderflow, vmEvtDivisionByConstZero, vmEvtArgNodeNotASymbol, vmEvtNodeNotASymbol, vmEvtNodeNotAProcSymbol, vmEvtIllegalConv, vmEvtIllegalConvFromXToY, vmEvtMissingCacheKey, vmEvtCacheKeyAlreadyExists, vmEvtFieldNotFound, vmEvtNotAField, vmEvtFieldUnavailable, vmEvtCannotCreateNode, vmEvtCannotSetChild, vmEvtCannotAddChild, vmEvtCannotGetChild, vmEvtNoType, vmEvtTooManyIterations
- Source Edit
VmEventKindAccessError = range[vmEvtAccessOutOfBounds .. vmEvtAccessNoLocation]
- Source Edit
VmExecTrace = object pc*: PrgCtr case kind*: VmExecTraceKind of vmTraceMin: nil of vmTraceFull: ra*, rb*, rc*: TRegisterKind
- Source Edit
VmExecTraceKind = enum vmTraceMin, ## minimal data execution trace, lighter/faster vmTraceFull ## full data execution trace, more info/heavier likely slower
- Source Edit
VmFunctionPtr = distinct int
- Source Edit
VmGenDiag = object location*: TLineInfo ## diagnostic location instLoc*: InstantiationInfo ## instantiation in VM Gen's source case kind*: VmGenDiagKind of vmGenDiagCannotImportc, vmGenDiagTooLargeOffset, vmGenDiagCannotCallMethod: sym*: PSym of vmGenDiagCannotCast: typeMismatch*: VmTypeMismatch of vmGenDiagMissingImportcCompleteStruct, vmGenDiagCodeGenUnhandledMagic: magic*: TMagic of vmGenDiagNotUnused, vmGenDiagCannotEvaluateAtComptime: ast*: PNode of vmGenDiagTooManyRegistersRequired: nil
- Diagnostic data from VM Gen, mostly errors Source Edit
VmGenDiagKind = enum vmGenDiagTooManyRegistersRequired, vmGenDiagNotUnused, vmGenDiagCannotEvaluateAtComptime, vmGenDiagMissingImportcCompleteStruct, vmGenDiagCodeGenUnhandledMagic, vmGenDiagCannotImportc, vmGenDiagTooLargeOffset, vmGenDiagCannotCallMethod, vmGenDiagCannotCast
- Source Edit
VmGenDiagKindAstRelated = range[vmGenDiagNotUnused .. vmGenDiagCannotEvaluateAtComptime]
- Source Edit
VmGenDiagKindMagicRelated = range[vmGenDiagMissingImportcCompleteStruct .. vmGenDiagCodeGenUnhandledMagic]
- Source Edit
VmGenDiagKindSymRelated = range[vmGenDiagCannotImportc .. vmGenDiagCannotCallMethod]
- Source Edit
VmHeap = object slots*: seq[HeapSlot] pending*: seq[int] ## the indices of the slots that are pending clean-up
-
VmHeap manages all ref-counted locations. These are used for new'ed values as well as globals.
Once a slot's ref-count reaches zero, the slot is not cleaned up and freed immediately, but is added to the pending list first. Pending slots are currently only freed at the end of a VM invocation, with cyclic references leading to memory leaks (no cycle detection is performed)
Source Edit VmMemoryManager = object allocator*: VmAllocator heap*: VmHeap
- Source Edit
VmMemoryRegion = openArray[byte]
- Source Edit
VmMemPointer = distinct ptr UncheckedArray[byte]
- A pointer into a memory region managed by the VM (i.e. guest memory) Source Edit
VmRawStackTrace = seq[tuple[sym: PSym, pc: PrgCtr]]
- Source Edit
VmSlice = object cell*: CellId ## the cell of which the locations are part of start*: VmMemPointer ## start of the slice (or ``nil``) len*: int ## the number of items typ* {.cursor.}: PVmType ## the item type
- A loaded slice. Stores all information that the VM needs to efficiently access the referenced guest memory. A VmSlice is not meant to be stored in guest memory, but rather for internal use by the VM. Source Edit
VmStackTrace = object currentExceptionA*, currentExceptionB*: PNode stacktrace*: seq[tuple[sym: PSym, location: TLineInfo]] skipped*: int
- Source Edit
VmType = object case kind*: AtomKind of akInt, akFloat, akPNode: nil of akPtr, akRef: targetType*: PVmType of akSet: setLength*: int ## The number of elements in the set of akSeq, akString: seqElemStride*: int seqElemType*: PVmType of akCallable: routineSig*: RoutineSigId of akDiscriminator: numBits*: int ## The amount of bits the discriminator value occupies of akObject: relFieldStart*: uint32 ## Encodes both the position (`PSym.position`) of ## the first field as well as whether the object ## has a base. If > 0, the object has a base. ## Subtracting 1 one from a `> 0` value yields the ## position objFields*: seq[tuple[offset: int, typ: PVmType]] branches*: seq[BranchListEntry] ## Empty if the object is no variant of akArray: elementCount*: int elementStride*: int elementType*: PVmType sizeInBytes*: uint alignment*: uint8 ## 1 shl `alignment` == real alignment value
- A VmType is a concrete type, holding the information necessary to interpret the raw bytes of a location. PTypes are mapped to VmTypes in vmtypegen. Source Edit
VmTypeId = uint32
- The unique ID of a VmType. Implementation-wise, it's an index into TypeInfoCache.types Source Edit
VmTypeInfo = object internal*: PVmType ## the `VmType` the type maps to nimType*: PType
- Stores run-time-type-information (RTTI) for types. Similiar to the PNimType used by the other backends, with the difference that VmTypeInfo is not exposed to user-code. Only used internally for implementing repr and opcConv Source Edit
VmTypeMismatch = object actualType*, formalType*: PType
- Source Edit
Consts
byteExcess = 128
- Source Edit
nimNodeFlag = 16
- Source Edit
pseudoAtomKinds = {akObject, akArray}
- Source Edit
realAtomKinds = {akInt..akPNode}
- Source Edit
regBxShift = 24'u64
- Source Edit
wordExcess = 8388608
- Source Edit
Procs
func `<=`(a, b: FieldIndex): bool {.borrow, ...raises: [], tags: [].}
- Source Edit
func `<`(a, b: FieldIndex): bool {.borrow, ...raises: [], tags: [].}
- Source Edit
func `==`(a, b: FieldIndex): bool {.borrow, ...raises: [], tags: [].}
- Source Edit
func `==`(a, b: FunctionIndex): bool {.borrow, ...raises: [], tags: [].}
- Source Edit
func `==`(a, b: RoutineSigId): bool {.borrow, ...raises: [], tags: [].}
- Source Edit
func applyOffset(p: VmMemPointer | CellPtr; offset: uint): VmMemPointer {.inline.}
- Source Edit
proc init(cache: var TypeInfoCache) {....raises: [], tags: [].}
- Source Edit
proc initCtx(module: PSym; cache: IdentCache; g: ModuleGraph; idgen: IdGenerator; tracer: TraceHandler = defaultTracer): TCtx {. ...raises: [], tags: [].}
- Source Edit
func newVmString(data: CellPtr; len: Natural): VmString {.inline, ...raises: [], tags: [].}
- Source Edit
func overlap(a, b: int; aLen, bLen: Natural): bool {....raises: [], tags: [].}
- Tests if the ranges [a, a+aLen) and [b, b+bLen) have elements in common Source Edit
func overlap(a, b: pointer; aLen, bLen: Natural): bool {....raises: [], tags: [].}
- Test if the memory regions overlap Source Edit
func packedConvDesc(op: NumericConvKind; dstbytes, srcbytes: range[1 .. 8]): uint16 {. ...raises: [], tags: [].}
- Packs the numeric conversion description into a single uint16. Source Edit
func safeCopyMem(dest: var openArray[byte | char]; src: openArray[byte | char]; numBytes: Natural) {.inline.}
- Source Edit
func safeCopyMemSrc(dest: var openArray[byte]; src: openArray[byte]) {.inline, ...raises: [], tags: [].}
- Source Edit
func safeZeroMem(dest: var openArray[byte]; numBytes: Natural) {....raises: [], tags: [].}
- Source Edit
func toFuncIndex(x: VmFunctionPtr): FunctionIndex {....raises: [], tags: [].}
- Source Edit
func toFuncPtr(x: FunctionIndex): VmFunctionPtr {....raises: [], tags: [].}
- Source Edit
func unpackedConvDesc(info: uint16): tuple[op: NumericConvKind, dstbytes, srcbytes: int] {....raises: [], tags: [].}
- Unpacks the numeric conversion description from info. Source Edit
Templates
template `-`(a: FieldIndex; b: int): FieldIndex
- Source Edit
template currentException(a: VmArgs): HeapSlotHandle
- A temporary workaround for the exception handle being stored as a pointer Source Edit
template currentException=(a: VmArgs; h: HeapSlotHandle)
- A temporary workaround for the exception handle being stored as a pointer Source Edit
template fieldAt(x: PVmType; i: FieldIndex): untyped
- Source Edit
template fieldAt(x: VmType; i: FieldIndex): untyped
- Source Edit
template fpos(x: int): FieldPosition
- Source Edit
template isNil(h: HeapSlotHandle): bool
- Source Edit
template isNil(p: VmMemPointer | CellPtr): bool
- Source Edit
template isNil(x: VmFunctionPtr): bool
- Source Edit
template isNotNil(h: HeapSlotHandle): bool
- Source Edit
template isValid(handle: LocHandle): bool
- Checks if handle is a valid handle, that is, whether it stores non-nil pointer and type information. NOTE: this is only meant for debugging and assertions -- the procedure does not check whether accessing the referenced memory location is legal Source Edit
template rawPointer(h: LocHandle): untyped
- Source Edit
template rawPointer(p: VmMemPointer | CellPtr): untyped
- Source Edit