The gate was using a system-wide Open Watcom that only ships the 32-bit
compiler. The project downloads its own toolchain, and .dosboxx/watcom/binnt
has the Windows-hosted 16-bit wcl.exe -- the same compiler and the same target
as the DOS build. Run that instead, with build-dos.bat's exact command.
The difference is not academic. Compiling 16-bit immediately reported three
functions the unification had orphaned: emit_type_helpers, which the M7
program emitter replaced, and emit_drop_helpers and emit_drop_fields, which it
was the only caller of. The 32-bit check had been clean.
Those warnings were going to the DOS screen, where the runner cannot see them:
COMMAND.COM redirects handle 1 only, and Watcom writes diagnostics to handle 2.
So the suite was green while the build was not quiet. Nothing in the runner
would ever have said so.
Found while chasing W210/W107 reported from a DOS screen, which this does not
yet explain -- those are not among what the compiler build emits now.
M1-M7: 214 passed.
Unifying the two engines exposed what the split had been hiding: every rule
that lived only in the M6 body was silently dropped for units the M7 half
claimed, and since M1-M6 sources never reached that half, nothing failed until
they all did. Eleven cases across m3, m5 and m6 caught it.
Checker, all from the M6 statement and lvalue cases:
- writing a struct field needs a writable place, so `p.x = 3` on a `let` is an
error again (m3-badfield)
- `let` cannot bind a mutable slice, a var with no initializer needs a type,
and a void expression cannot initialize (m3-bad-mlet)
- a returned reference must derive from a parameter or a static, and a void
expression cannot be returned from a value function (m6-badarg, badret,
badself, badtwo, badlocsl)
- rebinding a reference must not outlive its source scope, and must release the
previous borrow (m6-badscop)
- the loop case delegates to the core, which carries the flow capture and merge
that detects a value moved on every iteration; the M7 version had none of it
(m5-bad-loop)
Emitter:
- builtins other than the print family (@size_of, @align_of) and the str alias
methods are lowered by the core, which the M7 call path never reached, so
they were emitted verbatim into the C (m3-struct)
- the trim helper is emitted from the M7 type-helper pass as well, not only the
core one, or the call has no definition to link (m6-oktrim)
M1-M7 all green: 16, 19, 50, 20, 16, 57, 42.
There is an Open Watcom install on this host (C:\WATCOM19), and its Windows
build compiles the compiler's own sources in about a second. Every declaration
mismatch in the unification commits was found that way; each one would
otherwise have cost a DOSBox-X boot and a full compiler build to surface, with
a DOS-side message that names the wrong thing.
Add tools/tests/test_host_syntax.py: compile all twelve sources with the flags
build-dos.bat uses (-za -wx -wcd=202) and fail on any diagnostic. It skips when
Watcom is absent, so the suite still runs elsewhere. Two structural checks come
with it -- that build-dos.bat, the Makefile and the test agree on the source
list, and that no .c under fec/src is compiled by nothing. Both would have
caught check.c and emit_c.c quietly leaving the build when the M7 wrappers
included them textually.
ferro-test now runs these and the 8.3 name check before starting the VM, and
stops if they fail.
This is not verification and does not claim to be: wcc386 targets 32-bit where
the real build is 16-bit large model, so it sees syntax, types and declarations
and nothing about code generation. The DOS build and the milestone suite remain
the gate. It only moves the cheap failures earlier.
Compiling the merged sources with the Open Watcom install on this host
(C:\WATCOM19\binnt) found four things the merge got wrong, none of which any
amount of reading would have caught reliably:
- FE_M7_FLOW_CAP and <stdlib.h> lived in check_m7.c's preamble, above the
textual include, and were dropped with the wrapper.
- emit_error_return takes a const char *, not a FeNode *; the hand-written
forward declaration disagreed with the definition.
- emit_match was never defined by the M7 half, only called, so renaming it
alongside the other delegating pairs left a declared-but-undefined static.
- type_needs_drop and emit_lvalue are used a few hundred lines before the
declaration block, so their declarations had to be hoisted.
Also drop two locals that existed only to be cast to void.
All twelve compiler sources now compile with -za -wx -wcd=202 and produce no
warnings. That is a syntax and type check, not verification -- the DOS build
and the milestone suite remain the gate.
emitcm7.c was the same wrapper trick as the checker: nineteen #define renames,
a textual include of emit_c.c, and a per-unit feature scan choosing between two
emitters. Five of the M7 halves fell through to their M6 counterpart, so those
become the single entry point with the old body renamed to *_core; the other
thirteen never delegated at all and simply replace the M6 version.
The scan is gone from both places it was used -- the program entry and the
expression emitter, whose switch already ended in a default that delegates.
fe_emit_c_program_core went with the dispatch that was its only caller.
emitcm7.c is deleted and the build compiles emit_c.c directly. Nothing in the
compiler now selects an implementation by looking for `?` or `!` in a unit.
This is the pair to the checker commit; together they end the two-engine split
that produced four of the five defects fixed while getting M7 to pass.
check_m7.c textually included check.c, renamed three entry points aside, and
selected between two whole checkers by scanning each unit for `?`, `!`, `try`
and friends. A unit that mentioned any of them was checked by a second
implementation, so an M1-M6 rule fixed in check.c never reached it -- and the
split hid real defects, since the M7 half was reached by no existing fixture
until M7 cases were registered.
The split was cheaper to undo than it looked: every M7 dispatcher already
delegated to its M6 counterpart for nodes it did not handle. So the M7 entry
points become the single check_expr/check_stmt/check_lvalue/check_call, and
the former M6 bodies become check_expr_core/check_stmt_core/check_lvalue_core,
reached as the fallback. Recursion runs through the unified entry, which is
what makes an optional nested inside otherwise-M6 code get checked at all.
m7_check_fn and m7_check_method were identical to the M6 versions apart from
which check_stmt they called, so they are dropped. The feature scanner
(m7_type_ast, m7_node_feature, m7_program_feature) is gone with the dispatch
it fed, including the loop case that still consulted it. fe_check_program and
fe_check_expr_type keep the M7 bodies, which are supersets.
The build compiles check.c directly again.
No behaviour intended to change: the unified checker applies the union of the
rules to every unit, which for M1-M6 sources is what the M6 half already did.
The previous commit blamed core=dynamic for the M3 bounds cases exiting 0
instead of trapping. That was wrong. Both suspects were live at the time --
the recompiler and a cache hit that skipped BUILD.BAT's `set PATH` -- and the
core was ruled out before the PATH line was restored, so the test proved
nothing.
Retested with the PATH fix in place: core=dynamic gives 50 passed on m3, the
traps included. abort() reports its exit status fine under the recompiler.
Per milestone, warm cache, all green:
m1 2s m2 3s m3 7s m4 3s
m5 3s m6 6s m7 5s 29s total
against 65s on the interpreter.
Compiling okcatmov.fe's output crashed wcc386 hard enough to take DOSBox-X
down with it -- "no byte handler for write to ffffffc", emulator exit 1, the
whole suite lost rather than one case failing. `result catch fallback` in
return position lowered to `((tmp = X), tmp.e ? fallback : tmp.v)`, and with a
moved operand X is itself `(fe_live_x=0, x)`, so the compiler met a struct
assignment whose right side was a comma expression. Lower it as statements,
and clear a move flag on its own line rather than inside the assignment.
Revert core=dynamic from the previous commit. It was about 5x faster and it is
wrong: the recompiler loses abort()'s exit status, so a trapping program exits
0 and the M3 bounds cases stop reporting the trap they exist to prove. Checked
against a compiler built under core=normal, so the fault is in running the
generated program, not in building the compiler.
Also fix the build cache: BUILD.BAT puts the Watcom binaries on PATH, and
skipping it on a cache hit left the case commands without it, which silently
changed how the trap programs terminated. Reproduce that line.
Per milestone, warm cache: m1 2s, m2 5s, m3 20s, m4 7s, m5 6s, m6 14s, m7 11s.
All green, 220 cases.
Every run rebuilt fec from scratch inside DOS, about 14 seconds, even when no
source had changed. Key a cache on the hash of fec/src plus build-dos.bat and
restore FEC.EXE when it matches; the toolchain itself is pinned by
dosboxx.lock.json so it cannot drift under a hit. Only a passing build is
cached, and the batch skips BUILD.BAT on a hit because it would delete and
rebuild the executable it was just given.
Generated C included stdio.h unconditionally, but only the M4 writer runtime
reaches it. A 26-line unit was pulling in roughly 1900 lines of headers, paid
once per compile check. Emit it only when the runtime is emitted.
--only m6, 57 passed, over three changes:
31s before
17s core=dynamic
2s warm compiler cache
Cold runs still pay the build once.
Thirty-one cases only ever emitted C and never built it -- m5 owned/defer, and
every ok* acceptance fixture in m6 and m7. They asserted that fec produced a
file, not that the file was a program, so a malformed emission sat unnoticed.
Compile each one with wcl386 -c. Fixtures without a main cannot be run, so this
is the floor for them; it is a conformance check on the backend's output, not an
assertion about C, so a future backend swaps the command rather than the intent.
It found two defects in the M6 emitter immediately, both predating M7:
&s[0] took the address of an rvalue. Borrowing went through emit_expr, which
lowers an index to the bounds-checking accessor, so `&s[0]` became
`&fe_idx_slice_type_2(s, 0)`. emit_lvalue spells the same element as `s.p[0]`.
The M7 emitter already did this correctly; only the M6 path was wrong.
line.trim() was never lowered. SPEC lists trim among the built-in alias methods
on str and the checker accepts it, but no emitter case existed, so it emitted
`fe_l_line_0.trim()` -- a member access on a slice struct. Emit the helper, and
only for programs that actually trim.
Also set core=dynamic and cycles=max. core=auto uses the interpreter in real
mode, which is where the 16-bit compiler build spends its time; nothing here is
timing sensitive. m6 drops from 31s to 17s. Raise the DOSBox timeout to match a
whole-suite run, which now pays a process spawn per compile check.
Verified: --only m6, 57 passed.
SPEC.md allows `try` only inside a function returning an error union, but the
check sat in the FE_N_EXPR_STMT case, so it only ever saw a bare `try e;` and
walked past `var x = try e;` and `x = try e;`. Move it onto the try expression
in check_expr and drop the statement-level copy.
This could not land before M7: closing the hole forces m5/runtime.fe's `run` to
return an error union, and value returns from `-> !T` need contextual success
construction. That arrives with M7, and `run` is now `-> !i32`, so the rule can
be enforced. Supersedes the SPEC.AUDIT.md entry that recorded the blockage.
M1-M7: 183 passed.
m7_emit_call reimplements call emission and only carried over mem.destroy and
mem.replace, so every mem.create/mem.alloc_slice fell through to the generic
member path and emitted `fe_missing.create(0)`. wcc386 does not diagnose that
-- it terminates with exit 255, which wcl386 reports as "Unable to invoke
wcc386.exe" with no message at all.
The breakage covered all 17 allocation sites in m5/runtime.fe, and it was
invisible because m5-owned and m5-defer only emit C; m5-runtime is the one M5
fixture that compiles and links what was generated.
Also make runtime.fe legal: `run` used `try` while returning i32, which SPEC
allows only in a function returning an error union. It is `-> !i32` now, which
M7 accepts because contextual success construction lands with it, and
runtime.c takes the { error, value } struct the error union lowers to.
Found by calling wcc386 directly instead of through wcl386, which is the only
way to see a compiler crash here.
M1-M7: 183 passed.
emitcm7.c reimplements the emitter for sources that mention M7 syntax, and
three things were lost in the port. All of them only reached M4 fixtures,
because M1-M3 and M6 take the M6 fast path.
Local initializers named "0". The LET/VAR case passed the declaration node to
emit_lvalue, which matches only IDENT/MEMBER/INDEX and otherwise falls through
to the raw expression path -- a declaration node renders there as "0", so every
`var x = init;` emitted `0 = init;`. Teach emit_lvalue that a declaration names
its own storage, which also fixes the catch path that had the same call.
Aggregate initializers were not constant. A string-literal `const` lowered to a
maker call, but C89 requires a constant expression for aggregate initializers
at file scope and for automatics alike, and the build runs with -za. Restore
the braced form for both the local and the global path.
Slice helpers were never emitted. The final loop in m7_emit_type_helpers is
commented as reusing the M3 index/slice generator but only ported the index
half, so bodies called fe_slice_*/fe_full_*/fe_tail_* that no declaration
defined. Emit the three slicers for array and slice types.
The first defect masked the other two: wcc386 died on `0 = ...` before it could
reach them, and wcl386 reports that as "Unable to invoke wcc386.exe" with no
diagnostic, which is why this needed bisecting against master's output rather
than reading an error message.
M1-M7: 7 failed, 176 passed -> 3 failed, 180 passed. The remainder is m5
runtime, which is a separate fixture issue.
SPEC.md allows `try` only inside a function returning an error union, but
check.c only tests it in the FE_N_EXPR_STMT case, so `var x = try e;` and
`x = try e;` walk straight past. m5/owned.fe and m5/runtime.fe both depend on
that gap.
Moving the check onto the try expression is a four-line change and it is
correct, but it cannot land yet. runtime.fe's `run` allocates and returns a
value, so closing the hole forces it to return an error union -- and master
rejects `return <value>;` in `-> !i32` ("return type mismatch") as well as a
bare `return;` in `-> !void` ("void expression returned from value function").
Both need contextual success construction, which is M7 work. `catch` and
`@trap()`, the two spellings SPEC offers as alternatives, are also M7-only, so
there is no way to express `run` legally on master today. All three paths were
tried in DOSBox-X, not assumed.
Fix owned.fe now, since `main() -> !void` is legal today and matches
m4/try-fpr.fe, and leave the checker alone until M7 lands with the rest.
Verified: 155 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BScg8CF1sAAM2zVHAu5zvW
Every fec error message was written to stderr, and COMMAND.COM can only
redirect handle 1 -- ">" is the whole vocabulary, "2>" is not parsed at all.
So a failing compile recorded exit code 1 and a zero-byte log, and the actual
message went to a screen nobody reads. Confirmed directly: `FEC.EXE --check`
on a fixture that must fail produced rc=1 and 0 bytes of stdout.
That is why an unexpected compiler failure was undiagnosable. It also means
the M6 reject cases have only ever asserted "exit code was nonzero" -- the
error text they nominally check has never been observable to the runner.
Add fe_diag_stream(), which resolves once to stdout when FE_DIAG_STDOUT is set
and stderr otherwise, and route diag.c and driver.c through it. The default is
unchanged, so interactive use keeps writing to stderr; the runner sets the
variable in RUN.BAT. The stderr references in check.c and emit_c.c are the
Ferro language's own std.io.stderr writer and are deliberately untouched.
Verified in DOSBox-X: 155 passed. A rejecting compile now records its message,
source excerpt and caret in RESULTS\<key>.LOG.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BScg8CF1sAAM2zVHAu5zvW
Reviewing an M7 branch took six DOSBox-X runs to find three build blockers
that each take a second to explain. The runner threw away everything needed to
see them.
Capture the compiler build's output. Case commands were redirected to
RESULTS\<key>.LOG but `call BUILD.BAT` was not, so the step that fails first
and blocks every case left only BUILD.FAIL containing the string "FAIL". The
twelve wcl invocations inside it were invisible; finding "Unable to open
src\emit_c_m7.c" meant hand-editing build-dos.bat to add a redirect and
re-running the VM.
Record exit codes. The batch collapsed every outcome to `if errorlevel 1`, so
a compiler that aborted and one that exited 1 with a diagnostic were the same
FAIL. RC.BAT now walks a descending errorlevel ladder into RESULTS\<key>.RC
and the host derives pass/fail from it, which immediately separates an
ordinary rejection (1) from a trap (255). Note the space in `echo 0 >FILE`:
without it DOS parses `0>` as a redirect of handle 0.
Stop falling back to CONSOLE.LOG. That is DOSBox-X's own log -- display
enumeration and INT15 chatter -- so a crashed command reported fifty lines of
emulator noise instead of saying it produced no output.
Add tools/tests/test_dos_names.py. An over-long source name reaches the DOS
build as `Unable to open "src\..."`, which reads as a missing file rather than
a name FAT cannot represent, and only after a VM boot and ten object builds.
The check runs on the host in 0.03s and flags emit_c_m7.c (9-character stem)
on the branch that prompted this.
Also pass -k through to pytest so a single case can be re-run without its
whole milestone, and print the resolved ROOT at startup: an editable install
plus a git worktree will otherwise silently build a different checkout than
the one the shell is in.
Verified on master: 155 passed, unchanged. Recorded codes are 0 for success,
1 for rejections, 255 for the three bounds traps.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BScg8CF1sAAM2zVHAu5zvW
The bounds-no-checks cases emitted and built but never ran, so they only
proved that --no-checks produces compilable C -- not that it removes the
check, which is the entire point of the flag.
A run case could not simply be appended. BOUNDS.FE returns the out-of-bounds
element directly, so with checks removed its exit code is whatever sits past
the array on the stack and there is no correct status to assert. Asserting on
the generated C instead does not work either: emit_c.c defines fe_trap_bounds
unconditionally and --no-checks only suppresses the call sites.
Add NOCHK.FE, which reads one element past a [2]i32 and returns x - x. That
is 0 for whatever garbage the unchecked read produced, so the same source has
a defined outcome both ways: compiled with checks it must trap, compiled with
--no-checks it must run to completion and exit 0. Register both halves and
drop the two BOUNDS-N cases they supersede.
Verified in DOSBox-X: 155 passed, including m3-nochk-trap failing as expected
and m3-nochk-off-run succeeding.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BScg8CF1sAAM2zVHAu5zvW
The highest supported milestone was spelled out in six places across four
files: range(1, 7) and default="m6" in test_cli.py, the same pair in
test_milestones_dosboxx.py, and through=6 in both registry.py and suite.py.
Registering M7 meant finding all six, and missing one failed silently.
Derive MAX_MILESTONE and MILESTONES from CASES instead, and move the mN
selector parser to registry.milestone_number so the pytest module stops
carrying its own copy. Adding cases for a new milestone is now enough for
ferro-test to accept --through/--only for it.
No behaviour change: MAX_MILESTONE evaluates to 6, ferro-test still advertises
{m1..m6} with default m6, and the case snapshot is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BScg8CF1sAAM2zVHAu5zvW
registry.py was literally `cat registry_m1_m3.py registry_m4_m6.py`, which
left a duplicate `_c` definition and a stray module-level docstring at the
seam. Replace the hand-rolled append loops with five builders -- _emit, _wcl,
_triple, _rejects, _dump_ast -- and express the cases as one ordered list.
The irregularities are now parameters instead of one-off code, each with the
reason recorded:
- _triple(stem=) for M2 castwhil emitting CAST16
- _triple(build_source=) for M4 prop, which emits PROP.C but compiles
PROPTEST.C because that file #includes it
- _triple(run_suffix="trap", run_ok=False) for the M3 bounds cases
- _triple(emit_suffix=None) for M4, whose ids lack the -emit suffix
- _emit(output_first=True) for M6, which passes -o before the input
The three scattered 8.3 output-name mappings collapse into one _OUT83 table
keyed by (milestone, name). The key needs both: bad-type is BAD-TY in M2 but
BAD-TYP in M4, and bad-cond is BAD-CO in M2 but unshortened in M5. Values are
carried over verbatim -- the shortenings are inconsistent and several were
never required, but that is a separate decision.
suite.py keeps only Case and drops the all_cases forwarder, so the
registry -> suite -> registry cycle is gone along with the function-scoped
import that worked around it. dosboxx.py still imports Case from suite and is
untouched.
Verified behaviour-preserving by snapshotting (id, milestone, command,
expect_success) for all 150 cases in order before and after: diff is empty.
pytest collects the same 151 items.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BScg8CF1sAAM2zVHAu5zvW
coolguy
