# Dimentra3D: Assistant Expansion and Implementation Roadmap

**Audience:** coding assistants and contributors extending Dimentra3D
**Baseline:** v0.1 grammar/parser plus a narrow experimental CPU preview; no
general compiler or GPU/native renderer
**Purpose:** turn the language vision into a safe, testable implementation in
small increments without confusing syntax support with executable capability.

This document is an engineering guide, not a claim that the capabilities below
already exist. Read it with [`README.md`](README.md) and [`SPEC.md`](SPEC.md).
The spec defines what v0.1 accepts; this roadmap defines how to grow it.
For renderer-specific integration and Android preview criteria, also read
[`RENDERER-ROADMAP.md`](RENDERER-ROADMAP.md).

## Why v0.1 does not compile or render

The first milestone established an independent language identity, a small
grammar, parser diagnostics, examples, and a browser playground. A later
prototype now lowers one bounded scene subset to typed in-memory Scene IR and
draws it with the existing CPU rasterizer. It still deliberately does not turn
all declaration-shaped text into a runtime. That boundary is important:
accepting source syntax proves neither that the source has complete meaning
nor that it can safely or correctly run.

The current implementation makes the missing layers concrete:

- `dimentra3d.js` is a dependency-free, line-oriented lexer/parser with limited
  literal checks and cross-reference validation. It does not construct a full
  expression tree, resolve all names and types, or define execution semantics.
- Many values such as mesh arguments, positions, predicates, requirements and
  shader derivations are retained as source text. The parser does not evaluate
  them.
- There is no general typed intermediate representation, optimizer, code
  generator, target ABI, runtime, linker, or compiler build command. The
  renderer's version-1 Scene IR is a narrow preview data structure only.
- `dimentra3d-render.js` supports a bounded icosphere/material/light/camera
  subset, uses `mesh-render.js`, and returns a CPU pixel buffer. Other parsed
  mesh generators and camera modes remain unsupported by this preview.
- `android-preview.html` renders the sample subset in a local browser/WebView.
  The standalone Android APK source and Gradle-free build script exist, but
  this environment has no JDK/Android SDK, so no APK build or device run is
  verified yet.
- The original `playground.html` displays the AST and diagnostics. It is not a
  general Dimentra3D program runtime or renderer.
- There is no screen layout/event runtime, effect executor, Petri-flow
  scheduler, asset loader, or shader compiler.

In short, this is a **language front-end foundation**, not an incomplete claim
to a working compiler. A real compiler requires defined semantics and one
implemented target. A real renderer requires geometry, transforms, visibility,
shading, output, and tests for the rendered result. Those are substantial
engineering milestones, not features that a parser can imply.

## Non-negotiable design rules

1. **Dimentra3D owns its language.** Rustba, Gemba, Lavemba and tRust are
   conceptual/provenance references only. Do not accept their source syntax as
   compatibility syntax or make them required dependencies.
2. **Keep the frontend honest.** Separate lexing, parsing, name resolution,
   type checking, IR construction, lowering, and execution. Report exactly
   which phase succeeded. Never label an AST or serialized scene description
   as native compilation.
3. **Make expressions real before executing them.** Replace retained expression
   strings with a specified expression grammar, source-spanned AST nodes,
   typed checking, and bounded semantics. Do not use JavaScript `eval`,
   `Function`, generated JS, or string interpolation as an evaluator.
4. **Preserve safety boundaries.** Language source is data, never an
   unrestricted host command. Do not add ambient filesystem, network,
   process, shell, PowerShell, Electron, or native-host access. Effects must
   use explicit capabilities and a separately reviewed runtime gate.
5. **Keep parsing non-executing.** The browser playground and parse/validate
   commands must not run user programs, load arbitrary assets, invoke tools, or
   access host capabilities.
6. **Build a narrow vertical slice.** Implement and test one end-to-end subset
   before expanding the grammar. Unsupported constructs should fail with a
   located diagnostic; they must not silently become no-ops.
7. **Measure performance, never infer it.** A new language name, native
   backend, Deno, WebAssembly, or GPU API does not by itself make a program
   faster. Record named hardware, compiler/runtime versions, workload,
   correctness result, and repeatable timing before making comparisons.
8. **Keep output deterministic where promised.** Identical source, compiler
   version, target and declared options should produce equivalent behavior;
   reproducible artifact hashes are desirable for the first compiler subset.
9. **Protect the existing work.** Inspect the current files and repository
   state first. Preserve user changes, the original reference-program hashes,
   the parser's source limits, and existing geometry tools. Avoid unrelated
   changes.

## Target architecture

Keep language semantics independent of any single host or backend:

```text
 Dimentra3D source
        |
   lexer + parser  ----> located syntax tree
        |
 name resolution + type/effect checking
        |
 typed Dimentra3D IR  <---- canonical boundary for tools and backends
        |
  +-----+-------------------------+
  |                               |
 pure/runtime lowering        scene lowering
  |                               |
 native target backend       renderer interface
  |                         +-----+-----------+
 executable/runtime         |                 |
                       CPU reference      GPU backend
                       renderer           (later)
```

The initial implementation may be smaller than this architecture. The
separation is still useful: parsing must not depend on a graphics API, the
renderer must not parse strings, and a target backend must not redefine
language semantics.

### Frontend

- **Lexer:** token kinds, exact byte/code-unit and line/column spans, stable
  diagnostics, bounded input/token counts, and tests for malformed literals.
- **Parser:** a grammar-driven parser that builds explicit syntax nodes. Keep
  formatting and source locations; do not smuggle unparsed trailing text into
  fields.
- **Resolver:** symbol tables and declaration scopes for state, records,
  screens, effects, mesh/material/light/scene identifiers, shader names and
  flow places/transitions.
- **Type/effect checker:** assign a defined type to each expression, enforce
  conversions and operator rules, and verify that effect use is explicitly
  declared. A name match alone is not type checking.
- **Typed IR:** a documented, versioned internal model that contains resolved
  references and typed values, rather than source fragments. Do not make the
  internal IR an accidental stable public format before versioning it.

The current implementation is intentionally small. Before extending the
grammar, compare every EBNF production in `SPEC.md` to the actual parser and
tests. Fix disagreements explicitly; do not assume the spec is more complete
than the implementation.

### Compiler and runtime

A compiler milestone must name its supported source subset and target.
Recommended progression:

1. Define the first executable subset: scalar literals/expressions, typed
   records or state, and a simple deterministic entry point. Keep mobile UI,
   effects, arbitrary shader code, and platform-specific APIs outside that
   first subset unless they are implemented end to end.
2. Specify expression precedence, evaluation order, integer overflow,
   floating-point behavior, conversions, errors, and initialization. Add
   conformance tests before writing a backend.
3. Produce a typed IR and a small reference evaluator for pure operations.
   The evaluator is a correctness oracle for tests, not permission to evaluate
   arbitrary source with the host language.
4. Implement one explicit backend. A C11/native backend is a practical
   candidate if a trusted C compiler is available on the supported machines;
   verify actual toolchain availability before committing to it. Keep C
   emission structured and escaped, and test that hostile string literals
   cannot alter generated code. Do not silently run a system compiler from the
   parser or playground; invoking a compiler must be an explicit build action.
5. Add a CLI with separate `check` and `build` operations. `check` must not
   emit or run an executable. `build` must state the target and output,
   surface compiler failures, and never run the resulting program implicitly.
6. Compare backend results against the reference evaluator for the same
   conformance fixtures. Verify both successful outputs and diagnosed failures.

This compiler path is a proposal, not a settled requirement to use C. If tool
availability, portability, or later GPU/mobile targets make another backend a
better first target, record the evidence and decision in a small architecture
note before implementing it. Deno may be evaluated as a tooling/runtime host,
but it is not a compiler backend or a performance result by itself.

### 3D geometry and rendering

Keep source semantics and geometry math separate from rendering APIs:

1. **Typed scene data:** resolve mesh/material/light references and convert
   numeric arguments, transforms, camera and material values into typed,
   validated scene IR. Reject non-finite numbers, invalid ranges, missing
   references and unsupported features with source locations.
2. **Deterministic geometry:** implement a small generator set (for example
   cube and icosphere) with explicit parameter limits. Test vertex/index
   counts, bounds, winding, normals, degenerate cases and deterministic output.
   Reuse existing mathematical helpers only after inspecting their API and
   license/provenance; do not create a hidden coupling to the Polygon Studio UI.
3. **CPU reference renderer:** draw one small scene into an explicit pixel
   buffer. Start with camera projection, triangle coverage, depth testing,
   normals, a minimal documented light/material model and deterministic
   output. This provides a portable correctness reference before a GPU target.
4. **Visible playground preview:** connect a scene program to a canvas only
   after typed scene data and the renderer have tests. Make parse/diagnostics
   separate from preview; invalid programs should show diagnostics and no
   success-shaped stale frame.
5. **GPU backend:** only after the reference renderer is stable, define a
   backend interface and add a supported API (such as WebGPU) behind feature
   detection and clear fallback/error messages. Keep shader expressions
   constrained and typed. Do not compile arbitrary text as a shader.
6. **Native/mobile targets:** design separately around declared platform
   capabilities and resource budgets. The current CPU WebView preview is not
   proof of a native/GPU renderer, mobile performance, console support, or
   RTX-class performance.

The experimental visual milestone now renders a bounded subset of
`examples/scene.dimentra` in `android-preview.html` and is covered by pixel
buffer tests. The next steps are integrating that same pipeline into the
desktop playground and building/installing the standalone Android APK when an
Android toolchain is available. Do not silently ignore unsupported scene
features. Treat screenshots only as visual evidence; pair them with automated
pixel/geometry tests and label the renderer and limitations.

### UI, effects, flow and synthesis

- UI declarations need a documented widget tree, layout rules, state binding
  semantics, event ordering, accessibility behavior and a deterministic test
  harness before they become interactive.
- Effect declarations need a capability model, explicit host-provided
  implementations, permission/error semantics and review of every operation.
  Until then they are checked metadata, not operations.
- Petri-flow needs token types, transition enabling/firing rules, scheduling,
  termination/deadlock semantics and reproducible traces before a scheduler
  exists.
- 20D synthesis needs a mathematical definition for dimensions, folds,
  projections, numerical precision, bounds and expected results. Do not use
  the name "20D" as a substitute for a defined algorithm.
- Shader/material expressions need a typed, bounded expression language and
  known backend mapping. No arbitrary source evaluation.

These are separate semantic subsystems. Do not combine them into one
unreviewable runtime or claim that all syntax in the examples is executable
because a different subset compiles.

## Implementation sequence and done criteria

Treat each milestone as independently reviewable; update `SPEC.md`,
`README.md`, examples and tests with the implementation.

### Phase 0 — Baseline and spec agreement

- Run the Dimentra3D parser tests and Agent Workbench check.
- Inspect the parser, tests, spec, examples, playground and related geometry
  helpers. Check the Rustba reference hashes if those files are touched.
- Make a production list of parser-supported forms versus EBNF and example
  forms; record mismatches and correct them with tests.

**Done when:** baseline results are known, claims match code, and no existing
user changes were overwritten.

### Phase 1 — A real typed frontend

- Define source spans and grammar for expressions, vectors, transforms and
  function/generator arguments needed by the first slice.
- Parse expressions into AST nodes; add scope-aware resolution and a
  first-pass type checker.
- Add negative tests for unknown names/types, bad conversions, arity/ranges,
  malformed/nested input, resource limits and source locations.

**Done when:** every accepted expression has a defined type and AST shape;
unsupported or invalid input produces a stable located error rather than a
string accepted as meaningful.

### Phase 2 — Mathematical scene IR and correctness reference

- Define scene IR types for vertices/indices, transforms, camera, materials,
  lights and object references.
- Implement bounded deterministic mesh generation and transformations.
- Build a CPU reference renderer for the narrow scene subset, plus geometry
  invariants and pixel-level fixtures.

**Status:** the prototype covers a narrow icosphere scene subset using
`dimentra3d-render.js`; further generators, transforms, perspective,
clipping, richer lights and accuracy tests remain.
**Done when:** the supported scene set deterministically yields validated
geometry and tested image buffers, while unsupported declarations fail
explicitly.

### Phase 3 — First compiled subset

- Finalize a small executable language subset and its semantics.
- Implement a typed IR and one backend, with an explicit target/toolchain.
- Add a `check`/`build` CLI, compiler diagnostics, generated-output safety
  tests, and backend/reference equivalence tests.

**Done when:** a sample program is compiled to a named target artifact and
produces the same verified result as the reference semantics. Parsing alone
does not satisfy this phase.

### Phase 4 — Playground render integration

- Complete source editing, parse/type diagnostics, explicit preview action,
  scene loading from typed IR and visible output across the original desktop
  playground. The separate `android-preview.html` already exercises the
  current limited subset.
- Test malformed input, unsupported features, empty scenes, camera changes,
  depth overlap and deterministic redraws.

**Done when:** `examples/scene.dimentra` visibly renders through Dimentra3D's
own documented pipeline, and automated tests cover its math/output. Do not
present an unrelated imported viewer as Dimentra3D rendering.

### Phase 5 — Optimization and additional targets

- Profile named representative workloads first; establish a baseline and
  correctness equivalence.
- Optimize one measured bottleneck at a time; retain a reference path and
  report compiler/runtime/device versions and tolerances.
- Add GPU, mobile, system, UI, flow, synthesis, or console support only as
  separate scoped features with their own semantics, safety reviews and tests.

**Done when:** each claimed improvement meets predeclared correctness and
performance thresholds on named hardware and can be reproduced.

## Required validation for future changes

Run the smallest checks that cover the changed layer, then the integrated
language and Workbench checks:

```powershell
node --test workshop\dimentra3d\dimentra3d.test.cjs
node tools\agent-workbench.cjs run dimentra3d
node tools\agent-workbench.cjs run workbench
```

When compiler/runtime or renderer code is added, add and run its focused test
suite; do not rely on parser tests as a proxy. Also run `git diff --check`.
Tests that require a missing compiler/API should fail clearly or be reported as
not run; never convert an unavailable tool into a pass.

For renderer work, test exact dimensions and stable geometry invariants, then
compare pixels using documented color space/tolerance. For compiler work,
assert the target artifact and execute only a bounded fixture in an explicitly
invoked test process; compare it to the reference result. Keep benchmarks
separate from correctness tests and avoid flaky timing assertions.

## Starting instructions for the next assistant

1. Read this roadmap, `SPEC.md`, `README.md`, the parser/tests and all three
   examples before editing.
2. Inspect repository status and preserve unrelated user changes.
3. Choose exactly one next milestone. The recommended immediate task is
   **Phase 0 plus the expression/type-system design needed by Phase 1**; do
   not start an optimizer, GPU backend, mobile runtime, or system effects.
4. State the proposed source subset, semantics, files to change and tests.
   Ask the user before resolving significant language-design choices.
5. Implement the smallest end-to-end slice, update the spec and tests, run
   fixed checks, and describe what is still unsupported.
6. Do not claim compilation, rendering, learning, or speed improvements until
   the corresponding exit criteria above are met by executable evidence.

## Capability vocabulary

Use these terms consistently in code, docs and reports:

| Term | Required evidence |
|---|---|
| Parsed | Lexer/parser accepts source and returns located syntax/AST. |
| Checked | Names, types, constraints and references pass documented checks. |
| Lowered | Checked source has a defined typed IR representation. |
| Compiled | A backend emits a named executable/target artifact from the checked IR. |
| Executed | The artifact/runtime ran an explicitly selected bounded program. |
| Rendered | Dimentra3D scene semantics produced tested visible pixel output. |
| Faster | Correctness-equivalent workload beats a named baseline under a repeatable benchmark. |

Never use a higher-level term when only a lower-level condition has been
verified.
