← Catalog

RFC-011AcceptedToolchain

Testing Framework

Depends: RFC-001RFC-007RFC-009RFC-012

Blocks:

1. Abstract

This RFC specifies Aura’s built-in testing support: discovery via @test, assertions, async tests, unit vs integration layout, filtering, and coverage hooks. Tests run through aura test using the same compiler and runtime as production code.

Toolchain today (2026-07-16): aura test discovers @test functions; builtins assert / assert_eq (Int/String/Bool); std.assert package for package-mode asserts (C3d, C4h). No async tests, tags/filters, or coverage hooks yet.

2. Motivation

2.1 Problem statement

Testing bolted on via external-only frameworks fragments DX. First-party discovery and runner make TDD and CI default paths.

2.2 Why now

Language attributes (RFC-009) and CLI (RFC-012) need a concrete test model for Ship MVP.

2.3 Success metrics

MetricTarget
DiscoveryZero-config @test in package
Asyncasync @test works with runtime
CIJUnit-like / JSON report optional

3. Goals

  • Attribute-based discovery.
  • Rich asserts with good failure messages.
  • Parallel test execution by default where safe.
  • Integration test directory convention.
  • Coverage hooks (LLVM/source-based).

4. Non-goals

  • Full browser/e2e product testing.
  • Mandatory mocking framework in core (ecosystem).
  • Property-based testing in MVP (later).

5. Prior art & alternatives

SystemNotesTake
Rust #[test]SimpleInspiration
JUnitEcosystemReport formats
Go testingTable testsStyle
pytestFixtures powerLater ecosystem

6. Design

6.1 Overview

code
@test functions → compile test harness → run in process(es) → report

6.2 Discovery

aura
@test
fun adds() {
  assertEqual(1 + 1, 2)
}

@test
async fun fetches() {
  val v = await load()
  assertTrue(v > 0)
}
  • Package-private @test functions are discovered (no need for pub).
  • Ignore: @test(ignore) or @ignore.
  • Tags: @test(tag = "slow") + --tag filter.

6.3 Layout

KindLocation
UnitSame package, *_test.aura or inline
Integrationtests/*.aura as separate crates linking lib

Exact filename convention freeze later; document both.

6.4 Assertions (std.test)

aura
assertTrue(cond)
assertEqual(actual, expected)
assertNotNull(x)
assertFails { throw ... }
  • Failure captures file/line and pretty-prints values (Debug derive).

6.5 Runner

  • aura test builds test targets and executes.
  • Default: parallel by test function with isolation (process-per-test optional flag).
  • Filter: --test-name pattern, file path args.
  • Fail fast option.

6.6 Async & concurrency

  • Async tests run on runtime scheduler.
  • Timeouts per test configurable.
  • Race detector can enable under aura test --race when available.

6.7 Coverage

  • --coverage produces LCOV/HTML via instrumentation (LLVM).
  • Not required for MVP exit; design hooks reserved.

6.8 Reports

  • Human terminal default.
  • --format json / junit xml for CI.

6.9 Examples

code
aura test
aura test --release
aura test -- --tag slow
aura test packages/http

6.10 Error model / edge cases

CaseBehavior
Compile fail in testsNo run; show errors
Panic/uncaught exceptionTest fail + stack
HangTimeout fail
FlakyRetries not default

6.11 Compatibility & migration

  • Attribute names stable.
  • Runner flags semver careful.

7. Open questions

#QuestionOptionsOwnerStatus
1Inline vs *_test.aura onlyboth allowedTestResolved
2Process isolation defaultsame-process defaultTestResolved
3Fixture systemlaterTestDeferred — post-MVP; no core fixture system yet

8. Rationale & trade-offs

Built-in tests lower friction and unify CI. Keeping mocks out of core avoids framework wars. Parallel by default matches Go/Rust expectations; isolation trade-offs documented.

9. Unresolved / future work

  • Benchmark harness (@bench)
  • Snapshot testing
  • Property-based testing library

10. Security & safety considerations

  • Tests may execute untrusted project code—same trust as aura run.
  • Do not network to registry during test unless deps resolve already.
  • Coverage data should not embed secrets from env accidentally in reports.

11. Implementation plan (optional)

PhaseScopeExit criteria
T0@test + assertsaura test pass
T1Async + filterCI sample
T2Reports + coverage hookJSON junit

12. References


Changelog

DateAuthorChange
2026-07-16Defer fixture system post-MVP
2026-07-16Status → Accepted — Review: @test discovery + same-process MVP shipped and locked
2026-07-16Note aura test + assert MVP shipped
2026-07-15Initial skeleton
2026-07-15Solid draft: @test, runner, async
2026-07-15Lock discovery layout + same-process default