ADR 0102: Set-Theoretic Type Operators (Intersection and Negation) for Narrowing and Atom Exhaustiveness

Status

Implemented (Phases 1, 2, 4, 5, 2026-07-06) — Phase 1b (BT-2744, nominal-class difference + the class/isKindOf: false branch) deliberately deferred as needs-spec; not part of this implementation.

Implementation Tracking

Epic: BT-2738 Issues:

PhaseIssueTitleSizeBlocked by
1BT-2739intersect/difference operators + Negation variantM
2BT-2740Unify singleton_eq/is_nil narrowing; delete union_withoutMBT-2739
2BT-2741class_eq/is_kind_of true branch + Never-receiver policySBT-2739
1bBT-2744Nominal-class difference + class false branch (needs-spec)MBT-2741
4BT-2742\ difference annotation syntaxMBT-2739
4BT-2743& intersection syntax + stored Intersection variantMBT-2739
5BT-2745Advisory singleton-union match: exhaustivenessMBT-2740
6BT-2746E2E + docs + status flipSBT-2742, BT-2743, BT-2745

Status: Implemented — Phases 1, 2, 4, 5, and 6 (this issue) landed. Phase 1b (BT-2744) remains deferred pending its own design work on nominal-class difference semantics; it is not required for \/& surface syntax or the advisory match: exhaustiveness check, which only need atom-level (singleton) difference.

Context

Problem statement

Beamtalk's type checker narrows types by pattern-matching a fixed, growing list of AST idioms. Each recognised shape (isNil, class =:= Foo, isKindOf:, respondsTo:, x =:= #foo, isOk/isError) is a separate detector file under type_checker/narrowing/rules/, and the true/false branch types are computed by bespoke, per-idiom logic. Adding a new form of narrowing means adding a new file and a new entry in the RULES table (narrowing/rules/mod.rs).

Underneath these idioms, the checker is already performing set operations on types by hand — but only for the narrow cases each idiom needs:

There is no general algebra tying these together. The true branch of a test is computed one way, the false branch another, and each idiom re-derives both. ADR 0068 explicitly flagged the missing piece:

False-branch complement types (ifFalse: knowing "x is NOT Integer" — requires difference types)

as a known limitation. We have difference — but only for singletons removed from a flat union.

Current state

Singletons are represented as InferredType::Known { class_name: "#foo", .. } — the # prefix is the singleton marker — and treated as subtypes of Symbol (the "singleton-as-Symbol convention", type_resolver.rs:73, inference.rs:2035). Unions are a first-class InferredType::Union variant. There is no intersection type and no negation type. InferredType is otherwise nominal: Known, Union, Meta, Dynamic(reason), Never.

Dynamic(reason) is a checking hole — it means "skip checking", not a type with structure.

Constraints

Decision

Adopt the two missing set-theoretic operators as first-class members of InferredType — intersection and negation/difference — and express narrowing through a shared intersect/difference core where it applies (unifying singleton_eq and giving class/isKindOf: a principled true branch; §2 is explicit that this is not a uniform rewrite of all seven rules). Expose difference in surface type-annotation syntax, and use the algebra to drive advisory exhaustiveness checking for match: over singleton unions.

This is the middle of three possible commitments (see Alternatives): more than an internal-only refactor, less than a full CDuce-style semantic-subtyping engine.

1. Operators, mostly as normalising functions

The two operators are introduced primarily as normalising functionsintersect(A, B) and difference(A, B) — that return existing InferredType variants (Known / Union / Never) wherever the result is representable. This is deliberate: refine_singleton_narrowing today already returns the bare matched singleton, never a wrapped intersection, so most narrowing never needs new storage. Subtyping remains set inclusion as the mental model, but the equation A <: B iff difference(A, B) == Never holds only on the atom fragment this ADR defines (singletons, Symbol, unions of them) — it is the north-star invariant (contract #1), not a claim about these partial operators. Nominal subtyping (Integer <: Number) remains the hierarchy walk, and difference(Dynamic, P) = Dynamic deliberately says nothing about gradual subtyping.

Only irreducible results need new stored variants — and in Phase 1 there is exactly one:

enum InferredType {
    Known { class_name, type_args, provenance },
    Union { members, provenance },        // t1 | t2 | ...
    // NEW (Phase 1) — only when the result cannot collapse to the above:
    // `excluded` is an InferredType (a singleton or a normalised union of
    // singletons), NOT a raw atom name — `Symbol \ (#a | #b)` must be
    // representable. See the union-excluded and flattening rules below.
    Negation { base, excluded, provenance }, // co-finite complement, e.g. Symbol \ #foo
    // NEW (Phase 2, with `&` syntax) — see below:
    Intersection { members, provenance },  // irreducible conjunction: class ∩ protocol
    Meta { class_name, provenance },
    Dynamic(DynamicReason),                // unchanged: gradual hole
    Never,                                  // bottom / none()
}

The stored Intersection variant is deferred to Phase 2, by the same precedent that defers nominal-class difference: under single inheritance, class ∩ class always reduces (to the subclass, or to Never for unrelated sealed classes), so intersect() in Phase 1 is a pure function over existing variants. The only irreducible intersection is class ∩ protocol, and its only producers are ADR 0068's & protocol composition — which 0068 specifies (§Protocol Composition, Collection(Object) & Comparable) but which is not yet implemented (TypeAnnotation has no intersection variant) — and a future responds_to-narrowing refinement this ADR does not touch (§2). Storing Intersection before either producer exists would tax every exhaustive match for a variant nothing constructs. It lands in Phase 2 alongside the & surface syntax; see §3 for the reconciliation.

Normalisation rules (intersect / difference; not a blind mirror of union_of — the Dynamic handling is deliberately different, see below):

2. Narrowing expressed as intersect / difference — where it applies

The target shape for a test that a value has pattern-type P against current type T is:

This is an honest generalisation of singleton_eq, not a free port of all seven narrowing rules (narrowing/rules/mod.rs lists seven: is_nil, two is_result rules, responds_to, is_kind_of, class_eq, singleton_eq). They divide into three groups, and the ADR must not pretend otherwise:

  1. Already (nearly) this shape — singleton_eq, is_nil. refine_singleton_narrowing (inference.rs:2316) hand-picks matched for one branch and union_without for the other; it becomes literally:

    info.true_type  = intersect(&current_ty, &matched);
    info.false_type = Some(difference(&current_ty, &matched));  // swapped if negated
    

    type_admits_singleton becomes intersect(T, #foo) != Never. One deliberate corner changes: today the true branch is matched unconditionally (inference.rs:2326), so x :: Integer; x =:= #foo types the true branch #foo; under intersect it becomes Never. The impossible-comparison diagnostic already fires for exactly this case (check_impossible_singleton_comparison), so only the (unreachable) branch's type changes — pin with a new test. is_nil is the same shape with P = UndefinedObject — its difference engine, non_nil_type (inference.rs:3306), is the second hand-rolled subtraction this ADR unifies. One documented divergence: non_nil_type turns a nil-only union into Dynamic, not Never (test non_nil_type_all_nil_becomes_dynamic), a deliberate open-world softening. The port keeps that behaviour, and the mechanism is committed (not an implementation choice): difference itself stays puredifference(Union[Nil], UndefinedObject) is Never, because §4's exhaustiveness residual requires all-members-removed to mean Never — and the is_nil narrowing call-site maps NeverDynamic as its own open-world policy. Softening is a narrowing decision, not algebra; a difference-internal special case would corrupt residuals, and a private is_nil wrapper would keep a second hand-rolled subtraction alive, defeating the unification. Porting blindly "for uniformity" would be a silent behaviour change.

  2. Ported, but a genuine behaviour change — class_eq, is_kind_of. Today these set true_type = P unconditionally and false_type = None (class_eq.rs: true_type: Known(name), false_type: None). Moving the true branch to intersect(T, P) is new logic: x class =:= Bar where T and Bar are unrelated sealed classes would now yield Never and should route through the same impossible-comparison hint the singleton path already has (check_impossible_singleton_comparison, inference.rs:2354). One refinement, learned in implementation (BT-2741): the class hint is provenance-gated — it fires only when the receiver's type was inferred from value flow, and stays silent when it came from a declared annotation. Under gradual typing an annotation is an unverified promise, and isKindOf: is precisely how code verifies it at runtime, so a defensive guard against a declared type (stdlib SystemNavigation referencesTo:'s (aClass isKindOf: Symbol) check) is legitimate and must not warn; the true-branch narrowing to Never is not gated (sends there are silent per the Never-receiver policy, so it stays harmless). The singleton hint's behaviour (fires on declared types too, pinned by BT-2740) is unchanged precedent. Computing the false branch requires nominal-class difference (difference(T, Bar) over the class hierarchy), which §1 explicitly does not define. So closing the class/isKindOf: false-branch gap is its own design step (nominal negation semantics: membership, display, hover), tracked separately — not assumed done by this ADR.

  3. Not expressible as intersect/difference — is_result, responds_to. refine_result_narrowing keeps the full, unchanged Result(T, E) on both branches: it narrows a runtime tag that has no InferredType representation (difference(Result, Result) would wrongly yield Never). responds_to narrows to protocol conformance, which today has its own representation path; it is untouched by this ADR, but is the designated first consumer of the stored Intersection variant in Phase 2 (T & Protocol is exactly what a respondsTo: true branch means) — noted so the Phase 2 design has a real producer, not just syntax.

So the deliverable is: unify group 1 immediately (with the two documented corners), route group 2 through intersect for the true branch (false branch gated on nominal-difference design), and leave group 3 alone. The detectors in narrowing/rules/ all stay — they still recognise AST shape and produce P.

3. Surface syntax — reuse &, add a difference operator

Intersection already has a spelling — on paper. ADR 0068 §Protocol Composition specifies & in type annotations (sort: items :: Collection(Object) & Comparable) — semantically an intersection — but it is not yet implemented (no TypeAnnotation variant, no & handling in type position). This ADR therefore does not invent a second intersection keyword — implementing 0068's & and generalising it beyond protocol conjunction is Phase 2 work here, where class ∩ class reduces via the hierarchy (subclass, or Never for unrelated sealed classes — so Integer & String is defined: it is Never) and class ∩ protocol is the irreducible stored case. Only difference/complement is new surface syntax, written \ ("T without U"):

// "any symbol except the reserved ones"
tag :: Symbol \ (#reserved | #internal)

// residual after handling the known cases
handleRest: dir :: (#north | #south | #east | #west) \ #north => ...

// intersection is the existing 0068 operator, now general
thing :: Printable & Comparable

Precedence, lowest-binding to highest (all parsed only inside a type-annotation context, so none collides with a binary message selector — < stays a message, per ADR 0068):

OperatorMeaningBinding
|unionlowest
&intersectionmiddle
\differencemiddle (left-assoc, same tier as &)
(atomic type / (...))groupinghighest

Integer \| Symbol \ #foo therefore parses as Integer \| (Symbol \ #foo). Within one operator the tier is left-associative (A \ B \ C = (A \ B) \ C). Mixing & and \ without explicit grouping is a deliberate parse error — not left-associative resolution: A & B \ C is rejected with "parenthesise to disambiguate". This is a grammar rule (the "incompatible infix operators" pattern, as OCaml and Rust use for ambiguous operator mixes), chosen because (A & B) \ C and A & (B \ C) differ and neither reading is obviously dominant. Using operator symbols (not and/not identifiers) also sidesteps ADR 0068's rule that an unrecognised identifier in type position is an implicit method-local type parameter — a type param spelled and would otherwise silently shadow the operator.

Lexing is not symmetric with |, and the ADR must not pretend it is. | works in type position because it is a dedicated token (TokenKind::Pipe, token.rs:122), deliberately split from BinarySelector. \ and & currently lex as ordinary BinarySelectors (lexer.rs:436), and both are live expression operators&/| are boolean ops and \\ (double backslash) is the Smalltalk modulo selector (see is_pure_binary_op, lint_validators.rs). So the annotation parser must either special-case BinarySelector("\\") / BinarySelector("&") in type position, or promote them to dedicated tokens (which touches expression lexing). Additionally, because the lexer greedily consumes selector characters, the near-certain typo Symbol \\ #foo lexes as the modulo selector — Phase 2 must ship a targeted parse error for \\ in type position ("did you mean \?").

4. Exhaustiveness

Beamtalk has no Erlang-style multi-clause method heads — a selector maps to one method body, and case analysis happens inside it via the match: keyword message. match: already performs exhaustiveness checking (BT-1299) for sealed types with constructor patterns. This ADR extends that same check to singleton-union scrutinees, with the set-theoretic residual as its engine: because difference is now total, a match: over a singleton union can compute its residual type (difference(scrutinee_type, covered patterns)). When the residual is Never the match: is exhaustive; when it is non-Never the checker warns with the uncovered members.

compass :: #north | #south | #east | #west := readHeading

result := compass match: [
  #north -> "up";
  #south -> "down";
  #east  -> "right"
]
// ⚠️ non-exhaustive: `#west` is not handled  (residual type: `#west`)
//    — either add a `#west ->` arm or a `_ ->` wildcard

This is a type-based check, distinct from existing BT-1299 — and deliberately weaker. Today's match: exhaustiveness is pattern-based: it keys on the constructor patterns in the arms and emits a hard Diagnostic::error(), precisely because Beamtalk "is dynamically typed — there is no resolved scrutinee type available at compile time." This ADR proposes trusting the inferred scrutinee type instead, which is strictly less conservative: inference can be wrong, and — under gradual typing — annotations are not runtime-enforced, so a "provably exhaustive" static claim cannot promise the match: won't crash at runtime (e.g. an FFI call typed #north | #south that actually returns #west). Therefore:

REPL session

bt> x := someUnionReturning   "x :: Integer | #infinity"
bt> (x =:= #infinity) ifTrue: [x] ifFalse: [x + 1]
                                          ^^^^^
   in the false branch, x :: Integer  (#infinity removed) → x + 1 type-checks

Co-finite types surface through LSP hover (already in scope — hover rendering is listed under affected components), which displays Symbol \ #foo for a variable so narrowed. No new REPL command is added — a :type-style command would be its own surface-parity decision (docs/development/surface-parity.md) and is out of scope here.

Error example

d :: #north | #south := heading
d =:= #west
// ⚠️ comparison can never be true: `#west` is not a value of `#north | #south`
//    (this diagnostic already exists via check_impossible_singleton_comparison;
//     it is now a direct consequence of `intersect(T, #west) == Never`, not a
//     special-cased check)

Prior Art

Language / systemApproachWhat we take / leave
Elixir (1.17+) — gradual set-theoretic types (Castagna, Duboc, Valim)Types are sets of values; closed under or / and / not. Atoms are singleton types (:ok is both value and type); boolean() = true or false; nil, true, false are atoms. atom() and not (:foo or :bar) expresses co-finite atom sets. dynamic() is a range of types kept "at the root", enabling gradual typing that still tracks structure.Adopt: the three-operator algebra and singleton-as-set model — it is exactly what our #foo/Symbol/union_without code approximates. Adapt: we reuse ADR 0068's & for intersection and add \ for difference as surface syntax (not Elixir's and/not keywords), with internal intersect/difference operators, without adopting the full semantic-subtyping decision procedure. Leave (for now): replacing Dynamic(reason) with a structural dynamic(); BDD emptiness checking.
CDuce / semantic subtyping (the theory underneath Elixir)Subtyping = set inclusion decided via emptiness of a Boolean combination of type constructors, implemented with BDDs.The north-star. Rejected as the implementation today (XL, nominal-core mismatch); adopted as the mental model for our operators so a future upgrade is a deepening, not a rewrite.
TypeScriptDiscriminated unions + control-flow narrowing; literal (singleton) types; Exclude<T, U> at the type-operator level but no first-class negation type in values.We go slightly further than TS values by making negation a real InferredType, which is what makes atom false-branch narrowing and exhaustiveness fall out (nominal-class false-branch difference is deferred — §2).
Gleam (BEAM, Rust-implemented, our closest typed-BEAM peer)Sound nominal HM; no union/intersection/negation, exhaustiveness via nominal custom types.Confirms exhaustiveness is table-stakes on BEAM, but Gleam gets it from closed nominal sum types; we get it from the atom-union algebra, which fits Beamtalk's open, atom-rich style better.
Dialyzer (success typing)Post-compilation, optimistic; has union/negation internally but no IDE-time narrowing.Reaffirms why we compute this at edit time, not after codegen (type-system-design.md).
Smalltalk (Pharo/Squeak)No static types; #foo is just a Symbol, no singleton type.No prior art to preserve — this is purely additive tooling that does not touch message-passing semantics.

User Impact

Steelman Analysis

Chosen: Internal operators + surface syntax + exhaustiveness

Rejected A: Internal algebra only (no surface syntax, no exhaustiveness)

Rejected B: Full semantic subtyping (CDuce engine, structural dynamic())

Tension points

Alternatives Considered

Status quo — keep adding per-idiom rules

Every new narrowing shape stays a new file plus bespoke true/false logic, and false-branch complements remain impossible beyond the singleton case (ADR 0068's noted gap persists). Rejected: the idiom list grows unboundedly and duplicates set logic the codebase already contains four times (union_of, union_without, non_nil_type, type_admits_singleton).

Functions only — no stored Negation either

The true minimum between status quo and Alternative A: ship intersect / difference as normalising functions, and when a difference is co-finite (irreducible), fall back to today's behaviour (false_type = None — no narrowing). Zero new InferredType variants, zero new match arms. Rejected because the fallback silently discards exactly the new expressiveness: the Symbol-typed false branch of x =:= #foo stays un-narrowed, hover cannot display "everything except #foo", and Phase 2's \ syntax and residual-based exhaustiveness both need a representable co-finite result. Priced honestly: this saves one variant's worth of match arms, at the cost of the feature's point.

Internal algebra only

Add Intersection/Negation internally, refactor union_without / type_admits_singleton / branch computation onto them, but expose no syntax and add no exhaustiveness. Rejected as the terminal state (adopted as phase 1): the operators are the hard part, and the incremental cost of the surface syntax and exhaustiveness on top is small (chiefly TypeAnnotation variants and a residual check — see Implementation), so stopping short leaves most of the user value unrealised.

Full semantic subtyping engine

Replace nominal comparison with BDD-based emptiness checking and a structural dynamic(). Rejected as premature (see Steelman B); revisit if/when Beamtalk needs full arrow/intersection-function typing. Documented as the destination in docs/internal/set-theoretic-types-north-star.md.

Consequences

Positive

Negative

Neutral

Implementation

Phase 1 — internal operators + group-1 unification (Alternative A core) — ~M:

  1. Add intersect / difference normalising functions and the Negation variant only to InferredType (types.rs) — the stored Intersection is Phase 2 (§1). Encode the Dynamic asymmetry, the RHS-union fold, the exact-type_args rule, and the union_of absorption law (§1), each with a regression/property test; do not mirror union_of's Dynamic-absorbs rule.
  2. Update all exhaustive matches: PartialEq (canonical excluded ordering), display_for_diagnostic, provenance accessors, as_known, hover rendering, union_of.
  3. Rewrite refine_singleton_narrowing and is_nil's branch computation (group 1), delete union_without, redefine type_admits_singleton as intersect(T, s) != Never. Behaviour pinned by existing narrowing/union tests plus two new pins for the documented corners: the impossible singleton test's true branch becoming Never, and is_nil's nil-only-union → Dynamic softening being preserved.
  4. Route class_eq / is_kind_of true branch through intersect(T, P) (group 2, true branch only), wiring the impossible-comparison hint, and decide the Never-receiver send policy (silent-as-unreachable vs. hint). Leave their false branch, is_result, and responds_to (group 3) untouched.

Phase 1b — nominal-class difference (prerequisite for the class false-branch) — ~M (new semantics, design-heavy): 5. Define difference over the class hierarchy (membership, display, hover for Negation of a nominal base). Only then close the class_eq/is_kind_of false branch. Sequenced separately because it is new semantics, not a port.

Phase 2 — surface syntax + advisory exhaustiveness — ~M (parser + validator): 6. Implement ADR 0068's & (spec-only today) generalised to any-type intersection, add the stored Intersection variant (§1), and add \ (difference) to the annotation grammar — new TypeAnnotation variants (ast/expression.rs) plus every exhaustive match and type_resolver.rs. Handle the token asymmetry (§3): special-case BinarySelector("&")/("\\") in type position or promote to dedicated tokens; ship the \\-typo parse error. Add the precedence tier (§3). 7. Add a type-based, advisory singleton-union exhaustiveness check to match:: compute difference(scrutinee, covered); if non-Never, emit a Warning/Lint (never Error) gated by ADR 0100 completeness. Distinct from BT-1299's pattern-based error, which is unchanged.

Affected components: type checker (semantic_analysis/type_checker/*), annotation parser + TypeAnnotation AST, match: exhaustiveness validator, hover/diagnostic rendering (queries/hover_provider.rs). Not affected: codegen, runtime, REPL output values.

Migration Path

Backward compatible. singleton_eq/is_nil narrowing (Phase 1 group 1) produces identical results except two documented corners — the impossible singleton test's (already-diagnosed, unreachable) true branch becomes Never, and is_nil's nil-only-union → Dynamic softening is deliberately preserved — both pinned by new tests alongside the existing narrowing/union suites. class_eq/is_kind_of true-branch narrowing (group 2) is a deliberate refinement — it can newly report an impossible comparison where the old code silently kept P; this is the intended improvement, routed through the existing hint machinery, and any affected test is updated in the same change. &/\ syntax and the exhaustiveness diagnostic are additive; the latter is advisory (Warning/Lint), so no previously-clean build newly fails — even when a scrutinee is now inferred as a singleton union. No existing .bt source changes meaning.

References