Extensible data
Extensible data lets generic code read, build, deconstruct, and convert structs and enums through named fields and variants. It represents a struct as a product of fields and an enum as a sum of variants, with each name encoded at the type level.
Extensible-data derives expose a type’s structure to generic code. Ordinary struct literals and enum matches name every field or variant directly. The derives instead generate a type-level description and operations that construct, extract, and convert values through it.
Independent providers can then contribute fields or handle variants without knowing the complete type. This supports row polymorphism and structural sum types: generic operations depend on named parts of a type’s structure. Resolution happens at compile time. Builders, variant dispatchers, and structural casts use the generated operations to assemble and process contexts.
Records and variants use related representations. A record contains every field at once, while an enum contains one variant at a time. Their operations share marker, casting, and dispatch mechanisms, with different rules for representing absent fields and excluded variants.
The examples describe CGP v0.8.0 and assume use cgp::prelude::*;. The prelude carries the derives
but not the traits whose methods they generate, so the examples also import the builder, extractor,
and variant traits (HasBuilder, BuildField, FinalizeBuild, HasExtractor, ExtractField,
FinalizeExtractResult, and FromVariant) from cgp::core::field::traits, CanBuildFrom from
cgp::core::field::impls, and the handler names they wire from cgp::extra::handler and
cgp::extra::dispatch.
The umbrella derive and its shape-specific faces
Use #[derive(CgpData)] to generate the full extensible-data support for a struct or enum.
CgpRecord and CgpVariant generate the same support but accept only structs and enums
respectively. Applying either to the wrong item kind is an error.
Choose a narrower derive when only part of the support is needed. HasFields, HasField,
BuildField, ExtractField, and FromVariant each generate a subset. The umbrella derive covers
both shapes:
#[derive(CgpData)]
pub struct Person { pub first_name: String, pub last_name: String }
#[derive(CgpData)]
pub enum Shape { Circle(Circle), Rectangle(Rectangle) }
Generated operations identify fields and variants by type-level tags. Named fields and variants use
Symbol!("name"), while tuple-struct fields use positional tags such as Index<0>. The same tag
identifies an entry across its generated impls.
Variant construction and extraction require each variant to carry one unnamed payload or no fields.
Wrap multiple values in a dedicated struct to give the variant one payload type. A variant with no
fields, written Closed, Closed(), or Closed {}, carries the payload Nil: build it with
from_variant(PhantomData::<Symbol!("Closed")>, Nil), and extract it as Nil, &Nil, or
&mut Nil. Multi-field tuple variants and struct-style variants with fields are rejected, and
individual variants cannot opt out of the derive.
Variants with no fields have three practical consequences:
- Value dispatch merges them: a value matcher hands every such variant the same
Nil, so use a field matcher (MatchWithFieldHandlers), whose provider receivesField<Tag, Nil>, to tell them apart. #[cgp_auto_dispatch]cannot serve the enum: the dispatched trait would need an impl for the foreignNil, which conflicts with the macro's blanket impl. Use the dispatch combinators with a provider instead.- A
no_stdcrate needsBoxin scope: the mutable extractor builds the&mut NilwithBox::leak(Box::new(Nil)), resolvingBoxwhere the derive is used.
The builder and extractor companions (__Partial{Name}) carry none of the type's derives, and from
each field or variant they keep only documentation. A helper attribute of another derive, such as
#[serde(rename = "x")] beside Serialize, stays on the input type, so the two derives combine
freely.
#[derive(HasFields)] accepts all variant shapes because it describes their structure without
generating variant construction or extraction. A unit variant contributes Nil, a newtype variant
contributes its payload, a multi-field tuple variant contributes an Index<N>-keyed product, and a
named-field variant contributes a Symbol!-keyed product inside its Field entry. The variant
derives agree with it for a variant with no fields, which is Nil in both. An enum with multi-field
or named-field variants can therefore have a structural description even though the constructor and
extractor derives reject it.
Records: the whole-struct field view
#[derive(HasFields)] describes a struct through its Fields associated type. The type is a
Product! containing a Field<Symbol!("name"), Type> entry for each
field:
impl HasFields for Person {
type Fields = Product![
Field<Symbol!("first_name"), String>,
Field<Symbol!("last_name"), String>,
];
}
Write this shape as Struct! { first_name: String, last_name: String } wherever code names it;
the macro expands to exactly the derived Fields. See
type-level primitives.
The derive generates conversions between the struct and its field representation. ToFields
consumes a value to produce its field product, FromFields rebuilds the value, and ToFieldsRef
returns a product of references. Generic algorithms can use HasFields, ToFields, or FromFields
to process this structure without naming the concrete struct.
Derive HasField as well when code needs access to individual fields by tag. HasField<Tag>
supplies the per-field access used by getters, while HasFields describes the complete
structure. See functions and getters.
Records: building a struct field by field
#[derive(CgpRecord)] and #[derive(BuildField)] support incremental construction checked at
compile time. They generate a partial-record struct named __Partial{Name} with a MapType marker
for each field. IsPresent stores a field’s value, while IsNothing stores (). The builder
starts with every field absent and can be finalized once every field is present:
let employee: Employee = Employee::builder() // every field IsNothing
.build_from(person) // first_name + last_name now IsPresent
.build_field(PhantomData::<Symbol!("employee_id")>, 1) // employee_id now IsPresent
.finalize_build(); // exists only at the all-present configuration
HasBuilder::builder() creates an empty partial record, and BuildField::build_field changes one
field from IsNothing to IsPresent. FinalizeBuild::finalize_build exists only when every field
is present, so incomplete construction fails at compile time. Fields can be supplied in any order.
IntoBuilder converts a complete struct to an all-present partial record, and TakeField removes
fields individually. Both BuildField and TakeField use UpdateField<Tag, M>, which changes one
marker and returns the old value with the rebuilt partial record. The target marker M is not
inferred from the value, so a direct call names it, as UpdateField::<Tag, IsPresent>::update_field;
otherwise it fails with E0284. Import TakeField from cgp::core::field::traits to call
take_field; it is not in the prelude.
Generated __Partial… companions do not inherit the original struct or enum’s attributes. A
#[derive(Debug, Clone)] therefore does not make a partial record or extraction remainder printable
or cloneable. Tests using assert_eq! on an extract_field result can fail for this reason. Use
.ok(), .is_ok(), or a match as appropriate, and read present fields through the companion’s
per-field HasField impls.
Use the cgp-field-extra optional-field extension when fields may be absent or have defaults. It
reuses UpdateField to fill unset fields with Default::default() during finalization or represent
them as Option through IsOptional. Absence can then be a runtime value. The strict builder that
requires every field remains the default. On an all-optional builder, finalize_with_default needs
Default on every field's type, set or not, because a set field is still IsOptional;
finalize_optional needs none.
Records: the extensible builder pattern
CanBuildFrom merges a source struct’s fields into a target builder with one build_from call. It
walks the source’s field product, taking each field with TakeField and inserting it with
BuildField. Source and target share field names without needing to name each other. Every source
field must exist on the target; an extra one fails on an unsatisfied UpdateField bound rather than
being dropped.
The source must derive HasFields as well as BuildField. The recursion reads HasFields::Fields,
so deriving only BuildField produces a missing HasFields bound. The target needs only
BuildField.
The extensible builder pattern uses these operations to assemble a context from independent providers. Each subsystem produces a small output struct, and a dispatcher merges those outputs into the target builder before finalization. The providers need not know the final context type or one another:
delegate_components! {
FullAppBuilder {
HandlerComponent:
BuildAndMergeOutputs<App, Product![
BuildSqliteClient,
BuildHttpClient,
BuildOpenAiClient,
]>,
}
}
The dispatcher is generic over the target struct and provider list. Change one provider entry to
replace a subsystem, or use code-based dispatch to select among target structs. See
handlers for BuildAndMergeOutputs and related routing.
Variants: constructing and deconstructing an enum
#[derive(HasFields)] represents an enum’s variants as a Sum! of
Field<Symbol!("Variant"), Type> entries using Either and Void, written as an enum body with
Enum! { Circle(Circle), Rectangle(Rectangle) }. #[derive(FromVariant)]
supplies a constructor for each variant, selected by its type-level tag:
fn wrap_circle(circle: Circle) -> Shape {
Shape::from_variant(PhantomData::<Symbol!("Circle")>, circle) // == Shape::Circle(circle)
}
Write the tag whenever the enum has two or more variants; a bare PhantomData fails with E0283,
since the payload type does not select the impl. Only a one-variant enum infers it.
#[derive(ExtractField)] generates a partial-variant companion enum named __Partial{Name}. Each
variant carries a MapType marker. A present variant uses IsPresent; an excluded variant uses
IsVoid, which maps its payload to the uninhabited Void type.
HasExtractor::to_extractor starts with all variants marked IsPresent.
ExtractField::extract_field returns Ok(value) when the requested variant matches. Otherwise, it
returns Err(remainder) with that variant marked IsVoid, so later operations know it has been
ruled out:
fn area(shape: Shape) -> f64 {
match shape.to_extractor().extract_field(PhantomData::<Symbol!("Circle")>) {
Ok(circle) => core::f64::consts::PI * circle.radius * circle.radius,
Err(remainder) => {
// remainder's type now has Circle ruled out (IsVoid); try the next variant
let rect = remainder
.extract_field(PhantomData::<Symbol!("Rectangle")>)
.finalize_extract_result(); // remainder is now empty; this cannot fail
rect.width * rect.height
}
}
}
Variant extraction checks exhaustiveness at compile time. Each failed extraction rules out a
variant, and a remainder with every marker set to IsVoid cannot contain a value.
FinalizeExtract::finalize_extract handles that uninhabited remainder with an empty match.
FinalizeExtractResult::finalize_extract_result uses this guarantee to return the final Ok value
directly.
Adding an unhandled variant makes the final remainder inhabited and prevents compilation until the
new case is covered. This preserves the exhaustiveness guarantee of a concrete match without a
wildcard arm. HasExtractorRef and HasExtractorMut provide equivalent operations on borrowed
values, over a second companion __PartialRef{Name} that the same ExtractField and
FinalizeExtract impls cover, so a borrowed chain finalizes too. Extracting a variant twice fails
with a confusing E0308 (expected 9, found 6, the lengths of the two tags) rather than a message
about the variant.
Variants: the extensible visitor pattern
The extensible visitor pattern routes each enum value to a provider for its current variant. New variants can be added without changing existing handlers, and handlers can serve several enums that share variants. This addresses the expression problem by separating data variants from the operations on them.
A dispatcher derives extraction and handling steps from the enum’s Fields. The pipeline stops at
the matching variant and otherwise passes the remainder to the next step:
delegate_components! {
Interpreter {
open ComputerComponent;
@ComputerComponent.<Code> Code.MathExpr: DispatchEval, // the whole enum → dispatcher
@ComputerComponent.<Code> Code.Plus<MathExpr>: EvalAdd, // one provider per variant
@ComputerComponent.<Code> Code.Times<MathExpr>: EvalMultiply,
@ComputerComponent.<Code> Code.Literal<u64>: EvalLiteral,
}
}
Each key dispatches on the input type, the second segment of the path, whatever the Code. Older
code wires the same table as ComputerComponent: UseInputDelegate<new EvalComponents { … }>; see
wiring.
MathExpr selects a context-specific provider that calls the matcher combinator. This wrapper
breaks the trait-resolution cycle between the matcher and the variant providers it calls. See
handlers for MatchWithValueHandlers, its borrowed form, and the handler families.
The type-level lists underneath
Records and variants use nested type-level lists that generic providers can process one entry at a
time. A record’s Product![A, B, C] expands to Cons<A, Cons<B, Cons<C, Nil>>>. The final Nil is
constructible because an empty record is a valid value. Lowercase product![..] constructs a value
of that product type.
An enum’s Sum![A, B] expands to Either<A, Either<B, Void>>. Each step offers a choice, and the
final Void is uninhabited because an empty enum cannot contain a value. See type-level
primitives for both representations.
Import list operations from cgp::core::field::traits; they are not in the prelude. AppendProduct
adds a field to the end of a product, ConcatProduct joins products, and MapFields transforms
each entry. These type transformations are evaluated during type checking without runtime cost. The
derives generate partial records directly, without these traits; in the library, MapFields rewrites
provider lists, as PipeMonadic does with TryPromoteProviders.
MapFields supports both products and sums. Its result is named Mapped, while AppendProduct and ConcatProduct expose Output. The
library does not provide AppendSum.
Structural casts between records and variants
Structural casts convert between types with compatible named fields or variants. They match entries
by name without a handwritten From or TryFrom impl. Import CanUpcast, CanDowncast,
CanDowncastFields, and CanBuildFrom from cgp::core::field::impls; these traits are not in the
prelude.
CanUpcast converts a smaller enum into one containing all its variants. It always succeeds by
extracting the source variant and rebuilding it through FromVariant, so the source needs
HasFields and ExtractField while the target needs only FromVariant. CanDowncast narrows an
enum when its current variant exists in the target, returning a remainder otherwise; every target
variant must exist in the source, or the downcast fails to compile. Use CanDowncastFields, not a
second downcast, on that remainder to try another target, and close a chain whose candidates cover
every variant with finalize_extract_result.
CanBuildFrom performs the corresponding record operation by moving every source field into a
target builder; a source field the target lacks, or one already set, is a compile error at
build_from. The enum casts behave as follows:
use cgp::core::field::impls::{CanDowncast, CanUpcast}; // not in the prelude
let wide = FooBar::Foo(1).upcast(PhantomData::<FooBarBaz>); // always succeeds
assert_eq!(wide, FooBarBaz::Foo(1));
FooBarBaz::Bar("hi".into()).downcast(PhantomData::<FooBar>).ok(); // Some(FooBar::Bar(..))
FooBarBaz::Baz(true).downcast(PhantomData::<FooBar>).ok(); // None: FooBar lacks a Baz variant
A provider can construct a small local enum and upcast it into a larger type. This lets it work with only the variants it needs, much as a getter accesses one field without depending on the complete record.
Dispatching over extensible data
Dispatch combinators use the exposed data structure to select providers generically.
BuildAndMergeOutputs assembles record fields from provider outputs, while MatchWithValueHandlers
selects the handler for a value’s variant. These combinators derive and sequence operations from the
type’s Fields.
See handlers for the combinators and their handler families. The examples above
select providers through ordinary delegate_components! entries for each
component.
Further reference
These online references describe extensible data as of CGP v0.8.0:
- Extensible records: Generic record construction and access.
- Extensible variants: Generic enum construction and extraction.
- Derive references:
derive_cgp_data.md,derive_cgp_record.md,derive_cgp_variant.md,derive_has_fields.md,derive_build_field.md,derive_extract_field.md, andderive_from_variant.md.
Consult the trait references for the operations generated by those derives:
The type-macro references explain the underlying list representations: