Extend the language with native checksums
This program computes the same digest as the two before it with every stage native: an HTTP request, a SHA-256 checksum, and hexadecimal encoding. It is an example from Hypershell, a shell-scripting language whose programs are Rust types, built with Context-Generic Programming (CGP), a Rust library that lets a language be extended without editing its crates. The checksum stages are not part of the base language. The page shows how a language extension defined in other crates is joined by changing one namespace name.
New to CGP?
This page builds on http_checksum_client. For CGP itself, the Hello
World tutorial is the quickest start. The page uses CGP's
namespaces, reusable wiring tables that a namespace or a context can
inherit, and explains what it needs of them.
The problem
The task is the same digest again, with every stage native: an HTTP request, a SHA-256 checksum, and hexadecimal encoding. The base language has no checksum stage, so the task is really to extend a language from outside the crates that define it, and to run the extended language with as little change as possible.
Without CGP
A language whose program is an enum of instructions, interpreted by a match, can gain a checksum
instruction only in the crate that owns the enum: someone adds the variant, adds an arm to the
interpreter, and releases a new version. A user who cannot wait forks it. This page adds the
checksum stages in crates of their own, with their own syntax and providers, and a program joins the
extended language by naming a different namespace.
Run it
From the root of the Hypershell repository:
cargo run --example http_checksum_native
It needs the network and no other tools, and prints the same digest of the Nixpkgs manual page as
http_checksum_cli.
Two stages from an extension crate
The program replaces sha256sum and cut with Checksum and BytesToHex, from
http_checksum_native.rs:
use hypershell::prelude::*;
use hypershell_examples::namespaces::HypershellChecksumNamespace;
use hypershell_hash_components::dsl::{BytesToHex, Checksum};
use reqwest::Client;
use sha2::Sha256;
pub type Program = hypershell! {
StreamingHttpRequest<
GetMethod,
FieldArg<"url">,
WithHeaders[ ],
>
| Checksum<Sha256>
| BytesToHex
| StreamToStdout
};
The two new stages come from hypershell-hash-components, a crate that depends on neither the
Hypershell core nor Tokio. Checksum<Sha256> names its hash algorithm as a type from the sha2
crate, and produces the digest as raw bytes, which BytesToHex encodes as text for printing.
Joining the extended language
The context, the type the program runs on, differs from the previous example's in one name:
#[derive(HasField)]
pub struct MyApp {
pub http_client: Client,
pub url: String,
}
delegate_components! {
MyApp {
namespace HypershellChecksumNamespace;
}
}
HypershellNamespace does not route Checksum or BytesToHex, so a context joined to it cannot
run this program. HypershellChecksumNamespace is a namespace defined in the examples crate that
inherits HypershellNamespace and adds routes for the two new stages:
cgp_namespace! {
new HypershellChecksumNamespace: HypershellNamespace {
@cgp.extra.handler.HandlerComponent.[
<Hasher> Checksum<Hasher>,
BytesToHex,
]:
HypershellChecksumProvider,
}
}
cgp_namespace! defines the namespace, and the part after
the colon names the namespace it inherits. Its one entry routes both syntaxes to
HypershellChecksumProvider, a bundle of providers, or aggregate provider, also defined in the
examples crate:
delegate_components! {
new HypershellChecksumProvider {
open {HandlerComponent};
@HandlerComponent.<Hasher> Checksum<Hasher>:
PipeHandlers<Product![
HandleToFuturesStream,
HandleStreamChecksum,
]>,
@HandlerComponent.BytesToHex:
HandleBytesToHex,
}
}
HandleStreamChecksum and HandleBytesToHex are the providers from the hash crate that do the
work. HandleToFuturesStream in front of the checksum is an input dispatcher, the counterpart of
the one in http_checksum_client, so Checksum accepts bytes or
either kind of stream. The same two-layer shape, a bundle that maps syntax to providers and a
namespace route that points at the bundle, is how the whole base language is assembled; see
assembly.
Try a change
Remove the BytesToHex stage, so that the raw digest flows straight into StreamToStdout:
pub type Program = hypershell! {
StreamingHttpRequest<
GetMethod,
FieldArg<"url">,
WithHeaders[ ],
>
| Checksum<Sha256>
| StreamToStdout
};
The program then fails at its handle call, and cargo cgp check names
the missing entry, shown here with the long types abridged:
error[E0277]: [CGP-E001] missing entry for `@HandlerComponent.StreamToStdout.GenericArray<u8, …>` in provider `hypershell_tokio_components::providers::async_read::HandleToTokioAsyncRead`
= note: root cause: [CGP-E114] missing entry for `@HandlerComponent.StreamToStdout.GenericArray<u8, …>` in provider `hypershell_tokio_components::providers::async_read::HandleToTokioAsyncRead`
The digest is a GenericArray<u8, …>, and StreamToStdout's input dispatcher has no entry for that
type. The path in the message is the syntax followed by the input type, which is exactly the key the
dispatcher failed to match. The fix is to convert the value into a type the dispatcher lists, which
is what BytesToHex does. cargo cgp check leads with the root cause for the classes it
recognizes, and the tool does not yet reshape every class.
The pattern
This program shows extending a language through a namespace that inherits it. A library publishes its language as a namespace, and anyone can publish a larger language that inherits it and adds new syntax, from their own crates and without editing the library. A context chooses which language it runs by the namespace it joins, and extensions can build on each other: the examples crate has a second extension built on this one. The ideas are explained in Namespaces and Aggregate providers.
The limit is that an extension can only add. It cannot change what the syntax it inherits does, because CGP does not let an inheriting namespace override a choice its parent has made. The limitations page says more about what this means.
Where to go next
nix_manual: the next example, with a static URL and a mixed argument list.bluesky_websocket: an extension wired onto one context instead of a namespace.- Assembly: the bundles and routes the base language is built from.
- Namespaces: inheriting a namespace, and why an inherited binding cannot be overridden.
- Hypershell: a type-level DSL for shell-scripting in Rust: the post that announced Hypershell, whose last part builds this extension. Its code predates the current design.
An AI agent wrote this page using the CGP knowledge base. Its content was verified against the project's source. See How AI is used in this project.