📄️ How TypeStream Works
TypeStream compiles pipeline definitions into Kafka Streams topologies. Whether you define a pipeline through the CLI, a config file, or the visual GUI, the same compilation and execution engine processes it.
📄️ Three Ways to Build Pipelines
TypeStream offers three interfaces for building data pipelines. All three compile down to the same PipelineGraph proto and execute on the same Kafka Streams engine -- they differ only in how you define the pipeline.
📄️ Virtual Filesystem
TypeStream provides a UNIX-like virtual filesystem that maps Kafka infrastructure to a familiar directory structure. Topics become paths, schemas become types, and standard commands like ls, cd, and cat work as expected.
📄️ Schema Propagation
Schema propagation is TypeStream's compile-time pass that flows type information through the pipeline graph. It enables schema validation before execution, field-name display in the GUI, and type-safe pipeline configurations.
📄️ CDC and Debezium
Change Data Capture (CDC) is a technique for streaming database changes (inserts, updates, deletes) into Kafka topics in real time. TypeStream uses Debezium to capture changes from PostgreSQL (and other databases) via Kafka Connect.
📄️ Components
An overview of the components that make up TypeStream.