Aspirationally, Ember aims to be for Networks of Networks (NONs) and Compound AI Systems what PyTorch and XLA are for Neural Networks (NNs). It is a compositional framework that offers both eager execution and graph-based optimization. Ember empowers users to build complex NONs while supporting automatic parallelization and system-level optimizations.

The long-term vision for Ember is to enable the development of compound AI systems composed of millions—or even billions—of inference calls. Surprisingly simple patterns, such as best-of-N graphs, verifier-prover architectures, and ensemble models with voting-based aggregation, perform remarkably well across a wide range of scenarios.

Design Philosophy

Ember is built on these foundational principles:

    Composability First: The ability to combine, chain, and nest components (e.g. Operator components) is central to Ember's design
    Type Safety: Comprehensive type annotations ensure robustness and IDE support
    Testability: Components are designed with SOLID principles in mind, for easy isolation and testing
    Scalability: support for Parallel execution is built-in at the framework's core. This is more Tensorflow/JAX, than classic Torch spiritually
    Extensibility: Registry-based design makes it simple to add new components
    Skeurmophism: APIs follow familiar patterns from PyTorch/JAX, to somewhat control the learning curve
    Simple-over-easy: Minimal "magic" and a focus on explicitness