I built AgentSmith-HUB, an open-source security data pipeline platform with a built-in real-time threat detection engine.

What it is AgentSmith-HUB helps security teams process, enrich, and analyze massive amounts of security logs and alerts. It features:

A flexible XML-like rules engine (regex, thresholds, custom logic, dynamic fields)

Built-in plugin system with custom plugin support for enrichment, threat intel queries, and automated actions

Cluster/distributed mode for scaling to large data volumes

A full-featured web UI for building and testing detection workflows visually

Easy integration with Kafka, Elasticsearch, and major cloud logging services

Performance In our tests (8 complex rules), it handled ~40k messages/sec with sub-ms latency on a 2‑CPU, 4‑GB server.

Who is it for? Security engineers building custom detection workflows

Teams looking for a flexible, lightweight alternative to heavy SIEMs

Anyone needing a scalable, real-time log processing and threat detection pipeline

Links GitHub: https://github.com/EBWi11/AgentSmith-HUB

I’d love to hear your thoughts, especially on use cases or integrations you’d like to see supported!