I built AgentSmith-HUB, an open-source security data pipeline platform with a built-in real-time threat detection engine.
What it is AgentSmith-HUB helps security teams process, enrich, and analyze massive amounts of security logs and alerts. It features:
A flexible XML-like rules engine (regex, thresholds, custom logic, dynamic fields)
Built-in plugin system with custom plugin support for enrichment, threat intel queries, and automated actions
Cluster/distributed mode for scaling to large data volumes
A full-featured web UI for building and testing detection workflows visually
Easy integration with Kafka, Elasticsearch, and major cloud logging services
Performance In our tests (8 complex rules), it handled ~40k messages/sec with sub-ms latency on a 2‑CPU, 4‑GB server.
Who is it for? Security engineers building custom detection workflows
Teams looking for a flexible, lightweight alternative to heavy SIEMs
Anyone needing a scalable, real-time log processing and threat detection pipeline
Links GitHub: https://github.com/EBWi11/AgentSmith-HUB
I’d love to hear your thoughts, especially on use cases or integrations you’d like to see supported!