All Tools
N
DataFreeOpen Source
NSQ
Decentralized realtime messaging for data pipelines
MIT
ABOUT
Centralized brokers become a bottleneck and a single point of failure when you fan out jobs, logs, and inference events. NSQ is a decentralized messaging platform: producers talk to nsqd, consumers discover topics via nsqlookupd, and the topology scales by adding nodes instead of a shared cluster quorum.
INSTALL
brew install nsqINTEGRATION GUIDE
1. Fan out training-job events and dataset-ready notifications across workers
2. Stream inference telemetry and feature updates without a single broker bottleneck
3. Buffer scrape and ETL work behind nsqd so producers stay decoupled from consumers
4. Operate a lightweight pub-sub backbone for ML services that do not need Kafka
TAGS
messagingpub-substreaminggorealtimedata-pipelineopen-source