This is a small minimum viable product (MVP) to implement a security information and event management (SIEM) dashboard and to explore how other technologies can be combined to improve the SIEM output such as a large language model (LLM). My goal was to explore making it easier to parse logs that is generated via synthetic data using Cisco-Talos EvidenceForge to replicate real data and see how an LLM can help in flagging tasks
Quickwit was also used primarly since it is built with rust and python bindings and due to its performance.
Dash was used to make the dashboard which showcases all the information
For the LLM, gemma4:e2b was used from ollama with langchain.
flowchart LR
EF[Cisco Talos EvidenceForge<br/>Synthetic Logs + Ground Truth]
QW[(Quickwit<br/>Search Index)]
AI[LangChain Agent + Ollama<br/>Triage + Summarization]
IDX[(triage_alerts Index<br/>Enriched Alerts)]
UI[Dash + Console UI]
EF --> QW
QW --> AI
AI --> IDX
QW --> UI
IDX --> UI
classDef source fill:#dbeafe,stroke:#3b82f6,color:#1e3a5f;
classDef storage fill:#d1fae5,stroke:#10b981,color:#064e3b;
classDef ai fill:#fef3c7,stroke:#f59e0b,color:#78350f;
classDef output fill:#f0fdf4,stroke:#22c55e,color:#14532d;
class EF source;
class QW storage;
class IDX storage;
class AI ai;
class UI output;
Heres how the dashboard looks like:
A look at the Quickwit user interface which you can also access via the dashboard
EvidenceForge was used as step 0 to generate the synthetic data with the file under scenarios. There is an output folder already in this repo with some sample data in the folder called eforge-output.
To make your own synthetic data you can go check out EvidenceForge or do the following steps to use the local scenario:
Step 1:
# git clone the repository of evidence forge
git clone https://github.com/Cisco-Talos/EvidenceForge.git
cd EvidenceForge
uv sync
Step 2:
# run your scenario against it
uv run eforge generate scenarios/scenario.yaml --output eforge-output --force
Once thats done it will create a folder with synthetic data output based on the scenario.
Docker was used mainly since quickwit currently does not have support for windows when this project was made.
Please do the following steps to run the docker container:
Step 1:
docker compose up --build
Step 2 (load the data in a streaming manner):
docker compose --profile eforge run --rm eforge-ingest
Step 3: Navigate to the url https://localhost:8050 to see the dash website!
- Grafana can be added instead of dash itself
- mimic more scenarios as listed in evidenceforge itself
- reduce latency time between llm calls
- more visuals to showcase whats going on
MIT, feel free to play around with it!