OrcaAI

OrcaAI is a production-oriented AI platform scaffold for model training, inference, retrieval-augmented generation, multi-tenant API delivery, and internal system intelligence. This repository is structured to support a practical path from experimentation to deployment while keeping model, service, and infrastructure concerns separated.

What is included

• Training pipeline modules for LoRA and full fine-tuning
• Inference routing for vLLM and llama.cpp style backends
• FastAPI application entrypoints for chat, analysis, and code generation
• Internal intelligence orchestration for cloud operations, security, GITORC, drones, surveillance, and storage
• Retrieval components for document search and prompt augmentation
• Service modules for HPC, cloud, research, and operations workflows
• Config-driven layout for model, inference, training, and tenant settings

Project layout

orcaai/
├── config/              Runtime and deployment configuration
├── data/                Raw, processed, and instruction datasets
├── docs/                Architecture, API, training, and deployment docs
├── models/              Base models, adapters, and checkpoints
├── src/                 Python package source and CLI scripts
├── storage/             Vector store and document storage for RAG
└── tests/               API, training, and inference tests

Quick start

1. Create a virtual environment.
2. Install dependencies from requirements.txt.
3. Copy .env.example to .env and adjust settings.
4. Review the YAML files in config/.
5. Launch the API with uvicorn src.orcaai.api.main:app --reload from the project root, or run the scripts in src/scripts/.

For the full local training and transformers-based inference toolchain, also install requirements-ml.txt.

Training and inference integrations

The scaffold now uses real Hugging Face integrations for local model loading, tokenization, text generation, and fine-tuning:

• transformers powers model loading, tokenization, and text generation fallback.
• peft powers LoRA adapter attachment and saving.
• datasets provides in-memory training datasets from JSONL instruction data.
• vllm and llama-cpp-python are supported as optional inference extras.

The default config points to sshleifer/tiny-gpt2 so the project can be exercised with a small model before moving to larger checkpoints.

Containers

Use Docker for API packaging and local deployment:

docker compose up --build

The container image installs the base runtime dependencies and starts the FastAPI application on port 8000. To build an image with the optional ML stack, set INSTALL_ML=true in docker-compose.yml or build with --build-arg INSTALL_ML=true.

Development flow

• Put raw assets in data/raw/.
• Build normalized datasets with src/scripts/prepare_data.py.
• Train adapters with src/scripts/train_lora.py.
• Run local inference with src/scripts/run_inference.py.
• Deploy the API using src/scripts/deploy_api.py.

Internal intelligence engine

OrcaAI can now act as an internal system intelligence layer for OrcaStack. The current implementation includes:

• cross-domain signal detection for cloud, security, GITORC, drone, and storage telemetry
• automation action planning for remediation, scaling, incident response, and drone operations
• predictive risk generation for outages, breaches, deployment instability, mission failure, and data integrity events
• API endpoints for internal evaluation and capability discovery

Use POST /system/intelligence to submit operational snapshots and receive a structured response with signals, recommended actions, and predictions.

Live operational collection and workflow dispatch are also supported:

• config/connectors.yaml defines OpenStack, Ceph, Kubernetes, Prometheus, Loki, GITORC, and drone control connectors.
• config/system_models.yaml defines trainable anomaly and predictive model profiles.
• src/scripts/train_system_models.py can build anomaly baselines from historical telemetry.
• POST /system/intelligence accepts collect_live, execute_actions, and dry_run flags.

Operations UI

OrcaAI now exposes a control-room style operations dashboard at /ops.

It provides:

• auto-scanned platform availability for OpenStack, Slurm, Warewulf, Ceph, OpenHPC, database services, and automation workflows
• a live operational summary built from discovered platforms and collected telemetry
• active signals, action recommendations, predictive risks, and automation execution logs

Supporting APIs:

• GET /system/intelligence/platforms
• GET /system/intelligence/dashboard-state
• GET /ops

Documentation

• Architecture overview: docs/architecture/overview.md
• API guide: docs/api/overview.md
• Training workflow: docs/training/overview.md
• Deployment guide: docs/deployment/overview.md

Status

This scaffold is intentionally lightweight. It provides clear module boundaries, baseline implementations, sample configuration, and tests so the repository can be extended into a full production system.