Dimitri ML Project

This repository contains the code and data associated with the Dimitri ML project. The primary function of this codebase is to extract features from vibration data and train/evaluate Machine Learning models to detect anomalies and identify different fault modes.

Directory Structure

• DATOS/ - Original raw data source files (if applicable)
• RAW_DATABASE/ - Structured, original experimental data files (e.g., parquet files separated by fault mode)
• PROJECT_RESULTS/
  • CODE/ - All Python scripts for data processing, feature extraction, model training, and visualization
  • DATA/ - Generated intermediate features, signatures, and model test results (CSV)
  • MODELS/ - Pickled ML models (e.g., Isolation Forest, PCA, KMeans)
  • PLOTS/ - Generated visualizations summarizing ML model performance, anomaly severity, and exploration plots

Getting Started

1. Requirements

Ensure you have Python 3.8+ installed. You can set up the environment and install dependencies using the provided requirements.txt file.

# Optional: Create a virtual environment
python -m venv venv

# Activate the virtual environment (Windows)
.\venv\Scripts\activate

# Activate the virtual environment (macOS/Linux)
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

2. Execution Order

The scripts in PROJECT_RESULTS/CODE/ have been numerically prefixed to clarify their execution order if you want to reproduce the results from scratch. They automatically resolve data and output paths relative to the project root.

1. Data Preparation

• python PROJECT_RESULTS\CODE\01_raw_data_organizer.py
  • Organizes raw data into the RAW_DATABASE directory.

2. Feature Extraction

• python PROJECT_RESULTS\CODE\02a_feature_extractor_spectral.py (Standard Spectral)
• python PROJECT_RESULTS\CODE\02b_feature_extractor_advanced.py (Advanced Physics-Informed)
• python PROJECT_RESULTS\CODE\02c_profile_generator_signatures.py (Signatures)
  • Extracts features and profiles from the raw dataset.

3. Model Training and Evaluation

• python PROJECT_RESULTS\CODE\03a_model_trainer_spectral.py
• python PROJECT_RESULTS\CODE\03a_model_trainer_spectral_per_fm.py (Per-fault-mode, 20-25Hz)
• python PROJECT_RESULTS\CODE\03b_model_trainer_advanced.py
• python PROJECT_RESULTS\CODE\03b_model_trainer_advanced_per_fm.py (Per-fault-mode, 20-25Hz)
• python PROJECT_RESULTS\CODE\03c_model_trainer_signatures.py
  • Trains various ML models (PCA, Isolation Forest, K-Means, etc.) and evaluates their performance.

4. Visualization and Analysis

• python PROJECT_RESULTS\CODE\04a_visualizer_spectral.py
• python PROJECT_RESULTS\CODE\04b_visualizer_advanced.py
• python PROJECT_RESULTS\CODE\04c_visualizer_signatures.py
• python PROJECT_RESULTS\CODE\04d_feature_importance.py
• python PROJECT_RESULTS\CODE\04e_anomaly_severity.py
• python PROJECT_RESULTS\CODE\04f_exploration_plots.py
• python PROJECT_RESULTS\CODE\04g_visualizer_per_fm.py (Heatmaps for per-fault-mode)
  • Generates comparison graphs, feature importance plots, and exploration plots.

5. Inference

• python PROJECT_RESULTS\CODE\05_run_inference.py
  • Script to load saved ML .pkl models from PROJECT_RESULTS/MODELS and run predictions on new or synthetic data. Example:

  python PROJECT_RESULTS\CODE\05_run_inference.py --model isolation_forest_adv.pkl

Additional Utilities

• PROJECT_RESULTS\CODE\utils_check_spikes.py - Script to detect and inspect specific data spikes.

Note on Paths

All Python scripts now use dynamic path resolution. Assuming you run them from the repository root or any subfolder, they will properly detect the project structure and build paths automatically pointing to PROJECT_RESULTS/ or RAW_DATABASE/.