The goal of this project is to research and test a python library for working in the field of machine learning - scalecast.
We took 4 datasets and tested the existing models and analysis capabilities from this library on them.
The backend is written on fastapi (the logic of this application does not require the use of more complex backend systems). Part of the code (data analysis) works in real time, using caching (redis). The rest (model training and prediction) are ready-made results saved as a picture from a jupyter notebook.
The main data is stored in the database (container with postgres). The init container is responsible for creating tables and filling them (so as not to fill in the data manually after a new build).
It was also planned to make a connection with mlflow (mlflow, minio, postgres containers) for logging models. At the moment, you can follow the link to mlflow from the main application, but logging is not performed (models are not trained in real time, which would be very expensive in terms of time to view the results).
The frontend is written using the streamlit python library. It is simple and quite suitable for visualizing studies of this kind. It is also very convenient that this library allows you to create multi-page applications with navigation in the sidebar. Data from the backend is obtained using the requests library.
I also made self-signed certificates to proxy the application using nginx on https. As a result, you can enter the main application using the beautiful address https://konsin1988.dataresearch.ml/streamlit/ or the main address, and all other requests are redirected here. Except mlflow and minio, they have a dedicated proxy_pass.
