An end-to-end Production-Ready MLOps Pipeline built using Python, MongoDB, AWS, Docker, GitHub Actions, CI/CD, and Machine Learning that automates the complete lifecycle from data ingestion to cloud deployment.
Traditional Machine Learning projects usually end after training a model.
This project goes several steps further by implementing a complete production-grade MLOps architecture capable of:
- Automated Data Ingestion
- Data Validation
- Feature Engineering
- Model Training
- Model Evaluation
- Model Registry
- Model Versioning
- Cloud Storage
- Prediction Pipeline
- Docker Containerization
- Continuous Integration
- Continuous Deployment
- AWS Cloud Deployment
The entire workflow is designed following modular software engineering practices, making the project scalable, maintainable, and production-ready.
MongoDB Atlas
โ
โผ
Data Ingestion
โ
โผ
Data Validation
โ
โผ
Data Transformation
โ
โผ
Model Training
โ
โผ
Model Evaluation
โ
โผ
Model Registry (AWS S3)
โ
โผ
Prediction Pipeline
โ
โผ
Flask Web App
โ
โผ
Docker Container
โ
โผ
GitHub Actions
โ
โผ
AWS EC2 Deployment
- Fetches dataset directly from MongoDB Atlas
- Converts MongoDB documents into Pandas DataFrame
- Creates train-test split automatically
- Stores artifacts for downstream pipeline
- Schema Validation
- Missing Value Validation
- Data Drift Detection
- Column Validation
- Numerical Feature Validation
- Categorical Feature Validation
- Feature Engineering
- Missing Value Imputation
- Feature Scaling
- Encoding
- Pipeline Serialization
- Transformation Object Saving
Multiple algorithms can be trained and compared.
Supports:
- Random Forest
- Gradient Boosting
- XGBoost
- CatBoost
- Extra Trees
- Decision Tree
- AdaBoost
- Linear Models
Automatic Best Model Selection based on evaluation metrics.
Instead of replacing models blindly,
the project:
- Downloads previous production model
- Compares new model performance
- Uses Threshold Based Validation
- Registers model only if performance improves
This mimics a real production ML workflow.
Models are automatically:
- Uploaded to AWS S3
- Version Controlled
- Retrieved during prediction
- Managed without manual intervention
Production prediction pipeline supports:
- Single Prediction
- Batch Prediction
- Real-time Inference
Interactive web interface allowing users to
- Upload input
- Predict Insurance Claim
- Trigger Training Pipeline
- View Prediction Results
Entire application is containerized using Docker ensuring
- Environment consistency
- Easy deployment
- Platform independence
Fully automated deployment using
GitHub Actions
Workflow:
Code Push
โ
โผ
GitHub Actions
โ
โผ
Build Docker Image
โ
โผ
Push Image to AWS ECR
โ
โผ
Deploy on EC2
โ
โผ
Updated Application
No manual deployment required.
- Python
- HTML
- CSS
- Scikit-Learn
- Pandas
- NumPy
- MongoDB Atlas
- AWS EC2
- AWS S3
- AWS ECR
- AWS IAM
- Docker
- GitHub Actions
- CI/CD
- Model Registry
- Environment Variables
- Logging
- Exception Handling
- Flask
- Git
- GitHub
Vehicle-Insurance-Project/
โ
โโโ artifacts/
โโโ notebooks/
โโโ src/
โ
โโโ components/
โ โโโ Data Ingestion
โ โโโ Data Validation
โ โโโ Data Transformation
โ โโโ Model Trainer
โ โโโ Model Evaluation
โ โโโ Model Pusher
โ
โโโ configuration/
โโโ constants/
โโโ entity/
โโโ pipeline/
โโโ aws_storage/
โโโ utils/
โ
โโโ static/
โโโ templates/
โ
โโโ app.py
โโโ demo.py
โโโ requirements.txt
โโโ Dockerfile
โโโ setup.py
โโโ pyproject.toml
โโโ README.md
MongoDB
โ
โผ
Data Ingestion
โ
โผ
Data Validation
โ
โผ
Data Transformation
โ
โผ
Model Trainer
โ
โผ
Model Evaluation
โ
โผ
AWS S3 Model Registry
โ
โผ
Prediction Pipeline
Required environment variables
MONGODB_URL
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION
ECR_REPO- IAM
- EC2
- S3
- ECR
Developer
โ
GitHub Repository
โ
GitHub Actions
โ
Docker Build
โ
Push Docker Image
โ
AWS ECR
โ
EC2 Pulls Latest Image
โ
Application Deployment
git clone <repo_url>conda create -n vehicle python=3.10
conda activate vehiclepip install -r requirements.txtpython demo.pypython app.pyโ Modular Pipeline Architecture
โ Production-ready Folder Structure
โ Config Driven Development
โ Artifact Based Pipeline
โ Object-Oriented Design
โ Cloud Model Registry
โ Automatic Model Comparison
โ Automated Deployment
โ Logging System
โ Custom Exception Handling
โ Dockerized Infrastructure
โ Continuous Integration
โ Continuous Deployment
โ AWS Cloud Deployment
โ Environment Variable Management
- Clean Code Principles
- Modular Architecture
- Separation of Concerns
- Dependency Management
- Configuration Management
- Pipeline Design Pattern
- Artifact Management
- Model Versioning
- Cloud Integration
- CI/CD Automation
- Containerization
- Production Deployment
This project demonstrates hands-on experience with:
| Category | Skills Demonstrated |
|---|---|
| Machine Learning | End-to-End ML Pipeline |
| MLOps | Production Workflow |
| Backend | Flask |
| Cloud | AWS (EC2, S3, ECR, IAM) |
| Database | MongoDB Atlas |
| DevOps | Docker, GitHub Actions |
| CI/CD | Automated Deployment |
| Software Engineering | OOP, Modular Design, Logging, Exception Handling |
Ayush
Aspiring Machine Learning Engineer | MLOps Engineer
Focused on building production-grade AI systems using Machine Learning, Cloud Computing, and DevOps practices.