Minder¶

🧠 Overview¶

Minder is an intelligent assistant designed to interpret, analyze, and visualize my personal knowledge base using machine learning and MLOps principles. It connects with my notes from MindShelf to transform raw information into insights and interactive visualizations shown on DashLab.

🛠️ Technologies Used¶

Python
Pandas
FastAPI
Docker
MLFlow
AWS

🚀 Implementation Steps¶

Develop a simple interface to interact with the database using S3 buckets, enabling add, remove, and view operations on diverse data types stored in various database folders. This interface is designed with a broader scope in mind, offering a flexible data exchange solution that can be reused across multiple projects beyond this one.
Create a graph visualization of the data to facilitate exploration and better understanding of relationships within the dataset, laying the groundwork for graph-based models or advanced analytics in future iterations.
Prepare a data processing script to extract and transform MindShelf data, storing it in a structured database, with automation via GitLab CI/CD pipelines.
Build a dedicated API that accesses the database, and train ML models (e.g., RAG, GPT), with MLflow tracking for experiment management and visualization on the dashboard.
Create a separate deployment API for serving the trained ML models, which the dashboard can query to answer knowledge-based questions, provide relevant file links, and interact with users in real time.
Evaluate model performance on current data, monitor answer accuracy, collect feedback, and use insights to iteratively improve the model.
Continuously refine data processing, enhance tracking and monitoring tools, and add new features to boost overall efficiency and model performance.

📌 Challenges Encountered¶

Project currently ongoing — still exploring data preprocessing, model tuning, and deployment strategies.

✨ What I Learned¶

Practical MLOps pipeline setup and orchestration
Techniques for interpreting and visualizing knowledge graph data