I want to build the whole product (backend,frontend,APIs, models) with the standard…
- User
I want to build the whole product (backend,frontend,APIs, models) with the standard structure: Overview Emailer is an email management application that leverages Large Language Models (LLMs) to provide intelligent email classification, analysis, and reporting. Specifically Llama 3.1 served via the Ollama server on localhost, Emailer ensures data security while providing robust features for users to manage their emails efficiently. Scope Emailer is intended for individual users and organizations seeking an intelligent solution to manage large volumes of emails. The application provides features such as real-time email processing, customizable classification, detailed reporting, and secure data handling. Features 1. User Account Management Create Account: Users can sign up by selecting their email provider, entering a username, email, and password. 2. Email Integration Email Fetching: Support for multiple email providers (Gmail, Outlook, etc.) using IMAP and POP3 protocols. Real-Time Processing: Immediate fetching and processing of incoming emails. 3. Email Pre-processing Parsing: Extraction of email headers, body content, and attachments for thorough analysis. 4. User Customization Custom Classification Categories: Users can define their own categories for email classification. User-Defined Filters: Filters based on sender, date, subject, and topic. Manager Mail Addition: Ability to add manager email addresses for report distributions. 5. Intelligent Classification & Analysis LLM Integration: Utilization of Llama 3.1 for email classification. 6. Data Storage Database Management: Use of SQLite to store user profiles, emails, and classification results securely. 7. User Interface Email Display: Emails displayed with classification tags for easy identification. Visualization: Use of tables and charts to represent classification results. 8. Reporting & Notifications Scheduled Reports: Automatic generation and sending of email classification summaries to manager emails. Urgent Email Reminders: Notifications for emails marked as urgent, negative, or important based on user-defined thresholds. Technology Stack § Programming Language: Python § Large Language Model (LLM): Llama 3.1 § Model Server: Ollama § Database: SQLite § Email Protocols: IMAP, POP3 (based on email provider support) § Frontend Framework: PyQt6 § Backend Framework: FastAPI System Architected
Data Management § User Profiles: Store user information securely in the SQLite database. § Email Data: Store fetched emails, including headers, body, and attachments. § Classification Results: Store classification tags and analysis data linked to each email. § Data Backup: Implement regular backups of the SQLite database to prevent data loss. User Interface Requirements Dashboard: Overview of email statistics, recent classifications, and notifications. Email Viewer: Interface to view individual emails with classification tags. Filters and Search: Tools to filter and search emails based on user-defined criteria. Customization Settings: Sections for users to define classification categories and filters. Reporting Section: Access to generated reports and notification settings. Account Management: Interface for users to manage their account details and preferences. Integration and APIs Email Providers Integration: Utilize IMAP and POP3 protocols to fetch emails from supported providers. LLM Integration: Interface with the Ollama server to access and utilize Llama 3.1 for classification tasks. Reporting Services: Integrate with email services to send scheduled reports and urgent notifications. API Gateway (FastAPI): Centralizes API endpoints, facilitating communication between the frontend and backend services. Deployment Local Deployment: The application is designed to run locally on user machines, ensuring data remains secure. Environment Requirements: § Python environment with necessary dependencies. § Ollama server setup for Llama 3.1. § SQLite database setup. § FastAPI server setup. Installation Process: § Package the application with an installer that sets up the Python environment, dependencies, Ollama server, and FastAPI server. § Provide configuration guides for connecting to email providers. Week 1: Setup & User Management Objectives: · Establish the development environment. · Implement user account management. Key Tasks: · Project Initialization: o Configure the development environment (Python, FastAPI, PyQt6, Ollama). · System Architecture: o Finalize a high-level architecture diagram. o Design the initial SQLite database schema for user profiles. · User Account Management: o Develop backend endpoints for user signup and login using FastAPI. o Create frontend UI components in PyQt6 for account creation and login. Deliverables: · Functional user signup and login features. · Basic system architecture documentation. Week 2: Email Integration & Pre-processing Objectives: · Enable email fetching from multiple providers. · Develop email preprocessing capabilities. Key Tasks: · Email Integration: o Implement backend services to connect with email providers (Gmail, Outlook) using IMAP and POP3. o Set up real-time email fetching mechanisms. · Email Pre-processing: o Develop modules to parse email headers, body, and attachments. o Normalize and store fetched emails in the SQLite database. · Frontend Enhancements: o Update the UI to allow users to connect their email accounts. o Display status indicators for email fetching and pre-processing. Deliverables: · Ability to fetch and pre-process emails from supported providers. · Updated frontend for email integration. · Stored emails in the database. Week 3: Customization & Intelligent Classification Objectives: · Implement user customization features. · Integrate LLM for email classification. Key Tasks: · User Customization: o Develop functionalities for users to define custom classification categories and filters (sender, date, subject, topic). o Enable addition of manager email addresses for report distributions. · LLM Integration: o Interface with Llama 3.1 via the Ollama server for email classification. o Develop backend logic to send pre-processed emails to the LLM and handle classification results. · Store classification tags in the SQLite database. · Frontend Enhancements: o Update the email viewer to display classification tags. o Implement basic visualization (tables) for classification results. Deliverables: · User-defined classification categories and filters. · Integrated LLM-based classification system. · Enhanced email viewer with classification tags. Week 4: Reporting, Notifications & Deployment Objectives: · Implement reporting and notification features. · Prepare for deployment and finalize the user interface. Key Tasks: · Reporting & Notifications: o Develop backend services to generate and send scheduled reports summarizing email classifications. o Implement notifications for urgent, negative, or important emails based on user-defined thresholds. · Final UI Enhancements: o Polish the dashboard with email statistics and recent classifications. o Implement search and advanced filtering tools. o Ensure all UI components are user-friendly and intuitive. · Deployment Preparation: o Create an installer package that sets up the Python environment, dependencies, Ollama server, FastAPI server, and initializes the SQLite database. o Develop configuration guides for users to connect their email providers and set up the application. · Final Testing & Launch: o Conduct end-to-end testing of all features. o Perform security checks to ensure data privacy and secure communications. o Launch the application to initial users. Deliverables: · Functional reporting and notification systems. · Polished and fully functional user interface. · Ready-to-deploy installer package with user guides. · Successfully deployed Emailer application.
- Assistant
Building a comprehensive application like Emailer requires careful planning and structured development. Below is a detailed guide to help you execute your project based on the outlined structure and weekly objectives. This guide includes project setup, architecture design, implementation steps with code snippets, and best practices to ensure a smooth development process.
Project Overview
Emailer is an intelligent email management application leveraging Large Language Models (LLMs) to provide features like email classification, analysis, and reporting. It ensures data security and efficient email management for individual users and organizations.
Project Structure
Organizing your project with a clear structure is crucial for maintainability and scalability. Below is a recommended project structure:
emailer/ ├── backend/ │ ├── app/ │ │ ├── api/ │ │ │ ├── endpoints/ │ │ │ └── __init__.py │ │ ├── core/ │ │ ├── models/ │ │ ├── schemas/ │ │ ├── services/ │ │ ├── database.py │ │ └── main.py │ ├── tests/ │ ├── requirements.txt │ └── README.md ├── frontend/ │ ├── src/ │ │ ├── components/ │ │ ├── views/ │ │ ├── main.py │ │ └── utils.py │ ├── resources/ │ ├── tests/ │ ├── requirements.txt │ └── README.md ├── models/ ├── docs/ ├── scripts/ ├── .gitignore └── README.md
Weekly Development Plan
Week 1: Setup & User Management
Objectives:
- Establish the development environment.
- Implement user account management.
Key Tasks & Implementation Steps:
-
Project Initialization:
-
Backend Setup:
- Initialize FastAPI Project:
mkdir emailer cd emailer mkdir backend cd backend python -m venv venv source venv/bin/activate pip install fastapi uvicorn sqlalchemy pydantic passlib bcrypt pip freeze > requirements.txt - Create
main.py:# backend/app/main.py from fastapi import FastAPI app = FastAPI() @app.get("/") def read_root(): return {"message": "Welcome to Emailer Backend"} - Run the Backend Server:
uvicorn app.main:app --reload
- Initialize FastAPI Project:
-
Frontend Setup:
- Initialize PyQt6 Project:
cd .. mkdir frontend cd frontend python -m venv venv source venv/bin/activate pip install pyqt6 requests pip freeze > requirements.txt - Create Basic PyQt6 Application:
# frontend/src/main.py import sys from PyQt6.QtWidgets import QApplication, QLabel, QMainWindow class MainWindow(QMainWindow): def __init__(self): super().__init__() self.setWindowTitle("Emailer Frontend") label = QLabel("Welcome to Emailer") self.setCentralWidget(label) app = QApplication(sys.argv) window = MainWindow() window.show() sys.exit(app.exec())
- Initialize PyQt6 Project:
-
Version Control:
- Initialize Git and create
.gitignore:cd .. git init echo "venv/" >> .gitignore echo "__pycache__/" >> .gitignore echo "*.pyc" >> .gitignore git add . git commit -m "Initial commit with backend and frontend setup"
- Initialize Git and create
-
-
System Architecture:
-
High-Level Architecture Diagram:
- Use tools like Lucidchart, Draw.io, or Microsoft Visio to create the architecture diagram. Include components such as Frontend (PyQt6), Backend (FastAPI), Database (SQLite), LLM Integration (Ollama with Llama 3.1), and Email Providers (Gmail, Outlook via IMAP/POP3).
-
SQLite Database Schema Design:
- User Model:
# backend/app/models/user.py from sqlalchemy import Column, Integer, String from sqlalchemy.ext.declarative import declarative_base Base = declarative_base() class User(Base): __tablename__ = "users" id = Column(Integer, primary_key=True, index=True) username = Column(String, unique=True, index=True, nullable=False) email = Column(String, unique=True, index=True, nullable=False) hashed_password = Column(String, nullable=False) - Database Setup:
# backend/app/database.py from sqlalchemy import create_engine from sqlalchemy.orm import sessionmaker from .models.user import Base SQLALCHEMY_DATABASE_URL = "sqlite:///./emailer.db" engine = create_engine( SQLALCHEMY_DATABASE_URL, connect_args={"check_same_thread": False} ) SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine) def init_db(): Base.metadata.create_all(bind=engine) - Initialize the Database in
main.py:# backend/app/main.py from fastapi import FastAPI from .database import init_db app = FastAPI() @app.on_event("startup") def startup_event(): init_db() @app.get("/") def read_root(): return {"message": "Welcome to Emailer Backend"}
- User Model:
-
-
User Account Management:
-
Security:
- Password Hashing:
# backend/app/utils/security.py from passlib.context import CryptContext pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto") def hash_password(password: str) -> str: return pwd_context.hash(password) def verify_password(plain_password: str, hashed_password: str) -> bool: return pwd_context.verify(plain_password, hashed_password)
- Password Hashing:
-
Schemas:
# backend/app/schemas/user.py from pydantic import BaseModel, EmailStr class UserCreate(BaseModel): username: str email: EmailStr password: str class UserLogin(BaseModel): email: EmailStr password: str class UserResponse(BaseModel): id: int username: str email: EmailStr class Config: orm_mode = True -
CRUD Operations:
# backend/app/services/user_service.py from sqlalchemy.orm import Session from ..models.user import User from ..schemas.user import UserCreate from ..utils.security import hash_password def create_user(db: Session, user: UserCreate): hashed_pw = hash_password(user.password) db_user = User(username=user.username, email=user.email, hashed_password=hashed_pw) db.add(db_user) db.commit() db.refresh(db_user) return db_user def get_user_by_email(db: Session, email: str): return db.query(User).filter(User.email == email).first() -
API Endpoints:
# backend/app/api/endpoints/auth.py from fastapi import APIRouter, Depends, HTTPException, status from sqlalchemy.orm import Session from .. import schemas, services, database router = APIRouter( prefix="/auth", tags=["auth"] ) @router.post("/signup", response_model=schemas.UserResponse) def signup(user: schemas.UserCreate, db: Session = Depends(database.get_db)): existing_user = services.get_user_by_email(db, user.email) if existing_user: raise HTTPException(status_code=400, detail="Email already registered") new_user = services.create_user(db, user) return new_user @router.post("/login") def login(user: schemas.UserLogin, db: Session = Depends(database.get_db)): db_user = services.get_user_by_email(db, user.email) if not db_user: raise HTTPException(status_code=400, detail="Invalid credentials") if not services.verify_password(user.password, db_user.hashed_password): raise HTTPException(status_code=400, detail="Invalid credentials") # For simplicity, returning a success message. Implement JWT or session management as needed. return {"message": "Login successful"} -
Include Auth Endpoints in
main.py:# backend/app/main.py from fastapi import FastAPI from .database import init_db from .api.endpoints import auth app = FastAPI() @app.on_event("startup") def startup_event(): init_db() app.include_router(auth.router) -
Frontend User Account Management:
-
Using PyQt6, create UI components for signup and login forms.
-
Example Signup Form:
# frontend/src/components/signup.py from PyQt6.QtWidgets import QWidget, QVBoxLayout, QLabel, QLineEdit, QPushButton, QMessageBox import requests class SignupWidget(QWidget): def __init__(self): super().__init__() self.init_ui() def init_ui(self): layout = QVBoxLayout() self.username_input = QLineEdit(self) self.username_input.setPlaceholderText("Username") layout.addWidget(QLabel("Username:")) layout.addWidget(self.username_input) self.email_input = QLineEdit(self) self.email_input.setPlaceholderText("Email") layout.addWidget(QLabel("Email:")) layout.addWidget(self.email_input) self.password_input = QLineEdit(self) self.password_input.setPlaceholderText("Password") self.password_input.setEchoMode(QLineEdit.EchoMode.Password) layout.addWidget(QLabel("Password:")) layout.addWidget(self.password_input) self.signup_button = QPushButton("Sign Up", self) self.signup_button.clicked.connect(self.signup) layout.addWidget(self.signup_button) self.setLayout(layout) def signup(self): data = { "username": self.username_input.text(), "email": self.email_input.text(), "password": self.password_input.text() } try: response = requests.post("http://localhost:8000/auth/signup", json=data) if response.status_code == 200: QMessageBox.information(self, "Success", "Account created successfully") else: QMessageBox.warning(self, "Error", response.json().get("detail", "Signup failed")) except Exception as e: QMessageBox.critical(self, "Error", str(e)) -
Example Login Form:
# frontend/src/components/login.py from PyQt6.QtWidgets import QWidget, QVBoxLayout, QLabel, QLineEdit, QPushButton, QMessageBox import requests class LoginWidget(QWidget): def __init__(self): super().__init__() self.init_ui() def init_ui(self): layout = QVBoxLayout() self.email_input = QLineEdit(self) self.email_input.setPlaceholderText("Email") layout.addWidget(QLabel("Email:")) layout.addWidget(self.email_input) self.password_input = QLineEdit(self) self.password_input.setPlaceholderText("Password") self.password_input.setEchoMode(QLineEdit.EchoMode.Password) layout.addWidget(QLabel("Password:")) layout.addWidget(self.password_input) self.login_button = QPushButton("Login", self) self.login_button.clicked.connect(self.login) layout.addWidget(self.login_button) self.setLayout(layout) def login(self): data = { "email": self.email_input.text(), "password": self.password_input.text() } try: response = requests.post("http://localhost:8000/auth/login", json=data) if response.status_code == 200: QMessageBox.information(self, "Success", "Login successful") # Proceed to the dashboard or main application else: QMessageBox.warning(self, "Error", response.json().get("detail", "Login failed")) except Exception as e: QMessageBox.critical(self, "Error", str(e))
-
-
Deliverables:
- Functional user signup and login features.
- Basic system architecture documentation.
-
Week 2: Email Integration & Pre-processing
Objectives:
- Enable email fetching from multiple providers.
- Develop email preprocessing capabilities.
Key Tasks & Implementation Steps:
-
Email Integration:
-
Backend Email Fetching Services:
-
Install Dependencies:
pip install imaplib2 poplib email -
Implement Email Fetching:
# backend/app/services/email_service.py import imaplib import poplib from email import message_from_bytes from datetime import datetime from sqlalchemy.orm import Session from . import database from . import models def fetch_emails_imap(user_credentials): # Example for Gmail try: mail = imaplib.IMAP4_SSL("imap.gmail.com") mail.login(user_credentials.email, user_credentials.password) mail.select("inbox") status, messages = mail.search(None, 'ALL') email_ids = messages[0].split() emails = [] for eid in email_ids: status, msg_data = mail.fetch(eid, "(RFC822)") for response_part in msg_data: if isinstance(response_part, tuple): msg = message_from_bytes(response_part[1]) emails.append(msg) mail.logout() return emails except Exception as e: print(f"IMAP fetch error: {e}") return [] def fetch_emails_pop3(user_credentials): # Example for Outlook try: mail = poplib.POP3_SSL("outlook.office365.com") mail.user(user_credentials.email) mail.pass_(user_credentials.password) num_messages = len(mail.list()[1]) emails = [] for i in range(num_messages): response, lines, octets = mail.retr(i+1) msg_content = b'\n'.join(lines) msg = message_from_bytes(msg_content) emails.append(msg) mail.quit() return emails except Exception as e: print(f"POP3 fetch error: {e}") return [] def store_emails(db: Session, user_id: int, emails: list): for msg in emails: email_record = models.Email( user_id=user_id, subject=msg['subject'], sender=msg['from'], date=datetime.strptime(msg['date'], '%a, %d %b %Y %H:%M:%S %z'), body=get_body(msg), # Handle attachments as needed ) db.add(email_record) db.commit() def get_body(msg): if msg.is_multipart(): for part in msg.walk(): content_type = part.get_content_type() content_disposition = str(part.get("Content-Disposition")) if content_type == "text/plain" and "attachment" not in content_disposition: return part.get_payload(decode=True).decode() else: return msg.get_payload(decode=True).decode() -
Email Model:
# backend/app/models/email.py from sqlalchemy import Column, Integer, String, ForeignKey, DateTime, Text from sqlalchemy.orm import relationship from .user import Base class Email(Base): __tablename__ = "emails" id = Column(Integer, primary_key=True, index=True) user_id = Column(Integer, ForeignKey("users.id")) subject = Column(String, nullable=False) sender = Column(String, nullable=False) date = Column(DateTime, nullable=False) body = Column(Text, nullable=False) # Add attachment handling as needed user = relationship("User", back_populates="emails") -
Update User Model for Relationship:
# backend/app/models/user.py from sqlalchemy.orm import relationship # Existing imports and User class ... class User(Base): __tablename__ = "users" id = Column(Integer, primary_key=True, index=True) username = Column(String, unique=True, index=True, nullable=False) email = Column(String, unique=True, index=True, nullable=False) hashed_password = Column(String, nullable=False) emails = relationship("Email", back_populates="user")
-
-
Real-Time Email Fetching Mechanism:
- Use background tasks or schedulers like Celery or APScheduler to periodically fetch emails.
- Example with Background Tasks:
# backend/app/main.py from fastapi import FastAPI from .database import init_db, get_db from .api.endpoints import auth from .services.email_service import fetch_emails_imap, fetch_emails_pop3, store_emails from sqlalchemy.orm import Session import asyncio app = FastAPI() @app.on_event("startup") def startup_event(): init_db() asyncio.create_task(email_fetcher()) app.include_router(auth.router) async def email_fetcher(): while True: db = next(get_db()) users = db.query(models.User).all() for user in users: # Depending on user email provider, choose IMAP or POP3 if user.email_provider == "gmail": emails = fetch_emails_imap(user) elif user.email_provider == "outlook": emails = fetch_emails_pop3(user) else: emails = [] store_emails(db, user.id, emails) await asyncio.sleep(300) # Fetch every 5 minutes
-
-
Email Pre-processing:
-
Parsing, Normalization, and Storage:
- Parsing is already handled in the
fetch_emails_*functions. Ensure that emails are stored properly in the SQLite database as shown above.
- Parsing is already handled in the
-
Frontend Enhancements:
-
Connecting Email Accounts:
- UI to Add Email Credentials:
- Provide input fields for email provider, email address, and password.
- API Endpoint to Save User Email Settings:
# backend/app/schemas/user.py class EmailSettings(BaseModel): email_provider: str email_password: str # Consider encrypting sensitive data # backend/app/api/endpoints/email_settings.py from fastapi import APIRouter, Depends, HTTPException from sqlalchemy.orm import Session from .. import schemas, services, database router = APIRouter( prefix="/email", tags=["email"] ) @router.post("/settings") def set_email_settings(settings: schemas.EmailSettings, current_user: schemas.UserResponse = Depends(get_current_user), db: Session = Depends(database.get_db)): user = services.get_user_by_id(db, current_user.id) user.email_provider = settings.email_provider user.email_password = settings.email_password # Hash or encrypt in production db.commit() return {"message": "Email settings updated"}
- UI to Add Email Credentials:
-
Display Status Indicators:
- Show indicators like "Fetching emails..." or "Last fetched at [timestamp]" in the frontend.
-
-
-
Stored Emails in the Database:
- Ensure that emails fetched are stored in the
emailstable with proper associations to users.
- Ensure that emails fetched are stored in the
-
Deliverables:
- Ability to fetch and pre-process emails from supported providers.
- Updated frontend for email integration.
- Stored emails in the database.
Week 3: Customization & Intelligent Classification
Objectives:
- Implement user customization features.
- Integrate LLM for email classification.
Key Tasks & Implementation Steps:
-
User Customization:
-
Custom Classification Categories and Filters:
-
Schema for Categories and Filters:
# backend/app/models/custom_category.py from sqlalchemy import Column, Integer, String, ForeignKey from sqlalchemy.orm import relationship from .user import Base class CustomCategory(Base): __tablename__ = "custom_categories" id = Column(Integer, primary_key=True, index=True) user_id = Column(Integer, ForeignKey("users.id")) category_name = Column(String, nullable=False) user = relationship("User", back_populates="custom_categories")# backend/app/models/filter.py from sqlalchemy import Column, Integer, String, ForeignKey from sqlalchemy.orm import relationship from .user import Base class Filter(Base): __tablename__ = "filters" id = Column(Integer, primary_key=True, index=True) user_id = Column(Integer, ForeignKey("users.id")) filter_type = Column(String, nullable=False) # sender, date, subject, topic filter_value = Column(String, nullable=False) user = relationship("User", back_populates="filters") -
Update User Model:
# backend/app/models/user.py class User(Base): __tablename__ = "users" id = Column(Integer, primary_key=True, index=True) username = Column(String, unique=True, index=True, nullable=False) email = Column(String, unique=True, index=True, nullable=False) hashed_password = Column(String, nullable=False) emails = relationship("Email", back_populates="user") custom_categories = relationship("CustomCategory", back_populates="user") filters = relationship("Filter", back_populates="user") -
CRUD Endpoints for Categories and Filters:
# backend/app/api/endpoints/customization.py from fastapi import APIRouter, Depends, HTTPException from sqlalchemy.orm import Session from .. import schemas, services, database router = APIRouter( prefix="/customization", tags=["customization"] ) @router.post("/categories", response_model=schemas.CustomCategoryResponse) def add_category(category: schemas.CustomCategoryCreate, current_user: schemas.UserResponse = Depends(get_current_user), db: Session = Depends(database.get_db)): new_category = services.create_custom_category(db, current_user.id, category.category_name) return new_category @router.post("/filters", response_model=schemas.FilterResponse) def add_filter(filter: schemas.FilterCreate, current_user: schemas.UserResponse = Depends(get_current_user), db: Session = Depends(database.get_db)): new_filter = services.create_filter(db, current_user.id, filter.filter_type, filter.filter_value) return new_filter
-
-
Manager Mail Addition:
-
Schema and Model:
# backend/app/models/manager_email.py from sqlalchemy import Column, Integer, String, ForeignKey from sqlalchemy.orm import relationship from .user import Base class ManagerEmail(Base): __tablename__ = "manager_emails" id = Column(Integer, primary_key=True, index=True) user_id = Column(Integer, ForeignKey("users.id")) email = Column(String, nullable=False) user = relationship("User", back_populates="manager_emails") -
Update User Model:
# backend/app/models/user.py class User(Base): # Existing fields... manager_emails = relationship("ManagerEmail", back_populates="user") -
API Endpoints:
# backend/app/api/endpoints/manager_email.py from fastapi import APIRouter, Depends, HTTPException from sqlalchemy.orm import Session from .. import schemas, services, database router = APIRouter( prefix="/manager", tags=["manager"] ) @router.post("/emails", response_model=schemas.ManagerEmailResponse) def add_manager_email(manager_email: schemas.ManagerEmailCreate, current_user: schemas.UserResponse = Depends(get_current_user), db: Session = Depends(database.get_db)): new_manager_email = services.add_manager_email(db, current_user.id, manager_email.email) return new_manager_email
-
-
-
LLM Integration for Email Classification:
-
Integrate Llama 3.1 via Ollama:
- Ensure that Ollama server is running locally and serving Llama 3.1.
-
Install Ollama Client (if applicable):
- Depending on Ollama's API, use HTTP requests to interact with the model.
-
Backend Logic for Classification:
# backend/app/services/classification_service.py import requests def classify_email(body: str) -> str: try: response = requests.post("http://localhost:11434/v1/classify", json={"text": body}) if response.status_code == 200: classification = response.json().get("classification") return classification else: return "Unclassified" except Exception as e: print(f"LLM Classification error: {e}") return "Unclassified" def classify_and_store(db: Session, email_id: int): email = db.query(models.Email).filter(models.Email.id == email_id).first() if email: classification = classify_email(email.body) email.classification = classification db.commit() -
Update Email Model:
# backend/app/models/email.py class Email(Base): __tablename__ = "emails" id = Column(Integer, primary_key=True, index=True) # Existing fields... classification = Column(String, nullable=True) -
Trigger Classification After Email Fetching:
- Modify the
store_emailsfunction to classify each email after storing.# backend/app/services/email_service.py from .classification_service import classify_and_store def store_emails(db: Session, user_id: int, emails: list): for msg in emails: email_record = models.Email( user_id=user_id, subject=msg['subject'], sender=msg['from'], date=datetime.strptime(msg['date'], '%a, %d %b %Y %H:%M:%S %z'), body=get_body(msg), ) db.add(email_record) db.commit() db.refresh(email_record) classify_and_store(db, email_record.id)
- Modify the
-
-
Frontend Enhancements:
-
Display Classification Tags:
- Modify the email viewer to show classification results.
# frontend/src/components/email_viewer.py from PyQt6.QtWidgets import QWidget, QVBoxLayout, QLabel class EmailViewer(QWidget): def __init__(self, email_data): super().__init__() self.init_ui(email_data) def init_ui(self, email_data): layout = QVBoxLayout() subject = QLabel(f"Subject: {email_data['subject']}") sender = QLabel(f"From: {email_data['sender']}") classification = QLabel(f"Classification: {email_data.get('classification', 'Unclassified')}") body = QLabel(email_data['body']) layout.addWidget(subject) layout.addWidget(sender) layout.addWidget(classification) layout.addWidget(body) self.setLayout(layout)
- Modify the email viewer to show classification results.
-
Implement Basic Visualization (Tables):
- Use QTableView to display classification results.
# frontend/src/components/classification_table.py from PyQt6.QtWidgets import QWidget, QVBoxLayout, QTableView from PyQt6.QtCore import QAbstractTableModel, Qt class ClassificationTableModel(QAbstractTableModel): def __init__(self, data): super().__init__() self._data = data self.headers = ["ID", "Subject", "Sender", "Classification"] def rowCount(self, parent=None): return len(self._data) def columnCount(self, parent=None): return len(self.headers) def data(self, index, role=Qt.ItemDataRole.DisplayRole): if role == Qt.ItemDataRole.DisplayRole: row = self._data[index.row()] return row[index.column()] return None def headerData(self, section, orientation, role=Qt.ItemDataRole.DisplayRole): if orientation == Qt.Orientation.Horizontal and role == Qt.ItemDataRole.DisplayRole: return self.headers[section] return None class ClassificationTable(QWidget): def __init__(self, classification_data): super().__init__() self.init_ui(classification_data) def init_ui(self, classification_data): layout = QVBoxLayout() table = QTableView() model = ClassificationTableModel(classification_data) table.setModel(model) layout.addWidget(table) self.setLayout(layout)
- Use QTableView to display classification results.
-
-
Deliverables:
- User-defined classification categories and filters.
- Integrated LLM-based classification system.
- Enhanced email viewer with classification tags.
Week 4: Reporting, Notifications & Deployment
Objectives:
- Implement reporting and notification features.
- Prepare for deployment and finalize the user interface.
Key Tasks & Implementation Steps:
-
Reporting & Notifications:
-
Scheduled Reports:
-
Generate Report:
# backend/app/services/report_service.py import csv from datetime import datetime from . import models from sqlalchemy.orm import Session def generate_report(db: Session, user_id: int): emails = db.query(models.Email).filter(models.Email.user_id == user_id).all() report_data = [ ["ID", "Subject", "Sender", "Classification", "Date"], ] for email in emails: report_data.append([ email.id, email.subject, email.sender, email.classification, email.date.strftime("%Y-%m-%d %H:%M:%S") ]) report_filename = f"report_{user_id}_{datetime.now().strftime('%Y%m%d%H%M%S')}.csv" with open(report_filename, 'w', newline='') as f: writer = csv.writer(f) writer.writerows(report_data) return report_filename -
Send Report via Email:
# backend/app/services/email_notification.py import smtplib from email.message import EmailMessage def send_email(to_addresses: list, subject: str, body: str, attachment_path: str = None): msg = EmailMessage() msg['Subject'] = subject msg['From'] = "your_email@example.com" msg['To'] = ", ".join(to_addresses) msg.set_content(body) if attachment_path: with open(attachment_path, 'rb') as f: file_data = f.read() file_name = f.name msg.add_attachment(file_data, maintype='application', subtype='octet-stream', filename=file_name) with smtplib.SMTP_SSL('smtp.example.com', 465) as smtp: smtp.login("your_email@example.com", "your_password") smtp.send_message(msg) -
Scheduled Task to Generate and Send Reports:
- Integrate with the email fetcher or use a separate scheduler.
# backend/app/main.py from .services.report_service import generate_report from .services.email_notification import send_email async def report_scheduler(): while True: db = next(get_db()) users = db.query(models.User).all() for user in users: report_file = generate_report(db, user.id) manager_emails = [manager.email for manager in user.manager_emails] send_email( to_addresses=manager_emails, subject="Emailer - Weekly Email Classification Report", body="Please find attached the latest email classification report.", attachment_path=report_file ) await asyncio.sleep(604800) # Weekly
-
-
Urgent Email Reminders:
-
Define Thresholds:
- Allow users to set thresholds for urgency, negativity, or importance.
-
Detect and Notify:
# backend/app/services/notification_service.py from .email_notification import send_email from sqlalchemy.orm import Session def notify_users(db: Session, user_id: int): user = db.query(models.User).filter(models.User.id == user_id).first() urgent_emails = db.query(models.Email).filter(models.Email.user_id == user_id, models.Email.classification == "urgent").all() if urgent_emails: manager_emails = [manager.email for manager in user.manager_emails] send_email( to_addresses=manager_emails, subject="Urgent Emails Notification", body=f"You have {len(urgent_emails)} urgent emails. Please check your Emailer dashboard." ) -
Integrate with Email Fetcher:
# backend/app/services/email_service.py from .notification_service import notify_users def store_emails(db: Session, user_id: int, emails: list): for msg in emails: email_record = models.Email( user_id=user_id, subject=msg['subject'], sender=msg['from'], date=datetime.strptime(msg['date'], '%a, %d %b %Y %H:%M:%S %z'), body=get_body(msg), ) db.add(email_record) db.commit() db.refresh(email_record) classify_and_store(db, email_record.id) if email_record.classification == "urgent": notify_users(db, user_id)
-
-
-
Final UI Enhancements:
-
Polish Dashboard:
- Display email statistics (e.g., total emails, classifications distribution).
- Show recent classifications and notifications.
-
Implement Search and Advanced Filtering Tools:
- Search Bar:
- Allow users to search emails by subject, sender, or content.
- Advanced Filters:
- Implement filtering based on user-defined criteria like date range, sender, subject keywords, etc.
- Search Bar:
-
Ensure User-Friendly UI:
- Use consistent styling, intuitive navigation, and responsive design elements.
-
-
Deployment Preparation:
-
Create Installer Package:
- Use tools like PyInstaller or cx_Freeze to bundle the application.
# For Frontend pyinstaller --onefile frontend/src/main.py --name EmailerFrontend # For Backend pyinstaller --onefile backend/app/main.py --name EmailerBackend - Automate Setup:
- Create scripts to set up Python environments, install dependencies, and configure the Ollama server.
- Use tools like PyInstaller or cx_Freeze to bundle the application.
-
Configuration Guides:
- Provide step-by-step instructions for users to connect their email providers, including IMAP/POP3 settings and any required permissions.
-
-
Final Testing & Launch:
- End-to-End Testing:
- Test all features from user signup to email classification and report generation.
- Security Checks:
- Ensure data is securely stored and transmitted.
- Implement HTTPS for backend APIs if necessary.
- Launch Application:
- Distribute the installer package to initial users.
- Gather feedback and iterate as needed.
- End-to-End Testing:
-
Deliverables:
- Functional reporting and notification systems.
- Polished and fully functional user interface.
- Ready-to-deploy installer package with user guides.
- Successfully deployed Emailer application.
Best Practices and Recommendations
-
Version Control and Documentation:
- Regularly commit your code with clear messages.
- Maintain comprehensive documentation in the
docs/directory.
-
Environment Management:
- Use virtual environments to manage dependencies.
- Store sensitive information like API keys and passwords securely, possibly using environment variables or secret managers.
-
Error Handling and Logging:
- Implement robust error handling to manage exceptions gracefully.
- Use logging libraries like
loggingto monitor application behavior.
-
Security Considerations:
- Hash and salt user passwords.
- Use HTTPS for API communication.
- Validate and sanitize all user inputs to prevent injections.
-
Scalability:
- While SQLite is sufficient for initial stages, consider migrating to a more robust database like PostgreSQL for scalability.
- Modularize your codebase to facilitate future enhancements.
-
Testing:
- Write unit and integration tests to ensure code reliability.
- Automate testing where possible to streamline the development process.
Conclusion
Building Emailer involves integrating multiple components, each requiring careful implementation and testing. By following the structured weekly plan and adhering to best practices, you can develop a robust and secure email management application. Remember to iterate based on user feedback and continuously improve the application's features and performance.
If you need further assistance on specific components or encounter challenges during development, feel free to ask!