AI-Powered Phishing & Cyber Risk Intelligence

Project Overview

This dual-module AI project combines Natural Language Processing (NLP) and tabular machine learning to proactively detect phishing emails and score cyber vulnerability risks using CVSS metrics. It provides real-time predictions through Streamlit apps and supports security analysts with explainability (LIME), confidence-based thresholds, and risk prioritization.

The system is designed to strengthen enterprise and public cybersecurity workflows by addressing two of the most exploited attack surfaces: email threats and software vulnerabilities.

Objective & Business Context

Modern organizations face constant cyber threats from two directions:

• Phishing Emails: Tricking employees into clicking malicious links or revealing sensitive credentials.

• Software Vulnerabilities: Unpatched security flaws that attackers exploit via known CVEs.

To combat this, organizations need:

• Real-time AI models to screen emails for phishing attempts

• Risk scoring systems to prioritize vulnerabilities based on severity and exploit complexity

• Transparent decision support systems that can explain why an item is flagged as risky

This project simulates a real-world cybersecurity assistant that can be embedded into IT dashboards or used by SOC (Security Operations Center) analysts for early warnings and prioritization.

Business Value and Real-World Scope

This dual-layer solution enhances security operations by:

• Screening emails automatically before reaching users

• Helping security teams prioritize which CVEs need urgent patching

• Making AI predictions explainable to build trust

• Acting as a decision-support layer in security workflows

A. Phishing Detection Use Cases

• Finance & Banking: Helps filter scam emails that attempt to steal account credentials or execute invoice fraud schemes.

• Healthcare: Prevents credential theft and reduces the risk of ransomware attacks entering through phishing.

• EdTech / SaaS: Monitors employee inboxes and assists in conducting phishing simulation campaigns.

• E-commerce & Retail: Detects phishing emails that impersonate the brand or target internal systems and employee accounts.

B. Cyber Risk Scoring Use Cases

• Cloud & IT Services: Scores CVEs within infrastructure components and suggests patching priorities based on risk.

• Government: Assesses CVSS scores to identify nation-state threats and support internal audits.

• Healthcare: Evaluates cybersecurity risks in connected medical devices, especially firmware-level vulnerabilities.

• Enterprise Security: Acts as a triage tool to help teams assign severity levels and response SLAs for newly discovered vulnerabilities.

Implementation Flow

A. Phishing Email Dataset (phishing_emails.csv)

• Fields: Email body text (combined), label (0 = legit, 1 = phishing)

• Cleaning Steps: Lowercasing, punctuation removal, stop word filtering

• Features Used: TF-IDF vector of cleaned email text

B. Cyber Risk Dataset (cyber_risk_data.csv)

• Fields: CVSS score, complexity level, severity (target)

• Complexity Levels: LOW, MEDIUM, HIGH

• Severity Labels: LOW, MEDIUM, HIGH, CRITICAL

• Preprocessing: Label encoding, dummy row injection to balance class representation

Dataset Overview

Phishing Detection Pipeline

• Preprocessing – Clean and vectorize text using TF-IDF.

• Modeling – Random Forest Classifier with SMOTE for class imbalance.

• Threshold Tuning – Default decision threshold increased (e.g., 0.65) to reduce false positives.

• Inference Interface – Accepts raw email, returns prediction + confidence.

• Explainability – LIME explanations highlight suspicious words and influence.

Cyber Risk Scoring Pipeline

• Feature Engineering – Only 2 fields: CVSS (numeric) and complexity (encoded).

• Label Balancing – Missing classes like LOW are manually injected to ensure balanced learning.

• Standardization – Numeric inputs scaled using StandardScaler.

• Model – Random Forest Classifier with accuracy tuned for all 4 severity classes.

• Explainability – LIME (tabular) explains contribution of CVSS and complexity in predictions.

Phishing Module:

- Prediction label with confidence score

- LIME explanation showing the words influencing phishing detection

Risk Scoring Module:

- Predicted severity level: LOW / MEDIUM / HIGH / CRITICAL

- LIME visual explaining contribution of CVSS score and complexity

Note on LIME Visuals for Risk Scoring:
Since the cyber risk scoring model currently uses only two input features — CVSS score and attack complexity — the LIME explanations are inherently simple. The visualizations may appear repetitive, as LIME is limited to showing the relative influence of these two features on the predicted severity class. This behavior is expected and reflects the model's simplicity. For deeper interpretability, additional features such as exploitability score, attack vector, or privilege level can be integrated in future iterations.

Visualizations & Interpretations

✅ Cleaned and processed datasets (raw + processed)
✅ Trained ML models with serialization
✅ TF-IDF vectorizer, label encoders, and scalers saved
✅ Streamlit apps with real-time email and risk scoring UI
✅ LIME-based explainability for both NLP and tabular models
✅ Automated PDF report generator (optional extension)

Key Deliverables

Tools and Libraries Used

• pandas – Used for data loading, cleaning, and preprocessing CSV files in both phishing and cyber risk modules.

• numpy – Supported numerical computations and array transformations, especially for model inputs and explainability.

• matplotlib – Enabled plotting of model interpretability visuals like LIME outputs and classification charts.

• nltk, re, string – Combined for text preprocessing in phishing detection — removing stop words, punctuation, and cleaning raw email content.

• scikit-learn – Provided core ML tools: Random Forest Classifier, LabelEncoder, StandardScaler, along with evaluation metrics and train-test split functions.

• imblearn (SMOTE) – Used to balance phishing email classes and improve model generalization.

• lime – Delivered LIME-based explainability for both text (phishing) and tabular (CVSS scoring) models, highlighting feature importance.

• joblib – For saving/loading models, scalers, encoders, and vectorizers efficiently across training and inference.

• Streamlit – Built the user-facing apps for both phishing and cyber risk scoring, allowing real-time predictions and explanations.

• VS Code Terminal – Used throughout development for testing, script execution, and managing environments.

1. Model Upgrades:

   • Replace TF-IDF with BERT or transformer embeddings

   • Switch Random Forest with Gradient Boosting or LSTM for text

2. Data Extensions:

   • Add fields like exploitability, affected components, patch availability

3. Deployment:

   • Serve via FastAPI or Docker container for production use

   • Schedule phishing checks or batch vulnerability scoring via cron jobs

4. Security Integration:

   • Connect to email servers for real-time phishing alerts

   • Integrate with CVE/NIST feeds for daily vulnerability scoring

5. Enterprise Expansion:

   • Combine both modules into a Unified Cyber AI Agent

   • Build dashboards with alert management + email triage + patch recommendation

Possible Next Steps & Conclusion

Conclusion

This AI system serves as a prototype for next-generation cybersecurity tools that combine machine learning, explainability, and real-time feedback. Its dual architecture allows organizations to proactively address both social engineering attacks and software vulnerability risks.

With its foundation in interpretable AI and lightweight deployment, the system is:

• Easy to understand for non-technical teams

• Extendable for enterprise security workflows

• Ready to integrate with SIEM tools, ticketing systems, or LLM-based agents

In the future, this dual module could evolve into a plug-and-play cybersecurity assistant — auditing, classifying, and explaining digital threats in real-time.

Dive into the foundational concepts, algorithms, and real-world relevance behind this project. From machine learning principles to business strategy insights, this conceptual study bridges the gap between technical implementation and applied decision-making—helping you understand not just how it works, but why it matters.

Key Concepts

GitHub Repository

Want to dive deeper into how this project actually works?

We’ve made the complete codebase and resources available for you on GitHub

👉 Access the full repository here:

Whether you're a learner, recruiter, or collaborator — there's something for everyone.