Network Traffic Anomaly Detection
0.96 F1 · attack classHybrid intrusion-detection pipeline combining K-Means clustering, Isolation Forest anomaly scores and a Random Forest classifier, evaluated on UNSW-NB15 and cross-tested on CIC-IDS2017.
- Python
- Scikit-learn
- K-Means
- Isolation Forest
- Random Forest
- SMOTE
Problem
Traditional intrusion detection systems struggle with evolving attack patterns and with imbalanced attack classes.
Approach
K-Means groups similar network traffic profiles, Isolation Forest generates anomaly scores within each cluster, and a Random Forest uses the original features plus the anomaly score to classify traffic as Normal or Attack. SMOTE oversampling handles class imbalance, and custom ratio features (for example bytes and load ratios between source and destination) capture protocol-specific behaviour. UNSW-NB15 is used for training and validation; the DoS-Wednesday subset of CIC-IDS2017 is used for cross-dataset testing.
Architecture
K-Means clustering -> per-cluster Isolation Forest anomaly score -> Random Forest classifier (original features + anomaly score) -> Normal / Attack. Outputs include PCA cluster plots, ROC curves, confusion matrices and feature-importance plots.
Results
| Metric | Verified Value | Context |
|---|---|---|
| Evaluation data | UNSW-NB15 (training and validation); CIC-IDS2017 DoS-Wednesday subset (cross-dataset test) | — |
| F1-score (UNSW-NB15, attack class) | 0.96 | — |
| Precision (UNSW-NB15, attack class) | 1.00 | Zero false positives in primary testing |
| Accuracy (UNSW-NB15) | 95% | — |
What I Learned
Cross-testing on CIC-IDS2017 showed how feature alignment affects accuracy when a model trained on UNSW-NB15 is applied to a different network environment (domain adaptation).