AI-Driven Anomaly Detection With Wazuh-ELK

This project, titled AI-Driven Anomaly Detection With Wazuh-ELK, is a cybersecurity solution designed to identify and respond to suspicious network activity by integrating machine learning into industrial-standard security monitoring tools.

The system acts as an intelligent security layer that monitors system logs in real-time to detect threats that traditional rule-based systems might miss. It continuously fetches security alerts and logs generated by Wazuh, focusing on specific high-value data such as authentication attempts, file integrity changes, and network connection events. Instead of relying solely on predefined security rules, the system uses an AI model to analyze the statistical patterns of incoming logs, flagging 'anomalous' entries—activities that deviate significantly from normal behavior.

When suspicious activity is detected, the system automatically generates a new alert. These anomalies are indexed separately for immediate review by security teams. All detected anomalies and security trends are displayed on a centralized dashboard, allowing administrators to see a high-level overview of the system's security health.

The project is built on a sophisticated 'Security Information and Event Management' (SIEM) stack combined with Python-based data science tools. The ELK Stack serves as the foundation: Elasticsearch serves as the central database for storing vast amounts of log data and providing fast search capabilities; Logstash handles the processing and routing of log data between the security tools; and Kibana is used to create the visual interface and dashboards for monitoring security events.

Wazuh SIEM provides the core monitoring, including host-based intrusion detection and vulnerability detection. A custom Python application bridges the security stack and the AI model, using NumPy and Pandas for data processing and Joblib to load and run the pre-trained detection model. The system utilizes the Isolation Forest algorithm, which is highly effective at identifying outliers (anomalies) in large datasets without requiring labeled 'attack' data. The implementation features a loop that triggers every 60 seconds to fetch new logs, run them through the AI model, and post findings back to the security database for alerting.

• Developed AI-driven anomaly detection system integrating machine learning with security monitoring tools
• Implemented continuous log monitoring system fetching security alerts from Wazuh in real-time
• Built automated anomaly detection using AI models to analyze statistical patterns of incoming logs
• Created real-time alerting system automatically generating alerts for suspicious activities
• Designed incident visualization dashboard displaying anomalies and security trends
• Integrated ELK Stack (Elasticsearch, Logstash, Kibana) for log storage, processing, and visualization
• Utilized Wazuh SIEM for core monitoring including intrusion detection and vulnerability detection
• Developed Python application bridging security stack and AI model using NumPy and Pandas
• Implemented Isolation Forest algorithm for effective outlier detection without labeled attack data
• Created automated workflow triggering every 60 seconds for continuous log analysis
• Built specialized indexing system for anomaly alerts enabling immediate security team review
• Designed system focusing on high-value security data including authentication and file integrity events

Log Monitoring

Continuous monitoring system fetching security alerts and logs from Wazuh in real-time

Anomaly Detection

AI-powered anomaly detection analyzing statistical patterns to flag suspicious activities

Security Dashboard

Centralized dashboard displaying detected anomalies and security trends for administrators