ChatIDS: Using Generative AI to Turn Intrusion-Detection Alerts into Warnings Non-Experts Can Act On

It specifies the ChatIDS architecture: a network-based IDS generates alerts, the ChatIDS component embeds each alert into predefined prompt templates and sends it to the LLM component for translation into intuitive explanations, users can ask follow-up questions, and explanations are cached so the same one is not requested twice. Unlike prior expert-oriented security tooling (such as Microsoft Security Copilot) and static explanations of known attacks, it positions the LLM as a substitute for the cybersecurity expertise a home user lacks, and explicitly separates the roles of expert and user. A system and information-flow description (Figure 4) with a prompt template (Figure 5) and one full example output (Figure 6), forming the modeling step of a constructive research method.

It defines three testable requirements, R1 (assess the probability of a false alert), R2 (assess urgency), and R3 (identify appropriate measures), and scores 20 alerts drawn from the Snort, Suricata, Yara, and Sigma rulesets on a 5-point Likert scale from 2 to -2. It decomposes whether an alert is understandable to a non-expert into concrete scorable features (description, intuition, consequences, urgency, countermeasures, correctness) rather than discussing usability in general terms. 20 alerts across 6 scoring features, tested one by one; the authors describe this as a qualitative evaluation with a small sample.

Results show R1 and R2 were met rather well: with only one exception ChatIDS produced a good description of the security issue behind the alert and terminology was almost always intuitive, while consequences and urgency were conveyed almost always to complete satisfaction; R3 leaves room for improvement, as countermeasures were often too general and would not completely eliminate the threat described by the alert. It gives a tabular, per-alert comparison showing that LLM performance differs between explaining an alert and prescribing actionable countermeasures, with the latter being the weaker link. Table I lists scores for all 20 alerts across 6 features, including negative and zero entries; scoring was qualitative and performed by the authors, with no inter-rater agreement reported.

Through a pre-study with interdisciplinary AI experts it consolidates six areas of open issues before deployment, namely security, privacy, compliance, jurisprudence, trust, and ethics, for example that an external LLM becomes a new attack surface, that anonymization plus dummy alerts still permits inference, and that over-trust or manipulative explanations carry risk. It extends the discussion from technical feasibility to governance, and offers customizable prompt templates aimed at different cultural backgrounds, expertise levels, and protection needs (Appendix Figures 8 and 9 show two versions of the same alert). A qualitative consolidation from an expert pre-study covering applications, cybersecurity, ethics, jurisprudence, and privacy; the number of experts and any structured consensus method are not reported.

The result targets privately used networks with a pre-configured network-based IDS, such as smart homes and home offices, and requires a signature-based IDS so that alert messages are specific enough for the LLM. The method is realized with ChatGPT (gpt-3.5-turbo) and an interface to the open-source home automation software Home Assistant has been implemented, showing a short warning message on a Raspberry Pi. It aims to widen the three options a non-expert otherwise has in the Check phase, namely do nothing, turn off the device, or ask an expert, and it supports prompt templates tailored to different cultural backgrounds, expertise levels, and protection needs.

The authors state that more work on prompt engineering is needed to ensure intuitive explanations on the first attempt; whether anonymization removes relevant context or affects the report still needs analysis; and whether ChatIDS actually increases network security is difficult to measure because it depends on the user. The issues raised by experts, including security, privacy, compliance, jurisprudence, trust, and ethics, such as an external LLM becoming a new attack surface, inference remaining possible despite anonymization and dummy alerts, over-trust leading to wrong actions, and explanations that may be manipulative, are all listed as matters to resolve before deployment. In addition, the evaluation covers 20 alerts with qualitative scoring, reports no inter-rater agreement, and includes no real user study, so how non-experts actually understand and act on the warnings remains an open question.