As part of an internal project, we evaluated the operational boundaries of our custom-developed AI chatbot focused on the EU AI Act and GDPR through targeted manual testing and a collaborative technical workshop. The objective was to verify whether the system consistently adhered to its predefined operational boundaries.
Sector
Service & Technology
The Challenge
Our EU AI Act and GDPR AI chatbot is designed to provide clear, accessible information in response to natural-language questions about the EU AI Act and GDPR. The system is intended to assist users exclusively with these regulatory topics and must reject requests that fall outside its defined scope.
However, the flexibility of generative AI systems allows users to formulate instructions in countless different ways. For a chatbot developed for a specific purpose, it is therefore essential to verify that it cannot be prompted to perform tasks outside its intended scope, such as generating code or answering general-purpose questions.
Unintended use can affect both the predictability of the system’s behaviour and the operating costs of the service. The main testing challenge was to verify whether the chatbot consistently respected its operational boundaries when faced with a wide range of user requests, including potentially misleading ones.
The Solution
We evaluated the chatbot’s operational boundaries through a carefully structured manual testing process. Testers used requests of varying complexity and wording to try to make the system perform activities outside its defined scope.
Developing the testing approach required particular attention, as the responses of generative AI systems are not always predictable, and the same user intent can be expressed in numerous ways. Testing therefore focused on the expected system behaviour: the chatbot had to consistently reject requests unrelated to its designated topics.
Following individual testing, we organised an internal technical workshop in which several colleagues applied different approaches to try to bypass the chatbot’s restrictions. This collaborative effort provided an opportunity to broaden the range of test scenarios and share professional experience in testing generative AI systems.
Technology
ChatGPT (GPT4.1-mini), Angular, Python
The Result
Based on the findings from manual testing and the collaborative technical workshop:
- the testers were unable to make the chatbot deviate from its defined scope in the scenarios examined during the workshop;
- clearly defining the operational boundaries and expected responses provided specific criteria for evaluating system behaviour;
- testing a wide variety of user requests helped identify risks associated with unintended use;
- collaboration among multiple testers contributed to the development of the AI testing methodology and strengthened the team’s expertise.
The project confirmed that quality assurance for generative AI systems must go beyond verifying the accuracy of responses and also include consistent testing of operational boundaries.
