More capability. The same duty of control.
Asking an assistant to summarize an incident is not the same as allowing it to isolate a server. Both may involve artificial intelligence. The decisive difference is the authority we grant it, the data it can access, and the consequences of an incorrect response.
This distinction is our starting point for examining recent developments in AI and cybersecurity. Rather than choosing between enthusiasm and fear, we approach the issue as one of engineering, accountability, and risk management: where does AI add value, what exposure does it introduce, and how do we demonstrate that it remains under control?
CCN-CERT's BP/36 guide, published in June 2026, provides a useful starting point. It connects the acceleration of offensive techniques with the need to review processes, strengthen architecture, and govern defensive agents. Its message goes beyond buying tools: organizations need the ability to prevent, detect, respond, and recover. [1] [1.1]
A retrospective: from the model to the ecosystem
This concern did not begin with conversational assistants. In 2020, ENISA was already examining the assets, lifecycle, and threats of AI systems. In 2023, it organized good practices into three layers: cybersecurity foundations, AI-specific measures, and adaptation to the sector. AI security is therefore connected to the security practices organizations should already have in place. [2] [3]
Also in 2023, guidance led by the UK's NCSC and the US CISA structured protection from design through operation and maintenance. In 2024, France's ANSSI examined generative AI system security in greater depth, while NIST's generative AI profile extended risk management to concerns specific to these systems. [4] [5] [6]
In Portugal, the CNCS 2025 report on risks and conflicts, which mainly covers 2024, had already identified generative AI use in social engineering. This is a historical observation, not a universal measurement of AI-enabled attacks. In 2026, BP/36 and the European action plan on cybersecurity and AI continued this institutional attention. [7] [1] [8]
Our conclusion from this evolution is straightforward: the model alone can no longer be the unit of protection. We must consider the application, data, identities, integrations, people, and processes that enable it to act.
Three different problems, one shared responsibility
Speaking broadly about “AI risks” can hide very different problems. To decide where to invest, Cyberprotech proposes distinguishing three complementary areas. This is an editorial synthesis, informed by the risk families described by CCN-CERT and the technical risks identified by OWASP. [1] [9]
AI used by attackers
The first area is AI used to support reconnaissance, fraudulent messaging, impersonation, and code development. BP/36 describes it as a capability multiplier: familiar techniques can be prepared and repeated with less effort. This does not mean every attack is autonomous, human involvement disappears, or any model can exploit any system. [1]
The practical recommendation is to strengthen controls that do not depend on a message “looking genuine.” An IBAN change, a privileged-access request, or a financial instruction should be confirmed through independent procedures and channels, using previously verified contacts. A familiar voice should not replace verifiable authorization. The guide highlights this need for verification outside the original channel. [1]
AI adopted without the organization's knowledge
The second area is internal: unrecorded tools, personal accounts, extensions, and content copied to external services without assessment. The term shadow AI describes use outside formal governance mechanisms. Exposure does not require a sophisticated attacker: it may begin with an everyday decision about where to place a contract, customer conversation, or technical log. [1]
We propose combining clear rules with usable alternatives. A policy that only prohibits, without explanation or an approved path, is not in itself a technical control. Organizations should define authorized uses, excluded data, and how to request an assessment of a new tool.
AI systems themselves as targets
The third area arises when organizations connect models to documents, knowledge bases, or tools. OWASP identifies risks including prompt injection, sensitive information disclosure, supply chain problems, and excessive agency. Asking whether a model answers well is not enough; we must ask what the application allows it to do. [9]
Illustrative example
A support assistant reads an external document containing malicious instructions. If it can access internal data and send messages without restrictions, a misinterpretation may become a data leak. If access and operations are validated outside the model, the same error encounters an independent barrier.
This separation between content and authority should guide the architecture. OWASP emphasizes that there is no foolproof prevention of prompt injection, and that connecting a model to documentary sources through RAG does not, by itself, eliminate the problem. Defense requires multiple layers. [10]
SOC/CSIRT: start with assistance, not autonomous control.
In a security operations center (SOC) or computer security incident response team (CSIRT), our recommended starting point is decision support. AI can help organize information and prepare work; moving to actions on real systems requires a different assessment, proportionate to the impact. BP/36 recommends limiting scope, permissions, and autonomy while retaining oversight and the ability to interrupt execution. [1]
Initial use cases include summarizing alerts with links to evidence, preparing timelines, comparing hypotheses, and drafting reports. A suggested query can be reviewed before execution. A conclusion should distinguish observed facts, inferences, and missing information. The analyst remains responsible for validation, not merely for clicking “approve.”
The risk profile changes when a tool can block an account, change a network rule, isolate a device, close an incident, or send an external communication. In these cases, we recommend authorization that identifies the operation, target, scope, and conditions. Execution should not result solely from a sentence generated by the model.
Starting with assistance does not mean rejecting automation. A deterministic, tested, preauthorized procedure can contain a threat without waiting for manual approval on every occurrence. The distinction is between bounded automation and an agent's freedom to choose new targets, privileges, or actions. The guide also addresses preauthorized responses alongside control and resilience. [1]
Modes of use and authority limits
| Mode of use | Authority limit |
|---|---|
| Observe and test | Work with test data or authorized historical data, without changing production. |
| Assist | Access only what is needed, present sources, and submit conclusions to the analyst. |
| Execute with approval | Perform an identified, authorized action through permissions and validation outside the model. |
| Automate within limits | Execute only predefined cases, with tested monitoring, interruption, and recovery. |
Autonomy is a choice, not a requirement
Greater autonomy is not a mandatory objective. For some services, remaining in assistance mode will be the most appropriate decision. The criterion should be demonstrated value and accepted risk, not pressure to announce an “autonomous SOC.”
“Trust is measured not by the freedom granted to an agent, but by the ability to limit and verify what it does.”
Limits must exist outside the model
CCN-CERT's guidance on agents and the NCSC and ANSSI secure development recommendations point to controls throughout the lifecycle. In our view, applying them can begin with six concrete decisions. [1] [4] [5]
1. Define the purpose before integration
Each use should have an owner, a bounded task, and acceptance criteria. The inventory should identify the model and versions, supplier, data used, knowledge sources, available tools, and dependencies. A public-information assistant and an agent with administrative access should not automatically share the same risk profile.
2. Give each agent only the access it needs
A dedicated identity allows permissions to be assigned and revoked without relying on generic accounts. We recommend separating reading from writing, test environments from production, and different customers' information. Credentials should be managed outside prompts and code, with limited duration and scope wherever the integration permits. The principle is least privilege, not initial convenience.
3. Validate operations through independent rules
Components that execute actions should verify identity, authorization, destination, and parameters. Access control in integrations must follow the user and the real context. A response claiming authorization is not proof of authorization. There should also be limits on consumption, duration, number of calls, and permitted external destinations.
4. Govern the data and the supplier
Before sending information, clarify purpose, contractual terms, data locations and flows, retention, subprocessors, and possible training use. We propose starting with public information or suitable test data, avoiding customer data until controls have been validated. Local hosting or hosting in a European region does not remove the need to assess processing and dependencies.
5. Test failures, not only correct answers
Testing should include manipulated documents, out-of-scope requests, attempted unauthorized access, unsupported responses, and tool failures. Changes to models, configuration, data, or connectors justify reassessment. OWASP and MITRE ATLAS help structure adversarial scenarios; they do not replace tests tailored to the application and the consequences of its use. [9] [10] [11]
6. Retain the ability to stop and recover
We recommend testing credential revocation, integration shutdown, and an alternative procedure without AI. A switch that only closes the interface is insufficient if tasks remain running or authorizations remain valid. The plan should address effects already produced and operations in progress, as well as restoring the service.
These controls complement rather than replace the essentials: asset inventory, vulnerability remediation, strong authentication, segmentation, and recovery. BP/36 calls for prioritization based on exposure and active exploitation, without dispensing with validation or compensating measures. In industrial environments, passive monitoring and process continuity require specific care; changes should not be accelerated while ignoring operational risk. [1]
Traceability: knowing what happened without keeping everything
When an AI response influences a decision, a central question is whether we can reconstruct what happened. Keeping the final text is not enough. Relevant information includes the sources consulted, configuration in force, tools called, applicable authorization, and observed results. BP/36 connects traceability with monitoring, auditing, and incident response. [1]
To make this principle operational, we propose a minimum record for each relevant interaction: identifier, date and time, user or agent identity, application and model version, source references, requested operation, validation performed, and outcome. Granularity should increase with impact, not with an indiscriminate intention to collect everything.
This record does not necessarily reveal the model's internal reasoning or make a response true. Instead, it documents the application's observable path: authorized inputs, control decisions, actions, and effects. An AI-generated timeline should remain connected to original evidence, distinguishing observation from interpretation.
More records can also mean more exposure
Conversations, documents, and technical logs may contain personal data, confidential information, or access secrets. GDPR requires principles including purpose limitation, data minimization, and storage limitation. Security and data protection by design remain relevant when AI is added; an impact assessment is required where processing is likely to create a high risk to individuals. [14]
We therefore recommend selecting the necessary evidence, restricting access, and setting retention periods by purpose. Masking a secret is different from retaining its full copy “for auditing.” It is also important to distinguish routine operational logs from evidence preserved during an incident, which is subject to specific integrity, access, and retention criteria.
In April 2025, Portugal's CNPD highlighted a report made available by the European Data Protection Board with a methodology for managing privacy risks in AI systems. It is methodological support, not certification. The Board's Opinion 28/2024 clarifies that the anonymity of a model trained on personal data requires case-by-case assessment: it should not be presumed. [12] [13]
Evidence must withstand uncertainty
In our approach, preparing for investigation means defining in advance who preserves artifacts, how their integrity is protected, and who can access them. During an incident, the team should be able to identify the affected version, interrupt access, preserve relevant logs, and assess impact without relying exclusively on the system's own explanation.
“Useful traceability means demonstrating what was done, with which data and under which authorization, without creating an unnecessary archive of sensitive information.”
Portugal and Europe: choose the framework for the right question
No single document resolves governance, technical security, data protection, and compliance simultaneously. It is important to distinguish applicable legislation, management standards, and good-practice guidance. A long list of frameworks does not in itself demonstrate that controls are implemented.
The Portuguese starting point
Decree-Law 125/2025 approved Portugal's Cybersecurity Legal Framework, transposing the NIS2 Directive. Regulation 756/2026 approved the National Cybersecurity Reference Framework, the risk matrix, and minimum measures within its defined scope. For entities in scope, AI adoption should be considered in risk management and relevant processes rather than treated as a separate universe. [15] [16]
BP/36 is rooted in the Spanish context and the Esquema Nacional de Seguridad. Its technical value does not make the ENS the Portuguese framework or replace a national applicability assessment. Useful adaptation translates principles into controls and evidence suited to the entity, rather than importing obligations automatically. [1] [15] [16]
What each family of frameworks contributes
| Question | Useful references and purpose |
|---|---|
| How can the lifecycle be protected? | ENISA, NCSC/CISA, and ANSSI: secure design, development, integration, and operation. [3] [4] [5] |
| How can management be organized? | NIST AI RMF and ISO/IEC 42001: responsibilities, risk, evaluation, and continual improvement. [6] [18] |
| Which adversarial scenarios should be tested? | OWASP and MITRE ATLAS: technical risks and attack behaviors to include in tests. [9] [10] [11] |
| Which obligations should be checked? | National cybersecurity law, GDPR, and the AI Act, according to the entity, processing, role, and use. [14] [15] [16] [17] |
Management and applicability
ISO/IEC 42001 specifies requirements for an AI management system. It should be understood within that scope: it does not guarantee invulnerability or automatically approve each application. The voluntary NIST AI RMF complements management through Govern, Map, Measure, and Manage, connected to context and risk assessment. [18] [6]
The AI Act also requires a concrete analysis of the organization's role, purpose, and system classification. Not all AI is high-risk, and obligations are not identical. The analysis should consider the applicable wording and timetable, including the 2026 amendments, as well as relevant literacy and transparency provisions. An informational chatbot should not automatically be equated with a system that decides on rights or access to services. [17]
From intent to practice: an initial 90-day plan
An organization does not need to start with a large program. It needs to understand what it uses, choose a bounded use case, and establish decision criteria. The following plan is a Cyberprotech proposal: periods are indicative, not legal deadlines or a guarantee of delivery for every entity.
First 30 days: understand and define boundaries
Identify tools and integrations in use, assign owners, and classify the data involved. Review access, suppliers, retention, and contractual terms. Select a low-impact use case with measurable value and define who authorizes its development. In parallel, fix obvious exposures: misplaced keys, excessive permissions, or access without an owner.
Days 31–60: test in a controlled environment
Run the pilot without authority to change production. Compare assisted work with the existing procedure using equivalent cases and shared criteria. Test unsupported responses, content manipulation, unauthorized access, integration failures, and consumption limits. Train users to validate results and rehearse service shutdown. Document problems and corrections.
Days 61–90: decide based on evidence
Review results with security, technology, business, and data protection owners as needed. The decision may be to proceed, remain in assistance mode, narrow scope, or suspend. Any ability to act should add bounded authorization, technical controls, monitoring, and recovery. A convincing demonstration does not replace acceptance criteria or a record of residual risk.
Measure usefulness, not only speed
We propose measuring time to a validated conclusion, frequency of material errors, rework, and the percentage of operations with verifiable authorization and evidence. Also track attempted out-of-scope actions, data protection failures, cost per task, and interruption and recovery capability. Targets should be defined for the organization and use case, not copied from a commercial promise.
In a SOC, reducing report-writing time is a gain only if conclusion quality and evidence preservation are maintained. An apparent reduction in response time can conceal premature closures or work shifted to a later stage. Comparison should therefore consider validated outcomes and consequences, not only the number of tasks completed.
The decision to proceed
Before increasing autonomy, it should be possible to demonstrate who is accountable, what data is needed, which actions are permitted, how a failure is detected, and how execution is interrupted. When one of these answers is missing, the next step should strengthen control rather than expand privileges.
Research to adopt AI with confidence
BP/36 is useful for connecting processes, architecture, and agents. Responsible reading also requires distinguishing guidance from demonstrated results. The guide brings together references of different kinds and should not be used to extrapolate a single attack-success percentage or guarantee automation gains for a particular organization. For this analysis, its strongest contribution is its principles and adaptation program. [1]
This is the perspective through which Cyberprotech follows AI security developments and advances applied research and development. The aim is to turn technical knowledge and governance references into criteria that help organizations assess AI use, limit exposure, and preserve response capability.
This work focuses on visibility into tools and integrations, protection of shared information, control of identities and permissions, and traceability of operations. We are interested not only in what a system can produce, but also in the access it needs, the dependencies it introduces, and the evidence that remains when something goes wrong.
Applied research should enable approaches to be compared, limitations to be identified, and conditions of use to be tested before becoming recommendations. It does not replace contextual validation or justify zero-risk promises. Instead, it strengthens the technical preparation needed to support proportionate, informed, and verifiable adoption.
More capability without giving up responsibility
The lesson from this retrospective is not that every organization should build autonomous agents. It is that AI use should be accompanied by equivalent governance, protection, and verification capability. Technology can evolve quickly; responsibility for what we allow it to do still requires clear decisions.
For Cyberprotech, the path starts with bounded tasks, appropriate data, and verifiable results. It continues through testing, learning, and risk review. Only then does it make sense to discuss additional authority, and only when the benefit justifies the exposure and controls demonstrate they can contain it.
“SOC/CSIRT: start with assistance, not autonomous control.”
This principle gives innovation direction. Maturity is not measured by the number of agents deployed, but by the ability to demonstrate what each can do, with which data, under which authorization, and how it can be interrupted.
Reading note
This article is an original Cyberprotech analysis, not a translation of BP/36. The examples, modes of use, and 90-day plan are editorial proposals. Regulatory information must be applied to the specific case. The source document records consultation on September 15, 2026; the web edition was reviewed on October 6, 2026.
References to organizations, publications, and standards do not imply partnership, certification, or endorsement of this article. The tables and implementation proposal are original Cyberprotech syntheses.