AI Security and Emerging Cloud RisksIntroductionCloud computing has evolved from infrastructure rental into an environment in which organisations consume artificial intelligence (AI), machine learning (ML), generative AI, confidential-computing platforms, sovereign services and increasingly autonomous security controls. These capabilities create substantial value, but they also change the nature of cloud risk. Traditional security largely protected applications, networks, identities and stored data. AI-enabled cloud systems add models, prompts, embeddings, training pipelines, model outputs, inference interfaces and automated decisions to the protection problem. AI security is therefore not simply the application of conventional cybersecurity to an AI product. It requires assurance of the whole socio-technical system: data, models, infrastructure, users, suppliers, operational processes and governance. A secure model can still produce harmful outcomes if its training data is compromised, its access control is weak, or its outputs are used without human oversight. Conversely, a well-designed AI control can reduce detection and response time while introducing automation and concentration risks. This chapter examines AI/ML services, generative AI, model and data risks, shadow cloud, confidential computing, sovereign cloud, post-quantum readiness and autonomous security operations. It also considers governance, assurance, resilience and the future implications of these technologies. Learning ObjectivesBy the end of this chapter, a learner should be able to: - Distinguish major AI/ML cloud service models and identify their security boundaries.
- Analyse threats to AI data, models, prompts, outputs, pipelines and agents.
- Assess generative-AI risks, including prompt injection, data leakage and unsafe autonomy.
- Explain shadow cloud and design controls for unsanctioned services.
- Compare confidential computing and sovereign cloud as risk-management approaches.
- Develop a post-quantum migration perspective for cloud systems.
- Evaluate autonomous security operations and their governance requirements.
- Recommend a proportionate AI-security and emerging-risk governance programme.
1. The AI-Enabled Cloud Risk LandscapeAn AI service may be consumed as a complete application, an API, a managed model platform or a set of infrastructure components. Each arrangement allocates responsibility differently. A customer using a hosted conversational application may control accounts and prompts but have little control over model training or hardware. A customer deploying an open-source model on virtual machines controls more of the stack, but also assumes responsibility for patching, model provenance and runtime security. A useful distinction is between the technical attack surface and the decision surface. The technical attack surface includes endpoints, credentials, storage, libraries, containers, model files and networks. The decision surface includes the decisions made or influenced by the model: approving a payment, prioritising a cyber alert, denying access or generating code for production use. A minor technical compromise can become a major organisational incident when it affects a high-impact decision. AI risk should be evaluated across the lifecycle: | Lifecycle stage | Typical assets | Representative risks |
|---|
| Acquisition | Provider, model card, licence, data rights | Unclear provenance, unsuitable terms, hidden training data | | Preparation | Datasets, labels, feature pipelines | Poisoning, bias, personal-data exposure, weak quality | | Training/fine-tuning | Compute, code, checkpoints | Credential theft, malicious dependencies, model theft | | Deployment | APIs, endpoints, agents, prompts | Injection, denial of service, insecure authentication | | Operation | Logs, feedback, outputs | Drift, leakage, unsafe decisions, monitoring gaps | | Retirement | Models, backups, tokens, records | Residual data, unrevoked access, inability to reproduce decisions |
Security, privacy, safety, reliability and fairness overlap but are not identical. Security asks whether an unauthorised party can compromise the system. Privacy asks whether information is collected or disclosed appropriately. Safety asks whether the system causes unacceptable harm, including through error. Governance determines who is accountable and how evidence supports decisions. 2. AI/ML Cloud Service Models and Shared ResponsibilityCloud AI is commonly delivered through four patterns: - AI software as a service: A provider operates the application and model. The customer manages users, configuration, prompts and business use.
- Model as a service: A provider exposes a foundation or specialised model through an API. The customer manages application logic, input data, retrieval systems and output handling.
- Managed ML platform: The provider supplies notebooks, pipelines, feature stores, training and deployment services. The customer manages data, code, model configuration and often the resulting endpoint.
- Self-managed AI infrastructure: The customer operates virtual machines, accelerators, containers and model software. It gains control but bears substantially greater operational risk.
The shared-responsibility model must be extended beyond infrastructure. A contract should clarify whether customer prompts and outputs are retained, whether data is used for provider improvement, where processing occurs, how subprocessors are controlled, how models are changed, and what evidence is available after an incident. A practical service assessment asks: - What data enters the provider boundary?
- Which party controls model weights and system prompts?
- Can the provider alter the model without notice?
- Are outputs logged, retained or reviewed by humans?
- What controls prevent one tenant from influencing another?
- Can the customer export data, configurations and audit evidence?
- How are incidents, vulnerabilities and model failures reported?
3. Threats to Models, Data and AI Pipelines3.1 Data risksTraining and inference data may contain personal, confidential, regulated or copyrighted material. Risks include excessive collection, insecure storage, accidental inclusion in prompts, unauthorised retrieval and re-identification. Data poisoning occurs when an attacker introduces misleading or malicious records so that training or fine-tuning produces a desired behaviour. In a retrieval-augmented generation system, poisoned documents can influence answers even when the base model is unchanged. Controls include data minimisation, provenance records, access control, encryption, validation, segregation of environments, malicious-content scanning and review of high-impact labels. Data lineage should identify source, owner, transformation, permitted use and retention period. 3.2 Model risksModel files and parameters may be stolen, replaced or modified. A compromised model can contain a backdoor that activates for a particular trigger. Model extraction attempts to reconstruct a model by sending queries and observing responses. Inference attacks may reveal whether a person’s data was included in training or may recover sensitive features from outputs. Model integrity requires signed artefacts, controlled registries, reproducible build processes, dependency scanning, restricted export and verification before deployment. A model card or equivalent record should describe intended use, limitations, evaluation data, known failure modes and security assumptions. 3.3 Pipeline and supply-chain risksML systems rely on software packages, datasets, containers, notebooks, plugins, accelerator drivers and external model repositories. A vulnerable dependency or counterfeit model can enter through a development pipeline. Secure ML operations therefore combines DevSecOps with data and model governance. Important controls include isolated build environments, software bills of materials, dataset manifests, signed commits, secret management, least privilege and approval gates for production promotion. 3.4 Adversarial manipulationAdversarial examples are inputs deliberately constructed to cause incorrect classification or interpretation. Evasion attacks occur at inference time; poisoning attacks occur during preparation or training. These attacks are domain-dependent: a small image alteration, manipulated network telemetry or carefully crafted text may have different effects. Defence requires threat modelling and testing against realistic attack methods rather than assuming that a single benchmark proves robustness. 4. Generative AI and Agentic SystemsGenerative AI produces text, code, images, audio or other content. Its security problem is not limited to whether the output is factually correct. The system may disclose confidential context, follow malicious instructions embedded in retrieved content, invent citations, generate insecure code or create harmful content at scale. Prompt injection occurs when instructions in user input or external content manipulate the model into violating the intended task or policy. Direct injection comes from the user; indirect injection may be hidden in a web page, document or email that an AI system retrieves. Separating instructions from data, restricting tools, validating retrieved content and requiring approval for consequential actions reduce—but do not eliminate—this risk. System-prompt leakage occurs when an attacker induces disclosure of hidden instructions or configuration. A system prompt should not be treated as a secret capable of enforcing access control. Authorisation must be implemented in deterministic application logic and identity systems. Hallucination is the production of plausible but unsupported content. Retrieval, citations, constrained generation and output verification improve reliability, but none guarantees truth. High-impact uses require a human decision-maker or an independently verifiable rule. Tool and agent risk increases when a model can call APIs, execute code, send messages or modify records. The agent should receive narrowly scoped credentials, operate in a sandbox, apply transaction limits, maintain a tamper-resistant audit trail and request explicit approval for irreversible actions. A useful design principle is that the model may recommend an action, while a policy engine decides whether it is permitted. A generative-AI control set should include: - approved-use policies and prohibited data categories;
- identity-based access and tenant isolation;
- input and output filtering appropriate to the use case;
- retrieval-source validation and document trust levels;
- logging of prompts, tools, model versions and outputs, subject to privacy rules;
- red-team testing for injection, leakage, unsafe content and abuse;
- human review for high-impact or irreversible decisions; and
- a documented process for model, prompt and policy changes.
5. Risk Assessment and Governance FrameworkAI governance should connect organisational risk appetite to technical controls. A practical register records the use case, owner, data classification, affected people, model provider, model version, decision impact, threat scenarios, controls, residual risk and review date. One simple prioritisation model is expected loss: E(L) = P(I) × C(I) Here, E(L) is expected loss, P(I) is the estimated probability of an incident, and C(I) is its consequence in an agreed unit such as monetary loss, service downtime or a structured impact score. The formula assumes that probability and consequence are meaningfully estimated and that the selected consequence measure is comparable across scenarios. It is a prioritisation aid, not a precise prediction; correlated events, uncertainty and non-financial harms require additional analysis. A more complete assessment considers: - Impact: confidentiality, integrity, availability, privacy, safety, legal and reputational effects.
- Exposure: internet accessibility, privilege, data sensitivity, supplier dependence and concentration.
- Control strength: prevention, detection, response, recovery and assurance evidence.
- Uncertainty: unknown model behaviour, changing providers, incomplete inventories and limited test data.
Governance should assign three distinct forms of accountability: the business owner accountable for purpose and outcomes; the technical owner accountable for architecture and operation; and independent risk, legal, privacy or assurance functions accountable for challenge and review. Boards and senior managers should receive understandable measures such as high-risk use cases, unresolved findings, provider concentration, incident trends and percentage of models with documented lineage. Relevant standards and regulatory approaches vary by jurisdiction. Organisations commonly draw on risk-management standards, privacy law, sector regulation, security controls and emerging AI legislation. Compliance should not be confused with safety: a compliant process can still approve an unsuitable system if assumptions and evidence are weak. 6. Shadow Cloud and Unauthorised AI UseShadow cloud is the acquisition or use of cloud resources without the knowledge or approval of the organisation’s governance functions. Shadow AI is a related practice in which staff use public chatbots, coding assistants, image generators or unapproved APIs for business work. The underlying causes are usually practical: approved services are slow to obtain, staff need rapid experimentation, procurement processes are unclear, or employees do not understand data restrictions. Punitive blocking alone often drives use to less visible channels. A better response combines discovery, education, usable approved alternatives and proportionate enforcement. Controls include cloud access security brokers or equivalent monitoring, DNS and proxy analysis, identity federation, payment and procurement review, endpoint policies, data-loss prevention, API inventory, browser controls and provider allow-lists. Network monitoring must be balanced against privacy and labour requirements. The organisation should offer a sanctioned AI workspace with clear data handling, retention and logging settings. Incident response for shadow AI should determine what data was disclosed, which provider processed it, whether prompts were retained, whether credentials were exposed and whether generated code entered production. Legal and privacy teams may need to assess notification duties and contractual restrictions. 7. Confidential ComputingConfidential computing protects data while it is being processed by using hardware-supported trusted execution environments (TEEs). Conventional encryption protects data at rest and in transit, but plaintext normally exists in memory during computation. A TEE aims to isolate that memory from the host operating system, hypervisor and unauthorised administrators. Remote attestation allows a relying party to verify that an approved workload is running in an expected environment before releasing secrets. Confidential computing can support privacy-preserving ML, cross-organisation analytics and protection of model weights. However, it is not a complete security solution. Vulnerabilities in processors, firmware, enclave code, side channels, input data, identity services or application logic may remain. Attestation proves a measured configuration, not that the software is beneficial or free of design defects. Deployment decisions should examine: - What threat actor is excluded by the TEE?
- What exactly is measured and attested?
- How are keys released and rotated?
- Are debugging, logging and recovery compatible with confidentiality?
- What performance and accelerator limitations apply?
- How will the organisation respond to a hardware or firmware vulnerability?
Confidential computing is strongest when combined with minimisation, encryption, access control, secure development and independent assurance. 8. Sovereign Cloud and Data JurisdictionSovereign cloud seeks to ensure that data, infrastructure, operations and decision-making remain subject to specified national or regional control. Sovereignty may involve physical location, legal jurisdiction, local ownership, trusted personnel, operational independence, resilience and ability to continue service without foreign intervention. Sovereignty is multidimensional. A workload physically located in a country may still depend on foreign-owned software, support personnel, encryption services or parent-company legal obligations. Conversely, a globally operated provider may offer strong technical isolation but not satisfy a public-sector requirement for domestic control. A sovereignty assessment should map: | Dimension | Assessment question |
|---|
| Data | Where is data stored, backed up and processed? | | Legal | Which laws and disclosure orders may apply? | | Operations | Who can administer the environment? | | Supply chain | Who owns hardware, software and support? | | Continuity | Can services operate during geopolitical disruption? | | Exit | Can data and workloads be migrated without provider dependence? |
Sovereign cloud can reduce jurisdictional and strategic risks, but it may increase cost, reduce service choice and create fragmentation. It should be selected according to regulatory and threat requirements rather than used as a vague synonym for “secure cloud.” 9. Post-Quantum ReadinessQuantum computers capable of breaking widely deployed public-key schemes at scale are not generally available today, but migration planning is urgent because cryptographically protected data may be harvested now and decrypted later. This is known as the “harvest now, decrypt later” risk. Post-quantum cryptography (PQC) uses mathematical problems believed to resist quantum attacks. Organisations should not wait for a single universal algorithm or provider announcement. They should inventory cryptographic use, identify long-lived confidential data, map certificate and key dependencies, and require crypto-agility—the ability to replace algorithms and parameters without redesigning the entire service. Priority actions include: - catalogue algorithms, certificates, keys, protocols and libraries;
- identify public-key encryption and digital-signature uses;
- rank information by confidentiality lifetime and migration difficulty;
- test provider and application support for standardised PQC mechanisms;
- use hybrid approaches where appropriate during transition;
- update procurement requirements and third-party contracts; and
- plan certificate, firmware, identity and backup migration.
PQC does not solve every quantum risk. Symmetric cryptography generally requires parameter considerations rather than wholesale replacement, while integrity, randomness, key management and implementation vulnerabilities remain important. Migration must also consider larger keys or signatures, network overhead, hardware support and interoperability. 10. Autonomous Security OperationsAI can assist security operations by correlating alerts, summarising evidence, proposing detections, investigating events and executing bounded responses. Autonomous security operations go further by allowing systems to make or implement decisions with limited human intervention. The benefit is speed and scale. The risks include false positives, feedback loops, poisoned telemetry, model drift, excessive privileges and automation bias. An AI system that disables accounts or isolates workloads incorrectly can create an availability incident. An attacker who manipulates telemetry may cause the system to attack legitimate services or conceal malicious activity. A mature operating model uses graduated autonomy: | Level | Capability | Required control |
|---|
| Advisory | Summarises and recommends | Analyst review | | Assisted | Executes reversible low-risk actions | Policy guardrails and logging | | Conditional | Acts within defined thresholds | Independent approval or dual control | | Autonomous | Acts on high-volume, time-critical events | Continuous assurance, rollback and emergency stop |
Autonomous systems require least-privilege service identities, explicit action allow-lists, rate limits, simulation or shadow mode, rollback mechanisms, immutable logs, independent validation and periodic reassessment. Human oversight must be meaningful: the reviewer needs sufficient time, evidence and authority to reject the action. Merely placing a human nominally in the process does not provide effective control. Illustrative Case Study: Northbridge Health NetworkBackgroundNorthbridge Health Network, a fictional regional provider, operates hospitals and community clinics. It adopted a managed generative-AI service to summarise clinical correspondence and a security-AI platform to triage alerts. It also used a sovereign-cloud region for regulated records and confidential-computing features for a research partnership. ProblemStaff began using an unsanctioned public chatbot to rewrite patient letters. A researcher uploaded a dataset to an external notebook service. Meanwhile, the security-AI platform was granted permission to disable accounts automatically. An attacker later inserted a malicious instruction into a document indexed by the correspondence assistant. The assistant attempted to call a workflow tool with an excessively broad credential. The call was blocked by a policy gateway, but the event revealed multiple weaknesses. AnalysisThe incident was not caused by one defective model. It resulted from governance fragmentation. The organisation lacked a complete AI inventory, had not classified prompts as data, relied on system prompts instead of application authorisation, and had not tested indirect prompt injection. Provider contracts did not clearly specify retention and model-change notification. The security agent had autonomy beyond its tested operating envelope. Approach and solutionNorthbridge established an AI governance committee and required registration of every AI use case. It classified clinical text as restricted data, created an approved enterprise assistant with retention disabled where contractually possible, and blocked unapproved endpoints while providing a usable alternative. The retrieval system introduced trusted-source labels, document sanitisation and citations. Tool calls were mediated by a policy engine using short-lived, narrowly scoped credentials. The security agent was moved to advisory mode for thirty days, followed by assisted mode for reversible actions. High-impact actions required dual approval. The organisation signed model artefacts, recorded model and prompt versions, added AI-specific incident playbooks and began a cryptographic inventory for future PQC migration. OutcomeThe organisation reduced unsanctioned use, improved evidence for audits and detected several attempts to place malicious instructions in retrieved documents. It accepted that hallucination could not be eliminated and therefore restricted the assistant to summarisation with source links rather than autonomous clinical advice. Learning points- AI security is a lifecycle and governance problem, not only a model problem.
- A system prompt cannot replace access control.
- Approved alternatives are essential to controlling shadow AI.
- Tool permissions and autonomy must be narrower than the model’s apparent capabilities.
- Sovereignty, confidentiality and cryptographic readiness address different risk dimensions.
11. Implementation Roadmap and Best PracticesAn organisation beginning an emerging-risk programme can proceed in stages: Stage 1—Discover. Inventory AI applications, models, providers, data flows, agents, cloud accounts, cryptographic dependencies and business owners. Include informal and experimental use. Stage 2—Classify. Rate data sensitivity, decision impact, autonomy, external exposure, provider dependence and regulatory significance. Prohibit or escalate uses that cannot meet minimum requirements. Stage 3—Design. Apply identity, least privilege, segmentation, encryption, secure pipelines, model provenance, human review, logging, retention limits and exit planning. Treat prompts, embeddings and outputs as governed data. Stage 4—Test. Conduct threat modelling, red-team exercises, privacy assessments, adversarial testing, disaster recovery tests and provider assurance reviews. Test failure modes, not merely normal accuracy. Stage 5—Operate. Monitor drift, access, unusual prompts, data leakage, tool calls, model changes and control effectiveness. Maintain incident playbooks and exercise them. Stage 6—Improve. Review incidents, near misses, complaints, audit findings and technological change. Retire systems whose risk cannot be controlled economically or ethically. Good practice includes separating development and production data, using synthetic or minimised data for testing, keeping independent backups, requiring contractual notification of material model changes, documenting residual risk and preserving a practical exit route from critical providers. Challenges and LimitationsAI risk measurement remains difficult because probabilities are uncertain, harms may be non-linear and model behaviour can change with prompts, context or provider updates. Benchmarks may not represent the organisation’s environment. Explainability methods can be incomplete or misleading. Privacy-enhancing technologies may introduce performance costs. Sovereign environments can reduce flexibility and increase concentration within a small domestic supplier market. PQC migration can expose hidden dependencies and interoperability problems. Autonomous security can amplify both good and bad decisions at machine speed. The appropriate response is not to seek perfect prediction. It is to combine prevention, detection, human accountability, resilience, evidence and controlled experimentation. High-impact systems deserve stronger assurance than low-risk productivity tools. Future ImplicationsThe boundary between cloud service, software agent and organisational worker will continue to blur. Models may negotiate with other models, operate across providers and create infrastructure dynamically. This may improve productivity but will challenge identity, accountability, records management and legal attribution. Security teams will increasingly govern machine identities, model supply chains and automated decisions alongside human users. Future-ready organisations will be those that can change models, providers and cryptographic mechanisms without losing control of data or evidence. They will treat AI governance as an operational capability rather than a one-time approval. The central strategic question is not whether AI can be adopted, but under what conditions its benefits exceed the risks and how quickly the organisation can detect and recover when assumptions fail. Chapter SummaryAI-enabled cloud services expand security concerns from infrastructure and applications to data lineage, model integrity, prompts, outputs, agents and automated decisions. Shared responsibility must therefore include model operation, provider changes, data retention, tool permissions and audit evidence. Major AI threats include poisoning, extraction, adversarial manipulation, prompt injection, leakage, hallucination, supply-chain compromise and unsafe autonomy. Shadow cloud and shadow AI arise when official services do not meet operational needs or governance is unclear. Confidential computing protects data during processing but depends on sound attestation, hardware and application controls. Sovereign cloud addresses jurisdictional, operational and strategic control, not security in general. Post-quantum readiness requires cryptographic inventory, prioritisation and crypto-agility before large-scale quantum threats materialise. Autonomous security operations can improve speed but require graduated autonomy, least privilege, rollback, independent assurance and meaningful human oversight. Effective governance combines inventory, classification, threat modelling, secure engineering, contractual assurance, monitoring, incident response and periodic review. The most resilient approach is lifecycle-based, evidence-driven and proportionate to impact. Key TermsAI security: Protection of AI systems, data, models, users and decisions from malicious, accidental or harmful events. Adversarial example: An input deliberately crafted to cause an AI model to produce an incorrect result. Agentic system: An AI system able to plan or call tools to achieve a goal. Confidential computing: Protection of data during processing through hardware-supported trusted execution environments. Crypto-agility: The ability to replace cryptographic algorithms or parameters with limited redesign. Data poisoning: Deliberate manipulation of training or retrieval data to influence system behaviour. Generative AI: AI that produces new content such as text, code, images or audio. Hallucination: Plausible-looking but unsupported or false model output. Indirect prompt injection: Malicious instructions embedded in content retrieved by an AI system. Model extraction: Attempt to reconstruct a model through queries and observed outputs. Model drift: Change in model performance or data relationships over time. Model card: Documentation describing a model’s purpose, limitations, evaluation and risks. Post-quantum cryptography: Cryptographic techniques designed to resist attacks by quantum computers. Prompt injection: Manipulation of model instructions through crafted input or content. Remote attestation: Evidence that a specified workload is running in an expected trusted environment. Shadow cloud: Unauthorised or ungoverned use of cloud resources. Sovereign cloud: Cloud capability designed to satisfy specified national, legal and operational control requirements. Trusted execution environment: An isolated processing area intended to protect code and data from unauthorised access. Tool permission: The defined authority an AI system has to invoke an external service or take an action. Review QuestionsShort Answer Questions- What is the difference between a technical attack surface and a decision surface in an AI system?
- State two risks associated with training or retrieval data.
- Why can a system prompt not replace application-level authorisation?
- What does confidential computing protect, and what important limitation does it have?
- What is meant by crypto-agility?
Long Answer Questions- Explain how shared responsibility should be extended for a managed generative-AI cloud service.
- Discuss the principal security risks of generative AI and recommend appropriate controls.
- Compare confidential computing and sovereign cloud as approaches to cloud risk management.
- Explain why post-quantum migration should begin with cryptographic inventory and data-lifetime analysis.
- Evaluate the governance requirements for autonomous security operations.
Analytical / Application Questions- An employee uploads confidential product designs to an unapproved AI service. Design an incident-response and governance process for the organisation.
- A cloud agent is allowed to disable compromised accounts automatically but begins responding to poisoned telemetry. Analyse the failure and propose a safer autonomy architecture.
- A public-sector agency must choose between a globally operated AI platform with strong technical controls and a locally operated platform with fewer capabilities. Develop a decision framework addressing sovereignty, security, resilience, cost and exit risk.
Multiple Choice Questions with AnswersQuestion 1Which asset is specifically introduced by AI systems in addition to conventional application assets? A. Model weights and prompts B. Only physical server racks C. Only office furniture D. Only user passwords Correct Answer: A. Model weights and prompts Question 2What is indirect prompt injection? A. A hardware failure during model training B. Malicious instructions embedded in content retrieved by an AI system C. A routine software update approved by an administrator D. Encrypting a prompt before transmission Correct Answer: B. Malicious instructions embedded in content retrieved by an AI system Question 3Why should a system prompt not be treated as an access-control mechanism? A. Models cannot be hosted in the cloud B. System prompts are always publicly published C. Authorisation must be enforced by deterministic identity and policy controls D. System prompts prevent all data processing Correct Answer: C. Authorisation must be enforced by deterministic identity and policy controls Question 4What is a primary purpose of confidential computing? A. To protect data while it is being processed B. To eliminate all software vulnerabilities C. To guarantee that model outputs are correct D. To replace identity management Correct Answer: A. To protect data while it is being processed Question 5Which characteristic is central to sovereign cloud assessment? A. The colour of the provider’s user interface B. Jurisdiction, operational control and data location C. The number of available font styles D. The absence of all encryption Correct Answer: B. Jurisdiction, operational control and data location Question 6What does crypto-agility mean? A. The ability to change cryptographic mechanisms without extensive redesign B. The ability to remove all cryptography from a system C. The use of only one permanent encryption algorithm D. The automatic deletion of all keys after one minute Correct Answer: A. The ability to change cryptographic mechanisms without extensive redesign Question 7Which control is most appropriate for an AI agent that can perform irreversible actions? A. Unlimited administrator privileges B. No logging to preserve performance C. Narrow permissions, approval gates and rollback capability D. Reliance only on the model’s system prompt Correct Answer: C. Narrow permissions, approval gates and rollback capability Question 8What is a major risk of shadow AI? A. Unauthorised disclosure of organisational data to an external provider B. Guaranteed improvement in regulatory compliance C. Removal of all provider dependence D. Automatic validation of every generated output Correct Answer: A. Unauthorised disclosure of organisational data to an external provider Question 9In the expected-loss expression E(L) = P(I) × C(I), what does C(I) represent? A. The number of cloud instances B. The consequence of an incident C. The confidence interval of a model D. The cryptographic key length Correct Answer: B. The consequence of an incident Question 10Which approach best supports safe autonomous security operations? A. Immediate unrestricted autonomy for all actions B. Graduated autonomy with least privilege, monitoring and rollback C. Removing human review from high-impact decisions D. Allowing the model to change its own access policies Correct Answer: B. Graduated autonomy with least privilege, monitoring and rollback
|