Building Secure and Responsible LLM Applications Training Course

Overview of LLM Architecture and Attack Surface

• Understanding how LLMs are constructed, deployed, and accessed via APIs.
• Key components in LLM application stacks (e.g., prompts, agents, memory, APIs).
• Where and how security issues manifest in real-world usage.

Prompt Injection and Jailbreak Attacks

• Defining prompt injection and its potential dangers.
• Scenarios involving direct and indirect prompt injection.
• Techniques for jailbreaking to bypass safety filters.
• Strategies for detection and mitigation.

Data Leakage and Privacy Risks

• Risks of accidental data exposure through model responses.
• PII leaks and misuse of model memory.
• Designing privacy-conscious prompts and retrieval-augmented generation (RAG) structures.

LLM Output Filtering and Guarding

• Employing Guardrails AI for content filtering and validation.
• Defining output schemas and constraints.
• Monitoring and logging unsafe outputs.

Human-in-the-Loop and Workflow Approaches

• Identifying where and when to introduce human oversight.
• Managing approval queues, scoring thresholds, and fallback handling.
• Calibrating trust and the role of explainability.

Secure LLM App Design Patterns

• Implementing least privilege and sandboxing for API calls and agents.
• Applying rate limiting, throttling, and abuse detection mechanisms.
• Ensuring robust chaining with LangChain and prompt isolation.

Compliance, Logging, and Governance

• Ensuring the auditability of LLM outputs.
• Maintaining traceability and controlling prompt/version management.
• Aligning with internal security policies and regulatory requirements.

Summary and Next Steps