AI Red Teaming: Hands-on Demo
RTP Artificial Intelligence Group Think your LLM application is secure and reliable? Let's put it to the test! Join us for a live hands-on demo and retrospective on AI Red Teaming, where we'll explore how to identify vulnerabilities, stress-test LLM applications, and strengthen their security using open-source tools like Giskard and Garak. In this session, you'll learn:
How to perform red teaming on an LLM application from scratch. How to use Giskard and Garak to stress-test LLM applications. Why you can't always blindly trust an LLM judge when evaluating model outputs. How to harden prompts against role-play exploits and potential vulnerabilities. How to automate local regression testing to improve the reliability and security of LLM applications. Practical lessons learned and challenges encountered while testing and securing an LLM-powered bot.
This is a practical, hands-on session featuring a live demonstration and real-world lessons learned. You'll gain insights into identifying LLM vulnerabilities, improving prompt security, and building more robust AI applications. š¹ Format: Live Hands-on Demo + Retrospective + Q&A š¹ Presenter: Sai Thummala š¹ Tools Covered: Giskard and Garak š¹ Who's it for: AI engineers, developers, ML engineers, security enthusiasts, and anyone interested in building secure and reliable LLM applications. š¹ Takeaway: A practical understanding of LLM red teaming, prompt hardening, vulnerability testing, and automated regression testing using open-source tools.