Security News > 2024 > March > PyRIT: Open-source framework to find risks in generative AI systems

Python Risk Identification Tool is Microsoft's open-source automation framework that enables security professionals and machine learning engineers to find risks in generative AI systems.

It started as a collection of individual scripts used during the team's initial foray into red teaming generative AI systems in 2022.

The tool should not be seen as a substitute for the manual red teaming of generative AI systems.

"The biggest advantage we have found so far using PyRIT is our efficiency gain. For instance, in one of our red teaming exercises on a Copilot system, we were able to pick a harm category, generate several thousand malicious prompts, and use PyRIT's scoring engine to evaluate the output from the Copilot system all in the matter of hours instead of weeks," wrote Ram Shankar Siva Kumar, Microsoft AI Red Team Lead. PyRIT enables researchers to refine and enhance their defenses against various harms.

It adapts its strategy based on the feedback from the generative AI system, creating subsequent inputs for the AI system.

"Red teaming GenAI is important because companies don't want their AI systems manipulated by bad actors to say or take actions that would harm the company. PyRIT solves a problem many people struggle with. It will be most helpful for teams with the bandwidth to learn and set up a new framework. This isn't a replacement for manual testing by human red teamers. Still, it's a way to automate some testing so you can quickly iterate on prompts and other configurations to find the balance of safety and utility," said Joseph Thacker, principal AI engineer and security researcher at AppOmni.

News URL

https://www.helpnetsecurity.com/2024/03/04/pyrit-microsoft-framework-find-risks-generative-ai-systems/