Craig Martell, CTO at Cohesity

October 16, 2024

Presented by Cohesity

Balancing AI Implementation and Compliance in Federal Agencies

Craig Martell, Chief Technology Officer (CTO) at Cohesity and former Chief Digital and AI Officer at the DoD, offers insight into the challenges federal agencies face in merging compliance requirements with AI implementation. He shares the difficulty of balancing compliance and execution, particularly in AI systems where getting data right and performing accurate evaluations are crucial. Martell emphasizes that AI, being probabilistic by nature, is guaranteed to make errors. However, the acceptable margin of error varies based on the use case. In less critical applications like online advertising errors are tolerable, but in high-stakes domains such as telemedicine or autonomous vehicles, even a 3% error rate can be unacceptable. He advocates for a tailored risk management approach that aligns with the specific requirements of each AI use case. Additionally, Martell stresses the importance of data resilience and permissions management. Backing up data effectively and ensuring it complies with organizational access rules is key to both operational success and regulatory compliance. 

Key Takeaways:

1. Agencies must focus on breaking data silos, managing data permissions, and ensuring resilience to implement effective AI strategies.
   
   
2. AI errors are inevitable, but managing risk means ensuring errors are acceptable based on the use case, especially for high-stakes applications.
   
   
3. AI evaluation has become more complex with generative models, making it crucial to understand when AI gets it wrong and how to correct it effectively.