Monitoring is a Must
In October 2022, the Biden-Harris Administration released a blueprint for an “AI Bill of Rights”. This Bill of Rights is designed to ensure that the fundamental rights of Americans are protected while technology continues to advance. The framework will be used to inform policy-making.
We reviewed the framework and its five principles:
- Safe and Effective Systems
- Algorithmic Discrimination Protections
- Data Privacy
- Notice and Explanation
- Human Alternatives, Consideration and Fallback
While all of the details are interesting and important, here at DataPure we paid close attention to the Safe and Effective Systems principle and within that, the section called Ongoing Monitoring in particular because DataPure can support companies in their endeavor to meet the AI Bill of Rights requirements. See below.
"Ongoing monitoring. Automated systems should have ongoing monitoring procedures, including recalibration procedures, in place to ensure that their performance does not fall below an acceptable level over time, based on changing real-world conditions or deployment contexts, post-deployment modification, or unexpected conditions. This ongoing monitoring should include continuous evaluation of performance metrics and harm assessments, updates of any systems, and retraining of any machine learning models as necessary, as well as ensuring that fallback mechanisms are in place to allow reversion to a previously working system. Monitoring should take into account the performance of both technical system components (the algorithm as well as any hardware components, data inputs, etc.) and human operators. It should include mechanisms for testing the actual accuracy of any predictions or recommendations generated by a system, not just a human operator’s determination of their accuracy. Ongoing monitoring procedures should include manual, human-led monitoring as a check in the event there are shortcomings in automated monitoring systems. These monitoring procedures should be in place for the lifespan of the deployed automated system."
If you’re building an AI system and you want to stay in front of potential policy around this, reach out to us here at DataPure to discuss how we support teams just like yours 24 hours a day, 7 days a week, 365 days a year to ensure that your AI is performing as it was intended.
