The conversation around AI governance cannot stop at procurement, because, as this article explains well, outsourcing the technology does not outsource the liability. While the operational responsibilities of the system may be shared across the enterprise and vendors, leaders should consider that legal responsibility may not be divided in the same way. 

The contracts that govern these vendor relationships will be the first documents scrutinized when AI-driven decisions cause harm, so their provisions (i.e., indemnification, liability caps, and allocation-of-risk clauses) best be drafted with these new exposures in mind. What do your contracts say about who is accountable when AI is not used responsibly?  Does the liability sit with the vendor or the customer-facing enterprise?

In an area where the law is uncertain and trying to catch up to the ever-changing technology, it is my experience that the best thing clients can do is: 

(1) Understand the systems they are running and ensure their attorneys drafting their contracts understand it too; 

(2) Clearly map contractual responsibilities based on that understanding; and 

(3) Build controls within your organization that align with those responsibilities. 

The linked article is worth a read for anyone thinking about where AI risk sits.