How much do we need to know to trust an AI answer?

We have adopted Gen AI at extraordinary speed, without the training, shared norms and institutional practices that normally accompany a technology capable of changing how people work and learn. AI can improve our performance without necessarily improving our capabilities. For much of our working lives, we have assumed that simply performing a task is itself a form of training. Even when the work is imperfect, the process creates knowledge we can apply next time.

Gen AI changes that relationship. A person can now produce a sophisticated answer without necessarily understanding the reasoning that produced it.

This is where another concept from Littman’s field becomes relevant: the distinction between exploration and exploitation. AI must decide whether to use what it already knows to maximize an immediate result or explore an uncertain possibility that might provide information useful for future decisions. Humans face the same tradeoff. Exploration is how we learn, while exploitation allows us to apply what we already know.  Sreedhar Potarazu and Carin Isabel Knoop, The American Bazaar