Effective Human-AI Teams via Learned Natural Language Rules and Onboarding


People are starting to rely on AI agents to assist them with various tasks and thus forming human-AI teams. The human must know when to rely on the agent, collaborate with the agent, or ignore its suggestions. In this work, we propose to learn rules described in natural language and grounded in data regions that illustrate how we should collaborate with the AI agent. We propose a novel region discovery algorithm \algname that finds neighborhoods in the data that best help the human collaborate with the AI. Each region is then described using an iterative procedure where a large language model describes the region while distinguishing it from the rest of the data. We then teach these rules to the human via an onboarding stage and show the recommendations of the rules as part of an AI dashboard. Through user studies on object detection and question-answering tasks, we showcase that our method can sometimes lead to more accurate human-AI teams. We also evaluate our region discovery and description algorithm in synthetic setups on multiple datasets.

Advances in neural information processing systems (NeurIPS)
Hussein Mozannar
Hussein Mozannar
PhD Student

Hussein’s interests focus on human-centric aspects of machine learning, namely how to integrate expert decision makers into machine learning pipelines while ensuring fairness and an understanding of long-term consequences.

David Sontag
David Sontag
Professor of EECS

My research focuses on advancing machine learning and artificial intelligence, and using these to transform health care.