Happy to share that our workshop paper “Do You Even Know What I Want? Improving LLMs Ability to Detect Hidden User Intent” was accepted into the International Conference on Automated Planning and Scheduling (ICAPS) as a workshop paper.
One well-documented tendency of LLMs is that they often default to answering user queries rather then ask questions about them. With this, a known but often overlooked cause of value misalignment is a human user misspecifying their goals to an agent based on incorrect beliefs about its current state/capabilities resulting in behavior they might not have expected.
Given such things, in this paper we explore a specific human-robot goal misspecification problem: whether LLMs can reason about goal misalignment and identify good questions which resolve it. Though I’ve worked with LLMs in the past, in this project it was fun to use them in more of an AI Safety/Alignment context to see to what extent this was possible. To find out, you’ll of course have to read the paper…but it’s an interesting one, I promise!
If you happen to be around, the paper will be presented as part of the HAXP Workshop in Dublin, Ireland on June 28th 🍀.