大语言模型与交互式智能体：开放世界中的动态推理与规划.pdf

1、DataFunSummit#2023大语言模型与交互式智能体：开放世界中的动态推理与规划林禹臣-Allen Institute for AI-研究员Textual EnvironmentReal-world Situations for AgentsTask planning&execution interactive environmentALF Worldsample task:put A in Bmostly easy&shortaction space:very limitedScienceWorldComplex Setup-10 locations-25 action types-

2、200+object types-multiple states-random exceptions-30 high-level task typeshttps:/yuchenlin.xyz/swiftsage/FormulationBaseline methods:Reinforcement Learning DRRN:deep reinforcement relevance networkDRRN:deep reinforcement relevance networkAction State value for action i under state s at the time tKG

3、-A2C:add a dynamic graph to constrain the selection of actions+objects CALM:use a larger LM say GPT-2 to re-rank the action candidates after Q function is computed.Baseline methods:Imitation LearningBehavior Cloning with Transformer LMsBehavior Cloning with Transformer LMsDecision Transformer w/Caus

4、al Transformers Text Decision Transformers(Behavior Cloning)in the ScienceWorld paper Action History+Observations +Env t,t-1action t+1Oracle agent On training example tasks,I can search&generate golden paths for completing the tasks.Offline Training Data(in seq2seq mode)SayCanReflexionTaskTask:Your

5、task is to boil tin.For Each Timestep t:Action 1:go to kitchen you moved to kitchenAction 2:look around In this kitchen,you can see.Actiont-1:pick up metal pot metal pot in inventory nowDemoDemo:An oracle path for the task of boiling water.Kbest generations for Action tAction t:put metal pot on stov

6、eRerankingTaskTask:Your task is to boil tin.+the of previously failed trials in the last round.Action historyAction history(you moved to kitchenAction 2:look around In this kitchen,you can see.Action t-2:thinkthink:now I need to place the metal pot on a heater