Scene-graph Construction Optimized by Plan Error-correction for Robot Long-term Planning Datasets
(IROS 2026)
Abstract
Task and Motion Planning (TAMP) stands as a fundamental challenge in robotics, essential for enabling autonomous agents to execute complex, long-horizon tasks in diverse environments. While Large Language Models (LLMs) introduce a new paradigm for TAMP by leveraging extensive common sense, their inherent reasoning limitations hinder direct application, and fine-tuning strategies to enhance these capabilities are currently bottlenecked by the scarcity of high- quality datasets. Existing automated generation methods often suffer from homogeneity, limiting model generalization, and unsolvability, where hallucinated scene graphs create physical mismatches with task goals. To address these issues, we propose Scene-graph Construction Optimized by Plan Error-correction (SCOPE), an automated framework designed to generate diverse and executable TAMP datasets. SCOPE incorporates multi- level diversity metrics to maximize the information entropy of scenarios. Crucially, it introduces a Plan Feedback Error- correction pipeline that utilizes planner execution failures as feedback signals to retroactively rectify logical inconsistencies in scene graphs, ensuring physical feasibility. Experimental results demonstrate that SCOPE exhibits greater robustness to erroneous scene graphs and generates higher-quality data. When utilized for LLM fine-tuning, SCOPE-generated data yield a downstream TAMP model that achieves an absolute 12.33% gain in success rate over its counterpart fine-tuned with data generated by DELTA, a state-of-the-art PDDL-based planner.
Video
Method Overview
Diverse scene graph generation. To mitigate the homogeneity of LLM-generated scenes, SCOPE combines statistical feedback with dynamic sampling. A room composition tracker estimates the empirical distribution of generated room types, and an entropy-aware inverse-frequency weighting strategy encourages exploration of underrepresented layouts. At the asset level, SCOPE injects restricted furniture and object subsets into prompts, while a lightweight DSL constructs scenes through a coarse-to-fine process from rooms, to furniture, to object placement.
Chain-of-Thought driven instruction generation. Given a scene graph, SCOPE generates semantically rich and difficulty-graded task instructions through a strict reasoning sequence: Target Sampling, Action Chain Construction, and Instruction Language Embodiment. The LLM first identifies interactive objects and spatial relations, then builds a causal action chain that leads to the target state, and finally maps the chain into natural language instructions. A set-matching verification step filters out instructions whose target objects or locations are not grounded in the scene.
Plan feedback error-correction. SCOPE decomposes each instruction into PDDL sub-goals and uses a classical planner as both solver and verifier. When planning fails, the failed action is traced back to missing preconditions or invalid scene states. A Symbolic Rule-LLM repair module injects the required attributes or initial states into the scene graph, then replans until executable sub-plans are obtained. This closed loop yields the final tuple <SGfinal, I, Π> for robust downstream fine-tuning.
Visualizing SCOPE Planning Across Task Difficulties in AI2-THOR
Complex task: Please get the egg from the fridge and put it in the microwave. Heat it until it is cooked, then transfer it to a plate.
Vague task: I'm about to go to sleep. Could you check the desk lamp and computer for me?
Complex task: Please get the apple from the fridge and put it on the countertop. Slice it using the knife on the counter, then leave the knife there.
SCOPE's Prompt Templates for Each Module
BibTeX
@article{YourPaperKey2024,
title={Your Paper Title Here},
author={Anonymous authors},
journal={Conference/Journal Name},
year={2024},
url={https://your-domain.com/your-project-page}
}