In a groundbreaking study, scientists have introduced a novel approach to offline black-box optimization (BBO) by utilizing diffusion large language models (LLMs). This method addresses the challenge of finding optimal designs, such as DNA sequences or robotic configurations, when only limited labeled data is available. The research, led by Ye Yuan, Can Chen, Zipeng Sun, and their team, demonstrates how diffusion LLMs overcome traditional optimization limitations and deliver state-of-the-art performance, particularly in data-scarce environments.
Traditional optimization techniques often rely on task-specific proxy models or generative models, which struggle to capture complex dependencies across design spaces. Diffusion LLMs, however, leverage bidirectional modeling and iterative refinement to significantly improve the optimization process. By utilizing an in-context denoising module, the team effectively conditions the LLM on task descriptions and offline datasets, enhancing design generation from natural language prompts.
A New Approach to Offline Optimization with Diffusion LLMs
The key innovation in this research is the in-context denoising module. Researchers formatted both task descriptions and offline data as natural language prompts, allowing the diffusion LLM to iteratively refine and denoise masked designs. This method allows for more complex design problem-solving by capturing bidirectional dependencies, which left-to-right autoregressive models cannot manage effectively. By iteratively refining the designs, the model is able to generate improved candidates based on a minimal set of labeled data.
Additionally, the research team developed a masked diffusion tree search, a Monte Carlo Tree Search that dynamically balances exploration and exploitation. This innovative search method allows the model to efficiently navigate design spaces by prioritizing high-performing candidates while still exploring less-explored regions. The process evaluates candidates using expected improvement, leveraging a Gaussian Process trained on the offline dataset to guide the search.
State-of-the-Art Results in Few-Shot Optimization
The team’s method, dubbed dLLM, has achieved remarkable success in few-shot optimization, setting new benchmarks in the design-bench platform. By combining diffusion LLMs with masked diffusion tree search, the approach successfully tackles optimization problems in various fields, including DNA sequence design and robotics. This breakthrough demonstrates the power of LLMs to learn from limited offline data and generate high-performing solutions without requiring costly online evaluations.
The integration of diffusion LLMs with masked tree search marks a significant advancement in offline optimization. The ability to capture complex dependencies and dynamically search the design space opens up new possibilities in fields where labeled data is scarce, making this approach highly applicable in real-world scientific and engineering challenges.
