GFlowNets learn their own generation order: four of five runs converge on checksum tasks where a uniform backward policy never does
Synopsis
GFlowNets build discrete objects one component at a time, and the order determines which conditional distributions the network must learn. Previously the order was fixed by a preset or uniform backward policy, and standard training struggled to find useful orders. Learning-Order GFlowNet learns positions and symbols jointly without order labels, converging in four of five runs on checksum dependencies, against zero for a uniform backward policy and two of ten for fixed random orders.
Interpretation
Generation order becomes something trained rather than fixed before training. A GFlowNet fills a discrete object component by component, and the backward policy decides the erasure order, which fixes the forward generation order; that order was usually set to uniform erasure or a predefined sequence, or learned jointly with the forward policy without characterising what order emerged. Earlier work had not systematically tested whether existing GFlowNet algorithms discover interpretable generation orders, nor explained why such orders appear. On purpose-built checksum-dependency strings, fixed orders converge less often as the number of swaps grows, a uniformly sampled fixed random order converges in two of ten runs, and a uniform backward policy converges in none within the training budget.
The training recipe lets an order emerge on its own, in a form that matches the problem's dependency structure. It factorises the forward policy into a position head and a symbol head, penalises backward-policy entropy to favour deterministic erasure, zero-initialises the final layers of both order heads and lowers their learning rates, and can branch several continuations from a shared prefix. On checksum dependencies the full recipe converges in four of five runs at about 72,900 iterations on average; removing backward entropy regularisation leaves one run, removing policy factorisation or the reduced order-head learning rate leaves two, and default initialisation leaves three at about 88,650 iterations. Learned orders are locally consecutive on Markov-chain strings and prefix-early on checksum tasks.
Gradient signal-to-noise ratio is offered as an explanation for why good orders win out. It was previously unclear why training departs from random order, since a GFlowNet can occasionally converge even under a suboptimal order. At comparable validation loss, converged Learning-Order GFlowNet runs reach gradient SNR comparable to the identity order, and both exceed fixed orders with more swaps; the authors therefore characterise a good order as one giving higher gradient SNR at matched loss.
Perspective
The demonstrated scope is the authors' synthetic tasks: Markov-chain strings, string and directed-acyclic-graph checksum dependencies, and conditional Sudoku at two simplification levels; string experiments use length 12 and alphabet size 12, and graph experiments train on 4,000 graphs and evaluate on 500 unseen graphs. The authors propose extending the work to controlled problems where the optimal order is state-dependent, and note no convergence advantage on Sudoku.
The authors present gradient SNR as a possible explanation for order emergence, not an established causal account; on Sudoku the learned position policy does not match the convergence advantage of a solver-guided order, and confidence-based cell selection improves accuracy without the model learning to exploit that information. The constructed problems also have order preferences that do not vary with state, which may differ from real settings.