Pretrain on Small Synthetic Data, Scale Large for Free: Symmetry-Aware Foundation Model for Logic Rule Induction

Phua, Yin Jun

doi:10.48550/arXiv.2608.00383

How G-NRI works

Reordering examples or renaming variables should not change the underlying inference problem. G-NRI incorporates these symmetries, together with polarity flips and label swaps, into rule induction.

The method extends NRI through encoder changes, symmetrized training and evaluation, and canonical rule export. It retains the same model size and examples-to-rules interface.

Encoder symmetries

The encoder is modified to respect variable-renaming and polarity symmetries, without increasing the number of learned parameters.

Canonical rule export

The export procedure preserves score symmetries when converting model outputs into a discrete logical rule.

Label symmetry

Label-swap averaging and a symmetrized training objective account for both label encodings in binary classification.

Examples pass through a symmetry-aware NRI encoder and canonical rule export. Transforming the examples produces the corresponding transformation of the output rule. — The five changes from NRI to G-NRI. From Figure 1 of the paper; open full-size diagram.

Variable-count scaling

We train on synthetic logic problems with 6–12 variables and evaluate the same frozen checkpoints on problems with up to 1,024 variables. The plots report accuracy on the conditioning examples and rule fidelity on held-out inputs.

Accuracy on the conditioning examples as variable count grows. At 1,024 variables, G-NRI reaches 89.4%, compared with 48.7% for the NRI baseline. — **Support-set accuracy.** At 1,024 variables, G-NRI achieves 89.4%, compared with 48.7% for the NRI baseline.

The exported rule's agreement with the target rule on fresh inputs. G-NRI stays above the NRI baseline at 32, 128, and 1,024 variables. — **Held-out rule fidelity.** Agreement between the exported rule and the target rule on fresh inputs.

G-NRI NRI baseline Architecture changes only

Panels (a) and (b) of Figure 2, across eight training seeds. The dotted line marks the training maximum of 12 variables. These experiments increase variable count within the same rule family. View the full figure.

Benchmark results

Across 19 benchmark datasets, mean held-out accuracy is 76.2% for G-NRI and 70.2% for NRI, using frozen weights for both models. The datasets include tabular classification, MONK's logic tasks, molecular features, and symbolic scene descriptions.

Example rule

¬body_shape_octagon ∧ ¬jacket_blue

On MONK's-3, G-NRI returns this same two-literal rule across all eight seeds.

Results on 19 datasets

Held-out accuracy (%), averaged over eight seeds. NRI and G-NRI use frozen weights; the decision tree is trained per dataset. Results from Table 4 of the G-NRI paper.
Dataset	Variables	NRI	G-NRI	Decision tree
adult	105	65.6	64.4	81.5
breast-cancer	9	92.7	92.0	93.7
car	21	30.4	73.8	96.7
credit	46	70.7	80.7	80.6
diabetes	8	72.0	71.8	70.0
german	61	58.4	60.9	65.3
hepatitis	32	80.6	80.0	77.7
ionosphere	34	71.9	73.1	79.4
kr-vs-kp	73	66.8	69.9	99.6
mushroom	116	78.0	81.0	100.0
nursery	27	68.4	75.9	98.7
spambase	57	71.0	79.0	90.7
tic-tac-toe	27	60.1	69.9	93.1
vote	32	91.3	94.3	94.5
monks-1	17	74.1	74.6	98.4
monks-2	17	61.1	55.9	98.5
monks-3	17	90.3	96.4	96.8
mutag	51	63.9	73.1	84.6
clevr-hans3	105	66.4	81.5	100.0
Mean (19)	—	70.2	76.2	89.5

Code and reproducibility

The G-NRI repository includes synthetic training, rule export, benchmark evaluation, and scripts for the paper's tables and figures. Instructions are available for CPU training and experiment reproduction.

For the original model and its results, visit the NRI project page.

BibTeX

TBD — will be updated once the official NeSy 2026 proceedings are released and registered.