Representation Learning for Combinatorial Optimization

Forge: Foundational Optimization
Representations from Graph Embeddings

Representational learning for combinatorial optimization. Generate embeddings from MIP instances via pre-trained models, and fine-tune for downstream tasks such as predicting integrality gap, search guidance, backdoor prediction, and solver configuration.

Serdar Kadıoğlu^1,2 Zohair Shafi^2,3

¹ Department of Computer Science, Brown University ² AI Center of Excellence, Fidelity Investments ³ Northeastern University

TMLR'26 Code Slides 🤗 Models Dataset Demo

@misc{forge2026,
      title={FORGE: Foundational Optimization Representations from Graph Embeddings},
      author={Zohair Shafi and Serdar Kadioglu},
      journal={Transactions on Machine Learning Research},
      issn={2835-8856},
      year={2026},
      url={https://openreview.net/forum?id=7Uo1yRWUpo},
}

Overview

Forge (Foundational Optimization Representations from Graph Embeddings) is designed for representational learning in combinatorial optimization. It provides tools for:

Generating embeddings from Mixed Integer Programming (MIP) instances
Pre-training models using unsupervised reconstruction and commitment loss on bipartite constraint-variable graphs
Fine-tuning for specific downstream tasks such as predicting integral gap, search guidance, backdoor prediction, and solver configuration

Forge converts MIP instances into a bipartite graph representation (constraints ↔ variables), applies Graph Neural Networks (PyG SAGEConv) with Vector Quantization to learn compact, transferable embeddings that capture the structural properties of optimization problems.

How it Works

End-to-end workflow from MIP instances to embeddings and predictions.

Step 1

📐

MIP → MIPInfo

Parse MIP instances into feature tensors, edge indices, and edge weights using PyG COO format.

Step 2

🧠

Pre-Train

Unsupervised pre-training with graph reconstruction and VQ commitment loss on bipartite graphs.

Step 3

🎯

Fine-Tune

Fine-tune the pre-trained model for downstream tasks like integral gap prediction.

Step 4

📊

Embeddings

Generate instance, constraint, and variable-level embeddings for any MIP problem.

Models

Pre-trained and Fine-tuned models

Out-of-the-box models ready for generating instance/constraint/variable-level embeddings, and fine-tuned for downstream optimization tasks.

Zohair Shafi, Serdar Kadıoğlu

Pre-trained Models

🤗 Pre-trained MIP ModelPyG

Transfer Learning

Transfer Learning from Optimization to Satisfaction

Forge foundational optimization embeddings can transfer from optimization (MIP) to constraint satisfaction (SAT), as a step toward a unified representational framework for both optimization and decision problems.

Koyena Pal, Serdar Kadioglu

Paper Code

@misc{pal2026transferlearningfoundationaloptimization,
      title={Transfer Learning from Foundational Optimization Embeddings to Unsupervised SAT Representations},
      author={Koyena Pal and Serdar Kadioglu},
      year={2026},
      eprint={2604.15448},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2604.15448},
}

Citation

If you use FORGE in your research, please cite our paper.

@misc{forge2026,
      title={FORGE: Foundational Optimization Representations from Graph Embeddings},
      author={Zohair Shafi and Serdar Kadioglu},
      journal={Transactions on Machine Learning Research},
      issn={2835-8856},
      year={2026},
      url={https://openreview.net/forum?id=7Uo1yRWUpo},
}

Acknowledgments

We would like to thank Modal for their generous support through the provision of academic credits and computational infrastructure, which were instrumental in training the Forge model used in this research.