github.com/AidinHamedi/Optimizer-Benchmark ↗

About AidinHamedi/Optimizer-Benchmark

Optimizer Benchmark is a Python framework for systematically evaluating and comparing PyTorch optimization algorithms using 2D mathematical test functions as a standardized benchmark suite. The project leverages the pytorch_optimizer library to test various optimizers and uses Optuna for automated hyperparameter tuning, generating detailed trajectory visualizations and performance rankings for each optimizer-function combination.

The benchmark includes twelve standard mathematical test functions such as Ackley, Rastrigin, Rosenbrock, and Griewank, plus custom functions like Gradient Labyrinth and Neural Canyon. Results are compiled into rankings and interactive visualizations accessible through a project website, with all outputs configurable through a TOML configuration file. The entire workflow is automated, allowing users to run benchmarks locally and generate comprehensive performance reports saved to a results directory.

The project explicitly acknowledges an important limitation: results from 2D synthetic functions may not reflect real-world performance when training actual neural networks, so the rankings should be treated as reference material rather than definitive guidance for production use. This makes it a useful educational and comparative tool for optimizer researchers and developers interested in understanding algorithm behavior on simplified test landscapes.