On equivalence of algorithm's implementations

Rafał Biedrzycki

Abstract

When a new optimization algorithm is proposed, it is compared with state-of-the-art methods. That comparison is made using implementations of the algorithms, but names and versions of the implementations are usually not revealed. This paper compares five implementations of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) taken from a trusted source. The comparisons were performed using the Comparing Continuous Optimizers (COCO) platform. The results show that all examined implementations produce a different outcome. The variation of the results stems from differences in the auxiliary codes of the implementations and from implementing an algorithm which is still under development. It is therefore important to use an appropriate implementation for experiments. Using a weak implementation can lead to the wrong conclusions.
Author Rafał Biedrzycki (FEIT / IN)
Rafał Biedrzycki,,
- The Institute of Computer Science
Pages247-248
Publication size in sheets0.3
Book López-Ibáñez Manuel, Auger Anne, Stützle Thomas (eds.): GECCO '19 Proceedings of the Genetic and Evolutionary Computation Conference Companion, 2019, Association for Computing Machinery, ISBN 978-1-4503-6748-6, 2075 p.
Keywords in Englishexperiments replication, benchmarking, algorithm-implementation gap, CMA-ES
DOIDOI:10.1145/3319619.3322011
URL https://dl.acm.org/citation.cfm?id=3322011
Languageen angielski
File
rGecco.pdf 388.03 KB
Score (nominal)15
ScoreMinisterial score = 15.0, 20-07-2019, ChapterFromConference
Citation count*
Cite
Share Share

Get link to the record


* presented citation count is obtained through Internet information analysis and it is close to the number calculated by the Publish or Perish system.
Back