On equivalence of algorithm's implementations

Rafał Biedrzycki


When a new optimization algorithm is proposed, it is compared with state-of-the-art methods. That comparison is made using implementations of the algorithms, but names and versions of the implementations are usually not revealed. This paper compares five implementations of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) taken from a trusted source. The comparisons were performed using the Comparing Continuous Optimizers (COCO) platform. The results show that all examined implementations produce a different outcome. The variation of the results stems from differences in the auxiliary codes of the implementations and from implementing an algorithm which is still under development. It is therefore important to use an appropriate implementation for experiments. Using a weak implementation can lead to the wrong conclusions.
Author Rafał Biedrzycki (FEIT / IN)
Rafał Biedrzycki,,
- The Institute of Computer Science
Publication size in sheets0.3
Book López-Ibáñez Manuel, Auger Anne, Stützle Thomas (eds.): GECCO '19 Proceedings of the Genetic and Evolutionary Computation Conference Companion, 2019, Association for Computing Machinery, ISBN 978-1-4503-6748-6, 2075 p.
Keywords in Englishexperiments replication, benchmarking, algorithm-implementation gap, CMA-ES
URL https://dl.acm.org/citation.cfm?id=3322011
Languageen angielski
rGecco.pdf 388.03 KB
Score (nominal)140
Score sourceconferenceList
ScoreMinisterial score = 140.0, 17-06-2020, ChapterFromConference
Publication indicators Scopus Citations = 0
Citation count*
Share Share

Get link to the record

* presented citation count is obtained through Internet information analysis and it is close to the number calculated by the Publish or Perish system.
Are you sure?