On equivalence of algorithm's implementations
AbstractWhen a new optimization algorithm is proposed, it is compared with state-of-the-art methods. That comparison is made using implementations of the algorithms, but names and versions of the implementations are usually not revealed. This paper compares five implementations of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) taken from a trusted source. The comparisons were performed using the Comparing Continuous Optimizers (COCO) platform. The results show that all examined implementations produce a different outcome. The variation of the results stems from differences in the auxiliary codes of the implementations and from implementing an algorithm which is still under development. It is therefore important to use an appropriate implementation for experiments. Using a weak implementation can lead to the wrong conclusions.
|Publication size in sheets||0.3|
|Book||López-Ibáñez Manuel, Auger Anne, Stützle Thomas (eds.): GECCO '19 Proceedings of the Genetic and Evolutionary Computation Conference Companion, 2019, Association for Computing Machinery, ISBN 978-1-4503-6748-6, 2075 p.|
|Keywords in English||experiments replication, benchmarking, algorithm-implementation gap, CMA-ES|
|Score||= 15.0, 20-07-2019, ChapterFromConference|
* presented citation count is obtained through Internet information analysis and it is close to the number calculated by the Publish or Perish system.