[EN] How to Create Useful Graphs and Submit Solutions
Submission of solutions
Evolutionary algorithms are based on randomness, i.e. each run can have a slightly different outcome. Therefore, it is not possible to compare the results of only one run. Instead, we need to repeat the run a number of times and compare the performance in the average/best/worst case.
The submitted solution must contain the following:
- A short description of what you did and tried, what was the outcome. Around 5-10 sentences are enough.
- A plot showing the optimized criteria as a function of fitness evaluations. The plot must show the average (or median) value and first and third quartile. If you compare more algorithms, they all should be in a single plot.
- Do not zip the plots (unless there is a lot of them}. Upload them as pictures.
Scripts to create the plots
To create the plot with the fitness you can use the scripts
createGraphs.ps1 (for Windows PowerShell), or
createGraphs.sh(for Unix). Both scripts require to have
gnuplot installed and in
The scripts expect as an input the log of the performance of the algorithm. Each line in the file contains six numbers - the number of fitness evaluation up to this point, and 5 numbers describing the statistical properties of the fitness of the best individual in each of a number of runs - minimum, first quartile, average, third quartile, and maximum. Such files are produced by the available source codes, or you can produce them yourself (if you prefer programming in other languages and want to use the scripts to make the graphs).
Using the scripts is quite easy. They require only two parameters, which indicate where are their inputs (the outputs of the evolutionary algorithm) a how they should be named in the plot. Assume, we have outputs of two algorithms in files
logs/better.objective_stats. To compare these algorithms, we have to run
createGraphs.ps1 -logFileNames basic,better -legendNames Basic,Better
This yields the comparison of the two algorithms in one plot named
output.svg, the names in the legend will read “Basic” and “Better”.
This script has more useful parameters:
outputspecifies the name of the output file (default
titlesets the title of the plot (default “Objective value log”)
logScalespecifies, which axis use logarithmic scale, possible values are
y(default - both axis use linear scale)
pathspecifies the directory with the outputs (default
scalesets the scaling of the horizontal axis, e.g. if
-scale 1000is used, all the numbers are divided by 1000 (default 1)
barsEverysets the frequency of the error bars, as the number of lines in the input file (default 20)
limitis the upper limit of the horizontal axis (default
“”- no limit)
These parameters can be used to make the resulting plot more readable. Use them and experiment with them. Try to make the plots as readable as possible.
A Python script is also available to create the graphs. It does not need
gnuplot, but uses
matplotlib instead. The parameters of the script are similar to those of the scripts described above. Check them in the source code.
The following hints may help you to make better graphs
- If both algorithms converge long before they are stopped, the plot contains long almost horizontal lines and most information is contained on the left side -> use a suitable value of the
limitparameter to ignore the right part of the plot
- If one of the algorithms converges much faster than the other, use
logScale xyto emphasize the left part of the plot.
- If there is a large difference in the objective values in the beginning of the evolution and in the end, it can lead to almost vertical lines in the plot, use
logScale xyto make the small differences in the end more pronounced
- If error bars are too close to each other, use a larger value of the
barsEveryparameter (and vice versa)