Experiment 20231220-MTOA

Experiment design

20231220-MTOA

Date: 2023-12-20 (Andreas Kalaitzakis)

Hypotheses

Favoring the reproduction of agents with the lowest success rate (agents that agree less with their peers) will allow agent populations to equally explore all tasks. Hence, agent populations will become equally accurate on all tasks and thus improve their efficiency.

Experimental setting

36 agents; 3 tasks; 6 features (2 independent features per task); 4 decision classes; 10 runs; 80000 games; 20000 games per generation

Each agent initially trains on all tasks. The agents then carry all tasks. When the agents disagree the following take place:

(a) The agent with the lower score will adapt its knowledge accordingly. If memory capacity limit is attained, the agent will try to forget knowledge.

(b) The agent with the highest score will decide for both agents. If this agent's decision is correct, the agent will receive the points corresponding to both agents.

Every 20000 games, 9 new agents are born and 9 agents are removed. Each new agent will be trained with examples provided by the two parents.

The first parent is selected either randomly, or based on their collected points/success rate (low success rate, high success rate, low points, high points). The second parent is selected based on its success rate with respect to the first parent (parent 2 is the the agent that agrees more often with parent 1).

Agents undertake 3 tasks having a limited memory, enough for learning 1/3 tasks accurately (4 and 12 classes respectively)

Variables independent variables: ['birthCriteria','birthSortingOrder','knowledgeLimit','parentSelectionPolicy']

dependent variables: ['avg_min_accuracy', 'avg_accuracy', 'avg_max_accuracy', 'success_rate', 'correct_decision_rate', 'delegation_rate', 'accuracy_relative_standard_deviation', 'total_points', 'points_distribution', 'games_relative_standard_deviation']

Experiment

Date: 2023-12-20 (Andreas Kalaitzakis)

Computer: Dell Precision-5540 (CC: 12 * Intel(R) Core(TM) i7-9850H CPU @ 2.60GHz with 16GB RAM OS: Linux 5.4.0-92-generic)

Duration : 1080 minutes

Lazy lavender hash: ceb1c5d1ca8109373d293b687fc55953fce5241d

Parameter file: params.sh

Executed command (script.sh):

#!/bin/bash

. params.sh

CURRDIR=$(pwd)
OUTPUT=${CURRDIR}/${DIRPREF}
# cd ${LLPATH}
cd lazylav
# this sample runs ExperimentalPlan. It can be replaced with Monitor if parameters are not varied.
bash scripts/runexp.sh -p ${CURRDIR} -d ${DIRPREF} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.ExperimentalPlan -Dexperiment=fr.inria.exmo.lazylavender.decisiontaking.multitask.SelectiveAcceptanceSpecializationExperiment ${OPT} -DresultDir=${OUTPUT}

Analysis

Raw data

Full results can be found at: Zenodo DOI



Table 1: Final success rate values

Table 1 consists of the final achieved average success rate values, i.e., the average success rate after the last iteration.

Out[43]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.35060 0.35825 0.37915 0.41480 0.36525 0.36045 0.36640 0.41820 0.35610 0.39245
1 0.35085 0.34850 0.36850 0.36455 0.30640 0.36660 0.42720 0.40285 0.35840 0.37540
2 0.41825 0.34250 0.39315 0.38065 0.38570 0.35640 0.33190 0.36890 0.46100 0.37470
3 0.38080 0.39330 0.35625 0.37630 0.38960 0.36835 0.37710 0.40145 0.40775 0.40585
4 0.32200 0.36565 0.41225 0.36330 0.34715 0.35205 0.44745 0.37865 0.41400 0.43225
5 0.35585 0.36200 0.34815 0.41105 0.34810 0.37805 0.42945 0.38270 0.39700 0.39190
6 0.43565 0.36345 0.34915 0.37885 0.34875 0.34635 0.36420 0.41905 0.38945 0.38210
7 0.33955 0.36545 0.31995 0.37525 0.37855 0.35965 0.43590 0.37090 0.36340 0.40380
8 0.36045 0.36830 0.35445 0.41140 0.39820 0.36755 0.50410 0.38950 0.40755 0.36945
9 0.40925 0.36765 0.36860 0.37360 0.36340 0.34735 0.38655 0.40085 0.39635 0.36675

Table 2: Final average minimum accuracy values

Table 2 consists of the final average minimum accuracy values with respect to the worst task, i.e., the accuracy after the last iteration for the task for which the agent scores the lowest accuracies.

Out[44]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-COMPENSATION-RANDOM-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-RANDOM-0.075 ASCENDING-SUCCESS_RATE-RANDOM-0.225 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-RANDOM-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-RANDOM-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225 ASCENDING-COMPENSATION-RANDOM-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225
0 0.354167 0.341580 0.360243 0.250000 0.265625 0.353733 0.253472 0.260417 0.263021 0.256076 0.360677 0.335069 0.269097 0.350694 0.245660 0.335503
1 0.366753 0.374132 0.345920 0.260417 0.243056 0.344184 0.247396 0.254340 0.243056 0.259549 0.329861 0.331163 0.262153 0.374132 0.250868 0.329861
2 0.336372 0.344184 0.347656 0.264757 0.250000 0.346354 0.271701 0.262153 0.250000 0.262153 0.348524 0.361979 0.265625 0.356337 0.263889 0.342014
3 0.349392 0.358941 0.349826 0.241319 0.259549 0.343316 0.248264 0.263021 0.254340 0.252604 0.360243 0.355035 0.250868 0.333767 0.248264 0.356771
4 0.371528 0.367188 0.357205 0.245660 0.260417 0.357205 0.267361 0.253472 0.244792 0.256076 0.358941 0.368056 0.267361 0.309896 0.261285 0.342014
5 0.344618 0.316840 0.336806 0.256076 0.266493 0.360677 0.262153 0.263889 0.269097 0.244792 0.351997 0.329427 0.261285 0.327257 0.255208 0.340278
6 0.340712 0.357639 0.351128 0.256944 0.255208 0.343750 0.266493 0.263021 0.274306 0.250000 0.361545 0.368056 0.267361 0.353733 0.269965 0.360677
7 0.361979 0.331163 0.353299 0.254340 0.272569 0.361111 0.230903 0.264757 0.261285 0.256076 0.356771 0.340712 0.256076 0.340712 0.243056 0.358073
8 0.341580 0.341146 0.341580 0.264757 0.247396 0.346788 0.270833 0.250000 0.244792 0.250000 0.380642 0.358941 0.250000 0.349826 0.261285 0.357639
9 0.338108 0.368924 0.371528 0.262153 0.265625 0.366753 0.263889 0.258681 0.260417 0.254340 0.325521 0.353733 0.261285 0.362413 0.258681 0.352431
Out[8]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.253472 0.354167 0.256076 0.341580 0.250000 0.335503 0.260417 0.335069 0.269097 0.350694
1 0.247396 0.366753 0.259549 0.374132 0.260417 0.329861 0.254340 0.331163 0.262153 0.374132
2 0.271701 0.336372 0.262153 0.344184 0.264757 0.342014 0.262153 0.361979 0.265625 0.356337
3 0.248264 0.349392 0.252604 0.358941 0.241319 0.356771 0.263021 0.355035 0.250868 0.333767
4 0.267361 0.371528 0.256076 0.367188 0.245660 0.342014 0.253472 0.368056 0.267361 0.309896
5 0.262153 0.344618 0.244792 0.316840 0.256076 0.340278 0.263889 0.329427 0.261285 0.327257
6 0.266493 0.340712 0.250000 0.357639 0.256944 0.360677 0.263021 0.368056 0.267361 0.353733
7 0.230903 0.361979 0.256076 0.331163 0.254340 0.358073 0.264757 0.340712 0.256076 0.340712
8 0.270833 0.341580 0.250000 0.341146 0.264757 0.357639 0.250000 0.358941 0.250000 0.349826
9 0.263889 0.338108 0.254340 0.368924 0.262153 0.352431 0.258681 0.353733 0.261285 0.362413

Table 3: Final average accuracy values

Table 3 consists of the final achieved average ontology accuracy with respect to all tasks, i.e., the accuracy after the last iteration averaged on all tasks and agents.

Out[45]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.393519 0.495804 0.410590 0.554977 0.379919 0.507957 0.433160 0.539352 0.406250 0.550926
1 0.393519 0.506221 0.429398 0.526186 0.398148 0.507668 0.399884 0.537326 0.415799 0.529659
2 0.410590 0.487413 0.400463 0.528501 0.394387 0.496383 0.410880 0.529369 0.408565 0.534288
3 0.381944 0.529080 0.433738 0.542969 0.396412 0.525029 0.405671 0.532697 0.381655 0.550058
4 0.410301 0.489439 0.406829 0.514323 0.390336 0.504051 0.438947 0.526765 0.407407 0.556134
5 0.386285 0.502894 0.419560 0.540075 0.414063 0.532841 0.423611 0.513600 0.446470 0.553530
6 0.417245 0.504196 0.395544 0.547598 0.411748 0.485532 0.430556 0.568287 0.420428 0.535301
7 0.369502 0.536169 0.429687 0.501736 0.398148 0.523293 0.437211 0.510417 0.426794 0.525897
8 0.410590 0.515914 0.420718 0.559462 0.404514 0.501302 0.457755 0.541377 0.443287 0.524306
9 0.393519 0.490596 0.440394 0.523003 0.398727 0.491175 0.431424 0.533565 0.408854 0.530382

Table 4: Final average best task accuracy values

Table 4 consists of the final average best task accuracy values with respect to the best task, i.e., the accuracy after the last iteration for the task for which the agent score the highest accuracies.

Out[46]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-COMPENSATION-RANDOM-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-RANDOM-0.075 ASCENDING-SUCCESS_RATE-RANDOM-0.225 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-RANDOM-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-RANDOM-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225 ASCENDING-COMPENSATION-RANDOM-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225
0 0.674045 0.798611 0.733073 0.551215 0.645833 0.670573 0.565104 0.663194 0.609375 0.600694 0.786892 0.808160 0.576389 0.796875 0.576389 0.704861
1 0.694010 0.718316 0.659288 0.584201 0.559896 0.689670 0.567708 0.591146 0.681424 0.680556 0.704427 0.786024 0.640625 0.704861 0.609375 0.710938
2 0.661024 0.758247 0.710503 0.572049 0.586806 0.693142 0.600694 0.619792 0.651042 0.576389 0.717014 0.736111 0.581597 0.730035 0.613715 0.667101
3 0.725694 0.779080 0.646701 0.618924 0.628472 0.669705 0.551215 0.592014 0.701389 0.670139 0.695747 0.748264 0.559896 0.794705 0.539062 0.741753
4 0.623698 0.693142 0.654514 0.563368 0.594618 0.678385 0.576389 0.718750 0.590278 0.605903 0.750868 0.706597 0.564236 0.819878 0.578125 0.679688
5 0.691840 0.804688 0.704427 0.623264 0.553819 0.647135 0.546875 0.633681 0.635417 0.647569 0.783420 0.723524 0.708333 0.796007 0.556424 0.756076
6 0.700955 0.758247 0.671441 0.625000 0.576389 0.661892 0.600694 0.633681 0.670139 0.597222 0.735243 0.803819 0.641493 0.724392 0.563368 0.634115
7 0.738281 0.718316 0.661024 0.580729 0.563368 0.764757 0.539062 0.684028 0.710069 0.689236 0.775174 0.708333 0.684896 0.722656 0.543403 0.682292
8 0.717014 0.797743 0.660590 0.595486 0.564236 0.690972 0.568576 0.791667 0.671875 0.657986 0.733941 0.764323 0.704861 0.717014 0.616319 0.670139
9 0.682726 0.697917 0.677517 0.578125 0.565972 0.733941 0.545139 0.671007 0.657986 0.704861 0.776476 0.768663 0.593750 0.729601 0.586806 0.650174
Out[11]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.565104 0.674045 0.600694 0.798611 0.551215 0.704861 0.663194 0.808160 0.576389 0.796875
1 0.567708 0.694010 0.680556 0.718316 0.584201 0.710938 0.591146 0.786024 0.640625 0.704861
2 0.600694 0.661024 0.576389 0.758247 0.572049 0.667101 0.619792 0.736111 0.581597 0.730035
3 0.551215 0.725694 0.670139 0.779080 0.618924 0.741753 0.592014 0.748264 0.559896 0.794705
4 0.576389 0.623698 0.605903 0.693142 0.563368 0.679688 0.718750 0.706597 0.564236 0.819878
5 0.546875 0.691840 0.647569 0.804688 0.623264 0.756076 0.633681 0.723524 0.708333 0.796007
6 0.600694 0.700955 0.597222 0.758247 0.625000 0.634115 0.633681 0.803819 0.641493 0.724392
7 0.539062 0.738281 0.689236 0.718316 0.580729 0.682292 0.684028 0.708333 0.684896 0.722656
8 0.568576 0.717014 0.657986 0.797743 0.595486 0.670139 0.791667 0.764323 0.704861 0.717014
9 0.545139 0.682726 0.704861 0.697917 0.578125 0.650174 0.671007 0.768663 0.593750 0.729601

Table 5: Final average scope accuracy values

Table 5 consists of the final average scope accuracy values.

Out[47]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.393519 0.495804 0.410590 0.554977 0.379919 0.507957 0.433160 0.539352 0.406250 0.550926
1 0.393519 0.506221 0.429398 0.526186 0.398148 0.507668 0.399884 0.537326 0.415799 0.529659
2 0.410590 0.487413 0.400463 0.528501 0.394387 0.496383 0.410880 0.529369 0.408565 0.534288
3 0.381944 0.529080 0.433738 0.542969 0.396412 0.525029 0.405671 0.532697 0.381655 0.550058
4 0.410301 0.489439 0.406829 0.514323 0.390336 0.504051 0.438947 0.526765 0.407407 0.556134
5 0.386285 0.502894 0.419560 0.540075 0.414063 0.532841 0.423611 0.513600 0.446470 0.553530
6 0.417245 0.504196 0.395544 0.547598 0.411748 0.485532 0.430556 0.568287 0.420428 0.535301
7 0.369502 0.536169 0.429687 0.501736 0.398148 0.523293 0.437211 0.510417 0.426794 0.525897
8 0.410590 0.515914 0.420718 0.559462 0.404514 0.501302 0.457755 0.541377 0.443287 0.524306
9 0.393519 0.490596 0.440394 0.523003 0.398727 0.491175 0.431424 0.533565 0.408854 0.530382

Table 6: Final correct decision rate values

Table 6 consists of the final achieved correct decision rate values, i.e., the average correct decision rate after the last iteration.

Out[48]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.46275 0.57205 0.48480 0.62170 0.45885 0.58705 0.51385 0.62430 0.48815 0.65650
1 0.46775 0.57440 0.51355 0.60805 0.48095 0.59100 0.47750 0.61625 0.50725 0.61805
2 0.47010 0.55640 0.48425 0.62730 0.47700 0.57890 0.49230 0.61555 0.49405 0.60120
3 0.46590 0.61315 0.52595 0.64105 0.48055 0.61595 0.49575 0.62415 0.46745 0.63220
4 0.46875 0.55950 0.47220 0.59595 0.47430 0.56400 0.52030 0.60820 0.47545 0.64975
5 0.45910 0.57875 0.51580 0.62830 0.47980 0.64300 0.48830 0.59090 0.54295 0.65020
6 0.48510 0.58475 0.49405 0.63245 0.48000 0.55730 0.50500 0.65495 0.50390 0.62780
7 0.46880 0.59910 0.52755 0.59940 0.47135 0.58455 0.51960 0.57525 0.51295 0.58290
8 0.47835 0.59460 0.51925 0.64070 0.47755 0.58055 0.52190 0.61230 0.52090 0.60615
9 0.45525 0.57245 0.53485 0.60680 0.46135 0.57085 0.51090 0.60790 0.49445 0.60810

Table 7: Final accuracy for Task 1 values

Out[49]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.405382 0.483507 0.368924 0.499132 0.380208 0.510417 0.387153 0.393663 0.438368 0.642795
1 0.359375 0.486111 0.553819 0.535156 0.375000 0.450087 0.342882 0.518663 0.388021 0.558594
2 0.335069 0.503038 0.422743 0.633681 0.342882 0.556424 0.517361 0.513889 0.407986 0.614583
3 0.407118 0.429253 0.436632 0.467448 0.461806 0.509115 0.514757 0.689236 0.352431 0.364583
4 0.396701 0.506944 0.344618 0.482205 0.413194 0.511285 0.649306 0.533854 0.429688 0.704427
5 0.414931 0.611545 0.524306 0.546875 0.403646 0.509115 0.399306 0.384115 0.333333 0.425781
6 0.395833 0.544705 0.376736 0.596788 0.379340 0.481337 0.356771 0.672743 0.410590 0.606337
7 0.400174 0.558594 0.400174 0.481771 0.369792 0.474826 0.366319 0.446181 0.341146 0.661458
8 0.440972 0.637153 0.454861 0.373264 0.414931 0.499132 0.296875 0.684028 0.621528 0.535156
9 0.327257 0.593750 0.487847 0.535156 0.392361 0.457031 0.337674 0.461806 0.326389 0.539497

Table 8: Final accuracy for Task 0 values

Out[50]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.341146 0.570312 0.391493 0.425347 0.440104 0.437934 0.328125 0.750868 0.418403 0.555122
1 0.344618 0.512153 0.345486 0.531250 0.421875 0.611545 0.370660 0.376302 0.458333 0.518663
2 0.453993 0.505208 0.417535 0.473958 0.400174 0.467882 0.321181 0.528646 0.443576 0.473958
3 0.355903 0.656684 0.446181 0.561632 0.314236 0.552083 0.328993 0.425781 0.436632 0.608507
4 0.419271 0.493490 0.465278 0.520399 0.397569 0.538628 0.315972 0.417535 0.381076 0.639323
5 0.346354 0.479601 0.397569 0.733941 0.507812 0.454861 0.517361 0.665365 0.382812 0.561198
6 0.442708 0.496528 0.366319 0.557292 0.359375 0.470486 0.539062 0.602865 0.350694 0.532552
7 0.335938 0.483941 0.461806 0.611545 0.399306 0.533420 0.335938 0.475694 0.594618 0.514757
8 0.401910 0.435764 0.434028 0.709201 0.428819 0.539931 0.772569 0.493490 0.309028 0.448351
9 0.437500 0.402344 0.448785 0.523438 0.359375 0.447917 0.368056 0.704427 0.468750 0.470920

Table 9: Relative standard deviation of task undertakings

Out[51]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.700105 0.143003 0.469877 0.236425 0.386636 0.358465 0.271218 0.567960 0.462740 0.215254
1 0.139149 0.387375 0.303454 0.186103 0.347442 1.036929 0.874912 0.246653 1.000926 0.229749
2 0.457155 0.151688 0.345453 0.622472 0.439421 0.364147 1.034268 0.394069 0.544498 0.156255
3 0.885171 0.206859 0.610412 0.166930 0.249994 0.537131 0.257494 0.345168 0.954753 0.147521
4 0.441135 0.474957 0.130635 0.516832 0.078740 0.574753 0.534225 0.677871 0.177095 0.363864
5 0.521923 0.471132 0.266757 0.381719 0.789375 0.245070 0.557733 0.277269 0.960138 0.606839
6 0.688710 0.565321 0.088547 0.603974 0.195608 0.781353 0.366930 0.451875 0.075519 0.420104
7 0.151255 0.155834 0.455203 0.625290 1.088079 0.493448 0.427153 0.419512 0.211918 0.174132
8 0.042242 0.046469 0.086621 0.232103 0.960539 0.713633 0.413174 0.250368 0.598924 0.375179
9 0.582562 0.635927 0.092938 0.546157 0.651222 0.571522 0.544067 0.239713 0.511838 0.387702

Figures

Figure 1: Success rate


One-way ANOVA for success rate


4 classes - all selections
F : 3.000655586603843
p : 0.02807905700021931


##########################################


4 classes - only vs random
F : 5.936893631633058
p : 0.02543875344185936


############################################################################################


12 classes
F : 8.306828657992595
p : 4.171147520383266e-05


##########################################


12 classes - only vs random
F : 16.577967452840518
p : 0.0007160664591278651

Figure 1 displays the evolution of the average success rate (y-axis) as the number of iterations increases (x-axis), depending on the (a) birth criteria, (b) birth sorting order, (c) knowledge limit and (d) parent selection policy. Two things are shown. First, that favoring the reproduction of agents that are less successful or have accumulated less points, leads to a significantly lower success rate. This is to be expected according to the pressure exerted during the selection. Second, in both subfigures it is shown that the success rate stabilizes, but does not converge to 1. This indicates that either the agents' final ontologies do not allow them to agree on all decisions. Whether agents memory is equivalent to 4 or 12 classes, there is a statistically significant difference between the random selection of parents and the selection of the parents that have the lowest success rate (ASC_SRATE).

Figure 2: Average minimum accuracy (average accuracy of agents' accuracy on their worst task)


For the same mechanism and number of tasks, the number of generations has has a statistically significant impact only when mechanism = generalization and tasks are 1 or 2.


4 classes
F : 1.1890855233176356
p : 0.3285619354114727


##########################################


4 classes - only vs random
F : 2.7224296928587823
p : 0.11628811558028106


############################################################################################


12 classes
F : 0.18062623283809376
p : 0.9472613606446354


##########################################


12 classes - only vs random
F : 0.05881873727087633
p : 0.8111147358478134

Figure 2 displays the evolution of the average minimum accuracy (y-axis) as the number of iterations increases (x-axis), depending on the (a) birth criteria, (b) birth sorting order, (c) knowledge limit and (d) parent selection policy. It is shown that the (a) and (b) and (d) have no statistically significant impact on the observed average minimum accuracy. Whether agents memory is equivalent to 4 or 12 classes, there is no statistically significant difference between the random selection of parents and the selection of the parents that have the lowest success rate (ASC_SRATE).

Figure 3: Average accuracy


For the same mechanism and number of tasks, the number of generations has has a statistically significant impact only when mechanism = generalization and tasks are 1 or 2.


4 classes
F : 7.166566086506032
p : 0.00014893272678449983


##########################################


4 classes - only vs random
F : 6.960306391224073
p : 0.016698556200410084


############################################################################################


12 classes
F : 10.06298760441851
p : 6.633899885608674e-06


##########################################


12 classes - only vs random
F : 25.729541692732393
p : 7.939187028752312e-05

Figure 3 displays the evolution of the average accuracy over all tasks (y-axis) as the number of iterations increases (x-axis), depending on the (a) birth criteria, (b) birth sorting order, (c) knowledge limit and (d) parent selection policy. It is shown that favoring the reproduction of agents that are less successful or have accumulated less points, leads to a significantly lower average accuracy for a given knowledge limit. Whether agents memory is of 4 or 12 classes, there is a statistically significant difference between the random selection of parents and the selection of the parents that have the lowest success rate (ASC_SRATE).

Figure 4: Average maximum accuracy (average accuracy of agents' accuracy on their best task)


4 classes


Random forgetting, 1 task, same mechanism, different interval (number of generations)
F : 7.216516847086541
p : 0.0001406612965498181


##########################################


4 classes - only vs random
F : 3.256107588350092
p : 0.08791711840673119


############################################################################################


12 classes


Random forgetting, 1 task, same mechanism, different interval (number of generations)
F : 7.91864265707204
p : 6.386247003447367e-05


##########################################


12 classes - only vs random
F : 12.283742874240188
p : 0.0025295256337730058

Figure 4 displays the evolution of the average maximum accuracy over all agents (y-axis) as the number of iterations increases (x-axis), depending on the (a) birth criteria, (b) birth sorting order, (c) knowledge limit and (d) parent selection policy. It is shown that favoring the reproduction of agents that are less successful or have accumulated less points, leads to a significantly lower average maximum accuracy for a given knowledge limit. When agents memory holds 4 classes, there is no statistically significant difference between the random selection of parents and the selection of the parents that have the lowest success rate (ASC_SRATE). When the agents memory holds 12 classes, there is a statistically significant difference between the random selection of parents and the selection of parents that have the lowest success rate (ASC_RATE), i.e., agree less with their peers.

Figure 5: Correct decision rate


4 classes
F : 12.6347803193798
p : 5.673733827846321e-07


##########################################


4 classes - only vs random
F : 12.822047783318077
p : 0.0021365695348205347


############################################################################################


12 classes
F : 8.50952545328941
p : 3.349089187279674e-05


##########################################


12 classes - only vs random
F : 10.51670843627091
p : 0.004515007819233355

Figure 5 displays the evolution of the average correct decision rate (y-axis) as the number of iterations increases (x-axis), depending on the (a) birth criteria, (b) birth sorting order, (c) knowledge limit and (d) parent selection policy. It is shown that favoring the reproduction of agents that are less successful or have accumulated less points, leads to a significantly lower correct decision rate for a given knowledge limit. Whether agents memory is equivalent to 4 or 12 classes, there is a statistically significant difference between the random selection of parents and the selection of the parents that have the lowest success rate (ASC_SRATE).

Figure 6: Total points


4 classes
F : 7.0225536816030925
p : 0.00017573268067858802


##########################################


4 classes - only vs random
F : 7.667714974126427
p : 0.012646882319475935


############################################################################################


12 classes
F : 6.569929983846806
p : 0.0002977247431779161


##########################################


12 classes - only vs random
F : 13.162646753588865
p : 0.0019233981071808323

Figure 6 displays the evolution of the average accumulated points by all agents (y-axis) as the number of iterations increases (x-axis), depending on the (a) birth criteria, (b) birth sorting order, (c) knowledge limit and (d) parent selection policy. It is shown that the reproduction of agents that agree less is favored, agent societies collectively accumulate less points. Whether agents memory is equivalent to 4 or 12 classes, there is a statistically significant difference between the random selection of parents and the selection of the parents that have the lowest success rate (ASC_SRATE), i.e., the parents that agree less with their peers.

Figure 7: Points distribution