Experiment 20231120-MTOA

Experiment design

20231120-MTOA

Date: 2023-11-20 (Andreas Kalaitzakis)

Hypotheses

Favoring the reproduction of agents with the lowest success rate will improve the maximum accuracy and re-balance the tasks representation.

Experimental setting

36 agents; 3 tasks; 6 features (2 independent features per task); 4 decision classes; 10 runs; 80000 games; 20000 games per generation

Each agent initially trains on all tasks and then carries out one. When the agents disagree the following take place:

(a) The agent with the lower score will adapt its knowledge accordingly. If memory capacity limit is attained, the agent will try to forget knowledge (parent nodes with descendents that contain the same decisions are removed).

(b) The agent with the highest score will decide for both agents. If this agent's decision is correct, the agent will receive the points corresponding to both agents.

Agents undertake 1 task having a limited memory, enough for learning 3 tasks accurately (12 classes).

When this limit is attained, the agents will try to forget knowledge in favor of the undertaken task.

This is possible when two leaf nodes with the same parent have the same decisions for the task the agent undertakes.

Variables independent variables: ['birthCriteria','birthSortingOrder','knowledgeLimit','parentSelectionPolicy']

dependent variables: ['avg_min_accuracy', 'avg_accuracy', 'avg_max_accuracy', 'success_rate', 'correct_decision_rate', 'delegation_rate', 'accuracy_relative_standard_deviation', 'total_points', 'points_distribution', 'games_relative_standard_deviation']

Experiment

Date: 2023-01-01 (Andreas Kalaitzakis)

Computer: Dell Precision-5540 (CC: 12 * Intel(R) Core(TM) i7-9850H CPU @ 2.60GHz with 16GB RAM OS: Linux 5.4.0-92-generic)

Duration : 1080 minutes

Lazy lavender hash: ceb1c5d1ca8109373d293b687fc55953fce5241d

Parameter file: params.sh

Executed command (script.sh):

#!/bin/bash

. params.sh

CURRDIR=$(pwd)
OUTPUT=${CURRDIR}/${DIRPREF}
# cd ${LLPATH}
cd lazylav
# this sample runs ExperimentalPlan. It can be replaced with Monitor if parameters are not varied.
bash scripts/runexp.sh -p ${CURRDIR} -d ${DIRPREF} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.ExperimentalPlan -Dexperiment=fr.inria.exmo.lazylavender.decisiontaking.multitask.SelectiveAcceptanceSpecializationExperiment ${OPT} -DresultDir=${OUTPUT}

Analysis

Raw data

Full results can be found at: Zenodo DOI



Table 1: Final success rate values

Table 1 consists of the final achieved average success rate values, i.e., the average success rate after the last iteration.

Out[6]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.74485 0.63330 0.53475 0.72205 0.72135 0.59640 0.64450 0.69080 0.76490 0.76970
1 0.72475 0.57875 0.73375 0.75965 0.72610 0.43630 0.83995 0.73675 0.71760 0.81680
2 0.50445 0.64445 0.72670 0.75910 0.70145 0.50570 0.80190 0.78685 0.79015 0.83620
3 0.71705 0.52895 0.74910 0.83970 0.84710 0.73335 0.75895 0.73510 0.81755 0.81240
4 0.87505 0.63895 0.70235 0.71460 0.46630 0.53085 0.71115 0.78850 0.80965 0.89170
5 0.56475 0.45155 0.75875 0.84490 0.56190 0.76350 0.68415 0.84085 0.78995 0.85675
6 0.64440 0.48630 0.77855 0.93355 0.52435 0.43945 0.70755 0.77260 0.72000 0.76740
7 0.70650 0.72770 0.73495 0.76965 0.50310 0.56220 0.63145 0.77525 0.74395 0.88255
8 0.68990 0.60830 0.82825 0.82585 0.67595 0.45045 0.58900 0.54050 0.77350 0.72635
9 0.80985 0.69725 0.75975 0.58025 0.42695 0.43630 0.72760 0.73845 0.80260 0.75225

Table 2: Final average minimum accuracy values

Table 2 consists of the final average minimum accuracy values with respect to the worst task, i.e., the accuracy after the last iteration for the task for which the agent scores the lowest accuracies.

Out[7]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-COMPENSATION-RANDOM-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-RANDOM-0.075 ASCENDING-SUCCESS_RATE-RANDOM-0.225 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-RANDOM-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-RANDOM-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225 ASCENDING-COMPENSATION-RANDOM-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225
0 0.247830 0.224826 0.257378 0.241319 0.241319 0.259983 0.236111 0.206597 0.243056 0.246528 0.226562 0.249132 0.223958 0.253472 0.250000 0.247830
1 0.248264 0.246528 0.227431 0.222222 0.236111 0.276910 0.250000 0.232639 0.241319 0.239583 0.269531 0.250000 0.250000 0.253038 0.243056 0.260417
2 0.265191 0.215712 0.264757 0.246528 0.241319 0.259115 0.248264 0.236111 0.237847 0.227431 0.230035 0.222222 0.236111 0.233941 0.243056 0.226997
3 0.259115 0.244792 0.247396 0.246528 0.248264 0.240885 0.246528 0.243056 0.243056 0.237847 0.245660 0.229167 0.246528 0.256944 0.250000 0.242188
4 0.227431 0.234375 0.282552 0.227431 0.243056 0.265191 0.243056 0.217014 0.246528 0.217014 0.236111 0.236979 0.244792 0.230903 0.248264 0.253472
5 0.232639 0.220486 0.255208 0.225694 0.237847 0.253038 0.230903 0.229167 0.246528 0.215278 0.264757 0.243924 0.239583 0.253038 0.239583 0.255642
6 0.250000 0.228733 0.234809 0.227431 0.246528 0.266493 0.225694 0.227431 0.248264 0.232639 0.239149 0.239583 0.241319 0.239149 0.246528 0.246528
7 0.226562 0.241753 0.252170 0.246528 0.229167 0.255208 0.227431 0.241319 0.243056 0.243056 0.253906 0.232639 0.230903 0.236545 0.248264 0.261719
8 0.245660 0.211372 0.266927 0.243056 0.237847 0.265191 0.250000 0.222222 0.248264 0.236111 0.238281 0.251736 0.234375 0.244792 0.246528 0.255208
9 0.247396 0.249132 0.267361 0.250000 0.246528 0.250868 0.250000 0.225694 0.234375 0.232639 0.239583 0.235243 0.237847 0.250434 0.241319 0.235243
Out[8]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.236111 0.247830 0.246528 0.224826 0.241319 0.247830 0.206597 0.249132 0.223958 0.253472
1 0.250000 0.248264 0.239583 0.246528 0.222222 0.260417 0.232639 0.250000 0.250000 0.253038
2 0.248264 0.265191 0.227431 0.215712 0.246528 0.226997 0.236111 0.222222 0.236111 0.233941
3 0.246528 0.259115 0.237847 0.244792 0.246528 0.242188 0.243056 0.229167 0.246528 0.256944
4 0.243056 0.227431 0.217014 0.234375 0.227431 0.253472 0.217014 0.236979 0.244792 0.230903
5 0.230903 0.232639 0.215278 0.220486 0.225694 0.255642 0.229167 0.243924 0.239583 0.253038
6 0.225694 0.250000 0.232639 0.228733 0.227431 0.246528 0.227431 0.239583 0.241319 0.239149
7 0.227431 0.226562 0.243056 0.241753 0.246528 0.261719 0.241319 0.232639 0.230903 0.236545
8 0.250000 0.245660 0.236111 0.211372 0.243056 0.255208 0.222222 0.251736 0.234375 0.244792
9 0.250000 0.247396 0.232639 0.249132 0.250000 0.235243 0.225694 0.235243 0.237847 0.250434

Table 3: Final average accuracy values

Table 3 consists of the final achieved average ontology accuracy with respect to all tasks, i.e., the accuracy after the last iteration averaged on all tasks and agents.

Out[9]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.363426 0.439815 0.416667 0.477141 0.392361 0.456597 0.410301 0.476418 0.410880 0.494068
1 0.358796 0.458333 0.384838 0.484375 0.396991 0.423756 0.363715 0.439236 0.364583 0.487847
2 0.395255 0.464988 0.416667 0.473380 0.390625 0.438513 0.379630 0.478733 0.400752 0.487413
3 0.384838 0.442564 0.384259 0.500723 0.350694 0.477865 0.380498 0.473669 0.444444 0.498409
4 0.331597 0.438223 0.410880 0.480758 0.398148 0.442564 0.357350 0.493056 0.415509 0.510272
5 0.388889 0.405961 0.367477 0.482060 0.383681 0.491898 0.380208 0.506944 0.339699 0.516927
6 0.368634 0.436632 0.425926 0.502315 0.366030 0.420284 0.427662 0.481337 0.406829 0.505498
7 0.384259 0.461806 0.372975 0.468316 0.422454 0.462674 0.414931 0.485532 0.369792 0.486834
8 0.384259 0.410156 0.419560 0.480613 0.354167 0.443287 0.434028 0.436632 0.388889 0.482205
9 0.398148 0.463831 0.415509 0.463542 0.405671 0.424624 0.401042 0.469329 0.394676 0.453125

Table 4: Final average best task accuracy values

Table 4 consists of the final average best task accuracy values with respect to the best task, i.e., the accuracy after the last iteration for the task for which the agent score the highest accuracies.

Out[10]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-COMPENSATION-RANDOM-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-RANDOM-0.075 ASCENDING-SUCCESS_RATE-RANDOM-0.225 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-RANDOM-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-RANDOM-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225 ASCENDING-COMPENSATION-RANDOM-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225
0 0.773003 0.860243 0.838976 0.644097 0.630208 0.677083 0.590278 0.739583 0.685764 0.720486 0.914062 0.811632 0.711806 0.860677 0.526042 0.776910
1 0.766059 0.873264 0.774740 0.684028 0.672743 0.673177 0.527778 0.574653 0.668403 0.645833 0.899740 0.736545 0.586806 0.871094 0.605903 0.676215
2 0.789931 0.888455 0.585069 0.623264 0.666667 0.713542 0.640625 0.642361 0.687500 0.741319 0.875000 0.890191 0.653646 0.898438 0.649306 0.742188
3 0.708333 0.939670 0.791233 0.512153 0.623264 0.646267 0.602431 0.641493 0.793403 0.640625 0.845920 0.869792 0.791667 0.877170 0.585069 0.853733
4 0.753906 0.882378 0.564670 0.677083 0.609375 0.756510 0.480903 0.556424 0.685764 0.744792 0.800781 0.917535 0.718750 0.930990 0.673611 0.708333
5 0.649740 0.929253 0.666233 0.640625 0.659722 0.591580 0.640625 0.621528 0.654514 0.626736 0.757378 0.939670 0.505208 0.926215 0.638889 0.876736
6 0.744358 0.975260 0.815104 0.592882 0.612847 0.791667 0.616319 0.781250 0.706597 0.774306 0.875434 0.897135 0.690972 0.882812 0.581597 0.684462
7 0.837240 0.782986 0.559462 0.730903 0.597222 0.794705 0.623264 0.713542 0.739583 0.611979 0.882812 0.906250 0.588542 0.897569 0.602431 0.782552
8 0.625868 0.929253 0.648438 0.541667 0.633681 0.751736 0.607639 0.809028 0.654514 0.758681 0.859375 0.703559 0.657986 0.819878 0.642361 0.712240
9 0.833333 0.784288 0.578125 0.654514 0.598958 0.610677 0.645833 0.727431 0.680556 0.725694 0.873698 0.837240 0.668403 0.771701 0.557292 0.701823
Out[11]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.590278 0.773003 0.720486 0.860243 0.644097 0.776910 0.739583 0.811632 0.711806 0.860677
1 0.527778 0.766059 0.645833 0.873264 0.684028 0.676215 0.574653 0.736545 0.586806 0.871094
2 0.640625 0.789931 0.741319 0.888455 0.623264 0.742188 0.642361 0.890191 0.653646 0.898438
3 0.602431 0.708333 0.640625 0.939670 0.512153 0.853733 0.641493 0.869792 0.791667 0.877170
4 0.480903 0.753906 0.744792 0.882378 0.677083 0.708333 0.556424 0.917535 0.718750 0.930990
5 0.640625 0.649740 0.626736 0.929253 0.640625 0.876736 0.621528 0.939670 0.505208 0.926215
6 0.616319 0.744358 0.774306 0.975260 0.592882 0.684462 0.781250 0.897135 0.690972 0.882812
7 0.623264 0.837240 0.611979 0.782986 0.730903 0.782552 0.713542 0.906250 0.588542 0.897569
8 0.607639 0.625868 0.758681 0.929253 0.541667 0.712240 0.809028 0.703559 0.657986 0.819878
9 0.645833 0.833333 0.725694 0.784288 0.654514 0.701823 0.727431 0.837240 0.668403 0.771701

Table 5: Final average scope accuracy values

Table 5 consists of the final average scope accuracy values.

Out[12]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.583333 0.741319 0.715278 0.853299 0.637153 0.744792 0.736111 0.802517 0.710069 0.849392
1 0.527778 0.731337 0.644097 0.873264 0.684028 0.621962 0.565104 0.734809 0.586806 0.850260
2 0.631944 0.759549 0.741319 0.888455 0.605903 0.707899 0.642361 0.890191 0.653646 0.898438
3 0.595486 0.670573 0.623264 0.939670 0.506944 0.847656 0.641493 0.862847 0.788194 0.864149
4 0.480903 0.716580 0.744792 0.882378 0.670139 0.640191 0.554688 0.917535 0.710069 0.930122
5 0.628472 0.596788 0.619792 0.929253 0.623264 0.854167 0.612847 0.932726 0.501736 0.926215
6 0.602431 0.709201 0.774306 0.975260 0.552951 0.631944 0.781250 0.897135 0.690972 0.881510
7 0.619792 0.822917 0.605035 0.782118 0.725694 0.759115 0.713542 0.902778 0.581597 0.894097
8 0.590278 0.576823 0.758681 0.929253 0.534722 0.689670 0.805556 0.702257 0.647569 0.815538
9 0.638889 0.799479 0.725694 0.773872 0.637153 0.670573 0.727431 0.832465 0.668403 0.764757

Table 6: Final correct decision rate values

Table 6 consists of the final achieved correct decision rate values, i.e., the average correct decision rate after the last iteration.

Out[13]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.61630 0.87985 0.84605 0.92315 0.68055 0.87915 0.88615 0.86640 0.76145 0.92510
1 0.53630 0.84605 0.66935 0.95005 0.71110 0.69670 0.62835 0.78460 0.56755 0.95580
2 0.73795 0.88735 0.82165 0.95145 0.70300 0.78460 0.71505 0.97040 0.71480 0.96505
3 0.68110 0.75340 0.71670 0.96650 0.51670 0.93905 0.71020 0.94830 0.81655 0.96180
4 0.49465 0.85940 0.80505 0.93915 0.78625 0.80425 0.61120 0.95145 0.75215 0.98625
5 0.77695 0.71295 0.63925 0.97795 0.72735 0.92715 0.64860 0.97790 0.49415 0.98245
6 0.69230 0.70255 0.87535 0.99435 0.68155 0.72670 0.85810 0.95720 0.77100 0.94850
7 0.66365 0.92395 0.65235 0.80295 0.80695 0.84835 0.84970 0.95880 0.69310 0.97315
8 0.67880 0.64430 0.81975 0.97345 0.54875 0.73790 0.87425 0.74000 0.72165 0.92470
9 0.69560 0.88320 0.79005 0.86750 0.72500 0.70615 0.74805 0.93215 0.73085 0.89235

Table 7: Final accuracy for Task 1 values

Out[14]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.357639 0.727431 0.321181 0.448351 0.543403 0.756510 0.269097 0.427083 0.317708 0.414062
1 0.402778 0.473524 0.453125 0.414931 0.418403 0.592014 0.376736 0.391927 0.420139 0.483941
2 0.480903 0.772569 0.479167 0.461372 0.289931 0.586372 0.354167 0.720052 0.421875 0.338108
3 0.489583 0.640625 0.434028 0.593316 0.444444 0.810764 0.359375 0.410590 0.468750 0.286024
4 0.413194 0.293403 0.524306 0.882378 0.670139 0.447049 0.296875 0.339844 0.446181 0.778646
5 0.319444 0.298611 0.387153 0.379340 0.425347 0.332031 0.420139 0.849392 0.406250 0.584201
6 0.383681 0.493056 0.559028 0.346354 0.419271 0.394097 0.595486 0.344618 0.418403 0.444878
7 0.411458 0.284288 0.400174 0.375868 0.708333 0.352431 0.678819 0.403212 0.506944 0.316406
8 0.579861 0.308160 0.380208 0.346354 0.423611 0.417101 0.414931 0.337674 0.434028 0.641927
9 0.302083 0.484375 0.501736 0.457465 0.472222 0.456163 0.493056 0.773003 0.465278 0.616753

Table 8: Final accuracy for Task 0 values

Out[15]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 0.361111 0.292969 0.715278 0.492622 0.409722 0.395399 0.690972 0.656250 0.465278 0.644965
1 0.404514 0.334201 0.380208 0.401910 0.375000 0.387587 0.332465 0.441840 0.451389 0.659722
2 0.385417 0.342014 0.435764 0.668837 0.407986 0.500000 0.282986 0.454861 0.335938 0.534288
3 0.314236 0.313368 0.322917 0.429688 0.335069 0.408854 0.404514 0.333333 0.458333 0.624566
4 0.364583 0.641059 0.361111 0.361111 0.350694 0.286892 0.342882 0.449653 0.444444 0.417535
5 0.503472 0.641493 0.430556 0.600260 0.477431 0.499566 0.420139 0.402344 0.380208 0.457031
6 0.468750 0.557726 0.380208 0.975260 0.354167 0.647569 0.437500 0.832899 0.439236 0.440538
7 0.409722 0.486111 0.392361 0.546441 0.300347 0.780816 0.434028 0.625868 0.336806 0.328993
8 0.286458 0.620226 0.437500 0.833767 0.423611 0.567274 0.336806 0.433160 0.475694 0.335503
9 0.534722 0.292969 0.480903 0.447917 0.505208 0.360243 0.350694 0.407118 0.392361 0.419705

Table 9: Relative standard deviation of task undertakings

Out[16]:
ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 ASCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.075 DESCENDING-COMPENSATION-BEST_PARENT_BEST_MATE-0.225 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 ASCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.075 DESCENDING-SUCCESS_RATE-BEST_PARENT_BEST_MATE-0.225 DESCENDING-SUCCESS_RATE-RANDOM-0.075 DESCENDING-SUCCESS_RATE-RANDOM-0.225
0 58.525676 75.045113 108.593222 33.231691 95.156470 92.035579 94.040737 22.871384 49.856685 49.953298
1 74.150763 39.838326 51.083935 94.672963 28.049854 85.497424 78.980149 51.445386 57.688123 52.987217
2 62.908803 81.532242 70.058532 62.761431 42.264733 81.705369 47.310296 64.264987 41.309503 38.887940
3 49.960645 43.591604 96.577996 49.337827 17.804264 78.352808 100.645880 101.485149 32.651169 66.396204
4 45.879757 42.451106 67.303186 117.172355 124.056098 71.403224 48.380818 72.534779 6.838696 95.855433
5 37.193885 74.320200 80.910162 66.088488 83.962634 29.052912 36.512408 85.583788 65.651742 20.187370
6 28.107811 60.333560 53.960687 120.637698 78.200472 89.490954 36.669825 98.373273 32.439395 41.191324
7 10.542410 59.597945 75.132994 60.478338 69.789775 90.052919 53.298785 48.445645 75.512150 107.834015
8 84.557609 71.957357 20.898097 80.578076 66.121391 85.274890 58.910703 79.252957 40.585877 70.820636
9 43.983998 62.085701 56.126412 125.035295 61.694379 48.647343 37.082613 70.694791 33.909629 33.689658

Figures

Figure 1: Success rate


One-way ANOVA for success rate


4 classes
F : 3.8217010078929
p : 0.009304597491286107


##########################################


4 classes - only vs random
F : 12.31772966279751
p : 0.002502454132055644


############################################################################################


12 classes
F : 16.158090071497273
p : 2.8373871847621326e-08


##########################################


12 classes - only vs random
F : 39.67159960859607
p : 6.150571305327271e-06

Figure 1 displays the evolution of the average success rate (y-axis) as the number of iterations increases (x-axis), depending on the (a) birth criteria, (b) birth sorting order, (c) knowledge limit and (d) parent selection policy. Two things are shown. First, that favoring the reproduction of agents that are less successful or have accumulated less points, leads to a significantly lower success rate. This is to be expected according to the pressure exerted during the selection. Second, in both subfigures it is shown that the success rate stabilizes, but does not converge to 1. This indicates that either the agents' final ontologies do not allow them to agree on all decisions.

Figure 2: Average minimum accuracy (average accuracy of agents' accuracy on their worst task)


For the same mechanism and number of tasks, the number of generations has has a statistically significant impact only when mechanism = generalization and tasks are 1 or 2.


4 classes
F : 2.5876430052853423
p : 0.04939281702790428


##########################################


4 classes - only vs random
F : 0.043495070558669115
p : 0.8371376622799723


############################################################################################


12 classes
F : 3.353849358306764
p : 0.017398324681761116


##########################################


12 classes - only vs random
F : 0.5102273173412929
p : 0.4842014756983497

Figure 2 displays the evolution of the average minimum accuracy (y-axis) as the number of iterations increases (x-axis), depending on the (a) birth criteria, (b) birth sorting order, (c) knowledge limit and (d) parent selection policy. It is shown that the (a) and (b) and (d) have no statistically significant impact on the observed average minimum accuracy.

Figure 3: Average accuracy


For the same mechanism and number of tasks, the number of generations has has a statistically significant impact only when mechanism = generalization and tasks are 1 or 2.


4 classes
F : 1.6107527229886935
p : 0.18805154787114156


##########################################


4 classes - only vs random
F : 0.4030735350820971
p : 0.5334939090121481


############################################################################################


12 classes
F : 11.800941802672451
p : 1.224622681262782e-06


##########################################


12 classes - only vs random
F : 21.816386821000886
p : 0.00019024597441684202

Figure 3 displays the evolution of the average accuracy over all tasks (y-axis) as the number of iterations increases (x-axis), depending on the (a) birth criteria, (b) birth sorting order, (c) knowledge limit and (d) parent selection policy. It is shown that when agents memory consists of 12 classes, favoring the reproduction of agents that are less successful or have accumulated less points, leads to a significantly lower average accuracy for a given knowledge limit.

Figure 4: Average maximum accuracy (average accuracy of agents' accuracy on their best task)


4 classes


Random forgetting, 1 task, same mechanism, different interval (number of generations)
F : 3.2697655538947927
p : 0.019489700222426347


##########################################


4 classes - only vs random
F : 0.6805930311931964
p : 0.42018176667111884


############################################################################################


12 classes


Random forgetting, 1 task, same mechanism, different interval (number of generations)
F : 9.974126363820547
p : 7.2566051562134715e-06


##########################################


12 classes - only vs random
F : 20.762923336617682
p : 0.0002444986068284575

Figure 4 displays the evolution of the average maximum accuracy over all agents (y-axis) as the number of iterations increases (x-axis), depending on the (a) birth criteria, (b) birth sorting order, (c) knowledge limit and (d) parent selection policy. It is shown that when agents memory consists of 12 classes, favoring the reproduction of agents that are less successful or have accumulated less points, leads to a significantly lower average maximum accuracy for a given knowledge limit.

Figure 5: Correct decision rate


4 classes
F : 2.2357415905167763
p : 0.08009866411811911


##########################################


4 classes - only vs random
F : 0.10221577201358363
p : 0.7528676708809443


############################################################################################


12 classes
F : 8.491225987773396
p : 3.415842102159854e-05


##########################################


12 classes - only vs random
F : 23.94554223307667
p : 0.00011699205160926796

Figure 5 displays the evolution of the average correct decision rate (y-axis) as the number of iterations increases (x-axis), depending on the (a) birth criteria, (b) birth sorting order, (c) knowledge limit and (d) parent selection policy. It is shown that when agents memory consists of 12 classes, favoring the reproduction of agents that are less successful or have accumulated less points, leads to a significantly lower correct decision rate for a given knowledge limit.

Figure 6: Total points


4 classes
F : 3.0726834152105766
p : 0.02545814864329211


##########################################


4 classes - only vs random
F : 0.35317830123821753
p : 0.5597156162281557


############################################################################################


12 classes
F : 7.359225543323224
p : 0.00011956030332937621


##########################################


12 classes - only vs random
F : 16.025541357978057
p : 0.000833881152728394

Figure 6 displays the evolution of the average accumulated points by all agents (y-axis) as the number of iterations increases (x-axis), depending on the (a) birth criteria, (b) birth sorting order, (c) knowledge limit and (d) parent selection policy. It is shown that when agents memory consists of 12 classes, the reproduction of agents that have accumulated less points is favored, agent societies collectively accumulate less points.

Figure 7: Points distribution