Experiment 20210312-DOLA

Experiment design

Agent adapt ontologies to agree on decision taking

Date: 20210312 (Yasser Bourahla)

10 runs; 100000 games

Hypothesis: ['hypotheses of 20200623-DOLA are still verified for different adaptation operators', '']

Variation of: 20200623-DOLA

Decision tree learning algorithms: RAND,ID3

Variables

independent variables: ['op', 'trainer']

dependent variables: ['ssrate', 'accuracy', 'distance']

Experiment

Date: 20210312 (Yasser Bourahla)

LazyLavender hash: 37dffefd52354174aac791c6513cae2a564428f7

Link to lazylavender

Parameter file: params.sh

Executed command (script.sh):




BEWARE: REPRODUCING THE ANALYSIS MAY TAKE A CONSIDERABLE AMOUNT OF TIME.

#!/bin/bash

. params.sh

CURRDIR=$(pwd)
OUTPUT=${CURRDIR}/${DIRPREF}
# cd ${LLPATH}
cd lazylav
# this sample runs ExperimentalPlan. It can be replaced with Monitor if parameters are not varied.
bash scripts/runexp.sh -p ${CURRDIR} -d ${DIRPREF} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.ExperimentalPlan -Dexperiment=fr.inria.exmo.lazylavender.decisiontaking.Experiment ${OPT} -DresultDir=${OUTPUT}

Experimental plan

The independent variables have been varied as follows:

adaptation operator: ['allCom', 'noCom', 'oneCom']
ontology initialization: ['ID3', 'RAND']
adaptation operator ontology initialization
allCom noCom oneCom ID3 RAND
ssrate 1 0.65 0.55 0.30 0.63 0.37
1000000 0.90 0.80 0.84 0.87 0.83
accuracy 1 0.39 0.41 0.42 0.47 0.34
1000000 0.82 0.87 0.83 0.85 0.83
distance 1 0.71 0.70 0.69 0.64 0.76
1000000 0.46 0.41 0.44 0.45 0.42

Hypotheses testing:

Hypothesis 1:

hypothesis 1 verified

Hypothesis 2

There is a significant difference in the accuracy between the start and the end of the experiment.
Paired t-test results: t=-31.90 and p<0.01.
Hypothesis 2 verified.

Hypothesis 3

the percentage of final none zero distances to all distances is : 100.00 %
hypothesis 2 verified.

ANOVA Results:

influence of adaptation operator
PR(>F) Significance
ssrate 0.000000 True
accuracy 0.277567 False
distance 0.023671 False 






for ssrate
group1 group2 meandiff p-adj lower upper reject
0 allCom noCom -0.09 0.00 -0.12 -0.06 True
1 allCom oneCom -0.07 0.00 -0.10 -0.04 True
2 noCom oneCom 0.02 0.16 -0.01 0.06 False 
for accuracy
group1 group2 meandiff p-adj lower upper reject
0 allCom noCom 0.07 0.27 -0.04 0.17 False
1 allCom oneCom 0.05 0.48 -0.06 0.16 False
2 noCom oneCom -0.02 0.90 -0.12 0.09 False 
for distance
group1 group2 meandiff p-adj lower upper reject
0 allCom noCom -0.06 0.02 -0.11 -0.01 True
1 allCom oneCom -0.04 0.13 -0.09 0.01 False
2 noCom oneCom 0.02 0.67 -0.03 0.07 False

No significant differences between the operators.