Experimental Plan¶

Experiment 20230822-IKEM¶

Online: 'https://sake.re/20230822-IKEM'

Repo: 'http://felapton.inrialpes.fr/cakes/20230822-IKEM.git'

Experiment design¶

Agents interact by playing games with objects and adapt their ontologies to agree on decision taking. Intrinsic exploration motivation (individual / social and direct / indirect curiosity, creativity and non-exploration), that influences which object and, if the agent is social, partner(s) are chosen for the interaction, is introduced.

Date: 20230822 (Anaïs Siebers)

Hypothesis: 1) Agents will be able to fulfil their motivation in terms of increased exploration (exploratory motivations) and decreased exploration (non-exploratory motivation). 2) Agents with exploratory motivation will be more complete, but less accurate than the baseline and non-exploration. 3) Curious agents will be more accurate and complete, but converge slower in comparison with creativity. 4) In more complex settings (higher number of agents and properties), exploration leads to more completeness, accuracy, distance and faster convergence than the baseline. 5) Agents with non-exploratory motivation will be less accurate and complete than the baseline. 6) The indirect learning models have a higher accuracy and completeness than the direct models. 7) Agents with socially oriented intrinsic motivation will have less diverse knowledge, but agree more and converge faster than agents with individual intrinsic motivation. 8) Heterogeneously motivated agents will have a higher accuracy and completeness, but lower diversity and converge slower than homogeneously motivated agents.

5 runs; 40000 games

Experimental setting: Agents learn decision trees (transformed into ontologies); choose the objects (and partners) to interact with; adapt by splitting their leaf nodes

Dependent variables: ['number of agents', 'number of properties', 'motivated agents ratio', 'social', 'direct action choice']

Independent variables: ['success rate', 'completeness', 'accuracy', 'ontology distance', 'object exploration', 'partner exploration']

Experiment¶

Date: 20230822 (Anaïs Siebers)

Lazy lavender hash: db26577bee05f00f618e8b97f616ba24356a4afe

Parameter file: params.sh

Executed command (script.sh):

The independent variables have been varied as follows:

Number of Agents: [3,15,30]
Number of Properties: [4,6,8]
Motivated Agents Ratios: # [CURIOUS;CREATIVE;NON-EXPLORATORY]
                        [[0:0:0],[1:0:0],[0:1:0],[0:0:1],[0.5:0.5:0],[0.5:0:0.5],[0:0.5:0.5],[0.3:0.3:0.3]]  
Agent is Social: [true, false]
Direct Action Choice: [true, false]

Raw results¶

Style¶

Data exploration¶

Initialisation & helper function¶

Overview¶

Analysis¶

Full results are available on zenodo.¶

Hypotheses¶

Style¶

Helper functions¶

Proof direct baseline == indirect baseline¶

1) Agents will be able to fulfil their motivation in terms of increased exploration (exploratory motivations) and decreased exploration (non-exploratory motivation).¶

Statistical Analysis¶

Interpretation of the Results¶

The object exploration score is indeed higher for curious ([1:0:0]), creative ([0:1:0]) and even non-exploratory agents ([0:0:1]) compared to the baseline ([0:0:0]). The same applies for the partner exploration score.

One ANOVA was performed, testing only the direct models and baseline. The other ANOVA tested only the indirect models and baseline. Because of the difference of the approach behind the direct and indirect models, they are tested separately. It can be seen, that both the direct models and the indirect models have a statistically significant effect on the object exploration score and the partner exploration score. The post-hoc Tukey HSD test shows, that the influence of the exploration motivation is always significant regarding the baseline scenario.

Non-exploratory agents have the goal to explore less, and therefore the object exploration should be lower than the baseline. This is not the case. But the motivation of curious and creative agents to explore more objects is nevertheless given (see higher object exploration regarding the baseline).

2) Agents with exploratory motivation will be more complete, but less accurate than the baseline and non-exploration.¶

Statistical Analysis¶

Interpretation of the Results¶

It is shown that the baseline plot always converges to a higher knowledge accuracy and completeness than curiosity and creativity. The non-exploration plot always converges to a lower knowledge accuracy and completeness. Regarding non-exploration, exploratory agents have a higher completeness. But, they also have more knowledge accuracy, contrary to the hypothesis. It has to be considered that some results for knowledge accuracy are not statistically significant. Exploratory agents converge to a lower accuracy and completeness than the baseline.

Again, two ANOVA tests were performed, testing the results for the direct and indirect model. ANOVA reveals that there is a statistically significant effect of the different exploration motivations on the completeness and knowledge accuracy of the agents’ knowledge. According to the post-hoc Tukey HSD test, the exploratory motivations (curiosity and creativity) compared with the baseline (non- exploration and baseline) show a significant effect concerning completeness in both (direct and indirect) models. But no statistically significant effect for knowledge accuracy between the exploration motivations and non-exploration can be found. Knowledge accuracy is significant for the exploration motivations regarding the baseline.

3) Curious agents will be more accurate and complete, but converge slower in comparison with creativity.¶

Statistical Analysis¶

Interpretation of the Results¶

Convergence can be observed in the success rate. If the success rate stabilises, the agents do not disagree any more, and thus there are no changes in their knowledge any more. As before, the direct and indirect models are tested separately. It can be seen that curiosity and creativity with the direct models are very similar. ANOVA confirms that no significant effect can be found.

For the indirect models, on the other hand, a statistically significant effect can be found. As hypothesised, curiosity converges more slowly, but to higher knowledge accuracy and completeness than creativity.

4) In more complex settings (higher number of agents and properties), exploration leads to more completeness, accuracy, distance and faster convergence than the baseline.¶

Statistical Analysis¶