# Experiment 20230505-MTOA¶

## Experiment design¶

20230505-MTOA

Date: 2023-05-05 (Andreas Kalaitzakis)

### Hypotheses¶

1) The more deciding for different tasks relies on common properties, the more tackling additional tasks improves accuracy.

2) The more deciding for different tasks relies on common properties, the higher is success rate.

### Measures¶

Success rate evaluates the interoperability among agents. It is defined as the proportion of successful interactions, over all performed interactions until the $n^{th}$ interaction.

Task accuracy evaluates the quality of agent ontologies. It adapts the accuracy measure introduced in \cite{Bourahla2021a} to different tasks. It is defined as the proportion of object types for which a correct decision would be taken with respect to a task $t$, by an agent $\alpha$ on the $n^{th}$ iteration of the experiment. Task accuracy is used to measure the average and best task accuracy of agents.

\begin{align*} tacc(\alpha,n,t) = \frac{\vert\{o \in \mathcal{I} : h_n^\alpha(o,t) = h^*(o,t) \}\vert}{\vert \mathcal{I} \vert} \end{align*}

### Experimental setting¶

The experiment is executed under 6 setups. Each setup is run 20 times and its results are averaged. One run consists of 80000 interactions with each interaction taking place among two agents. These two agents are randomly selected out of a total population of 18 agents. Their environment contains 64 different object types, each one perceivable through 6 different binary properties. The agents are initially trained with respect to all $|\mathcal{T}|=\{3\}$ tasks. Deciding with respect to each task relies on 2 out of the 6 perceivable binary properties. These properties are either the same for all tasks, or different for each task. Agents induce an initial ontology based on a random 10 \% of all existing labeled examples. The agents are assigned 1 to 3 assigned tasks ($|\mathcal{T}_{ass}|=\{1,2,3\}$). For each task, 4 different decisions exist. Between two consecutive interactions, the environment attributes a score to each agent. This score is calculated taking into account the 60 \% of all samples.

dependent variables: ['avg_min_accuracy', 'avg_accuracy', 'avg_max_accuracy', 'success_rate', 'correct_decision_rate', 'delegation_rate']

## Experiment¶

Date: 2023-04-01 (Andreas Kalaitzakis)

Computer: Dell Precision-5540 (CC: 12 * Intel(R) Core(TM) i7-9850H CPU @ 2.60GHz with 16GB RAM OS: Linux 5.4.0-92-generic)

Duration : 720 minutes

Lazy lavender hash: ceb1c5d1ca8109373d293b687fc55953fce5241d

Parameter file: params.sh

Executed command (script.sh):

#!/bin/bash

. params.sh

CURRDIR=$(pwd) OUTPUT=${CURRDIR}/${DIRPREF} # cd${LLPATH}
cd lazylav
# this sample runs ExperimentalPlan. It can be replaced with Monitor if parameters are not varied.

### CONCLUSIONS¶

Based on the results, two conclusions are drawn. The first is that when agents tackle additional tasks relying on common properties, the agents may transfer knowledge from one task to another. The second is that when agents tackle additional tasks that rely on different properties, the number of assigned tasks does not affect their accuracy on their best task.