Experiment 20250611-CLWG¶

Fewer cards drawn and risky strategy generate more diversity, not longer transmission chains

This experiment simulates transmission chain with the Class? game. For that purpose agents transmit a classifiction through the communication of card samples and their distribution in the classification leaves. From a classification and a set of cards, their distribution can be computed deterministically. From a distribution, the classification can only be guessed, hence there are several outcomes creating variation.

The goal of the experiment is to assess the diversity of the classifications obtained through transmission chains.

For that purpose, from a classification $C$, $N=$10 transmission chains of length $I=$20 are played in parallel so that it is possible to compute the diversity across the $N$ chains. This experiment is repeated $R=$10 times and the diversity values averaged.

Three parameters are tested in this experiment which are:

the length of the chain: from 1 to 20,
the size $S$ of sample cards: from 1 to 25,
the strategy to compute the classification: 0=conservative or 1=risky

Diversity is assessed using our knowledge diversity measure [Bourahla et al., 2022c].

Experiment design¶

Date: 2025-06-11

Designer: Adrien Bonnardel (UGA)

Hypotheses: **(1) The longer the transmission chain, the higher the diversity,

(2) The fewer cards drawn, the higher the diversity,

(3) The riskier the strategy, the higher the diversity**

10 runs; 20 games

There are 3 different seed classifications used (classif0, classif1, classif2)

Experimental setting: Computer-based transmission chain experiment in which agents communicate a classification through providing examples of class assignments

Variables¶

fixed variables: SEED NBITERATIONS NBAGENTS NBRUNS

controled variables: NBCARDS STRATEGY

dependent variables: DIVERSITY

NBCARDS takes values between 1 and 25

STRATEGY takes 1 for risky and 0 for conservative

Measures¶

The main measure that is computed is the diversity of the set of ontologies that agents have.

More on the diversity measure can be found at the knowledge diversity software site (https://gitlab.inria.fr/moex/kdiv) which is used here.

The measure is computed withing the experiment.

Compilation¶

The experiment is 'described' wi The code has to be compiled.

It relies on the classapp and kdiv libraries and the experiments where run with the following hashes:

KDiv hash: 41476a476601bd38b0695136863d05d0fc2feb16

Classapp hash: f640e28ef9ba4baa83d4b6a634fadf36ec8a8a4f

They can be checked out at the correct hash and recompiled before compiling the experiment sources (in Java).

This compilation process may be carried out by the script (compile.sh) whose content is displayed below:

#!/bin/bash

. params.sh

# if submodules dirs are empty:
# git submodule update --init --recursive --rebase --force

cd kdiv
git checkout ${KDIVHASH}
ant jar
cd ..

cd classapp
git checkout ${CLASSHASH}
ant compileall
cd ..

javac -d ./build/classes -classpath ${CLASSPATH} ./src/fr/inria/moex/transchains/Expe.java
mkdir -p build/jars
jar -c -f ./build/jars/TransChain.jar  --main-class=fr.inria.moex.transchains.Expe -C build/classes .

A legacy ant script (build.xml) is also available.

Experiment¶

Date: 2025-07-10 (Jerome Euzenat (INRIA))

Parameter file: params.sh

Executed command (script.sh):

#!/bin/bash

. params.sh

mkdir -p ${RESDIR}

# run

date > ${RESDIR}/log.txt

java -cp ${CLASSPATH}:./build/jars/TransChain.jar fr.inria.moex.transchains.Expe

date >> ${RESDIR}/log.txt

# analyse

Raw results¶

results/
  log.txt
results/classif0/
  data_classif.csv
  data_classif.txt
results/classif1/
  data_classif.csv
  data_classif.txt
results/classif2/
  data_classif.csv
  data_classif.txt

Analysis¶

Experiment classif2 analysis

	sum_sq	df	F	PR(>F)
C(strategy)	0.003167	1.0	2.443692	1.186572e-01
C(cards)	1.295272	24.0	41.641957	3.246402e-101
C(iteration)	0.021521	20.0	0.830274	6.769304e-01
C(strategy):C(cards)	1.013039	24.0	32.568377	4.978958e-85
C(strategy):C(iteration)	0.001957	20.0	0.075492	1.000000e+00
C(cards):C(iteration)	0.352389	480.0	0.566451	1.000000e+00
Residual	0.622100	480.0	NaN	NaN

No description has been provided for this image

Experiment classif1 analysis

	sum_sq	df	F	PR(>F)
C(strategy)	1.931813	1.0	1019.745857	7.908406e-121
C(cards)	4.747806	24.0	104.425967	2.157602e-173
C(iteration)	3.265008	20.0	86.174952	1.372595e-144
C(strategy):C(cards)	3.398293	24.0	74.743995	2.793380e-145
C(strategy):C(iteration)	0.244034	20.0	6.440918	1.434139e-15
C(cards):C(iteration)	1.461375	480.0	1.607115	1.167404e-07
Residual	0.909315	480.0	NaN	NaN

Experiment classif0 analysis

	sum_sq	df	F	PR(>F)
C(strategy)	3.072718	1.0	3109.900219	7.417347e-212
C(cards)	6.202686	24.0	261.572604	4.940576e-258
C(iteration)	1.528620	20.0	77.355848	5.474587e-136
C(strategy):C(cards)	2.081565	24.0	87.781379	1.443506e-158
C(strategy):C(iteration)	0.239887	20.0	12.139493	1.112971e-31
C(cards):C(iteration)	1.122760	480.0	2.367387	7.334664e-21
Residual	0.474261	480.0	NaN	NaN

All the experiments together¶

	sum_sq	df	F	PR(>F)
C(strategy)	1.058523	1.0	1437.279068	1.891396e-146
C(cards)	1.424889	24.0	80.614026	1.795155e-151
C(iteration)	1.084646	20.0	73.637444	3.871935e-132
C(strategy):C(cards)	0.829790	24.0	46.945907	1.217000e-109
C(strategy):C(iteration)	0.099002	20.0	6.721343	2.131719e-16
C(cards):C(iteration)	0.291156	480.0	0.823617	9.831204e-01
Residual	0.353509	480.0	NaN	NaN

Conclusion¶

Considering the raised hypotheses:

(1) The longer the transmission chain, the higher the diversity
- The results do not strictly support this hypothesis: diversity first increases largely and then progressively decreases. The reason is due to the degradation of information which make resulting classifications more similar (this can be tested by adapting the number of cards to the size of the classification).
(2) The fewer cards drawn, the higher the diversity
- This is supported, but not strictly monotonically, likely for the same reasons as before.
(3) The riskier the strategy, the higher the diversity
- From the plots above, this hypothesis is supported.

There are some caveats to these results:

there are run only with three different (relatively small) classifications
they are random and not seed controlled (however very robust)
the results with the number of cards drawn are very dependent on the size of the classification

Hence, more experiments are needed.

This file can be retrieved from URL https://sake.re/20250611-CLWG