20180927-NOOR

Experiment design

Designer: Jérôme Euzenat (INRIA) (2017-10-01)

Revision of networks of ontologies with 5 agents relaxation, expansion and generation (5 agents; 10 runs; 100000 games; delete/replace/refine/add/addjoin/refadd)

Hypotheses: With 5 agents processes converge within 100000 games

Experimental setting: Same as 20171231-NOOR after correction of expansion (48054fa4b73f6eb5aebb5fcda4bd7c6831f0040f) and add behaviour. Results are also compared with the same set of games (and obviously different initial situations)

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2018-09-27)

Date: 2018-09-27 -- 2018-10-29

Lazy lavender hash: 759ff097b96520c12aa84f3749927f9a22022e62

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20171231-NOOR

Parameters: params.sh

Command line (script.sh):

. params.sh

# bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-nothing java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=nothing -DsaveDir=input/expeRun -DsaveGames -DloadDir=input/expeRun -DloadEnv -DloadAgents

for op in ${OPS}
do
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-gen java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-gen-empty java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative -Dstartempty
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -Dstartempty
done

# bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-nothing2 java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=nothing -DsaveDir=input/expeRun -DloadGames -DloadDir=input/expeRun -DsaveEnv -DloadAgents

for op in ${OPS}
do
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-gen-real java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative -Drealistic
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-real java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -Drealistic
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: 24 * Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz with 20GB RAM / Linux ProxMox 2 / Linux 4.15.17-1-pve / Java Java(TM) SE Runtime Environment 1.8.0_151 with 4.33G max heap size

Note: This experiment has been set up to first generate the runs and then play them with different configuration. To strictly repeat it, it is necessary to reuse the same input and not generate a new one. Hence, the input is saved in the repository, the generating line above must be commented (including in Docker).

Raw results

5-100000-add-clever-nr-gen-empty.tsv 5-100000-add-clever-nr-gen-empty.txt 5-100000-add-clever-nr-gen-real.tsv 5-100000-add-clever-nr-gen-real.txt 5-100000-add-clever-nr-gen.tsv 5-100000-add-clever-nr-gen.txt 5-100000-add-clever-nr-im80-gen-empty.tsv 5-100000-add-clever-nr-im80-gen-empty.txt 5-100000-add-clever-nr-im80-gen-real.tsv 5-100000-add-clever-nr-im80-gen-real.txt 5-100000-add-clever-nr-im80-gen.tsv 5-100000-add-clever-nr-im80-gen.txt 5-100000-addjoin-clever-nr-gen-empty.tsv 5-100000-addjoin-clever-nr-gen-empty.txt 5-100000-addjoin-clever-nr-gen-real.tsv 5-100000-addjoin-clever-nr-gen-real.txt 5-100000-addjoin-clever-nr-gen.tsv 5-100000-addjoin-clever-nr-gen.txt 5-100000-addjoin-clever-nr-im80-gen-empty.tsv 5-100000-addjoin-clever-nr-im80-gen-empty.txt 5-100000-addjoin-clever-nr-im80-gen-real.tsv 5-100000-addjoin-clever-nr-im80-gen-real.txt 5-100000-addjoin-clever-nr-im80-gen.tsv 5-100000-addjoin-clever-nr-im80-gen.txt 5-100000-delete-clever-nr-gen-empty.tsv 5-100000-delete-clever-nr-gen-empty.txt 5-100000-delete-clever-nr-gen-real.tsv 5-100000-delete-clever-nr-gen-real.txt 5-100000-delete-clever-nr-gen.tsv 5-100000-delete-clever-nr-gen.txt 5-100000-delete-clever-nr-im80-gen-empty.tsv 5-100000-delete-clever-nr-im80-gen-empty.txt 5-100000-delete-clever-nr-im80-gen-real.tsv 5-100000-delete-clever-nr-im80-gen-real.txt 5-100000-delete-clever-nr-im80-gen.tsv 5-100000-delete-clever-nr-im80-gen.txt 5-100000-nothing.tsv 5-100000-nothing.txt 5-100000-nothing2.tsv 5-100000-nothing2.txt 5-100000-refadd-clever-nr-gen-empty.tsv 5-100000-refadd-clever-nr-gen-empty.txt 5-100000-refadd-clever-nr-gen-real.tsv 5-100000-refadd-clever-nr-gen-real.txt 5-100000-refadd-clever-nr-gen.tsv 5-100000-refadd-clever-nr-gen.txt 5-100000-refadd-clever-nr-im80-gen-empty.tsv 5-100000-refadd-clever-nr-im80-gen-empty.txt 5-100000-refadd-clever-nr-im80-gen-real.tsv 5-100000-refadd-clever-nr-im80-gen-real.txt 5-100000-refadd-clever-nr-im80-gen.tsv 5-100000-refadd-clever-nr-im80-gen.txt 5-100000-refine-clever-nr-gen-empty.tsv 5-100000-refine-clever-nr-gen-empty.txt 5-100000-refine-clever-nr-gen-real.tsv 5-100000-refine-clever-nr-gen-real.txt 5-100000-refine-clever-nr-gen.tsv 5-100000-refine-clever-nr-gen.txt 5-100000-refine-clever-nr-im80-gen-empty.tsv 5-100000-refine-clever-nr-im80-gen-empty.txt 5-100000-refine-clever-nr-im80-gen-real.tsv 5-100000-refine-clever-nr-im80-gen-real.txt 5-100000-refine-clever-nr-im80-gen.tsv 5-100000-refine-clever-nr-im80-gen.txt 5-100000-replace-clever-nr-gen-empty.tsv 5-100000-replace-clever-nr-gen-empty.txt 5-100000-replace-clever-nr-gen-real.tsv 5-100000-replace-clever-nr-gen-real.txt 5-100000-replace-clever-nr-gen.tsv 5-100000-replace-clever-nr-gen.txt 5-100000-replace-clever-nr-im80-gen-empty.tsv 5-100000-replace-clever-nr-im80-gen-empty.txt 5-100000-replace-clever-nr-im80-gen-real.tsv 5-100000-replace-clever-nr-im80-gen-real.txt 5-100000-replace-clever-nr-im80-gen.tsv 5-100000-replace-clever-nr-im80-gen.txt

Result exploration

The same curves as 20171231-NOOR are given, first without and then with relaxation:

operator test success
rate
network
size
incoherence
degree
semantic
precision
semantic
F-measure
semantic
recall
maximum
convergence
nothing 0.17 170 0.43 0.06 0.11 0.97 1
expansion (clever-nr) + generation (gen)
delete random 0.97 240 0.18 0.78 0.65 0.56 95849
realistic 0.97 237 0.17 0.78 0.64 0.54 87557
empty 0.97 256 0.17 0.78 0.67 0.58 73697
replace random 0.97 234 0.16 0.80 0.65 0.55 69611
realistic 0.97 237 0.15 0.81 0.66 0.56 95856
empty 0.97 251 0.15 0.80 0.67 0.57 79465
refine random 0.96 256 0.18 0.77 0.71 0.65 80344
realistic 0.96 260 0.19 0.77 0.72 0.68 87104
empty 0.97 267 0.20 0.76 0.74 0.71 76069
add random 0.95 268 0.24 0.70 0.76 0.82 83031
realistic 0.95 268 0.24 0.70 0.76 0.83 78558
empty 0.96 285 0.18 0.77 0.74 0.72 82763
addjoin random 0.98 268 0.23 0.71 0.76 0.83 63770
realistic 0.98 271 0.23 0.71 0.76 0.82 88130
empty 0.98 275 0.16 0.80 0.72 0.65 77049
refadd random 0.97 286 0.25 0.68 0.76 0.87 49105
realistic 0.97 289 0.26 0.67 0.76 0.89 54091
empty 0.97 278 0.18 0.76 0.75 0.73 79059
expansion (clever-nr) + generation (gen) + relaxation (im80)
delete random 0.97 195 0.00 1.00 0.47 0.31 87745
realistic 0.97 192 0.00 1.00 0.46 0.30 65303
empty 0.97 205 0.00 1.00 0.48 0.31 83689
replace random 0.97 201 0.00 1.00 0.48 0.32 79061
realistic 0.97 196 0.00 1.00 0.46 0.30 79363
empty 0.97 207 0.00 1.00 0.48 0.31 77274
refine random 0.96 220 0.00 1.00 0.61 0.44 85428
realistic 0.96 219 0.00 1.00 0.60 0.43 87937
empty 0.96 236 0.00 1.00 0.62 0.45 66947
add random 0.94 243 0.00 1.00 0.65 0.48 90625
realistic 0.95 245 0.00 1.00 0.66 0.49 98672
empty 0.96 268 0.00 1.00 0.62 0.45 87619
addjoin random 0.98 246 0.00 1.00 0.66 0.49 64363
realistic 0.98 244 0.00 1.00 0.66 0.49 86706
empty 0.98 256 0.00 1.00 0.62 0.45 97299
refadd empty 0.97 289 0.00 1.00 0.73 0.57 72572
realistic 0.97 292 0.00 1.00 0.76 0.61 72614
random 0.97 296 0.00 1.00 0.76 0.62 60733

Analysis

Analyst: Jérôme Euzenat (INRIA) (2018-10-29)

Key points:

Further experiments:


This file can be retrieved from URL https://sake.re/20180927-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20180927-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr.

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20180927-NOOR.zip