20180917-NOOR

Experiment design

Designer: Jérôme Euzenat (INRIA) (2017-10-01)

Revision of networks of ontologies with 5 agents relaxation, expansion and generation (5 agents; 10 runs; 50000 games; delete/replace/refine/add/addjoin/refadd)

Hypotheses: Threads observed with four agents are the same after fixing expansion and add behaviour

Experimental setting: Same as 20171128-NOOR after correction of expansion and add behaviour

The experiment has been in fact performed in three batches due to mistakes in software or setting

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2018-09-17)

Date: 2018-05-04, 2018-08-30 and 2018-09-17

Lazy lavender hash: 759ff097b96520c12aa84f3749927f9a22022e62

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20171128-NOOR

Parameters: params.sh

Command line (script.sh):

. params.sh

for op in ${OPS}
do
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80 java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-gen java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-gen-empty java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative -Dstartempty
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -Dstartempty
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: 24 * Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz with 20GB RAM / Linux ProxMox 2 / Linux 4.4.98-3-pve and 4.15.17-1-pve / Java Java(TM) SE Runtime Environment 1.8.0_151 with 4.33G max heap size

Raw results

5-50000-add-clever-nr-gen-empty.tsv 5-50000-add-clever-nr-gen-empty.txt 5-50000-add-clever-nr-gen.tsv 5-50000-add-clever-nr-gen.txt 5-50000-add-clever-nr-im80-gen-empty.tsv 5-50000-add-clever-nr-im80-gen-empty.txt 5-50000-add-clever-nr-im80-gen.tsv 5-50000-add-clever-nr-im80-gen.txt 5-50000-add-clever-nr-im80.tsv 5-50000-add-clever-nr-im80.txt 5-50000-add-clever-nr.tsv 5-50000-add-clever-nr.txt 5-50000-addjoin-clever-nr-gen-empty.tsv 5-50000-addjoin-clever-nr-gen-empty.txt 5-50000-addjoin-clever-nr-gen.tsv 5-50000-addjoin-clever-nr-gen.txt 5-50000-addjoin-clever-nr-im80-gen-empty.tsv 5-50000-addjoin-clever-nr-im80-gen-empty.txt 5-50000-addjoin-clever-nr-im80-gen.tsv 5-50000-addjoin-clever-nr-im80-gen.txt 5-50000-addjoin-clever-nr-im80.tsv 5-50000-addjoin-clever-nr-im80.txt 5-50000-addjoin-clever-nr.tsv 5-50000-addjoin-clever-nr.txt 5-50000-delete-clever-nr-gen-empty.tsv 5-50000-delete-clever-nr-gen-empty.txt 5-50000-delete-clever-nr-gen.tsv 5-50000-delete-clever-nr-gen.txt 5-50000-delete-clever-nr-im80-gen-empty.tsv 5-50000-delete-clever-nr-im80-gen-empty.txt 5-50000-delete-clever-nr-im80-gen.tsv 5-50000-delete-clever-nr-im80-gen.txt 5-50000-delete-clever-nr-im80.tsv 5-50000-delete-clever-nr-im80.txt 5-50000-delete-clever-nr.tsv 5-50000-delete-clever-nr.txt 5-50000-nothing.tsv 5-50000-nothing.txt 5-50000-refadd-clever-nr-gen-empty.tsv 5-50000-refadd-clever-nr-gen-empty.txt 5-50000-refadd-clever-nr-gen.tsv 5-50000-refadd-clever-nr-gen.txt 5-50000-refadd-clever-nr-im80-gen-empty.tsv 5-50000-refadd-clever-nr-im80-gen-empty.txt 5-50000-refadd-clever-nr-im80-gen.tsv 5-50000-refadd-clever-nr-im80-gen.txt 5-50000-refadd-clever-nr-im80.tsv 5-50000-refadd-clever-nr-im80.txt 5-50000-refadd-clever-nr.tsv 5-50000-refadd-clever-nr.txt 5-50000-refine-clever-nr-gen-empty.tsv 5-50000-refine-clever-nr-gen-empty.txt 5-50000-refine-clever-nr-gen.tsv 5-50000-refine-clever-nr-gen.txt 5-50000-refine-clever-nr-im80-gen-empty.tsv 5-50000-refine-clever-nr-im80-gen-empty.txt 5-50000-refine-clever-nr-im80-gen.tsv 5-50000-refine-clever-nr-im80-gen.txt 5-50000-refine-clever-nr-im80.tsv 5-50000-refine-clever-nr-im80.txt 5-50000-refine-clever-nr.tsv 5-50000-refine-clever-nr.txt 5-50000-replace-clever-nr-gen-empty.tsv 5-50000-replace-clever-nr-gen-empty.txt 5-50000-replace-clever-nr-gen.tsv 5-50000-replace-clever-nr-gen.txt 5-50000-replace-clever-nr-im80-gen-empty.tsv 5-50000-replace-clever-nr-im80-gen-empty.txt 5-50000-replace-clever-nr-im80-gen.tsv 5-50000-replace-clever-nr-im80-gen.txt 5-50000-replace-clever-nr-im80.tsv 5-50000-replace-clever-nr-im80.txt 5-50000-replace-clever-nr.tsv 5-50000-replace-clever-nr.txt

Result exploration

test success
rate		 network
size		 incoherence
degree		 semantic
precision		 semantic
F-measure		 semantic
recall		 maximum
convergence
nothing 0.17 170 0.43 0.06 0.11 0.97 1
expansion
delete 0.99 41 0.01 0.97 0.15 0.08 7205
replace 0.98 77 0.07 0.88 0.31 0.19 6193
refine 0.96 123 0.10 0.86 0.50 0.35 12917
add 0.96 209 0.24 0.70 0.69 0.67 19114
addjoin 0.98 212 0.25 0.69 0.68 0.68 6910
refadd 0.97 278 0.26 0.65 0.74 0.86 23806
expansion+relaxation
delete 0.99 40 0.00 1.00 0.15 0.08 7619
replace 0.98 72 0.00 1.00 0.21 0.12 9368
refine 0.96 113 0.00 1.00 0.41 0.26 14543
add 0.96 173 0.00 1.00 0.50 0.33 22214
addjoin 0.98 176 0.00 1.00 0.50 0.33 22402
refadd 0.95 292 0.00 1.00 0.76 0.62 48460
expansion+generation
delete 0.94 232 0.17 0.78 0.64 0.54 49727
replace 0.94 236 0.16 0.78 0.66 0.57 49029
refine 0.93 252 0.17 0.79 0.71 0.66 49795
add 0.90 270 0.23 0.70 0.76 0.82 49991
addjoin 0.96 267 0.25 0.70 0.76 0.83 48167
refadd 0.94 287 0.25 0.66 0.76 0.87 49752
expansion+generation+relaxation
delete 0.94 190 0.00 1.00 0.44 0.28 49872
replace 0.94 201 0.00 1.00 0.47 0.31 49933
refine 0.92 216 0.00 1.00 0.60 0.43 49976
add 0.89 249 0.00 0.99 0.65 0.48 49961
addjoin 0.95 244 0.00 1.00 0.66 0.49 49721
refadd 0.93 295 0.00 1.00 0.76 0.62 49909
expansion+generation+startempty
delete 0.94 243 0.17 0.79 0.65 0.56 49876
replace 0.94 245 0.17 0.78 0.65 0.55 49960
refine 0.93 270 0.19 0.77 0.72 0.67 48570
add 0.92 283 0.19 0.77 0.73 0.71 49967
addjoin 0.96 276 0.16 0.80 0.72 0.65 49569
refadd 0.94 279 0.19 0.76 0.75 0.75 49663
expansion+generation+startempty+relaxation
delete 0.94 208 0.00 1.00 0.46 0.30 49723
replace 0.94 201 0.00 1.00 0.46 0.30 49418
refine 0.92 237 0.00 1.00 0.63 0.46 49848
add 0.91 269 0.01 0.99 0.62 0.45 49999
addjoin 0.96 258 0.00 1.00 0.62 0.45 49681
refadd 0.93 293 0.00 1.00 0.74 0.59 49480

Analysis

Analyst: Jérôme Euzenat (INRIA) (2018-09-19)

Key points:

Further experiments:

This file can be retrieved from URL https://sake.re/20180917-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20180917-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20180917-NOOR.zip