Revision of networks of ontologies with most specific strenghening (4 agents; 10 runs; 2000 games; add/addjoin)Designer: Jérôme Euzenat (INRIA) (2018-05-03)
Revision of networks of ontologies with most specific strenghening (4 agents; 10 runs; 2000 games; add/addjoin)
Hypotheses: Addjoin achieves the same as add but faster
Experimental setting: Same as 20180305-NOOR after correction of expansion... (and likely realistic size estimation)
Experimenter: Jérôme Euzenat (INRIA) (2018-08-26)
Date: 20180826
Lazy lavender hash: 759ff097b96520c12aa84f3749927f9a22022e62
Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.
OS: stretch
Variation of: 20180305-NOOR
Parameters: params.sh
Command line (script.sh):
. params.sh
# bash scripts/runexp.sh -d ${DIRPREF}-nothing java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=nothing -DsaveDir=expeRun -DsaveInit -DsaveParams -DsaveGames
bash scripts/runexp.sh -d ${DIRPREF}-add java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=add ${LOADOPT}
bash scripts/runexp.sh -d ${DIRPREF}-addjoin java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=addjoin ${LOADOPT}
bash scripts/runexp.sh -d ${DIRPREF}-add-syntactic java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=add -Dsyntactic=true ${LOADOPT} -DsaveDir=addRes -DsaveFinal
bash scripts/runexp.sh -d ${DIRPREF}-addjoin-syntactic java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=addjoin -Dsyntactic=true ${LOADOPT} -DsaveDir=addjoinRes -DsaveFinal
# bash scripts/runexp.sh -d ${DIRPREF}-nothing-real java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=nothing -DsaveDir=expeRun -DsaveInit -DsaveParams -DsaveGames
bash scripts/runexp.sh -d ${DIRPREF}-add-real java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=add ${LOADOPT}
bash scripts/runexp.sh -d ${DIRPREF}-addjoin-real java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=addjoin ${LOADOPT}
bash scripts/runexp.sh -d ${DIRPREF}-add-syntactic-real java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=add -Dsyntactic=true ${LOADOPT} -DsaveDir=addRes -DsaveFinal
bash scripts/runexp.sh -d ${DIRPREF}-addjoin-syntactic-real java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=addjoin -Dsyntactic=true ${LOADOPT} -DsaveDir=addjoinRes -DsaveFinal
Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor
Execution environment: This is the first time that an experiment is rerun using the Docker environment instead as being replayed directly. The Docker file generated for 20180305-NOOR is found in the zip file. It has been tweaked, in particular to run the experiments on 2000 runs and with non realistic size
Note: This experiment has been set up to first generate the runs and then play them with different configuration. To strictly repeat it, it is necessary to reuse the same input and not generate a new one. Hence, the input is saved in the repository, the generating line above must be commented (including in Docker).
4-2000-add-syntactic.tsv 4-2000-add-syntactic.txt 4-2000-add.tsv 4-2000-add.txt 4-2000-addjoin-syntactic.tsv 4-2000-addjoin-syntactic.txt 4-2000-addjoin.tsv 4-2000-addjoin.txt 4-2000-nothing.tsv 4-2000-nothing.txt 4-2000-add-syntactic-real.tsv 4-2000-add-syntactic-real.txt 4-2000-add-real.tsv 4-2000-add-real.txt 4-2000-addjoin-syntactic-real.tsv 4-2000-addjoin-syntactic-real.txt 4-2000-addjoin-real.tsv 4-2000-addjoin-real.txt 4-2000-nothing-real.tsv 4-2000-nothing-real.txt
Results without -Drealistic, i.e., commensurate with most experiments so far:
First results with -Drealistic as in 20180305-NOOR:
| test | success rate |
network size |
incoherence degree |
semantic precision |
semantic F-measure |
semantic recall |
syntactic precision |
syntactic F-measure |
syntactic recall |
maximum convergence |
|---|---|---|---|---|---|---|---|---|---|---|
| 4-2000-nothing | 0.26 | 54 | 0.36 | 0.11 | 0.20 | 0.89 | 1 | |||
| 4-2000-add | 0.91 | 32 | 0.14 | 0.80 | 0.50 | 0.36 | 0.65 | 0.35 | 0.24 | 1572 |
| 4-2000-addjoin | 0.95 | 32 | 0.14 | 0.80 | 0.50 | 0.36 | 0.65 | 0.35 | 0.24 | 1361 |
| 4-2000-nothing-real | 0.53 | 92 | 0.36 | 0.24 | 0.38 | 0.90 | 1 | |||
| 4-2000-add-real | 0.94 | 45 | 0.16 | 0.81 | 0.57 | 0.44 | 0.49 | 0.34 | 0.26 | 1261 |
| 4-2000-addjoin-real | 0.97 | 45 | 0.16 | 0.81 | 0.57 | 0.44 | 0.49 | 0.34 | 0.26 | 1012 |
Analyst: Jérôme Euzenat (INRIA) (2018-08-26)
Key points: This experiment fully confirms 20180305-NOOR, i.e., addjoin is now working properly, with the observation that:
Further experiments: None
This file can be retrieved from URL https://sake.re/20180826-NOOR
It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20180826-NOOR.git
This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)
The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20180826-NOOR.zip