20180826-NOOR

Experiment design

Designer: Jérôme Euzenat (INRIA) (2018-05-03)

Revision of networks of ontologies with most specific strenghening (4 agents; 10 runs; 2000 games; add/addjoin)

Hypotheses: Addjoin achieves the same as add but faster

Experimental setting: Same as 20180305-NOOR after correction of expansion... (and likely realistic size estimation)

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2018-08-26)

Date: 20180826

Lazy lavender hash: 759ff097b96520c12aa84f3749927f9a22022e62

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20180305-NOOR

Parameters: params.sh

Command line (script.sh):

. params.sh

# bash scripts/runexp.sh -d ${DIRPREF}-nothing java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=nothing -DsaveDir=expeRun -DsaveInit -DsaveParams -DsaveGames
bash scripts/runexp.sh -d ${DIRPREF}-add java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=add ${LOADOPT}
bash scripts/runexp.sh -d ${DIRPREF}-addjoin java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=addjoin ${LOADOPT}
bash scripts/runexp.sh -d ${DIRPREF}-add-syntactic java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=add -Dsyntactic=true ${LOADOPT} -DsaveDir=addRes -DsaveFinal
bash scripts/runexp.sh -d ${DIRPREF}-addjoin-syntactic java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=addjoin -Dsyntactic=true ${LOADOPT} -DsaveDir=addjoinRes -DsaveFinal

# bash scripts/runexp.sh -d ${DIRPREF}-real-nothing java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=nothing -DsaveDir=expeRun -DsaveInit -DsaveParams -DsaveGames
bash scripts/runexp.sh -d ${DIRPREF}-real-add java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=add ${LOADOPT}
bash scripts/runexp.sh -d ${DIRPREF}-real-addjoin java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=addjoin ${LOADOPT}
bash scripts/runexp.sh -d ${DIRPREF}-real-add-syntactic java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=add -Dsyntactic=true ${LOADOPT} -DsaveDir=addRes -DsaveFinal
bash scripts/runexp.sh -d ${DIRPREF}-real-addjoin-syntactic java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=addjoin -Dsyntactic=true ${LOADOPT} -DsaveDir=addjoinRes -DsaveFinal

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: This is the first time that an experiment is rerun using the Docker environment instead as being replayed directly. The Docker file generated for 20180305-NOOR is found in the zip file. It has been tweaked, in particular to run the experiments on 2000 runs and with non realistic size

Note: This experiment has been set up to first generate the runs and then play them with different configuration. To strictly repeat it, it is necessary to reuse the same input and not generate a new one. Hence, the input is saved in the repository, the generating line above must be commented (including in Docker).

Raw results

4-2000-add-syntactic.tsv 4-2000-add-syntactic.txt 4-2000-add.tsv 4-2000-add.txt 4-2000-addjoin-syntactic.tsv 4-2000-addjoin-syntactic.txt 4-2000-addjoin.tsv 4-2000-addjoin.txt 4-2000-nothing.tsv 4-2000-nothing.txt 4-2000-real-add-syntactic.tsv 4-2000-real-add-syntactic.txt 4-2000-real-add.tsv 4-2000-real-add.txt 4-2000-real-addjoin-syntactic.tsv 4-2000-real-addjoin-syntactic.txt 4-2000-real-addjoin.tsv 4-2000-real-addjoin.txt 4-2000-real-nothing.tsv 4-2000-real-nothing.txt

Result exploration

Results without -Drealistic, i.e., commensurate with most experiments so far:

First results with -Drealistic as in 20180305-NOOR:

test success
rate
network
size
incoherence
degree
semantic
precision
semantic
F-measure
semantic
recall
syntactic
precision
syntactic
F-measure
syntactic
recall
maximum
convergence
4-2000-nothing 0.26 54 0.36 0.11 0.20 0.89 1
4-2000-add 0.91 32 0.14 0.80 0.50 0.36 0.65 0.35 0.24 1572
4-2000-addjoin 0.95 32 0.14 0.80 0.50 0.36 0.65 0.35 0.24 1361
4-2000-real-nothing 0.53 92 0.36 0.24 0.38 0.90 1
4-2000-real-add 0.94 45 0.16 0.81 0.57 0.44 0.49 0.34 0.26 1261
4-2000-real-addjoin 0.97 45 0.16 0.81 0.57 0.44 0.49 0.34 0.26 1012

Analysis

Analyst: Jérôme Euzenat (INRIA) (2018-08-26)

Key points: This experiment fully confirms 20180305-NOOR, i.e., addjoin is now working properly, with the observation that:

Further experiments: None


This file can be retrieved from URL https://sake.re/20180826-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20180826-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr.

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20180826-NOOR.zip