20190606-NOOR

Experiment design

Designer: Jérôme Euzenat (INRIA) (2018-09-18)

Revision of networks of ontologies with most general and most specific strenghening (4 agents; 10 runs; 20000 games; delete/replace/refine/add/addjoin/refadd)

Hypotheses: Strenghening with refadd, relaxation, expansion, generation reaches 100% recall

Experimental setting: Repeating 20180913-NOOR with rewritten software

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2019-08-28)

Date: 2019-08-28

Lazy lavender hash: 8c16edba8faa0eae88508f2893896ca701f9d9c3

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20180913-NOOR

Parameters: params.sh

Command line (script.sh):

. params.sh

OUTPUT=${OUTPUT}${LABEL}

for op in ${OPS}
do
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-im80${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -DstartEmpty ${PARAMS}
done

PARAMS="-Dstrengthen=mostspecific"
postfix=-strspc
for op in ${OPS}
do
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-im80${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -DstartEmpty ${PARAMS}
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: 24 * Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz with 20GB RAM / Linux 4.15.17-1-pve / Java(TM) SE Runtime Environment 1.8.0_151 with 4.33G max heap size

Input: Input required for reproducibility can be retrieved from: https://files.inria.fr/sakere/input/expeRun2.zip Then unzip expeRun2.zip -d input

Raw results

4-20000-add-clever-nr-im80-gen-empty-strgen.tsv 4-20000-add-clever-nr-im80-gen-empty-strgen.txt 4-20000-add-clever-nr-im80-gen-empty-strspc.tsv 4-20000-add-clever-nr-im80-gen-empty-strspc.txt 4-20000-add-clever-nr-im80-gen-strgen.tsv 4-20000-add-clever-nr-im80-gen-strgen.txt 4-20000-add-clever-nr-im80-gen-strspc.tsv 4-20000-add-clever-nr-im80-gen-strspc.txt 4-20000-add-clever-nr-im80-strgen.tsv 4-20000-add-clever-nr-im80-strgen.txt 4-20000-add-clever-nr-im80-strspc.tsv 4-20000-add-clever-nr-im80-strspc.txt 4-20000-add-clever-nr-strgen.tsv 4-20000-add-clever-nr-strgen.txt 4-20000-add-clever-nr-strspc.tsv 4-20000-add-clever-nr-strspc.txt 4-20000-add-im80-strgen.tsv 4-20000-add-im80-strgen.txt 4-20000-add-im80-strspc.tsv 4-20000-add-im80-strspc.txt 4-20000-addjoin-clever-nr-im80-gen-empty-strgen.tsv 4-20000-addjoin-clever-nr-im80-gen-empty-strgen.txt 4-20000-addjoin-clever-nr-im80-gen-empty-strspc.tsv 4-20000-addjoin-clever-nr-im80-gen-empty-strspc.txt 4-20000-addjoin-clever-nr-im80-gen-strgen.tsv 4-20000-addjoin-clever-nr-im80-gen-strgen.txt 4-20000-addjoin-clever-nr-im80-gen-strspc.tsv 4-20000-addjoin-clever-nr-im80-gen-strspc.txt 4-20000-addjoin-clever-nr-im80-strgen.tsv 4-20000-addjoin-clever-nr-im80-strgen.txt 4-20000-addjoin-clever-nr-im80-strspc.tsv 4-20000-addjoin-clever-nr-im80-strspc.txt 4-20000-addjoin-clever-nr-strgen.tsv 4-20000-addjoin-clever-nr-strgen.txt 4-20000-addjoin-clever-nr-strspc.tsv 4-20000-addjoin-clever-nr-strspc.txt 4-20000-addjoin-im80-strgen.tsv 4-20000-addjoin-im80-strgen.txt 4-20000-addjoin-im80-strspc.tsv 4-20000-addjoin-im80-strspc.txt 4-20000-delete-clever-nr-im80-gen-empty-strgen.tsv 4-20000-delete-clever-nr-im80-gen-empty-strgen.txt 4-20000-delete-clever-nr-im80-gen-empty-strspc.tsv 4-20000-delete-clever-nr-im80-gen-empty-strspc.txt 4-20000-delete-clever-nr-im80-gen-strgen.tsv 4-20000-delete-clever-nr-im80-gen-strgen.txt 4-20000-delete-clever-nr-im80-gen-strspc.tsv 4-20000-delete-clever-nr-im80-gen-strspc.txt 4-20000-delete-clever-nr-im80-strgen.tsv 4-20000-delete-clever-nr-im80-strgen.txt 4-20000-delete-clever-nr-im80-strspc.tsv 4-20000-delete-clever-nr-im80-strspc.txt 4-20000-delete-clever-nr-strgen.tsv 4-20000-delete-clever-nr-strgen.txt 4-20000-delete-clever-nr-strspc.tsv 4-20000-delete-clever-nr-strspc.txt 4-20000-delete-im80-strgen.tsv 4-20000-delete-im80-strgen.txt 4-20000-delete-im80-strspc.tsv 4-20000-delete-im80-strspc.txt 4-20000-refadd-clever-nr-im80-gen-empty-strgen.tsv 4-20000-refadd-clever-nr-im80-gen-empty-strgen.txt 4-20000-refadd-clever-nr-im80-gen-empty-strspc.tsv 4-20000-refadd-clever-nr-im80-gen-empty-strspc.txt 4-20000-refadd-clever-nr-im80-gen-strgen.tsv 4-20000-refadd-clever-nr-im80-gen-strgen.txt 4-20000-refadd-clever-nr-im80-gen-strspc.tsv 4-20000-refadd-clever-nr-im80-gen-strspc.txt 4-20000-refadd-clever-nr-im80-strgen.tsv 4-20000-refadd-clever-nr-im80-strgen.txt 4-20000-refadd-clever-nr-im80-strspc.tsv 4-20000-refadd-clever-nr-im80-strspc.txt 4-20000-refadd-clever-nr-strgen.tsv 4-20000-refadd-clever-nr-strgen.txt 4-20000-refadd-clever-nr-strspc.tsv 4-20000-refadd-clever-nr-strspc.txt 4-20000-refadd-im80-strgen.tsv 4-20000-refadd-im80-strgen.txt 4-20000-refadd-im80-strspc.tsv 4-20000-refadd-im80-strspc.txt 4-20000-refine-clever-nr-im80-gen-empty-strgen.tsv 4-20000-refine-clever-nr-im80-gen-empty-strgen.txt 4-20000-refine-clever-nr-im80-gen-empty-strspc.tsv 4-20000-refine-clever-nr-im80-gen-empty-strspc.txt 4-20000-refine-clever-nr-im80-gen-strgen.tsv 4-20000-refine-clever-nr-im80-gen-strgen.txt 4-20000-refine-clever-nr-im80-gen-strspc.tsv 4-20000-refine-clever-nr-im80-gen-strspc.txt 4-20000-refine-clever-nr-im80-strgen.tsv 4-20000-refine-clever-nr-im80-strgen.txt 4-20000-refine-clever-nr-im80-strspc.tsv 4-20000-refine-clever-nr-im80-strspc.txt 4-20000-refine-clever-nr-strgen.tsv 4-20000-refine-clever-nr-strgen.txt 4-20000-refine-clever-nr-strspc.tsv 4-20000-refine-clever-nr-strspc.txt 4-20000-refine-im80-strgen.tsv 4-20000-refine-im80-strgen.txt 4-20000-refine-im80-strspc.tsv 4-20000-refine-im80-strspc.txt 4-20000-replace-clever-nr-im80-gen-empty-strgen.tsv 4-20000-replace-clever-nr-im80-gen-empty-strgen.txt 4-20000-replace-clever-nr-im80-gen-empty-strspc.tsv 4-20000-replace-clever-nr-im80-gen-empty-strspc.txt 4-20000-replace-clever-nr-im80-gen-strgen.tsv 4-20000-replace-clever-nr-im80-gen-strgen.txt 4-20000-replace-clever-nr-im80-gen-strspc.tsv 4-20000-replace-clever-nr-im80-gen-strspc.txt 4-20000-replace-clever-nr-im80-strgen.tsv 4-20000-replace-clever-nr-im80-strgen.txt 4-20000-replace-clever-nr-im80-strspc.tsv 4-20000-replace-clever-nr-im80-strspc.txt 4-20000-replace-clever-nr-strgen.tsv 4-20000-replace-clever-nr-strgen.txt 4-20000-replace-clever-nr-strspc.tsv 4-20000-replace-clever-nr-strspc.txt 4-20000-replace-im80-strgen.tsv 4-20000-replace-im80-strgen.txt 4-20000-replace-im80-strspc.tsv 4-20000-replace-im80-strspc.txt

Result exploration

All results (most specific: plain, most general: dashed).

First recall is compared to no strenghening (expansion 20190523-NOOR, relaxation: 20180311-NOOR, expansion+relaxation 20190524-NOOR, expansion+relaxation+generation 20190624-NOOR, expansion+relaxation+generation+start empty 20190712-NOOR).

Comparison with the previous results. Here we only compare the most specific values (plain) with 20181110-NOOR (dashed). It is also possible to compare the most specific with 20180829-NOOR, but the values of each pairs are so close that this would not help readability.

test success
rate
network
size
incoherence
degree
semantic
precision
semantic
F-measure
semantic
recall
maximum
convergence
relaxation+most general strenghening
4-20000-delete-im80-strgen 1.00 6 0.00 1.00 0.13 0.07 737
4-20000-replace-im80-strgen 1.00 6 0.00 1.00 0.13 0.07 816
4-20000-refine-im80-strgen 0.99 6 0.00 1.00 0.13 0.07 1483
4-20000-add-im80-strgen 0.98 25 0.00 1.00 0.46 0.30 4869
4-20000-addjoin-im80-strgen 0.99 25 0.00 1.00 0.46 0.30 4178
4-20000-refadd-im80-strgen 0.98 60 0.00 1.00 0.82 0.70 8286
relaxation+most specific strenghening
4-20000-delete-im80-strspc 1.00 6 0.00 1.00 0.13 0.07 737
4-20000-replace-im80-strspc 1.00 6 0.00 1.00 0.13 0.07 849
4-20000-refine-im80-strspc 0.99 6 0.00 1.00 0.13 0.07 1911
4-20000-add-im80-strspc 0.98 25 0.00 1.00 0.46 0.30 4435
4-20000-addjoin-im80-strspc 0.99 25 0.00 1.00 0.46 0.30 5974
4-20000-refadd-im80-strspc 0.98 59 0.00 1.00 0.82 0.69 6738
expansion+most general strenghening (Compare to 20190523-NOOR)
4-20000-delete-clever-nr-strgen 0.99 6 0.00 1.00 0.13 0.07 2309
4-20000-replace-clever-nr-strgen 0.98 6 0.00 1.00 0.13 0.07 2391
4-20000-refine-clever-nr-strgen 0.97 8 0.00 1.00 0.18 0.10 4540
4-20000-add-clever-nr-strgen 0.97 38 0.26 0.69 0.60 0.54 5803
4-20000-addjoin-clever-nr-strgen 0.98 45 0.31 0.64 0.65 0.67 3362
4-20000-refadd-clever-nr-strgen 0.97 64 0.37 0.60 0.74 0.94 3999
expansion+most specific strenghening (Compare to 20190523-NOOR)
4-20000-delete-clever-nr-strspc 0.99 6 0.00 1.00 0.13 0.07 1821
4-20000-replace-clever-nr-strspc 0.98 6 0.00 1.00 0.13 0.07 2231
4-20000-refine-clever-nr-strspc 0.97 8 0.01 0.98 0.17 0.10 4428
4-20000-add-clever-nr-strspc 0.97 40 0.28 0.66 0.63 0.60 3373
4-20000-addjoin-clever-nr-strspc 0.98 46 0.30 0.65 0.66 0.66 2901
4-20000-refadd-clever-nr-strspc 0.97 64 0.35 0.62 0.75 0.95 3816
expansion+relaxation+most general strenghening (Compare to 20190524-NOOR)
4-20000-delete-clever-nr-im80-strgen 0.99 6 0.00 1.00 0.13 0.07 1911
4-20000-replace-clever-nr-im80-strgen 0.98 6 0.00 1.00 0.13 0.07 2712
4-20000-refine-clever-nr-im80-strgen 0.97 7 0.00 1.00 0.17 0.09 4164
4-20000-add-clever-nr-im80-strgen 0.97 30 0.00 1.00 0.52 0.35 6999
4-20000-addjoin-clever-nr-im80-strgen 0.98 35 0.00 1.00 0.59 0.42 4406
4-20000-refadd-clever-nr-im80-strgen 0.96 81 0.00 1.00 0.97 0.95 13037
expansion+relaxation+most specific strenghening (Compare to 20190524-NOOR)
4-20000-delete-clever-nr-im80-strspc 0.99 6 0.00 1.00 0.13 0.07 2357
4-20000-replace-clever-nr-im80-strspc 0.98 6 0.00 1.00 0.13 0.07 1983
4-20000-refine-clever-nr-im80-strspc 0.97 7 0.00 1.00 0.16 0.09 4818
4-20000-add-clever-nr-im80-strspc 0.97 30 0.00 1.00 0.52 0.35 7513
4-20000-addjoin-clever-nr-im80-strspc 0.98 37 0.00 1.00 0.61 0.44 5948
4-20000-refadd-clever-nr-im80-strspc 0.96 81 0.00 1.00 0.98 0.95 9562
expansion+relaxation+generation+most general strenghening (Compare to 20190624-NOOR)
4-20000-delete-clever-nr-im80-gen-strgen 0.95 26 0.00 1.00 0.47 0.31 14374
4-20000-replace-clever-nr-im80-gen-strgen 0.95 29 0.00 1.00 0.52 0.35 9872
4-20000-refine-clever-nr-im80-gen-strgen 0.94 31 0.00 1.00 0.53 0.36 13341
4-20000-add-clever-nr-im80-gen-strgen 0.93 61 0.00 1.00 0.83 0.71 16064
4-20000-addjoin-clever-nr-im80-gen-strgen 0.96 65 0.00 1.00 0.87 0.76 13124
4-20000-refadd-clever-nr-im80-gen-strgen 0.95 84 0.00 1.00 0.99 0.98 17229
expansion+relaxation+generation+most specific strenghening (Compare to 20190624-NOOR)
4-20000-delete-clever-nr-im80-gen-strspc 0.95 30 0.00 1.00 0.53 0.36 12077
4-20000-replace-clever-nr-im80-gen-strspc 0.95 31 0.00 1.00 0.53 0.36 14652
4-20000-refine-clever-nr-im80-gen-strspc 0.93 33 0.00 1.00 0.56 0.39 13945
4-20000-add-clever-nr-im80-gen-strspc 0.93 59 0.00 1.00 0.82 0.69 16589
4-20000-addjoin-clever-nr-im80-gen-strspc 0.96 66 0.00 1.00 0.87 0.77 15729
4-20000-refadd-clever-nr-im80-gen-strspc 0.95 84 0.00 1.00 0.99 0.98 13216
expansion+relaxation+generation+empty+most general strenghening (Compare to 20190712-NOOR)
4-20000-delete-clever-nr-im80-gen-empty-strgen 0.95 34 0.00 1.00 0.49 0.33 12508
4-20000-replace-clever-nr-im80-gen-empty-strgen 0.95 35 0.00 1.00 0.52 0.35 9924
4-20000-refine-clever-nr-im80-gen-empty-strgen 0.93 39 0.00 1.00 0.56 0.39 12673
4-20000-add-clever-nr-im80-gen-empty-strgen 0.93 75 0.00 1.00 0.90 0.81 15128
4-20000-addjoin-clever-nr-im80-gen-empty-strgen 0.96 74 0.00 1.00 0.88 0.79 10769
4-20000-refadd-clever-nr-im80-gen-empty-strgen 0.95 89 0.00 1.00 0.98 0.97 18576
expansion+relaxation+generation+empty+most specific strenghening (Compare to 20190712-NOOR)
4-20000-delete-clever-nr-im80-gen-empty-strspc 0.95 34 0.00 1.00 0.50 0.34 13447
4-20000-replace-clever-nr-im80-gen-empty-strspc 0.95 37 0.00 1.00 0.54 0.37 10673
4-20000-refine-clever-nr-im80-gen-empty-strspc 0.93 41 0.00 1.00 0.58 0.41 10760
4-20000-add-clever-nr-im80-gen-empty-strspc 0.94 73 0.00 1.00 0.88 0.78 16353
4-20000-addjoin-clever-nr-im80-gen-empty-strspc 0.96 74 0.00 1.00 0.89 0.80 11969
4-20000-refadd-clever-nr-im80-gen-empty-strspc 0.95 87 0.00 1.00 0.97 0.95 12417

Analysis

Analyst: Jérôme Euzenat (INRIA) (2019-09-10)

Key points: There are many hypotheses that may be tested with this data:

Further experiments:


This file can be retrieved from URL https://sake.re/20190606-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20190606-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr.

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20190606-NOOR.zip