20190606-NOOR

Experiment design

Designer: Jérôme Euzenat (INRIA) (2018-09-18)

Revision of networks of ontologies with most general and most specific strenghening (4 agents; 10 runs; 20000 games; delete/replace/refine/add/addjoin/refadd)

Hypotheses: Strenghening with refadd, relaxation, expansion, generation reaches 100% recall

Experimental setting: Repeating 20180913-NOOR with rewritten software

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2019-08-28)

Date: 2019-08-28

Lazy lavender hash: 8c16edba8faa0eae88508f2893896ca701f9d9c3

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20180913-NOOR

Parameters: params.sh

Command line (script.sh):

. params.sh

OUTPUT=${OUTPUT}${LABEL}

for op in ${OPS}
do
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-im80${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -DstartEmpty ${PARAMS}
done

PARAMS="-Dstrengthen=mostspecific"
postfix=-strspc
for op in ${OPS}
do
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-im80${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -DstartEmpty ${PARAMS}
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: 24 * Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz with 20GB RAM / Linux 4.15.17-1-pve / Java(TM) SE Runtime Environment 1.8.0_151 with 4.33G max heap size

Input: Input required for reproducibility can be retrieved from: https://files.inria.fr/sakere/input/expeRun2.zip Then unzip expeRun2.zip -d input

Raw results

4-20000-add-clever-nr-im80-gen-empty-strgen.tsv 4-20000-add-clever-nr-im80-gen-empty-strgen.txt 4-20000-add-clever-nr-im80-gen-empty-strspc.tsv 4-20000-add-clever-nr-im80-gen-empty-strspc.txt 4-20000-add-clever-nr-im80-gen-strgen.tsv 4-20000-add-clever-nr-im80-gen-strgen.txt 4-20000-add-clever-nr-im80-gen-strspc.tsv 4-20000-add-clever-nr-im80-gen-strspc.txt 4-20000-add-clever-nr-im80-strgen.tsv 4-20000-add-clever-nr-im80-strgen.txt 4-20000-add-clever-nr-im80-strspc.tsv 4-20000-add-clever-nr-im80-strspc.txt 4-20000-add-clever-nr-strgen.tsv 4-20000-add-clever-nr-strgen.txt 4-20000-add-clever-nr-strspc.tsv 4-20000-add-clever-nr-strspc.txt 4-20000-add-im80-strgen.tsv 4-20000-add-im80-strgen.txt 4-20000-add-im80-strspc.tsv 4-20000-add-im80-strspc.txt 4-20000-addjoin-clever-nr-im80-gen-empty-strgen.tsv 4-20000-addjoin-clever-nr-im80-gen-empty-strgen.txt 4-20000-addjoin-clever-nr-im80-gen-empty-strspc.tsv 4-20000-addjoin-clever-nr-im80-gen-empty-strspc.txt 4-20000-addjoin-clever-nr-im80-gen-strgen.tsv 4-20000-addjoin-clever-nr-im80-gen-strgen.txt 4-20000-addjoin-clever-nr-im80-gen-strspc.tsv 4-20000-addjoin-clever-nr-im80-gen-strspc.txt 4-20000-addjoin-clever-nr-im80-strgen.tsv 4-20000-addjoin-clever-nr-im80-strgen.txt 4-20000-addjoin-clever-nr-im80-strspc.tsv 4-20000-addjoin-clever-nr-im80-strspc.txt 4-20000-addjoin-clever-nr-strgen.tsv 4-20000-addjoin-clever-nr-strgen.txt 4-20000-addjoin-clever-nr-strspc.tsv 4-20000-addjoin-clever-nr-strspc.txt 4-20000-addjoin-im80-strgen.tsv 4-20000-addjoin-im80-strgen.txt 4-20000-addjoin-im80-strspc.tsv 4-20000-addjoin-im80-strspc.txt 4-20000-delete-clever-nr-im80-gen-empty-strgen.tsv 4-20000-delete-clever-nr-im80-gen-empty-strgen.txt 4-20000-delete-clever-nr-im80-gen-empty-strspc.tsv 4-20000-delete-clever-nr-im80-gen-empty-strspc.txt 4-20000-delete-clever-nr-im80-gen-strgen.tsv 4-20000-delete-clever-nr-im80-gen-strgen.txt 4-20000-delete-clever-nr-im80-gen-strspc.tsv 4-20000-delete-clever-nr-im80-gen-strspc.txt 4-20000-delete-clever-nr-im80-strgen.tsv 4-20000-delete-clever-nr-im80-strgen.txt 4-20000-delete-clever-nr-im80-strspc.tsv 4-20000-delete-clever-nr-im80-strspc.txt 4-20000-delete-clever-nr-strgen.tsv 4-20000-delete-clever-nr-strgen.txt 4-20000-delete-clever-nr-strspc.tsv 4-20000-delete-clever-nr-strspc.txt 4-20000-delete-im80-strgen.tsv 4-20000-delete-im80-strgen.txt 4-20000-delete-im80-strspc.tsv 4-20000-delete-im80-strspc.txt 4-20000-refadd-clever-nr-im80-gen-empty-strgen.tsv 4-20000-refadd-clever-nr-im80-gen-empty-strgen.txt 4-20000-refadd-clever-nr-im80-gen-empty-strspc.tsv 4-20000-refadd-clever-nr-im80-gen-empty-strspc.txt 4-20000-refadd-clever-nr-im80-gen-strgen.tsv 4-20000-refadd-clever-nr-im80-gen-strgen.txt 4-20000-refadd-clever-nr-im80-gen-strspc.tsv 4-20000-refadd-clever-nr-im80-gen-strspc.txt 4-20000-refadd-clever-nr-im80-strgen.tsv 4-20000-refadd-clever-nr-im80-strgen.txt 4-20000-refadd-clever-nr-im80-strspc.tsv 4-20000-refadd-clever-nr-im80-strspc.txt 4-20000-refadd-clever-nr-strgen.tsv 4-20000-refadd-clever-nr-strgen.txt 4-20000-refadd-clever-nr-strspc.tsv 4-20000-refadd-clever-nr-strspc.txt 4-20000-refadd-im80-strgen.tsv 4-20000-refadd-im80-strgen.txt 4-20000-refadd-im80-strspc.tsv 4-20000-refadd-im80-strspc.txt 4-20000-refine-clever-nr-im80-gen-empty-strgen.tsv 4-20000-refine-clever-nr-im80-gen-empty-strgen.txt 4-20000-refine-clever-nr-im80-gen-empty-strspc.tsv 4-20000-refine-clever-nr-im80-gen-empty-strspc.txt 4-20000-refine-clever-nr-im80-gen-strgen.tsv 4-20000-refine-clever-nr-im80-gen-strgen.txt 4-20000-refine-clever-nr-im80-gen-strspc.tsv 4-20000-refine-clever-nr-im80-gen-strspc.txt 4-20000-refine-clever-nr-im80-strgen.tsv 4-20000-refine-clever-nr-im80-strgen.txt 4-20000-refine-clever-nr-im80-strspc.tsv 4-20000-refine-clever-nr-im80-strspc.txt 4-20000-refine-clever-nr-strgen.tsv 4-20000-refine-clever-nr-strgen.txt 4-20000-refine-clever-nr-strspc.tsv 4-20000-refine-clever-nr-strspc.txt 4-20000-refine-im80-strgen.tsv 4-20000-refine-im80-strgen.txt 4-20000-refine-im80-strspc.tsv 4-20000-refine-im80-strspc.txt 4-20000-replace-clever-nr-im80-gen-empty-strgen.tsv 4-20000-replace-clever-nr-im80-gen-empty-strgen.txt 4-20000-replace-clever-nr-im80-gen-empty-strspc.tsv 4-20000-replace-clever-nr-im80-gen-empty-strspc.txt 4-20000-replace-clever-nr-im80-gen-strgen.tsv 4-20000-replace-clever-nr-im80-gen-strgen.txt 4-20000-replace-clever-nr-im80-gen-strspc.tsv 4-20000-replace-clever-nr-im80-gen-strspc.txt 4-20000-replace-clever-nr-im80-strgen.tsv 4-20000-replace-clever-nr-im80-strgen.txt 4-20000-replace-clever-nr-im80-strspc.tsv 4-20000-replace-clever-nr-im80-strspc.txt 4-20000-replace-clever-nr-strgen.tsv 4-20000-replace-clever-nr-strgen.txt 4-20000-replace-clever-nr-strspc.tsv 4-20000-replace-clever-nr-strspc.txt 4-20000-replace-im80-strgen.tsv 4-20000-replace-im80-strgen.txt 4-20000-replace-im80-strspc.tsv 4-20000-replace-im80-strspc.txt

Result exploration

All results (most specific: plain, most general: dashed).

First recall is compared to no strenghening (expansion 20190523-NOOR, relaxation: 20180311-NOOR, expansion+relaxation 20190524-NOOR, expansion+relaxation+generation 20190624-NOOR, expansion+relaxation+generation+start empty 20190712-NOOR).

Comparison with the previous results. Here we only compare the most specific values (plain) with 20181110-NOOR (dashed). It is also possible to compare the most specific with 20180829-NOOR, but the values of each pairs are so close that this would not help readability.

Analysis

Analyst: Jérôme Euzenat (INRIA) (2019-09-10)

Key points: There are many hypotheses that may be tested with this data:

• That the new implemetation is on par with the previous ones (i.e. 20181110-NOOR for strspc, 20180829-NOOR for strgen and 20180913-NOOR for both on gen and gen-empty).
• That strenghening increases recall, at least for add, addjoin and refadd, like for 20180913-NOOR.
• That the recall is higher but still does not reach 100% (that was 20180913-NOOR). This can be observed on the table above. Even, 20190604-NOOR reaches a 99% recall, but clearly not 100%.
• That the most general or most specific strategy provide the same results. This can indeed be observed but will be further detailed on 20190604-NOOR (using the data of this experiment).

Further experiments:

• It seems that there are discrepancies in the case of delete/replace/refine. They may be due to the new software. But this should be clarified.

This file can be retrieved from URL https://sake.re/20190606-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20190606-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20190606-NOOR.zip