20181110-NOOR

Experiment design

Designer: Iris Lohja (INRIA) (2018-06-01)

Revision of networks of ontologies with most specific strenghening (fixed implementation) (4 agents; 10 runs; 10000 games; delete/replace/refine/add/addjoin/refadd)

Hypotheses: Strenghening improves recall over expansion even better when implementation is fixed

Experimental setting: Repeating 20180828-NOOR replaying the runs of 20180308-NOOR with most specific strengthening; the software has been improved for taking into account equivalence correspondences which were treated exactly like subsumption correspondences. They are now treated the more correctly.

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2018-11-10)

Date: 2018-11-10

Lazy lavender hash: d230f8c69b7c97c4c570785e554aabcb2bbe09e1

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20180828-NOOR

Parameters: params.sh

Command line (script.sh):

. params.sh

OUTPUT=${OUTPUT}${LABEL}

for op in ${OPS}
do
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-im80-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -DstartEmpty ${PARAMS}
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: 24 * Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz with 20GB RAM / Linux 4.15.17-1-pve / Java(TM) SE Runtime Environment 1.8.0_151 with 4.33G max heap size

Input: Input required for reproducibility can be retrieved from: https://files.inria.fr/sakere/input/expeRun.zip Then unzip expeRun.zip -d input

Raw results

4-10000-add-clever-nr-im80-gen-empty-strspc.tsv 4-10000-add-clever-nr-im80-gen-empty-strspc.txt 4-10000-add-clever-nr-im80-gen-strspc.tsv 4-10000-add-clever-nr-im80-gen-strspc.txt 4-10000-add-clever-nr-im80-strspc.tsv 4-10000-add-clever-nr-im80-strspc.txt 4-10000-add-clever-nr-strspc.tsv 4-10000-add-clever-nr-strspc.txt 4-10000-add-im80-strspc.tsv 4-10000-add-im80-strspc.txt 4-10000-add-strspc.tsv 4-10000-add-strspc.txt 4-10000-addjoin-clever-nr-im80-gen-empty-strspc.tsv 4-10000-addjoin-clever-nr-im80-gen-empty-strspc.txt 4-10000-addjoin-clever-nr-im80-gen-strspc.tsv 4-10000-addjoin-clever-nr-im80-gen-strspc.txt 4-10000-addjoin-clever-nr-im80-strspc.tsv 4-10000-addjoin-clever-nr-im80-strspc.txt 4-10000-addjoin-clever-nr-strspc.tsv 4-10000-addjoin-clever-nr-strspc.txt 4-10000-addjoin-im80-strspc.tsv 4-10000-addjoin-im80-strspc.txt 4-10000-addjoin-strspc.tsv 4-10000-addjoin-strspc.txt 4-10000-delete-clever-nr-im80-gen-empty-strspc.tsv 4-10000-delete-clever-nr-im80-gen-empty-strspc.txt 4-10000-delete-clever-nr-im80-gen-strspc.tsv 4-10000-delete-clever-nr-im80-gen-strspc.txt 4-10000-delete-clever-nr-im80-strspc.tsv 4-10000-delete-clever-nr-im80-strspc.txt 4-10000-delete-clever-nr-strspc.tsv 4-10000-delete-clever-nr-strspc.txt 4-10000-delete-im80-strspc.tsv 4-10000-delete-im80-strspc.txt 4-10000-delete-strspc.tsv 4-10000-delete-strspc.txt 4-10000-refadd-clever-nr-im80-gen-empty-strspc.tsv 4-10000-refadd-clever-nr-im80-gen-empty-strspc.txt 4-10000-refadd-clever-nr-im80-gen-strspc.tsv 4-10000-refadd-clever-nr-im80-gen-strspc.txt 4-10000-refadd-clever-nr-im80-strspc.tsv 4-10000-refadd-clever-nr-im80-strspc.txt 4-10000-refadd-clever-nr-strspc.tsv 4-10000-refadd-clever-nr-strspc.txt 4-10000-refadd-im80-strspc.tsv 4-10000-refadd-im80-strspc.txt 4-10000-refadd-strspc.tsv 4-10000-refadd-strspc.txt 4-10000-refine-clever-nr-im80-gen-empty-strspc.tsv 4-10000-refine-clever-nr-im80-gen-empty-strspc.txt 4-10000-refine-clever-nr-im80-gen-strspc.tsv 4-10000-refine-clever-nr-im80-gen-strspc.txt 4-10000-refine-clever-nr-im80-strspc.tsv 4-10000-refine-clever-nr-im80-strspc.txt 4-10000-refine-clever-nr-strspc.tsv 4-10000-refine-clever-nr-strspc.txt 4-10000-refine-im80-strspc.tsv 4-10000-refine-im80-strspc.txt 4-10000-refine-strspc.tsv 4-10000-refine-strspc.txt 4-10000-replace-clever-nr-im80-gen-empty-strspc.tsv 4-10000-replace-clever-nr-im80-gen-empty-strspc.txt 4-10000-replace-clever-nr-im80-gen-strspc.tsv 4-10000-replace-clever-nr-im80-gen-strspc.txt 4-10000-replace-clever-nr-im80-strspc.tsv 4-10000-replace-clever-nr-im80-strspc.txt 4-10000-replace-clever-nr-strspc.tsv 4-10000-replace-clever-nr-strspc.txt 4-10000-replace-im80-strspc.tsv 4-10000-replace-im80-strspc.txt 4-10000-replace-strspc.tsv 4-10000-replace-strspc.txt

Result exploration

Results below provide the values for the current strenghening with most specific (plain), values without strenghening (dashed), and previous 20181109-NOOR strenghening with most specific (dotted). The dotted lines are very often hidden by the plain ones because they indeed coincide.

test success
rate		 network
size		 incoherence
degree		 semantic
precision		 semantic
F-measure		 semantic
recall		 maximum
convergence
strenghen=specific (compare to 20180308-NOOR)
delete 0.99 6 0.00 1.00 0.13 0.07 789
replace 0.99 6 0.00 1.00 0.13 0.07 1224
refine 0.99 6 0.00 1.00 0.13 0.07 1224
add 0.97 31 0.19 0.75 0.54 0.43 2227
addjoin 0.98 31 0.19 0.75 0.54 0.43 2153
refadd 0.97 52 0.30 0.67 0.68 0.70 2766
expansion + strenghen=specific (compare to 20180529-NOOR)
delete 0.98 6 0.00 1.00 0.13 0.07 2044
replace 0.97 6 0.00 1.00 0.13 0.07 2425
refine 0.95 6 0.00 1.00 0.14 0.08 2466
add 0.94 58 0.32 0.63 0.67 0.72 4323
addjoin 0.96 59 0.31 0.65 0.70 0.75 2959
refadd 0.95 72 0.37 0.60 0.71 0.89 4633
relaxation + strenghen=specific (compare to 20180311-NOOR)
delete 0.99 6 0.00 1.00 0.13 0.07 789
replace 0.99 6 0.00 1.00 0.13 0.07 1224
refine 0.99 6 0.00 1.00 0.13 0.07 1224
add 0.97 25 0.00 1.00 0.46 0.30 3928
addjoin 0.98 25 0.00 1.00 0.46 0.30 3874
refadd 0.97 57 0.00 1.00 0.80 0.67 7974
expansion + relaxation + strenghen=specific (compare to 20180530-NOOR)
delete 0.98 6 0.00 1.00 0.13 0.07 2552
replace 0.97 6 0.00 1.00 0.13 0.07 2114
refine 0.95 6 0.00 1.00 0.14 0.08 3397
add 0.93 47 0.00 1.00 0.71 0.55 7011
addjoin 0.96 46 0.00 1.00 0.70 0.54 6781
refadd 0.94 80 0.00 1.00 0.96 0.93 9796
expansion + generation + relaxation + strenghen=specific (compare to 20180601-NOOR)
delete 0.91 56 0.00 1.00 0.71 0.56 9624
replace 0.91 56 0.00 1.00 0.71 0.55 7874
refine 0.89 58 0.00 1.00 0.72 0.56 9827
add 0.91 75 0.00 1.00 0.89 0.80 9191
addjoin 0.93 68 0.00 1.00 0.88 0.78 6905
refadd 0.93 84 0.00 1.00 0.99 0.98 7307
expansion + generation + empty + relaxation + strenghen=specific (compare to 20180827-NOOR)
delete 0.91 64 0.00 1.00 0.73 0.58 7770
replace 0.91 64 0.00 1.00 0.74 0.58 7371
refine 0.89 64 0.00 1.00 0.72 0.57 8613
add 0.89 63 0.00 1.00 0.85 0.73 9739
addjoin 0.94 75 0.00 1.00 0.89 0.81 6718
refadd 0.93 86 0.00 1.00 0.97 0.93 6565

Analysis

Analyst: Jérôme Euzenat (INRIA) (2018-11-22)

Key points:

Further experiments: none

This file can be retrieved from URL https://sake.re/20181110-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20181110-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20181110-NOOR.zip