20181109-NOOR

Experiment design

Designer: Iris Lohja (INRIA) (2018-06-01)

Revision of networks of ontologies with most specific strenghening (improved code) (4 agents; 10 runs; 10000 games; delete/replace/refine/add/addjoin/refadd)

Hypotheses: Strenghening improves recall over expansion

Experimental setting: Repeating 20180828-NOOR replaying the runs of 20180308-NOOR with most specific strengthening on a reengineered strenghening code

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2018-11-09)

Date: 2018-11-09

Lazy lavender hash: 619edd1134aa281afa1a5dc101ac541be5f021f4

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20180828-NOOR

Parameters: params.sh

Command line (script.sh):

. params.sh

OUTPUT=${OUTPUT}${LABEL}

for op in ${OPS}
do
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-im80-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -DstartEmpty ${PARAMS}
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: 24 * Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz with 20GB RAM / Linux 4.15.17-1-pve / Java(TM) SE Runtime Environment 1.8.0_151 with 4.33G max heap size

Input: Input required for reproducibility can be retrieved from: https://files.inria.fr/sakere/input/expeRun.zip Then unzip expeRun.zip -d input

Raw results

4-10000-add-clever-nr-im80-gen-empty-strspc.tsv 4-10000-add-clever-nr-im80-gen-empty-strspc.txt 4-10000-add-clever-nr-im80-gen-strspc.tsv 4-10000-add-clever-nr-im80-gen-strspc.txt 4-10000-add-clever-nr-im80-strspc.tsv 4-10000-add-clever-nr-im80-strspc.txt 4-10000-add-clever-nr-strspc.tsv 4-10000-add-clever-nr-strspc.txt 4-10000-add-im80-strspc.tsv 4-10000-add-im80-strspc.txt 4-10000-add-strspc.tsv 4-10000-add-strspc.txt 4-10000-addjoin-clever-nr-im80-gen-empty-strspc.tsv 4-10000-addjoin-clever-nr-im80-gen-empty-strspc.txt 4-10000-addjoin-clever-nr-im80-gen-strspc.tsv 4-10000-addjoin-clever-nr-im80-gen-strspc.txt 4-10000-addjoin-clever-nr-im80-strspc.tsv 4-10000-addjoin-clever-nr-im80-strspc.txt 4-10000-addjoin-clever-nr-strspc.tsv 4-10000-addjoin-clever-nr-strspc.txt 4-10000-addjoin-im80-strspc.tsv 4-10000-addjoin-im80-strspc.txt 4-10000-addjoin-strspc.tsv 4-10000-addjoin-strspc.txt 4-10000-delete-clever-nr-im80-gen-empty-strspc.tsv 4-10000-delete-clever-nr-im80-gen-empty-strspc.txt 4-10000-delete-clever-nr-im80-gen-strspc.tsv 4-10000-delete-clever-nr-im80-gen-strspc.txt 4-10000-delete-clever-nr-im80-strspc.tsv 4-10000-delete-clever-nr-im80-strspc.txt 4-10000-delete-clever-nr-strspc.tsv 4-10000-delete-clever-nr-strspc.txt 4-10000-delete-im80-strspc.tsv 4-10000-delete-im80-strspc.txt 4-10000-delete-strspc.tsv 4-10000-delete-strspc.txt 4-10000-refadd-clever-nr-im80-gen-empty-strspc.tsv 4-10000-refadd-clever-nr-im80-gen-empty-strspc.txt 4-10000-refadd-clever-nr-im80-gen-strspc.tsv 4-10000-refadd-clever-nr-im80-gen-strspc.txt 4-10000-refadd-clever-nr-im80-strspc.tsv 4-10000-refadd-clever-nr-im80-strspc.txt 4-10000-refadd-clever-nr-strspc.tsv 4-10000-refadd-clever-nr-strspc.txt 4-10000-refadd-im80-strspc.tsv 4-10000-refadd-im80-strspc.txt 4-10000-refadd-strspc.tsv 4-10000-refadd-strspc.txt 4-10000-refine-clever-nr-im80-gen-empty-strspc.tsv 4-10000-refine-clever-nr-im80-gen-empty-strspc.txt 4-10000-refine-clever-nr-im80-gen-strspc.tsv 4-10000-refine-clever-nr-im80-gen-strspc.txt 4-10000-refine-clever-nr-im80-strspc.tsv 4-10000-refine-clever-nr-im80-strspc.txt 4-10000-refine-clever-nr-strspc.tsv 4-10000-refine-clever-nr-strspc.txt 4-10000-refine-im80-strspc.tsv 4-10000-refine-im80-strspc.txt 4-10000-refine-strspc.tsv 4-10000-refine-strspc.txt 4-10000-replace-clever-nr-im80-gen-empty-strspc.tsv 4-10000-replace-clever-nr-im80-gen-empty-strspc.txt 4-10000-replace-clever-nr-im80-gen-strspc.tsv 4-10000-replace-clever-nr-im80-gen-strspc.txt 4-10000-replace-clever-nr-im80-strspc.tsv 4-10000-replace-clever-nr-im80-strspc.txt 4-10000-replace-clever-nr-strspc.tsv 4-10000-replace-clever-nr-strspc.txt 4-10000-replace-im80-strspc.tsv 4-10000-replace-im80-strspc.txt 4-10000-replace-strspc.tsv 4-10000-replace-strspc.txt

Result exploration

Results below provide the values for the current strenghening with most specific (plain), values without strenghening 20180601-NOOR (dashed), and previous 20180828-NOOR strenghening with most specific (dotted).

The table below can be directly compared to that of 20180828-NOOR

test success
rate		 network
size		 incoherence
degree		 semantic
precision		 semantic
F-measure		 semantic
recall		 maximum
convergence
strenghen=specific
delete 1.00 6 0.00 1.00 0.13 0.07 789
replace 0.99 6 0.00 1.00 0.13 0.07 1224
refine 0.99 6 0.00 1.00 0.13 0.07 1224
add 0.97 28 0.14 0.81 0.50 0.36 2227
addjoin 0.98 28 0.14 0.81 0.50 0.36 2153
refadd 0.98 45 0.25 0.71 0.66 0.61 3132
expansion + strenghen=specific
delete 0.98 6 0.00 1.00 0.13 0.07 1712
replace 0.97 6 0.00 1.00 0.13 0.07 2131
refine 0.96 6 0.00 1.00 0.14 0.08 2897
add 0.94 53 0.29 0.66 0.66 0.66 4075
addjoin 0.97 51 0.27 0.69 0.67 0.66 2504
refadd 0.96 69 0.34 0.62 0.72 0.85 3158
relaxation + strenghen=specific
delete 1.00 6 0.00 1.00 0.13 0.07 789
replace 0.99 6 0.00 1.00 0.13 0.07 1224
refine 0.99 6 0.00 1.00 0.13 0.07 1224
add 0.97 24 0.00 1.00 0.44 0.28 5256
addjoin 0.98 24 0.00 1.00 0.44 0.28 4031
refadd 0.97 50 0.00 1.00 0.74 0.59 7548
expansion + relaxation + strenghen=specific
delete 0.98 6 0.00 1.00 0.13 0.07 1878
replace 0.97 6 0.00 1.00 0.13 0.07 1927
refine 0.95 6 0.00 1.00 0.15 0.08 2576
add 0.93 46 0.00 1.00 0.70 0.54 6488
addjoin 0.96 43 0.00 1.00 0.67 0.51 6031
refadd 0.94 78 0.00 1.00 0.95 0.91 7575
expansion + generation + relaxation + strenghen=specific
delete 0.91 55 0.00 1.00 0.70 0.53 9148
replace 0.91 57 0.00 1.00 0.72 0.56 7162
refine 0.89 60 0.00 1.00 0.75 0.60 9418
add 0.89 64 0.00 1.00 0.85 0.74 8317
addjoin 0.93 69 0.00 1.00 0.89 0.81 6992
refadd 0.93 84 0.00 1.00 0.99 0.98 8515
expansion + generation + empty + relaxation + strenghen=specific
delete 0.91 64 0.00 1.00 0.73 0.58 6689
replace 0.91 62 0.00 1.00 0.72 0.56 7288
refine 0.89 66 0.00 1.00 0.74 0.59 7450
add 0.91 75 0.00 1.00 0.89 0.80 9979
addjoin 0.94 75 0.00 1.00 0.89 0.81 7545
refadd 0.93 87 0.00 1.00 0.97 0.94 8478

Analysis

Analyst: Jérôme Euzenat (INRIA) (2018-11-21)

Key points:

We attribute this effect to the merging of the two caches.

Further experiments: none

This file can be retrieved from URL https://sake.re/20181109-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20181109-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20181109-NOOR.zip