20180829-NOOR

Experiment design

Designer: Iris Lohja (INRIA) (2018-06-01)

Revision of networks of ontologies with most general strengthening (4 agents; 10 runs; 10000 games; delete/replace/refine/add/addjoin/refadd)

Hypotheses: Strengthening (most general) improves recall over expansion, just like most specific strengthening

Experimental setting: Same as 20180828-NOOR replaying the same runs as 20180308-NOOR with most general strengthening.

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2018-08-29)

Date: 2018-08-29

Lazy lavender hash: 759ff097b96520c12aa84f3749927f9a22022e62

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20180601-NOOR

Parameters: params.sh

Command line (script.sh):

. params.sh

OUTPUT=${OUTPUT}${LABEL}

for op in ${OPS}
do
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} ${PARAMS}
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy ${PARAMS}
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-im80-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DimmediateRatio=80 ${PARAMS}
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 ${PARAMS}
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative ${PARAMS}
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -DstartEmpty ${PARAMS}
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: 24 * Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz with 20GB RAM / Linux ProxMox 2 / Linux 4.15.17-1-pve / Java Java(TM) SE Runtime Environment 1.8.0_151 with 4.33G max heap size

Note: These experiments are an independent rerun of Iris Lohja (INRIA)'s MSc thesis experiment 7. Actually, experiment 7 was only concerned with The tables and plots above cover these results and confirm them. Hence the analysis is conform to hers.

Input: Input required for reproducibility can be retrieved from: https://files.inria.fr/sakere/input/expeRun.zip Then unzip expeRun.zip -d input

Raw results

4-10000-add-clever-nr-im80-gen-empty-strgen.tsv 4-10000-add-clever-nr-im80-gen-empty-strgen.txt 4-10000-add-clever-nr-im80-gen-strgen.tsv 4-10000-add-clever-nr-im80-gen-strgen.txt 4-10000-add-clever-nr-im80-strgen.tsv 4-10000-add-clever-nr-im80-strgen.txt 4-10000-add-clever-nr-strgen.tsv 4-10000-add-clever-nr-strgen.txt 4-10000-add-im80-strgen.tsv 4-10000-add-im80-strgen.txt 4-10000-add-strgen.tsv 4-10000-add-strgen.txt 4-10000-addjoin-clever-nr-im80-gen-empty-strgen.tsv 4-10000-addjoin-clever-nr-im80-gen-empty-strgen.txt 4-10000-addjoin-clever-nr-im80-gen-strgen.tsv 4-10000-addjoin-clever-nr-im80-gen-strgen.txt 4-10000-addjoin-clever-nr-im80-strgen.tsv 4-10000-addjoin-clever-nr-im80-strgen.txt 4-10000-addjoin-clever-nr-strgen.tsv 4-10000-addjoin-clever-nr-strgen.txt 4-10000-addjoin-im80-strgen.tsv 4-10000-addjoin-im80-strgen.txt 4-10000-addjoin-strgen.tsv 4-10000-addjoin-strgen.txt 4-10000-delete-clever-nr-im80-gen-empty-strgen.tsv 4-10000-delete-clever-nr-im80-gen-empty-strgen.txt 4-10000-delete-clever-nr-im80-gen-strgen.tsv 4-10000-delete-clever-nr-im80-gen-strgen.txt 4-10000-delete-clever-nr-im80-strgen.tsv 4-10000-delete-clever-nr-im80-strgen.txt 4-10000-delete-clever-nr-strgen.tsv 4-10000-delete-clever-nr-strgen.txt 4-10000-delete-im80-strgen.tsv 4-10000-delete-im80-strgen.txt 4-10000-delete-strgen.tsv 4-10000-delete-strgen.txt 4-10000-refadd-clever-nr-im80-gen-empty-strgen.tsv 4-10000-refadd-clever-nr-im80-gen-empty-strgen.txt 4-10000-refadd-clever-nr-im80-gen-strgen.tsv 4-10000-refadd-clever-nr-im80-gen-strgen.txt 4-10000-refadd-clever-nr-im80-strgen.tsv 4-10000-refadd-clever-nr-im80-strgen.txt 4-10000-refadd-clever-nr-strgen.tsv 4-10000-refadd-clever-nr-strgen.txt 4-10000-refadd-im80-strgen.tsv 4-10000-refadd-im80-strgen.txt 4-10000-refadd-strgen.tsv 4-10000-refadd-strgen.txt 4-10000-refine-clever-nr-im80-gen-empty-strgen.tsv 4-10000-refine-clever-nr-im80-gen-empty-strgen.txt 4-10000-refine-clever-nr-im80-gen-strgen.tsv 4-10000-refine-clever-nr-im80-gen-strgen.txt 4-10000-refine-clever-nr-im80-strgen.tsv 4-10000-refine-clever-nr-im80-strgen.txt 4-10000-refine-clever-nr-strgen.tsv 4-10000-refine-clever-nr-strgen.txt 4-10000-refine-im80-strgen.tsv 4-10000-refine-im80-strgen.txt 4-10000-refine-strgen.tsv 4-10000-refine-strgen.txt 4-10000-replace-clever-nr-im80-gen-empty-strgen.tsv 4-10000-replace-clever-nr-im80-gen-empty-strgen.txt 4-10000-replace-clever-nr-im80-gen-strgen.tsv 4-10000-replace-clever-nr-im80-gen-strgen.txt 4-10000-replace-clever-nr-im80-strgen.tsv 4-10000-replace-clever-nr-im80-strgen.txt 4-10000-replace-clever-nr-strgen.tsv 4-10000-replace-clever-nr-strgen.txt 4-10000-replace-im80-strgen.tsv 4-10000-replace-im80-strgen.txt 4-10000-replace-strgen.tsv 4-10000-replace-strgen.txt

Result exploration

We reproduce the same plots as with (20180828-NOOR):

test success
rate
network
size
incoherence
degree
semantic
precision
semantic
F-measure
semantic
recall
maximum
convergence
strenghen=general (compare to 20180308-NOOR)
delete 1.00 6 0.00 1.00 0.13 0.07 835
replace 0.99 6 0.00 1.00 0.13 0.07 1224
refine 0.99 6 0.00 1.00 0.13 0.07 1224
add 0.97 28 0.14 0.81 0.50 0.36 3178
addjoin 0.99 28 0.14 0.81 0.50 0.36 1843
refadd 0.98 45 0.25 0.71 0.66 0.61 2467
expansion + strenghen=general (compare to 20180529-NOOR)
delete 0.98 6 0.00 1.00 0.13 0.07 1880
replace 0.97 6 0.00 1.00 0.13 0.07 2425
refine 0.96 6 0.00 1.00 0.13 0.07 2868
add 0.94 54 0.27 0.69 0.68 0.67 4593
addjoin 0.97 53 0.29 0.67 0.66 0.65 2611
refadd 0.96 68 0.35 0.62 0.72 0.86 4076
relaxation + strenghen=general (compare to 20180311-NOOR)
delete 1.00 6 0.00 1.00 0.13 0.07 835
replace 0.99 6 0.00 1.00 0.13 0.07 1224
refine 0.99 6 0.00 1.00 0.13 0.07 1224
add 0.97 24 0.00 1.00 0.44 0.28 5212
addjoin 0.99 24 0.00 1.00 0.44 0.28 3026
refadd 0.97 52 0.00 1.00 0.76 0.61 5628
expansion + relaxation + strenghen=general (compare to 20180530-NOOR)
delete 0.98 6 0.00 1.00 0.13 0.07 2729
replace 0.97 6 0.00 1.00 0.13 0.07 2165
refine 0.95 6 0.00 1.00 0.13 0.07 3077
add 0.93 44 0.00 1.00 0.68 0.52 7808
addjoin 0.96 45 0.00 1.00 0.69 0.53 4439
refadd 0.94 76 0.00 1.00 0.94 0.89 6453
expansion + generation + relaxation + strenghen=general (compare to 20180601-NOOR)
delete 0.91 56 0.00 1.00 0.59 0.41 8366
replace 0.90 58 0.00 1.00 0.59 0.41 7660
refine 0.88 58 0.00 1.00 0.61 0.44 9421
add 0.89 66 0.00 1.00 0.87 0.77 9916
addjoin 0.93 67 0.00 1.00 0.88 0.79 7459
refadd 0.93 84 0.00 1.00 0.99 0.98 8141
expansion + generation + empty + relaxation + strenghen=general (compare to 20180827-NOOR)
delete 0.90 63 0.00 1.00 0.58 0.41 8080
replace 0.90 64 0.00 1.00 0.59 0.42 8560
refine 0.88 66 0.00 1.00 0.62 0.45 9975
add 0.90 73 0.00 1.00 0.88 0.79 9784
addjoin 0.94 74 0.00 1.00 0.89 0.80 5636
refadd 0.93 86 0.00 1.00 0.97 0.94 8082

Analysis

Analyst: Jérôme Euzenat (INRIA) (2018-09-13)

Key points:

Further experiments:


This file can be retrieved from URL https://sake.re/20180829-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20180829-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr.

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20180829-NOOR.zip