20181112-NOOR

Experiment design

Designer: Iris Lohja (INRIA) (2018-06-01)

Revision of networks of ontologies with random strenghening (4 agents; 10 runs; 10000 games; delete/replace/refine/add/addjoin/refadd)

Hypotheses: The random version of strengthening does not differ much from the most specific and most general ones

Experimental setting: Repeating 20180828-NOOR replaying the runs of 20180308-NOOR with random strengthening

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2018-11-12)

Date: 2018-11-12

Lazy lavender hash: d230f8c69b7c97c4c570785e554aabcb2bbe09e1

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20180828-NOOR

Parameters: params.sh

Command line (script.sh):

. params.sh

OUTPUT=${OUTPUT}${LABEL}

for op in ${OPS}
do
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-im80-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty-${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -DstartEmpty ${PARAMS}
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: 24 * Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz with 20GB RAM / Linux 4.15.17-1-pve / Java(TM) SE Runtime Environment 1.8.0_151 with 4.33G max heap size

Input: Input required for reproducibility can be retrieved from: https://files.inria.fr/sakere/input/expeRun.zip Then unzip expeRun.zip -d input

Raw results

4-10000-add-clever-nr-im80-gen-empty-strrdm.tsv 4-10000-add-clever-nr-im80-gen-empty-strrdm.txt 4-10000-add-clever-nr-im80-gen-strrdm.tsv 4-10000-add-clever-nr-im80-gen-strrdm.txt 4-10000-add-clever-nr-im80-strrdm.tsv 4-10000-add-clever-nr-im80-strrdm.txt 4-10000-add-clever-nr-strrdm.tsv 4-10000-add-clever-nr-strrdm.txt 4-10000-add-im80-strrdm.tsv 4-10000-add-im80-strrdm.txt 4-10000-add-strrdm.tsv 4-10000-add-strrdm.txt 4-10000-addjoin-clever-nr-im80-gen-empty-strrdm.tsv 4-10000-addjoin-clever-nr-im80-gen-empty-strrdm.txt 4-10000-addjoin-clever-nr-im80-gen-strrdm.tsv 4-10000-addjoin-clever-nr-im80-gen-strrdm.txt 4-10000-addjoin-clever-nr-im80-strrdm.tsv 4-10000-addjoin-clever-nr-im80-strrdm.txt 4-10000-addjoin-clever-nr-strrdm.tsv 4-10000-addjoin-clever-nr-strrdm.txt 4-10000-addjoin-im80-strrdm.tsv 4-10000-addjoin-im80-strrdm.txt 4-10000-addjoin-strrdm.tsv 4-10000-addjoin-strrdm.txt 4-10000-delete-clever-nr-im80-gen-empty-strrdm.tsv 4-10000-delete-clever-nr-im80-gen-empty-strrdm.txt 4-10000-delete-clever-nr-im80-gen-strrdm.tsv 4-10000-delete-clever-nr-im80-gen-strrdm.txt 4-10000-delete-clever-nr-im80-strrdm.tsv 4-10000-delete-clever-nr-im80-strrdm.txt 4-10000-delete-clever-nr-strrdm.tsv 4-10000-delete-clever-nr-strrdm.txt 4-10000-delete-im80-strrdm.tsv 4-10000-delete-im80-strrdm.txt 4-10000-delete-strrdm.tsv 4-10000-delete-strrdm.txt 4-10000-refadd-clever-nr-im80-gen-empty-strrdm.tsv 4-10000-refadd-clever-nr-im80-gen-empty-strrdm.txt 4-10000-refadd-clever-nr-im80-gen-strrdm.tsv 4-10000-refadd-clever-nr-im80-gen-strrdm.txt 4-10000-refadd-clever-nr-im80-strrdm.tsv 4-10000-refadd-clever-nr-im80-strrdm.txt 4-10000-refadd-clever-nr-strrdm.tsv 4-10000-refadd-clever-nr-strrdm.txt 4-10000-refadd-im80-strrdm.tsv 4-10000-refadd-im80-strrdm.txt 4-10000-refadd-strrdm.tsv 4-10000-refadd-strrdm.txt 4-10000-refine-clever-nr-im80-gen-empty-strrdm.tsv 4-10000-refine-clever-nr-im80-gen-empty-strrdm.txt 4-10000-refine-clever-nr-im80-gen-strrdm.tsv 4-10000-refine-clever-nr-im80-gen-strrdm.txt 4-10000-refine-clever-nr-im80-strrdm.tsv 4-10000-refine-clever-nr-im80-strrdm.txt 4-10000-refine-clever-nr-strrdm.tsv 4-10000-refine-clever-nr-strrdm.txt 4-10000-refine-im80-strrdm.tsv 4-10000-refine-im80-strrdm.txt 4-10000-refine-strrdm.tsv 4-10000-refine-strrdm.txt 4-10000-replace-clever-nr-im80-gen-empty-strrdm.tsv 4-10000-replace-clever-nr-im80-gen-empty-strrdm.txt 4-10000-replace-clever-nr-im80-gen-strrdm.tsv 4-10000-replace-clever-nr-im80-gen-strrdm.txt 4-10000-replace-clever-nr-im80-strrdm.tsv 4-10000-replace-clever-nr-im80-strrdm.txt 4-10000-replace-clever-nr-strrdm.tsv 4-10000-replace-clever-nr-strrdm.txt 4-10000-replace-im80-strrdm.tsv 4-10000-replace-im80-strrdm.txt 4-10000-replace-strrdm.tsv 4-10000-replace-strrdm.txt

Result exploration

Results below provide the values for the current strenghening with random (plain), most specific (dashed 20181110-NOOR), and most general (dotted 20180829-NOOR) strenghening. All curves roughly coincide, except for some most general recall which was obtained before using the same cache for expanding and strenghening).

The dotted lines are very often hidden by the plain ones because they indeed coincide.

test success
rate		 network
size		 incoherence
degree		 semantic
precision		 semantic
F-measure		 semantic
recall		 maximum
convergence
strenghen=random (compare to 20180308-NOOR)
delete 0.99 6 0.00 1.00 0.13 0.07 789
replace 0.99 6 0.00 1.00 0.13 0.07 1224
refine 0.99 6 0.00 1.00 0.13 0.07 1324
add 0.97 31 0.19 0.75 0.54 0.43 3145
addjoin 0.98 31 0.19 0.75 0.54 0.43 2153
refadd 0.98 52 0.30 0.67 0.68 0.70 2766
expansion + strenghen=random (compare to 20180529-NOOR)
delete 0.98 6 0.00 1.00 0.13 0.07 1764
replace 0.97 6 0.00 1.00 0.13 0.07 2854
refine 0.95 6 0.00 1.00 0.14 0.07 2876
add 0.93 59 0.31 0.65 0.70 0.75 3580
addjoin 0.96 60 0.34 0.63 0.69 0.75 2789
refadd 0.95 74 0.38 0.59 0.72 0.93 2985
relaxation + strenghen=random (compare to 20180311-NOOR)
delete 0.99 6 0.00 1.00 0.13 0.07 835
replace 0.99 6 0.00 1.00 0.13 0.07 1224
refine 0.99 6 0.00 1.00 0.13 0.07 1324
add 0.97 25 0.00 1.00 0.46 0.30 3831
addjoin 0.98 25 0.00 1.00 0.46 0.30 3248
refadd 0.97 57 0.00 1.00 0.80 0.66 7014
expansion + relaxation + strenghen=random (compare to 20180530-NOOR)
delete 0.98 6 0.00 1.00 0.13 0.07 2490
replace 0.97 6 0.00 1.00 0.13 0.07 1887
refine 0.96 6 0.00 1.00 0.14 0.08 2884
add 0.92 47 0.00 1.00 0.71 0.55 6307
addjoin 0.96 47 0.00 1.00 0.71 0.56 6168
refadd 0.94 79 0.00 1.00 0.96 0.93 8092
expansion + generation + relaxation + strenghen=random (compare to 20180601-NOOR)
delete 0.91 54 0.00 1.00 0.71 0.55 6580
replace 0.91 56 0.00 1.00 0.71 0.56 8247
refine 0.89 56 0.00 1.00 0.70 0.53 6745
add 0.89 63 0.00 1.00 0.85 0.73 9942
addjoin 0.93 67 0.00 1.00 0.88 0.78 7481
refadd 0.93 84 0.00 1.00 0.99 0.98 9138
expansion + generation + empty + relaxation + strenghen=random (compare to 20180827-NOOR)
delete 0.91 65 0.00 1.00 0.75 0.60 6082
replace 0.91 66 0.00 1.00 0.75 0.60 7134
refine 0.90 66 0.00 1.00 0.75 0.60 6295
add 0.91 74 0.00 1.00 0.88 0.79 8472
addjoin 0.94 77 0.00 1.00 0.90 0.83 8810
refadd 0.93 86 0.00 1.00 0.97 0.94 7648

Analysis

Analyst: Jérôme Euzenat (INRIA) (2018-11-22)

Key points:

Further experiments: none

This file can be retrieved from URL https://sake.re/20181112-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20181112-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20181112-NOOR.zip