20190604-NOOR

Experiment design

Designer: Iris Lohja (INRIA) (2018-06-01)

Revision of networks of ontologies with random strenghening (4 agents; 10 runs; 20000 games; delete/replace/refine/add/addjoin/refadd)

Hypotheses: The random version of strengthening does not differ much from the most specific and most general ones

Experimental setting: Repeating 20180601-NOOR with rewritten software and 20000 iterations

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2019-07-22)

Date: 2019-07-22

Lazy lavender hash: 4665e226cf61479a716576db43d55fa4b8189578

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20180601-NOOR

Parameters: params.sh

Command line (script.sh):

. params.sh

OUTPUT=${OUTPUT}${LABEL}

for op in ${OPS}
do
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-im80${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative ${PARAMS}
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty${postfix} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -DstartEmpty ${PARAMS}
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: 24 * Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz with 20GB RAM / Linux 4.15.17-1-pve / Java(TM) SE Runtime Environment 1.8.0_151 with 4.33G max heap size

Input: Input required for reproducibility can be retrieved from: https://files.inria.fr/sakere/input/expeRun2.zip Then unzip expeRun2.zip -d input

Raw results

4-20000-add-clever-nr-im80-gen-empty-strrdm.tsv 4-20000-add-clever-nr-im80-gen-empty-strrdm.txt 4-20000-add-clever-nr-im80-gen-strrdm.tsv 4-20000-add-clever-nr-im80-gen-strrdm.txt 4-20000-add-clever-nr-im80-strrdm.tsv 4-20000-add-clever-nr-im80-strrdm.txt 4-20000-add-clever-nr-strrdm.tsv 4-20000-add-clever-nr-strrdm.txt 4-20000-add-im80-strrdm.tsv 4-20000-add-im80-strrdm.txt 4-20000-add-strrdm.tsv 4-20000-add-strrdm.txt 4-20000-addjoin-clever-nr-im80-gen-empty-strrdm.tsv 4-20000-addjoin-clever-nr-im80-gen-empty-strrdm.txt 4-20000-addjoin-clever-nr-im80-gen-strrdm.tsv 4-20000-addjoin-clever-nr-im80-gen-strrdm.txt 4-20000-addjoin-clever-nr-im80-strrdm.tsv 4-20000-addjoin-clever-nr-im80-strrdm.txt 4-20000-addjoin-clever-nr-strrdm.tsv 4-20000-addjoin-clever-nr-strrdm.txt 4-20000-addjoin-im80-strrdm.tsv 4-20000-addjoin-im80-strrdm.txt 4-20000-addjoin-strrdm.tsv 4-20000-addjoin-strrdm.txt 4-20000-delete-clever-nr-im80-gen-empty-strrdm.tsv 4-20000-delete-clever-nr-im80-gen-empty-strrdm.txt 4-20000-delete-clever-nr-im80-gen-strrdm.tsv 4-20000-delete-clever-nr-im80-gen-strrdm.txt 4-20000-delete-clever-nr-im80-strrdm.tsv 4-20000-delete-clever-nr-im80-strrdm.txt 4-20000-delete-clever-nr-strrdm.tsv 4-20000-delete-clever-nr-strrdm.txt 4-20000-delete-im80-strrdm.tsv 4-20000-delete-im80-strrdm.txt 4-20000-delete-strrdm.tsv 4-20000-delete-strrdm.txt 4-20000-refadd-clever-nr-im80-gen-empty-strrdm.tsv 4-20000-refadd-clever-nr-im80-gen-empty-strrdm.txt 4-20000-refadd-clever-nr-im80-gen-strrdm.tsv 4-20000-refadd-clever-nr-im80-gen-strrdm.txt 4-20000-refadd-clever-nr-im80-strrdm.tsv 4-20000-refadd-clever-nr-im80-strrdm.txt 4-20000-refadd-clever-nr-strrdm.tsv 4-20000-refadd-clever-nr-strrdm.txt 4-20000-refadd-im80-strrdm.tsv 4-20000-refadd-im80-strrdm.txt 4-20000-refadd-strrdm.tsv 4-20000-refadd-strrdm.txt 4-20000-refine-clever-nr-im80-gen-empty-strrdm.tsv 4-20000-refine-clever-nr-im80-gen-empty-strrdm.txt 4-20000-refine-clever-nr-im80-gen-strrdm.tsv 4-20000-refine-clever-nr-im80-gen-strrdm.txt 4-20000-refine-clever-nr-im80-strrdm.tsv 4-20000-refine-clever-nr-im80-strrdm.txt 4-20000-refine-clever-nr-strrdm.tsv 4-20000-refine-clever-nr-strrdm.txt 4-20000-refine-im80-strrdm.tsv 4-20000-refine-im80-strrdm.txt 4-20000-refine-strrdm.tsv 4-20000-refine-strrdm.txt 4-20000-replace-clever-nr-im80-gen-empty-strrdm.tsv 4-20000-replace-clever-nr-im80-gen-empty-strrdm.txt 4-20000-replace-clever-nr-im80-gen-strrdm.tsv 4-20000-replace-clever-nr-im80-gen-strrdm.txt 4-20000-replace-clever-nr-im80-strrdm.tsv 4-20000-replace-clever-nr-im80-strrdm.txt 4-20000-replace-clever-nr-strrdm.tsv 4-20000-replace-clever-nr-strrdm.txt 4-20000-replace-im80-strrdm.tsv 4-20000-replace-im80-strrdm.txt 4-20000-replace-strrdm.tsv 4-20000-replace-strrdm.txt

Result exploration

Results (plain) are compared with the the old implementation (on 10000 games) 20181112-NOOR (dash-dotted) and those of the most specific (dashed) and most general (dotted) strenghening strategies 20190606-NOOR.

Results for the simplest modality are not available for gen and spc, so not compared.

test success
rate		 network
size		 incoherence
degree		 semantic
precision		 semantic
F-measure		 semantic
recall		 maximum
convergence
random strenghening
4-20000-delete-strrdm 1.00 6 0.00 1.00 0.13 0.07 737
4-20000-replace-strrdm 1.00 6 0.00 1.00 0.13 0.07 816
4-20000-refine-strrdm 0.99 6 0.00 1.00 0.13 0.07 1911
4-20000-add-strrdm 0.98 31 0.19 0.75 0.54 0.43 2480
4-20000-addjoin-strrdm 0.99 31 0.19 0.75 0.54 0.43 2075
4-20000-refadd-strrdm 0.99 53 0.31 0.66 0.69 0.71 3103
expansion+random strenghening
4-20000-delete-clever-nr-strrdm 0.99 6 0.00 1.00 0.13 0.07 2680
4-20000-replace-clever-nr-strrdm 0.98 6 0.00 1.00 0.13 0.07 2295
4-20000-refine-clever-nr-strrdm 0.97 7 0.00 1.00 0.16 0.09 4065
4-20000-add-clever-nr-strrdm 0.97 38 0.26 0.68 0.61 0.55 4493
4-20000-addjoin-clever-nr-strrdm 0.98 46 0.31 0.65 0.65 0.66 2787
4-20000-refadd-clever-nr-strrdm 0.97 64 0.36 0.61 0.74 0.95 3751
relaxation+random strenghening
4-20000-delete-im80-strrdm 1.00 6 0.00 1.00 0.13 0.07 737
4-20000-replace-im80-strrdm 1.00 6 0.00 1.00 0.13 0.07 816
4-20000-refine-im80-strrdm 0.99 6 0.00 1.00 0.13 0.07 1911
4-20000-add-im80-strrdm 0.98 25 0.00 1.00 0.46 0.30 4866
4-20000-addjoin-im80-strrdm 0.99 25 0.00 1.00 0.46 0.30 3644
4-20000-refadd-im80-strrdm 0.98 59 0.00 1.00 0.82 0.69 7623
relaxation+expansion+random strenghening
4-20000-delete-clever-nr-im80-strrdm 0.99 6 0.00 1.00 0.13 0.07 2075
4-20000-replace-clever-nr-im80-strrdm 0.98 6 0.00 1.00 0.13 0.07 2118
4-20000-refine-clever-nr-im80-strrdm 0.97 7 0.00 1.00 0.15 0.08 3247
4-20000-add-clever-nr-im80-strrdm 0.97 29 0.00 1.00 0.52 0.35 6267
4-20000-addjoin-clever-nr-im80-strrdm 0.98 37 0.00 1.00 0.61 0.44 6210
4-20000-refadd-clever-nr-im80-strrdm 0.96 82 0.00 1.00 0.98 0.96 10535
relaxation+generation+expansion+random strenghening
4-20000-delete-clever-nr-im80-gen-strrdm 0.95 29 0.00 1.00 0.51 0.34 12897
4-20000-replace-clever-nr-im80-gen-strrdm 0.95 26 0.00 1.00 0.48 0.31 14192
4-20000-refine-clever-nr-im80-gen-strrdm 0.94 34 0.00 1.00 0.57 0.40 13709
4-20000-add-clever-nr-im80-gen-strrdm 0.93 62 0.00 1.00 0.84 0.73 16268
4-20000-addjoin-clever-nr-im80-gen-strrdm 0.96 65 0.00 1.00 0.86 0.76 17728
4-20000-refadd-clever-nr-im80-gen-strrdm 0.95 84 0.00 1.00 0.99 0.99 13561
relaxation+generation+expansion+starting empty+random strenghening
4-20000-delete-clever-nr-im80-gen-empty-strrdm 0.95 35 0.00 1.00 0.52 0.35 12297
4-20000-replace-clever-nr-im80-gen-empty-strrdm 0.95 37 0.00 1.00 0.53 0.37 11164
4-20000-refine-clever-nr-im80-gen-empty-strrdm 0.93 38 0.00 1.00 0.55 0.38 10298
4-20000-add-clever-nr-im80-gen-empty-strrdm 0.93 73 0.00 1.00 0.88 0.79 13849
4-20000-addjoin-clever-nr-im80-gen-empty-strrdm 0.96 73 0.00 1.00 0.88 0.79 16560
4-20000-refadd-clever-nr-im80-gen-empty-strrdm 0.95 86 0.00 1.00 0.97 0.94 11624

Analysis

Analyst: Jérôme Euzenat (INRIA) (2019-09-10)

Key points:

Further experiments: *

This file can be retrieved from URL https://sake.re/20190604-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20190604-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20190604-NOOR.zip