20170216-NOOR

Although, the hypothesis is not invalidated, the results for addjoin and refadd were not correct (see 20180529-NOOR)

Experiment design

Revision of networks of ontologies with expansion (4 agents; 10000 iterations)

Hypotheses:

Experimental setting: Repetition of 20170214a-NOOR and 20170214b-NOOR with expansion operators So all operators: delete/replace/refine/add/addjoin/refadd and modalities random/protected/clever with nonRedundancy and ontology look-up variants

Experiment

Experimenter: Jérôme Euzenat (INRIA)

Date: 20170215-20170216

Lazy lavender hash: dc342dca26a5a6feff895b2a1829b04e7dfaa5ae

Parameters: params.sh

Command line (script.sh):

for mod in delete replace refine add addjoin refadd
do
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${mod} -DexpandAlignments=random
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${mod} -DexpandAlignments=protected
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${mod} -DexpandAlignments=protected -DnonRedundancy
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${mod} -DexpandAlignments=clever
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${mod} -DexpandAlignments=clever -DnonRedundancy
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${mod} -DexpandAlignments=clever -DnonRedundancy -DontoLookup
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: Debian Linux virtual machine configured with four processors and 20GB of RAM running under a Dell PowerEdge T610 with 4*Intel Xeon Quad Core 1.9GHz E5-2420 processors, under Linux ProxMox 2 (Debian). - Java 1.8.0 HotSpot

Raw results

20170216-NOOR-4-10000-add-clever-nr-comp.tsv 20170216-NOOR-4-10000-add-clever-nr-comp.txt 20170216-NOOR-4-10000-add-clever-nr.tsv 20170216-NOOR-4-10000-add-clever-nr.txt 20170216-NOOR-4-10000-add-clever.tsv 20170216-NOOR-4-10000-add-clever.txt 20170216-NOOR-4-10000-add-protected-nr.tsv 20170216-NOOR-4-10000-add-protected-nr.txt 20170216-NOOR-4-10000-add-protected.tsv 20170216-NOOR-4-10000-add-protected.txt 20170216-NOOR-4-10000-add-random.tsv 20170216-NOOR-4-10000-add-random.txt 20170216-NOOR-4-10000-addjoin-clever-nr-comp.tsv 20170216-NOOR-4-10000-addjoin-clever-nr-comp.txt 20170216-NOOR-4-10000-addjoin-clever-nr.tsv 20170216-NOOR-4-10000-addjoin-clever-nr.txt 20170216-NOOR-4-10000-addjoin-clever.tsv 20170216-NOOR-4-10000-addjoin-clever.txt 20170216-NOOR-4-10000-addjoin-protected-nr.tsv 20170216-NOOR-4-10000-addjoin-protected-nr.txt 20170216-NOOR-4-10000-addjoin-protected.tsv 20170216-NOOR-4-10000-addjoin-protected.txt 20170216-NOOR-4-10000-addjoin-random.tsv 20170216-NOOR-4-10000-addjoin-random.txt 20170216-NOOR-4-10000-delete-clever-nr-comp.tsv 20170216-NOOR-4-10000-delete-clever-nr-comp.txt 20170216-NOOR-4-10000-delete-clever-nr.tsv 20170216-NOOR-4-10000-delete-clever-nr.txt 20170216-NOOR-4-10000-delete-clever.tsv 20170216-NOOR-4-10000-delete-clever.txt 20170216-NOOR-4-10000-delete-protected-nr.tsv 20170216-NOOR-4-10000-delete-protected-nr.txt 20170216-NOOR-4-10000-delete-protected.tsv 20170216-NOOR-4-10000-delete-protected.txt 20170216-NOOR-4-10000-delete-random.tsv 20170216-NOOR-4-10000-delete-random.txt 20170216-NOOR-4-10000-refadd-clever-nr-comp.tsv 20170216-NOOR-4-10000-refadd-clever-nr-comp.txt 20170216-NOOR-4-10000-refadd-clever-nr.tsv 20170216-NOOR-4-10000-refadd-clever-nr.txt 20170216-NOOR-4-10000-refadd-clever.tsv 20170216-NOOR-4-10000-refadd-clever.txt 20170216-NOOR-4-10000-refadd-protected-nr.tsv 20170216-NOOR-4-10000-refadd-protected-nr.txt 20170216-NOOR-4-10000-refadd-protected.tsv 20170216-NOOR-4-10000-refadd-protected.txt 20170216-NOOR-4-10000-refadd-random.tsv 20170216-NOOR-4-10000-refadd-random.txt 20170216-NOOR-4-10000-refine-clever-nr-comp.tsv 20170216-NOOR-4-10000-refine-clever-nr-comp.txt 20170216-NOOR-4-10000-refine-clever-nr.tsv 20170216-NOOR-4-10000-refine-clever-nr.txt 20170216-NOOR-4-10000-refine-clever.tsv 20170216-NOOR-4-10000-refine-clever.txt 20170216-NOOR-4-10000-refine-protected-nr.tsv 20170216-NOOR-4-10000-refine-protected-nr.txt 20170216-NOOR-4-10000-refine-protected.tsv 20170216-NOOR-4-10000-refine-protected.txt 20170216-NOOR-4-10000-refine-random.tsv 20170216-NOOR-4-10000-refine-random.txt 20170216-NOOR-4-10000-replace-clever-nr-comp.tsv 20170216-NOOR-4-10000-replace-clever-nr-comp.txt 20170216-NOOR-4-10000-replace-clever-nr.tsv 20170216-NOOR-4-10000-replace-clever-nr.txt 20170216-NOOR-4-10000-replace-clever.tsv 20170216-NOOR-4-10000-replace-clever.txt 20170216-NOOR-4-10000-replace-protected-nr.tsv 20170216-NOOR-4-10000-replace-protected-nr.txt 20170216-NOOR-4-10000-replace-protected.tsv 20170216-NOOR-4-10000-replace-protected.txt 20170216-NOOR-4-10000-replace-random.tsv 20170216-NOOR-4-10000-replace-random.txt

Result exploration

This is a lot of data.

First we look at convergence. It appears that random and protected do not seem to converge:

Convergence random protected protected-nr clever clever-nr clever-nr-comp
delete 9997 9998 9991 1952 1764 1778
replace 9999 9997 9995 2810 2690 2203
refine 9996 9998 9998 2498 4133 2519
add 9993 9995 9998 3534 3665 3542
addjoin 9996 9994 9994 2621 2292 2322
refadd 9985 9998 9999 4421 3405 3383

Hence from now on, we will only analyse the results of the clever modality (all data is in the files below).

To have an idea of the evolution of alignments, here is the size of the network for clever-nr on the 3000 first runs.

The figures for coherence/precision/recall/F-measure are as follows:

Coherence clever clever-nr Clever-nr-comp
delete 0.05 0.05 0.01
replace 0.11 0.11 0.08
refine 0.17 0.16 0.08
add 0.21 0.21 0.22
addjoin 0.22 0.19 0.20
refadd 0.25 0.23 0.22
Precision clever clever-nr Clever-nr-comp
delete 0.91 0.91 0.97
replace 0.84 0.84 0.86
refine 0.79 0.78 0.88
add 0.72 0.73 0.71
addjoin 0.72 0.75 0.74
refadd 0.69 0.71 0.72
Recall clever clever-nr Clever-nr-comp
delete 0.22 0.22 0.16
replace 0.39 0.38 0.30
refine 0.56 0.55 0.41
add 0.63 0.66 0.68
addjoin 0.63 0.59 0.57
refadd 0.78 0.76 0.73
F-measure clever clever-nr Clever-nr-comp
delete 0.35 0.35 0.27
replace 0.53 0.52 0.44
refine 0.66 0.65 0.56
add 0.67 0.69 0.69
addjoin 0.67 0.66 0.64
refadd 0.73 0.73 0.72

F-measure evolution on clever (solid), clever-nr (dashed) and clever-nr-com (dotted), first 5000 games:

(drawn from data scaled every 100 measures)

Analysis

Key points:

Further experiments:


This file can be retrieved from URL https://sake.re/20170216-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20170216-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr.