20170208b-NOOR

Experiment design

Revision of networks of ontologies with logical repair systems only

Hypothesis: Alignment repair systems used alone do not achieve good success rate

Experimental setting: Repetion of 20150218-NOOR with

10000 iterations
4 agents
10 runs

Experiment

Experimenter: Jérôme Euzenat (INRIA)

Date: 20170208

Lazy lavender hash: 738f55c3e7dc6fc16ebcfe070adc314dd4703d21

Parameters: params.sh

Command line (script.sh):

java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DrevisionModality=nothing -DrawRepair=logmap -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec

java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DrevisionModality=nothing -DrawRepair=logmap -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec

java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DrevisionModality=nothing -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor, LogMapRepair, AlcomoRepair

Execution environment: Debian Linux virtual machine configured with four processors and 20GB of RAM running under a Dell PowerEdge T610 with 4*Intel Xeon Quad Core 1.9GHz E5-2420 processors, under Linux ProxMox 2 (Debian). - Java 1.8.0 HotSpot

Raw results

20170208-NOOR-4-10000-alcomo.tsv 20170208-NOOR-4-10000-alcomo.txt 20170208-NOOR-4-10000-logmap.tsv 20170208-NOOR-4-10000-logmap.txt 20170208-NOOR-4-10000-none.tsv 20170208-NOOR-4-10000-none.txt

Result exploration

modality	size	success	convergence	Incoherence degree	Precision	F-measure	Recall
Initial	54	.24	-	.34	.11	.20	.89
Alcomo	28	.43	-	0.	.21	.26	.33
LogMap	29	.51	-	0.	.24	.26	.29

In blue the average success rate (over 10 runs) of the initial alignment;
In green the average success rate (over 10 runs) of the LogMap repaired alignment;
In red the average success rate (over 10 runs) of the Alcomo repaired alignment;

Analysis

Key points:

The results in terms of success rates are not converging and establish at level of 50% for the best repair system

Further experiments:

We will test the performance, in terms of precision and recall, of using LogMap and Alcomo together with another operator

This file can be retrieved from URL https://sake.re/20170208b-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20170208b-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)