20140311-NOOR

Experiment design

Hypothesis: Confirm the hypothesis that in 20140306-NOOR the difference between LogMap and Alcomo and the AlignmentAdjustingAgent approach cannot be found in simple terms of precision and recall, but are due to the semantic precision and recall.

Experimental setting: Very same setting as 20140306-NOOR with classical precision and recall, but the lines implementing semantic precision and recall have been replaced by those with classical precision and recall (commented in the software)

Experiment

Experimenter: Jérôme Euzenat (INRIA)

Date: 2014-03-11

Lazy lavender hash: 2aec5fe496c2b95760dba0ef87e82ac13264879b

Parameters: params.sh

Command line (script.sh):

 java -Xms500M -Xmx1G -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.0.9.jar:lib/slf4j/logback-core-1.0.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DrevisionModality=delete -DnbRuns=10 -DnbAgents=4 -DnbIterations=2000 > log4-2000-del.txt

java -Xms500M -Xmx1G -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.0.9.jar:lib/slf4j/logback-core-1.0.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DrevisionModality=replace -DnbRuns=10 -DnbAgents=4 -DnbIterations=2000 > log4-2000-repl.txt

java -Xms500M -Xmx1G -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.0.9.jar:lib/slf4j/logback-core-1.0.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DrevisionModality=add -DnbRuns=10 -DnbAgents=4 -DnbIterations=2000 > log4-2000-add.txt

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor.

Execution environment: MacBook Pro 15, C2D 2.66GHz, 8GB - MacOS X 6.8 - Java 1.6.0 HotSpot

Raw results

20140311-NOOR-4add.tsv 20140311-NOOR-4add.txt 20140311-NOOR-4del.tsv 20140311-NOOR-4del.txt 20140311-NOOR-4repl.tsv 20140311-NOOR-4repl.txt

Result exploration

Top:

Bottom:

modality size success convergence Incoherence degree F-measure
Reference 70 0.00 1.00
Initial 54 0.49 0.20
Delete 6 0.98 330 0.0 0.16
Replace 6 0.95 860 0.0 0.16
Add 12.2 0.88 1280 0.0 0.16
Alcomo 27.3 1 0.0 0.14
LogMap 35.7 1 0.0 0.14

Analysis

Key points: The hypothesis is confirmed:

Further experiments:


This file can be retrieved from URL https://sake.re/20140311-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20140311-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr.