20140224-NOOR

Experiment design

Hypothesis: This is a run for the record

Experimental setting: This is a largely new different setting from the previous experiment 20140208-NOOR:

F-measure uses Semantic precision and recall instead of syntactic
The matrix is symmetrics (the findings of one agent are communicated to the other one)

Results with this setting have several constraints:

The number of agents is 4, because with three agents in this setting they simply empty the correspondences (this is due to the structure of the network)
As a consequence, the number of iterations for converging seems to be 350 (rougthly)
It is not possible to compute the curve of incoherence (conjecture: Alcomo has memory leaks)

Experiment

Experimenter: Jérôme Euzenat (INRIA)

Date: 2014-02-24

Lazy lavender hash: 402eae910f33cf73a4663fe83308d0b0be6d7b5b

Parameters: params.sh

Command line (script.sh):

java -Xms500M -Xmx1G -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.0.9.jar:lib/slf4j/logback-core-1.0.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DrevisionModality=add -DnbRuns=5 -DnbAgents=4 -DnbIterations=350

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor.

Execution environment: MacBook Pro 15, C2D 2.66GHz, 8GB - MacOS X 6.8 - Java 1.6.0 HotSpot

Raw results

20140224-NOOR.tsv 20140224-NOOR.txt

Result exploration

In blue the average success rate (over 5 runs);
In dotted, the minimum and maximum success rate (over the same 5 runs);
In red, the average semantic F-measure.
In black, the best static semantic F-measure (Alcomo: 26%).

Analysis

Key points:

Agents in this setting are able to reach the quality of repair techniques
There is variability among different experiments (not always the winner (see logs)

Further experiments:

Compare with more agents
Compare with different modalities

This file can be retrieved from URL https://sake.re/20140224-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20140224-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)