Experiment 20140224-NOOR

Agents achieve comensurate semantic F-measure as logical repairer (LogMap/Alcomo)

Experiment design

DockerOS DockerEXP

Date: 2014-02-24

Hypotheses: This is a run for the record

Variation of: 20140208-NOOR

4 agents; 5 runs; 350 games

Experimental setting: This is a largely new different setting from the previous experiment [20140208-NOOR] F-measure uses Semantic precision and recall instead of syntactic. The matrix is symmetrics (the findings of one agent are communicated to the other one). Results with this setting have several constraints. The number of agents is 4, because with three agents in this setting they simply empty the correspondences (this is due to the structure of the network). As a consequence, the number of iterations for converging seems to be 350 (rougthly). It is not possible to compute the curve of incoherence (conjecture: Alcomo has memory leaks).

Variables

controled variables: []

dependent variables: ['srate', 'fmeas', 'conv']

Experiment

Date: 2014-02-24

Performer: Jérôme Euzenat (INRIA)

Lazy lavender hash: 402eae910f33cf73a4663fe83308d0b0be6d7b5b

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: wheezy

Parameter file: params.sh

Executed command (script.sh):

#!/bin/bash

. params.sh

java -Xms500M -Xmx1G -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor -DrevisionModality=add -DnbRuns=5 -DnbAgents=4 -DnbIterations=350

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor.

Execution environment: MacBook Pro 15, C2D 2.66GHz, 8GB - MacOS X 6.8 - Java 1.6.0 HotSpot

Raw results

results/
  20140224-NOOR.tsv
  20140224-NOOR.txt

Analysis

Initial results

Discussion

Key points:

  • Agents in this setting are able to reach the quality of repair techniques
  • There is variability among different experiments (not always the winner (see logs)

Further experiments:

  • Compare with more agents
  • Compare with different modalities

This file can be retrieved from URL https://sake.re/20140224-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20140224-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)

See original markdown (20140224-NOOR.md) or HTML (20140224-NOOR.html) files.