20170227-NOOR

The results for 'clever' (expansion with memory) were ineffective (see 20170305-NOOR)

Experiment design

Application of repair algorithms every 400 games (4 agents; 10000 iterations)

Hypotheses:

Using repair systems does not improve F-measure whatever the level achieved by agents.

Experimental setting: Repetition of 20170209-NOOR, 20170215a-NOOR, 20170215b-NOOR and 20170216-NOOR for refadd with logmap and alcomo revision every 400 games

Experiment

Experimenter: Jérôme Euzenat (INRIA)

Date: 2017-02-27

Lazy lavender hash: a0ed90e78c55040a5514ed57e750d5a73e4472f5

Parameters: params.sh

Command line (script.sh):

java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=refadd -DrepeatRepairs=400 -Drepairers=logmap,alcomo

java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=refadd -DrepeatRepairs=400 -Drepairers=logmap,alcomo -DexpandAlignments=clever -DnonRedundancy

java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=refadd -DrepeatRepairs=400 -Drepairers=logmap,alcomo -DimmediateRatio=80

java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.1.9.jar:lib/slf4j/logback-core-1.1.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=refadd -DrepeatRepairs=400 -Drepairers=logmap,alcomo -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor, AlcomoRepair, LogMapRepair

Execution environment: Debian Linux virtual machine configured with four processors and 20GB of RAM running under a Dell PowerEdge T610 with 4*Intel Xeon Quad Core 1.9GHz E5-2420 processors, under Linux ProxMox 2 (Debian). - Java 1.8.0 HotSpot

Raw results

20170227-NOOR-4-10000-refadd-clever-nr-im80-rr.tsv 20170227-NOOR-4-10000-refadd-clever-nr-im80-rr.txt 20170227-NOOR-4-10000-refadd-clever-nr-rr.tsv 20170227-NOOR-4-10000-refadd-clever-nr-rr.txt 20170227-NOOR-4-10000-refadd-im80-rr.tsv 20170227-NOOR-4-10000-refadd-im80-rr.txt 20170227-NOOR-4-10000-refadd-rr.tsv 20170227-NOOR-4-10000-refadd-rr.txt

Result exploration

No expansion, no immediate consumption probability:

Memory expansion, no immediate consumption probability:

No expansion, immediate consumption probability 80%:

Memory expansion, immediate consumption probability 80%:

(drawn from data scaled every 400 measures)

Analysis

Key points:

Except for the last case, it is clear that the logical repairers do not improve over the agents
However, the fact that incoherence is not reduced to 0.0 in the second and last case, let suspect a bug

Further experiments:

redo after correcting the bug (see 20170305-NOOR)

This file can be retrieved from URL https://sake.re/20170227-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20170227-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)