20180308-NOOR

Experiment design

Hypothesis:

Experimental setting: Same as the initial experiments 20170214a-NOOR and 20170214b-NOOR with:

Experiment

Experimenter: Jérôme Euzenat (INRIA)

Date: 2018-03-08

Lazy lavender hash: fa69a4569fb2ecd7d8f24759d1ce0cd71080811a

Command line (script.sh):

JPATH=lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.
OPT="-DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec"

# Dry test only for generating the runs
# bash scripts/runexp.sh -d 4-10000-nothing java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=nothing -DsaveDir=input/expeRun -DsaveInit -DsaveParams -DsaveGames

LOADOPT="-DloadDir=input/expeRun -DloadEnv -DloadAgents -DreplayGames"

for mod in delete replace refine add addjoin refadd
do
   bash scripts/runexp.sh -d 4-10000-${mod} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${mod}
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor.

Execution environment: Debian Linux virtual machine configured with four processors and 20GB of RAM running under a Dell PowerEdge T610 with 4*Intel Xeon Quad Core 1.9GHz E5-2420 processors, under Linux ProxMox 2 (Debian). - Java 1.8.0_151-b12 HotSpot

Note: This experiment has been set up to first generate the runs and then play them with different configuration (it has generated the expeRun dataset). To strictly repeat it, it is necessary to reuse the same input and not generate a new one. Hence, the input is saved in the repository, the generating line above must be commented (including in Docker).

Input: Input required for reproducibility can be retrieved from: https://files.inria.fr/sakere/input/expeRun.zip Then unzip expeRun.zip -d input

Raw results

4-10000-add.tsv 4-10000-add.txt 4-10000-addjoin.tsv 4-10000-addjoin.txt 4-10000-delete.tsv 4-10000-delete.txt 4-10000-nothing.tsv 4-10000-nothing.txt 4-10000-refadd.tsv 4-10000-refadd.txt 4-10000-refine.tsv 4-10000-refine.txt 4-10000-replace.tsv 4-10000-replace.txt

Result exploration

opsuccess
rate
network
size
incoherence
degree
semantic
precision
semantic
F-measure
semantic
recall
maximum
convergence
delete1.0060.001.000.130.07290
replace0.99120.020.960.250.141224
refine0.99200.030.950.380.241224
add0.98310.160.790.520.391526
addjoin0.99310.160.790.520.391526
refadd0.99480.150.790.670.581554

Analysis

Key points:

Further experiments:

Remarks:


This file can be retrieved from URL https://sake.re/20180308-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20180308-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr.

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20180308-LOG.zip