20171001-NOOR

2018-09-17: expansion did not work properly since commit 562f5c4b96893d30e2dc9fd59d935e013ce5219f (see 20180917-NOOR)

Experiment design

Hypothesis:

Experimental setting: Same as 20170530-NOOR and 20170531-NOOR with 5 agents and 20000 iterations

Experiment

Experimenter: Jérôme Euzenat (INRIA)

Date: 2017-09-28 to 2017-10-08

Lazy lavender hash: 562f5c4b96893d30e2dc9fd59d935e013ce5219f

Parameters: params.sh

Command line (script.sh):

for op in delete replace refine add addjoin refadd
do
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=5 -DnbIterations=20000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=5 -DnbIterations=20000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=5 -DnbIterations=20000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=5 -DnbIterations=20000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=5 -DnbIterations=20000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative -Dstartempty
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=5 -DnbIterations=20000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -Dstartempty
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: Debian Linux virtual machine configured with four processors and 20GB of RAM running under a Dell PowerEdge T610 with 4*Intel Xeon Quad Core 1.9GHz E5-2420 processors, under Linux ProxMox 2 (Debian). - Java 1.8.0_131-b11 HotSpot

Raw results

5-20000-add-clever-nr-gen-empty.tsv 5-20000-add-clever-nr-gen-empty.txt 5-20000-add-clever-nr-gen.tsv 5-20000-add-clever-nr-gen.txt 5-20000-add-clever-nr-im80-gen-empty.tsv 5-20000-add-clever-nr-im80-gen-empty.txt 5-20000-add-clever-nr-im80-gen.tsv 5-20000-add-clever-nr-im80-gen.txt 5-20000-add-clever-nr-im80.tsv 5-20000-add-clever-nr-im80.txt 5-20000-add-clever-nr.tsv 5-20000-add-clever-nr.txt 5-20000-addjoin-clever-nr-gen-empty.tsv 5-20000-addjoin-clever-nr-gen-empty.txt 5-20000-addjoin-clever-nr-gen.tsv 5-20000-addjoin-clever-nr-gen.txt 5-20000-addjoin-clever-nr-im80-gen-empty.tsv 5-20000-addjoin-clever-nr-im80-gen-empty.txt 5-20000-addjoin-clever-nr-im80-gen.tsv 5-20000-addjoin-clever-nr-im80-gen.txt 5-20000-addjoin-clever-nr-im80.tsv 5-20000-addjoin-clever-nr-im80.txt 5-20000-addjoin-clever-nr.tsv 5-20000-addjoin-clever-nr.txt 5-20000-delete-clever-nr-gen-empty.tsv 5-20000-delete-clever-nr-gen-empty.txt 5-20000-delete-clever-nr-gen.tsv 5-20000-delete-clever-nr-gen.txt 5-20000-delete-clever-nr-im80-gen-empty.tsv 5-20000-delete-clever-nr-im80-gen-empty.txt 5-20000-delete-clever-nr-im80-gen.tsv 5-20000-delete-clever-nr-im80-gen.txt 5-20000-delete-clever-nr-im80.tsv 5-20000-delete-clever-nr-im80.txt 5-20000-delete-clever-nr.tsv 5-20000-delete-clever-nr.txt 5-20000-refadd-clever-nr-gen-empty.tsv 5-20000-refadd-clever-nr-gen-empty.txt 5-20000-refadd-clever-nr-gen.tsv 5-20000-refadd-clever-nr-gen.txt 5-20000-refadd-clever-nr-im80-gen-empty.tsv 5-20000-refadd-clever-nr-im80-gen-empty.txt 5-20000-refadd-clever-nr-im80-gen.tsv 5-20000-refadd-clever-nr-im80-gen.txt 5-20000-refadd-clever-nr-im80.tsv 5-20000-refadd-clever-nr-im80.txt 5-20000-refadd-clever-nr.tsv 5-20000-refadd-clever-nr.txt 5-20000-refine-clever-nr-gen-empty.tsv 5-20000-refine-clever-nr-gen-empty.txt 5-20000-refine-clever-nr-gen.tsv 5-20000-refine-clever-nr-gen.txt 5-20000-refine-clever-nr-im80-gen-empty.tsv 5-20000-refine-clever-nr-im80-gen-empty.txt 5-20000-refine-clever-nr-im80-gen.tsv 5-20000-refine-clever-nr-im80-gen.txt 5-20000-refine-clever-nr-im80.tsv 5-20000-refine-clever-nr-im80.txt 5-20000-refine-clever-nr.tsv 5-20000-refine-clever-nr.txt 5-20000-replace-clever-nr-gen-empty.tsv 5-20000-replace-clever-nr-gen-empty.txt 5-20000-replace-clever-nr-gen.tsv 5-20000-replace-clever-nr-gen.txt 5-20000-replace-clever-nr-im80-gen-empty.tsv 5-20000-replace-clever-nr-im80-gen-empty.txt 5-20000-replace-clever-nr-im80-gen.tsv 5-20000-replace-clever-nr-im80-gen.txt 5-20000-replace-clever-nr-im80.tsv 5-20000-replace-clever-nr-im80.txt 5-20000-replace-clever-nr.tsv 5-20000-replace-clever-nr.txt

Result exploration

testsuccess
rate
network
size
incoherence
degree
semantic
precision
semantic
F-measure
semantic
recall
maximum
convergence
5-20000-delete-clever-nr0.99100.001.000.060.031095
5-20000-replace-clever-nr0.98250.010.990.110.062345
5-20000-refine-clever-nr0.97490.020.960.230.134576
5-20000-add-clever-nr0.96960.160.790.460.328048
5-20000-addjoin-clever-nr0.98530.070.880.280.174698
5-20000-refadd-clever-nr0.971160.100.860.500.365435
5-20000-delete-clever-nr-im800.99100.001.000.060.031576
5-20000-replace-clever-nr-im800.98230.001.000.100.053531
5-20000-refine-clever-nr-im800.97500.001.000.220.138504
5-20000-add-clever-nr-im800.96810.001.000.280.1615004
5-20000-addjoin-clever-nr-im800.98470.001.000.180.103268
5-20000-refadd-clever-nr-im800.971150.001.000.450.297870
5-20000-delete-clever-nr-gen0.892340.001.000.500.3319903
5-20000-replace-clever-nr-gen0.892300.000.990.520.3519962
5-20000-refine-clever-nr-gen0.892310.010.990.560.3919958
5-20000-add-clever-nr-gen0.832590.100.760.670.6019998
5-20000-addjoin-clever-nr-gen0.902290.030.970.630.4619987
5-20000-refadd-clever-nr-gen0.912340.060.930.700.5619998
5-20000-delete-clever-nr-im80-gen0.892310.001.000.500.3319920
5-20000-replace-clever-nr-im80-gen0.892310.000.990.520.3519958
5-20000-refine-clever-nr-im80-gen0.892290.000.990.560.3919971
5-20000-add-clever-nr-im80-gen0.842540.020.880.580.4320000
5-20000-addjoin-clever-nr-im80-gen0.902300.000.990.590.4219896
5-20000-refadd-clever-nr-im80-gen0.912410.000.990.660.4919975
5-20000-delete-clever-nr-gen-empty0.902600.001.000.540.3719955
5-20000-replace-clever-nr-gen-empty0.902600.001.000.530.3719956
5-20000-refine-clever-nr-gen-empty0.902600.001.000.530.3619982
5-20000-add-clever-nr-gen-empty0.882790.000.930.520.3619989
5-20000-addjoin-clever-nr-gen-empty0.922610.001.000.560.3919988
5-20000-refadd-clever-nr-gen-empty0.922610.000.990.560.3919968
5-20000-delete-clever-nr-im80-gen-empty0.902600.001.000.540.3719820
5-20000-replace-clever-nr-im80-gen-empty0.902610.001.000.540.3719518
5-20000-refine-clever-nr-im80-gen-empty0.902600.001.000.530.3619812
5-20000-add-clever-nr-im80-gen-empty0.882770.000.940.530.3719999
5-20000-addjoin-clever-nr-im80-gen-empty0.922610.000.990.560.3919997
5-20000-refadd-clever-nr-im80-gen-empty0.922610.000.990.560.3919958

(drawn from data scaled every 100 measures)

Analysis

Key points:

Further experiments:


This file can be retrieved from URL https://sake.re/20171001-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20171001-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20171001-NOOR.zip