20170917-NOOR

2018-05-16: Under scrutiny because expansion did not work since commit 562f5c4b96893d30e2dc9fd59d935e013ce5219f

Experiment design

Hypothesis:

Experimental setting: Same as 20170530-NOOR and 20170531-NOOR with 6 agents

Experiment

Experimenter: Jérôme Euzenat (INRIA)

Date: 2017-09-17 to 19

Lazy lavender hash: 562f5c4b96893d30e2dc9fd59d935e013ce5219f

Parameters: params.sh

Command line (script.sh):

for op in delete replace refine add addjoin refadd
do
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=6 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=6 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=6 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=6 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=6 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative -Dstartempty
   java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DnbAgents=6 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -Dstartempty
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: Debian Linux virtual machine configured with four processors and 20GB of RAM running under a Dell PowerEdge T610 with 4*Intel Xeon Quad Core 1.9GHz E5-2420 processors, under Linux ProxMox 2 (Debian). - Java 1.8.0_131-b11 HotSpot

Raw results

6-10000-add-clever-nr-gen-empty.tsv 6-10000-add-clever-nr-gen-empty.txt 6-10000-add-clever-nr-gen.tsv 6-10000-add-clever-nr-gen.txt 6-10000-add-clever-nr-im80-gen-empty.tsv 6-10000-add-clever-nr-im80-gen-empty.txt 6-10000-add-clever-nr-im80-gen.tsv 6-10000-add-clever-nr-im80-gen.txt 6-10000-add-clever-nr-im80.tsv 6-10000-add-clever-nr-im80.txt 6-10000-add-clever-nr.tsv 6-10000-add-clever-nr.txt 6-10000-addjoin-clever-nr-gen-empty.tsv 6-10000-addjoin-clever-nr-gen-empty.txt 6-10000-addjoin-clever-nr-gen.tsv 6-10000-addjoin-clever-nr-gen.txt 6-10000-addjoin-clever-nr-im80-gen-empty.tsv 6-10000-addjoin-clever-nr-im80-gen-empty.txt 6-10000-addjoin-clever-nr-im80-gen.tsv 6-10000-addjoin-clever-nr-im80-gen.txt 6-10000-addjoin-clever-nr-im80.tsv 6-10000-addjoin-clever-nr-im80.txt 6-10000-addjoin-clever-nr.tsv 6-10000-addjoin-clever-nr.txt 6-10000-delete-clever-nr-gen-empty.tsv 6-10000-delete-clever-nr-gen-empty.txt 6-10000-delete-clever-nr-gen.tsv 6-10000-delete-clever-nr-gen.txt 6-10000-delete-clever-nr-im80-gen-empty.tsv 6-10000-delete-clever-nr-im80-gen-empty.txt 6-10000-delete-clever-nr-im80-gen.tsv 6-10000-delete-clever-nr-im80-gen.txt 6-10000-delete-clever-nr-im80.tsv 6-10000-delete-clever-nr-im80.txt 6-10000-delete-clever-nr.tsv 6-10000-delete-clever-nr.txt 6-10000-refadd-clever-nr-gen-empty.tsv 6-10000-refadd-clever-nr-gen-empty.txt 6-10000-refadd-clever-nr-gen.tsv 6-10000-refadd-clever-nr-gen.txt 6-10000-refadd-clever-nr-im80-gen-empty.tsv 6-10000-refadd-clever-nr-im80-gen-empty.txt 6-10000-refadd-clever-nr-im80-gen.tsv 6-10000-refadd-clever-nr-im80-gen.txt 6-10000-refadd-clever-nr-im80.tsv 6-10000-refadd-clever-nr-im80.txt 6-10000-refadd-clever-nr.tsv 6-10000-refadd-clever-nr.txt 6-10000-refine-clever-nr-gen-empty.tsv 6-10000-refine-clever-nr-gen-empty.txt 6-10000-refine-clever-nr-gen.tsv 6-10000-refine-clever-nr-gen.txt 6-10000-refine-clever-nr-im80-gen-empty.tsv 6-10000-refine-clever-nr-im80-gen-empty.txt 6-10000-refine-clever-nr-im80-gen.tsv 6-10000-refine-clever-nr-im80-gen.txt 6-10000-refine-clever-nr-im80.tsv 6-10000-refine-clever-nr-im80.txt 6-10000-refine-clever-nr.tsv 6-10000-refine-clever-nr.txt 6-10000-replace-clever-nr-gen-empty.tsv 6-10000-replace-clever-nr-gen-empty.txt 6-10000-replace-clever-nr-gen.tsv 6-10000-replace-clever-nr-gen.txt 6-10000-replace-clever-nr-im80-gen-empty.tsv 6-10000-replace-clever-nr-im80-gen-empty.txt 6-10000-replace-clever-nr-im80-gen.tsv 6-10000-replace-clever-nr-im80-gen.txt 6-10000-replace-clever-nr-im80.tsv 6-10000-replace-clever-nr-im80.txt 6-10000-replace-clever-nr.tsv 6-10000-replace-clever-nr.txt

Result exploration

testsuccess
rate
network
size
incoherence
degree
semantic
precision
semantic
F-measure
semantic
recall
maximum
convergence
6-10000-delete-clever-nr0.95150.001.000.020.015722
6-10000-replace-clever-nr0.91460.010.970.050.039685
6-10000-refine-clever-nr0.811130.020.950.160.099416
6-10000-add-clever-nr0.703360.190.730.510.409991
6-10000-addjoin-clever-nr0.901030.040.930.160.099967
6-10000-refadd-clever-nr0.802980.070.890.400.269988
6-10000-delete-clever-nr-im800.95150.001.000.020.013179
6-10000-replace-clever-nr-im800.91440.001.000.050.029269
6-10000-refine-clever-nr-im800.811030.000.980.140.079919
6-10000-add-clever-nr-im800.693000.050.880.280.179995
6-10000-addjoin-clever-nr-im800.90920.000.990.100.059953
6-10000-refadd-clever-nr-im800.802770.010.970.320.199994
6-10000-delete-clever-nr-gen0.566030.000.910.310.1910000
6-10000-replace-clever-nr-gen0.565880.000.900.310.199998
6-10000-refine-clever-nr-gen0.545880.000.890.350.229999
6-10000-add-clever-nr-gen0.547320.130.540.550.5610000
6-10000-addjoin-clever-nr-gen0.646730.020.890.460.319997
6-10000-refadd-clever-nr-gen0.637060.030.870.550.4010000
6-10000-delete-clever-nr-im80-gen0.565990.000.910.310.1810000
6-10000-replace-clever-nr-im80-gen0.565950.000.900.310.199997
6-10000-refine-clever-nr-im80-gen0.535760.000.890.340.2110000
6-10000-add-clever-nr-im80-gen0.527200.050.610.390.2910000
6-10000-addjoin-clever-nr-im80-gen0.646900.000.890.420.2710000
6-10000-refadd-clever-nr-im80-gen0.627120.000.900.500.3510000
6-10000-delete-clever-nr-gen-empty0.596910.000.910.330.2010000
6-10000-replace-clever-nr-gen-empty0.596920.000.920.330.209998
6-10000-refine-clever-nr-gen-empty0.596870.000.920.330.2010000
6-10000-add-clever-nr-gen-empty0.629040.000.800.370.2410000
6-10000-addjoin-clever-nr-gen-empty0.688150.000.920.410.279997
6-10000-refadd-clever-nr-gen-empty0.688160.000.920.420.279999
6-10000-delete-clever-nr-im80-gen-empty0.606970.000.920.320.2010000
6-10000-replace-clever-nr-im80-gen-empty0.596980.000.920.320.209999
6-10000-refine-clever-nr-im80-gen-empty0.606940.000.920.330.2010000
6-10000-add-clever-nr-im80-gen-empty0.629030.000.800.380.2510000
6-10000-addjoin-clever-nr-im80-gen-empty0.688190.000.920.410.2710000
6-10000-refadd-clever-nr-im80-gen-empty0.678120.000.920.410.279999

(drawn from data scaled every 100 measures)

Analysis

Key points:

Further experiments:


This file can be retrieved from URL https://sake.re/20170917-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20170917-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr.

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20170917-NOOR.zip