20171231-NOOR

Mostly incorrect results because expansion did not work since commit 562f5c4b96893d30e2dc9fd59d935e013ce5219f: see 20180927-NOOR

Experiment design

Hypothesis: If the initial network contains the correct number of expected correspondences, then its results will be closer to what is obtained with empty alignment, than the standard initial network

Experimental setting: Same as 20171128-NOOR with 100000 iterations and additionally -Drealistic option

Experiment

Experimenter: Jérôme Euzenat (INRIA)

Date: 2018-01-16 -- 2018-02-12 (The 2017-12-31 test was not sufficient)

Lazy lavender hash: ef0e80766f2b2177c5d3c1aadba6d0ca556780ef

Parameters: params.sh

Command line (script.sh):

JPATH=lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OPT=" -DnbAgents=5 -DnbIterations=100000 -DnbRuns=10 -DreportPrecRec"

for op in delete replace refine add addjoin refadd
do
   # clever-nr
  bash scripts/runexp.sh -d 5-100000-real-${op}-clever-nr-gen java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative -Drealistic=1
  bash scripts/runexp.sh -d 5-100000-${op}-clever-nr-gen java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative
  bash scripts/runexp.sh -d 5-100000-${op}-clever-nr-gen-empty java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative -Dstartempty

  # clever-nr plus im80
  bash scripts/runexp.sh -d 5-100000-real-${op}-clever-nr-im80-gen java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative -Drealistic=1
  bash scripts/runexp.sh -d 5-100000-${op}-clever-nr-im80-gen java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative
  bash scripts/runexp.sh -d 5-100000-${op}-clever-nr-im80-gen-empty java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy  -DimmediateRatio=80 -Dgenerative -Dstartempty

done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor.

Execution environment: Debian Linux virtual machine configured with four processors and 20GB of RAM running under a Dell PowerEdge T610 with 4*Intel Xeon Quad Core 1.9GHz E5-2420 processors, under Linux ProxMox 2 (Debian). - Java 1.8.0_151-b12 HotSpot (max heap 4.33G)

Raw results

5-100000-add-clever-nr-gen-empty.tsv 5-100000-add-clever-nr-gen-empty.txt 5-100000-add-clever-nr-gen.tsv 5-100000-add-clever-nr-gen.txt 5-100000-add-clever-nr-im80-gen-empty.tsv 5-100000-add-clever-nr-im80-gen-empty.txt 5-100000-add-clever-nr-im80-gen.tsv 5-100000-add-clever-nr-im80-gen.txt 5-100000-addjoin-clever-nr-gen-empty.tsv 5-100000-addjoin-clever-nr-gen-empty.txt 5-100000-addjoin-clever-nr-gen.tsv 5-100000-addjoin-clever-nr-gen.txt 5-100000-addjoin-clever-nr-im80-gen-empty.tsv 5-100000-addjoin-clever-nr-im80-gen-empty.txt 5-100000-addjoin-clever-nr-im80-gen.tsv 5-100000-addjoin-clever-nr-im80-gen.txt 5-100000-delete-clever-nr-gen-empty.tsv 5-100000-delete-clever-nr-gen-empty.txt 5-100000-delete-clever-nr-gen.tsv 5-100000-delete-clever-nr-gen.txt 5-100000-delete-clever-nr-im80-gen-empty.tsv 5-100000-delete-clever-nr-im80-gen-empty.txt 5-100000-delete-clever-nr-im80-gen.tsv 5-100000-delete-clever-nr-im80-gen.txt 5-100000-real-add-clever-nr-gen.tsv 5-100000-real-add-clever-nr-gen.txt 5-100000-real-add-clever-nr-im80-gen.tsv 5-100000-real-add-clever-nr-im80-gen.txt 5-100000-real-addjoin-clever-nr-gen.tsv 5-100000-real-addjoin-clever-nr-gen.txt 5-100000-real-addjoin-clever-nr-im80-gen.tsv 5-100000-real-addjoin-clever-nr-im80-gen.txt 5-100000-real-delete-clever-nr-gen.tsv 5-100000-real-delete-clever-nr-gen.txt 5-100000-real-delete-clever-nr-im80-gen.tsv 5-100000-real-delete-clever-nr-im80-gen.txt 5-100000-real-refadd-clever-nr-gen.tsv 5-100000-real-refadd-clever-nr-gen.txt 5-100000-real-refadd-clever-nr-im80-gen.tsv 5-100000-real-refadd-clever-nr-im80-gen.txt 5-100000-real-refine-clever-nr-gen.tsv 5-100000-real-refine-clever-nr-gen.txt 5-100000-real-refine-clever-nr-im80-gen.tsv 5-100000-real-refine-clever-nr-im80-gen.txt 5-100000-real-replace-clever-nr-gen.tsv 5-100000-real-replace-clever-nr-gen.txt 5-100000-real-replace-clever-nr-im80-gen.tsv 5-100000-real-replace-clever-nr-im80-gen.txt 5-100000-refadd-clever-nr-gen-empty.tsv 5-100000-refadd-clever-nr-gen-empty.txt 5-100000-refadd-clever-nr-gen.tsv 5-100000-refadd-clever-nr-gen.txt 5-100000-refadd-clever-nr-im80-gen-empty.tsv 5-100000-refadd-clever-nr-im80-gen-empty.txt 5-100000-refadd-clever-nr-im80-gen.tsv 5-100000-refadd-clever-nr-im80-gen.txt 5-100000-refine-clever-nr-gen-empty.tsv 5-100000-refine-clever-nr-gen-empty.txt 5-100000-refine-clever-nr-gen.tsv 5-100000-refine-clever-nr-gen.txt 5-100000-refine-clever-nr-im80-gen-empty.tsv 5-100000-refine-clever-nr-im80-gen-empty.txt 5-100000-refine-clever-nr-im80-gen.tsv 5-100000-refine-clever-nr-im80-gen.txt 5-100000-replace-clever-nr-gen-empty.tsv 5-100000-replace-clever-nr-gen-empty.txt 5-100000-replace-clever-nr-gen.tsv 5-100000-replace-clever-nr-gen.txt 5-100000-replace-clever-nr-im80-gen-empty.tsv 5-100000-replace-clever-nr-im80-gen-empty.txt 5-100000-replace-clever-nr-im80-gen.tsv 5-100000-replace-clever-nr-im80-gen.txt

Result exploration

The figures reproduce those of [Euzenat 2017b] with 100000 runs, 5 agents and a realistic size of networks.

It displays precision, size and recall to be more precise.

Superimposition of the precision (left), size (centre) and recall (right) curves for operators with standard random network (red) random network with realistic size (black) and and empty ones (blue) [expansion+generative; 5 agents; 10 runs; 100000 games].

The first figure is without relaxation; the second one with a relaxation immediacy rate of 80%.

Observations:

Analysis

Key points:

Further experiments:


This file can be retrieved from URL https://sake.re/20171231-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20171231-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20171231-LOG.zip