Experiment 20180827-NOOR¶

The generative modality starting with empty networks is on par with random networks
2019-12-07: This experiment does not allow to conclude

Experiment design¶

Date: 20180827

Designer: Jérôme Euzenat (INRIA)

Hypotheses: Starting from scratch and generating correspondences when needed should not be different from starting with initial alignments

Variation of: 20170531-NOOR

4 agents; 10 runs; 10000 games

Adaptation operators: delete replace refine add addjoin refadd

Experimental setting: Same as [[20170531-NOOR]] replaying the same runs as [[20180308-NOOR]] and after fixing addjoin and expansion.

Variables¶

controled variables: ['revisionModality', 'immediateRatio']

dependent variables: ['srate', 'size', 'inc', 'prec', 'fmeas', 'rec', 'conv']

Experiment¶

Date: 20180827

Performer: Jérôme Euzenat (INRIA)

Lazy lavender hash: 759ff097b96520c12aa84f3749927f9a22022e62

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Parameter file: params.sh

Executed command (script.sh):

#!/bin/bash

. params.sh

for op in ${OPS}
do
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-${POSTFIX} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative -DstartEmpty
   bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-${POSTFIX} java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -DstartEmpty
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: 24 * Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz with 20GB RAM / Linux ProxMox 2 / Linux 4.15.17-1-pve / Java Java(TM) SE Runtime Environment 1.8.0_151 with 4.33G max heap size

Input: Input required for reproducibility can be retrieved from: https://files.inria.fr/sakere/input/expeRun.zip or Then unzip expeRun.zip -d input

Raw results¶

Analysis¶

Results are compared to that of 20180601-NOOR (in dashed) which uses the same configurations of parameters but start with random (non realistic) alignments.

Relaxation	Operator	Success rate	Size	Incoherence	Semantic precision	Semantic F-measure	Semantic recall	Convergence
0	delete	0.94	72	0.13	0.85	0.70	0.60	9899
0	replace	0.94	72	0.14	0.83	0.70	0.61	9817
0	refine	0.94	78	0.16	0.82	0.76	0.70	9888
0	add	0.93	83	0.15	0.82	0.76	0.72	9666
0	addjoin	0.96	80	0.13	0.84	0.75	0.68	9951
0	refadd	0.94	81	0.15	0.81	0.78	0.76	9315
1	delete	0.94	61	0.00	1.00	0.61	0.44	9910
1	replace	0.94	63	0.00	1.00	0.65	0.48	9781
1	refine	0.94	74	0.00	1.00	0.78	0.64	8250
1	add	0.93	77	0.00	1.00	0.77	0.63	9233
1	addjoin	0.95	77	0.00	1.00	0.78	0.64	7848
1	refadd	0.94	80	0.00	1.00	0.82	0.69	7318

Discussion¶

Analyst: Jérôme Euzenat (INRIA) (2018-08-28)

Key points¶

The results are remarkably similar to those of 20170531-NOOR
Looking at the curves and figures, they are not necessarily that close to 20180601-NOOR.

Further experiments¶

Maybe starting with a realistic size of network

This file can be retrieved from URL https://sake.re/20180827-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20180827-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20180827-NOOR.zip

See original markdown (20180827-NOOR.md) or HTML (20180827-NOOR.html) files.