20180531-NOOR

Experiment design

Designer: Jérôme Euzenat (INRIA) (2018-03-07)

Revision of networks of ontologies with expansion and expansion starting with a realistic alignment size (4 agents; 10 runs; 10000 games; delete/replace/refine/add/addjoin/refadd)

Hypotheses: Starting with a realistic alignment size improves the final size (hence recall and F-measure)

Experimental setting: Same as 20180307-NOOR replaying the same runs as 20180308-NOOR (with different initial alignments) and after fixing addjoin, expansion and real size estimation.

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2018-05-31)

Date: 2018-05-31

Lazy lavender hash: d50e70f87bca76951ec2f149dc8ae1d42b9a1a28

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20180307-NOOR

Parameters: params.sh

Command line (script.sh):

#!/bin/bash

JPATH=lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OPT="-DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec"

LOADOPT="-DloadDir=input/expeRun -DloadEnv -DloadAgents -DreplayGames"

for op in delete replace refine add addjoin refadd
do
  bash scripts/runexp.sh -d 4-10000-${op}-real java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op}
  bash scripts/runexp.sh -d 4-10000-${op}-im80-real java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DimmediateRatio=80
  bash scripts/runexp.sh -d 4-10000-${op}-clever-nr-real java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy
  bash scripts/runexp.sh -d 4-10000-${op}-clever-nr-im80-real java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: Debian Linux virtual machine configured with four processors and 20GB of RAM running under a Dell PowerEdge T610 with 4*Intel Xeon Quad Core 1.9GHz E5-2420 processors, under Linux ProxMox 2 (Debian). - Java 1.8.0 HotSpot

Raw results

4-10000-add-clever-nr-im80-real.tsv 4-10000-add-clever-nr-im80-real.txt 4-10000-add-clever-nr-real.tsv 4-10000-add-clever-nr-real.txt 4-10000-add-im80-real.tsv 4-10000-add-im80-real.txt 4-10000-add-real.tsv 4-10000-add-real.txt 4-10000-addjoin-clever-nr-im80-real.tsv 4-10000-addjoin-clever-nr-im80-real.txt 4-10000-addjoin-clever-nr-real.tsv 4-10000-addjoin-clever-nr-real.txt 4-10000-addjoin-im80-real.tsv 4-10000-addjoin-im80-real.txt 4-10000-addjoin-real.tsv 4-10000-addjoin-real.txt 4-10000-delete-clever-nr-im80-real.tsv 4-10000-delete-clever-nr-im80-real.txt 4-10000-delete-clever-nr-real.tsv 4-10000-delete-clever-nr-real.txt 4-10000-delete-im80-real.tsv 4-10000-delete-im80-real.txt 4-10000-delete-real.tsv 4-10000-delete-real.txt 4-10000-nothing.tsv 4-10000-nothing.txt 4-10000-refadd-clever-nr-im80-real.tsv 4-10000-refadd-clever-nr-im80-real.txt 4-10000-refadd-clever-nr-real.tsv 4-10000-refadd-clever-nr-real.txt 4-10000-refadd-im80-real.tsv 4-10000-refadd-im80-real.txt 4-10000-refadd-real.tsv 4-10000-refadd-real.txt 4-10000-refine-clever-nr-im80-real.tsv 4-10000-refine-clever-nr-im80-real.txt 4-10000-refine-clever-nr-real.tsv 4-10000-refine-clever-nr-real.txt 4-10000-refine-im80-real.tsv 4-10000-refine-im80-real.txt 4-10000-refine-real.tsv 4-10000-refine-real.txt 4-10000-replace-clever-nr-im80-real.tsv 4-10000-replace-clever-nr-im80-real.txt 4-10000-replace-clever-nr-real.tsv 4-10000-replace-clever-nr-real.txt 4-10000-replace-im80-real.tsv 4-10000-replace-im80-real.txt 4-10000-replace-real.tsv 4-10000-replace-real.txt

Result exploration

Systematic comparison of realistic (plain) with plain (20180308-NOOR), clever-nr (20180529-NOOR), im80 (20180311-NOOR) and clever-nr-im80 (20180530-NOOR) (all dashed):

testsuccess
rate
network
size
incoherence
degree
semantic
precision
semantic
F-measure
semantic
recall
maximum
convergence
4-10000-delete-real0.99250.060.910.330.201030
4-10000-replace-real0.99250.060.910.330.201030
4-10000-refine-real0.99340.060.920.460.311030
4-10000-add-real0.99490.190.780.590.481066
4-10000-addjoin-real0.99490.190.780.590.48941
4-10000-refadd-real0.99630.190.780.700.64941
4-10000-delete-clever-nr-real0.98460.130.820.560.422039
4-10000-replace-clever-nr-real0.98440.130.840.540.391468
4-10000-refine-clever-nr-real0.98530.140.820.640.522681
4-10000-add-clever-nr-real0.98780.250.710.730.751852
4-10000-addjoin-clever-nr-real0.99780.240.720.740.751517
4-10000-refadd-clever-nr-real0.99870.260.700.760.841066
4-10000-delete-im80-real0.99230.001.000.260.151659
4-10000-replace-im80-real0.99230.001.000.260.151648
4-10000-refine-im80-real0.99320.001.000.410.261659
4-10000-add-im80-real0.99400.001.000.460.305642
4-10000-addjoin-im80-real0.99400.001.000.460.303635
4-10000-refadd-im80-real0.99590.001.000.690.523256
4-10000-delete-clever-nr-im80-real0.98360.001.000.410.261787
4-10000-replace-clever-nr-im80-real0.98360.001.000.420.262078
4-10000-refine-clever-nr-im80-real0.98460.001.000.560.392887
4-10000-add-clever-nr-im80-real0.97660.001.000.680.528959
4-10000-addjoin-clever-nr-im80-real0.98650.001.000.680.518338
4-10000-refadd-clever-nr-im80-real0.98810.001.000.830.717733

Analysis

Analyst: Jérôme Euzenat (INRIA) (2018-08-16)

Key points:

All this is comparable to the results obtained in 20180307-NOOR.

Further experiments:


This file can be retrieved from URL https://sake.re/20180531-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20180531-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr.

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20180531-NOOR.zip