20180601-NOOR

Experiment design

Designer: Jérôme Euzenat (INRIA) (2017-05-30)

Revision of networks of ontologies with expansion, relaxation and generation starting with random network (4 agents; 10 runs; 10000 games; delete/replace/refine/add/addjoin/refadd)

Hypotheses: Generating correspondences when needed should increase results quality (recall) with respect to expansion

Experimental setting: Same as 20170530-NOOR replaying the same runs as 20180308-NOOR and after fixing addjoin and expansion.

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2018-06-01)

Date: 2018-06-01

Lazy lavender hash: d50e70f87bca76951ec2f149dc8ae1d42b9a1a28

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20170530-NOOR

Parameters: params.sh

Command line (script.sh):

#!/bin/bash

JPATH=lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OPT="-DnbAgents=4 -DnbIterations=10000 -DnbRuns=10 -DreportPrecRec"

LOADOPT="-DloadDir=input/expeRun -DloadEnv -DloadAgents -DreplayGames"

for op in delete replace refine add addjoin refadd
do
  bash scripts/runexp.sh -d 4-10000-${op}-clever-nr-gen java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -Dgenerative
  bash scripts/runexp.sh -d 4-10000-${op}-clever-nr-im80-gen java -Dlog.level=INFO -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:. fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: Debian Linux virtual machine configured with four processors and 20GB of RAM running under a Dell PowerEdge T610 with 4*Intel Xeon Quad Core 1.9GHz E5-2420 processors, under Linux ProxMox 2 (Debian). - Java 1.8.0 HotSpot

Input: Input required for reproducibility can be retrieved from: https://files.inria.fr/sakere/input/expeRun.zip Then unzip expeRun.zip -d input

Raw results

4-10000-add-clever-nr-gen.tsv 4-10000-add-clever-nr-gen.txt 4-10000-add-clever-nr-im80-gen.tsv 4-10000-add-clever-nr-im80-gen.txt 4-10000-addjoin-clever-nr-gen.tsv 4-10000-addjoin-clever-nr-gen.txt 4-10000-addjoin-clever-nr-im80-gen.tsv 4-10000-addjoin-clever-nr-im80-gen.txt 4-10000-delete-clever-nr-gen.tsv 4-10000-delete-clever-nr-gen.txt 4-10000-delete-clever-nr-im80-gen.tsv 4-10000-delete-clever-nr-im80-gen.txt 4-10000-refadd-clever-nr-gen.tsv 4-10000-refadd-clever-nr-gen.txt 4-10000-refadd-clever-nr-im80-gen.tsv 4-10000-refadd-clever-nr-im80-gen.txt 4-10000-refine-clever-nr-gen.tsv 4-10000-refine-clever-nr-gen.txt 4-10000-refine-clever-nr-im80-gen.tsv 4-10000-refine-clever-nr-im80-gen.txt 4-10000-replace-clever-nr-gen.tsv 4-10000-replace-clever-nr-gen.txt 4-10000-replace-clever-nr-im80-gen.tsv 4-10000-replace-clever-nr-im80-gen.txt

Result exploration

Systematic comparison of clever-nr (dashed, 20180529-NOOR) to clever-nr-gen (plain) and clever-nr-im80 (dashed, 20180530-NOOR) to clever-nr-im80-gen (plain):

testsuccess
rate
network
size
incoherence
degree
semantic
precision
semantic
F-measure
semantic
recall
maximum
convergence
4-10000-delete-clever-nr-gen0.94580.120.850.670.558076
4-10000-replace-clever-nr-gen0.94610.130.840.710.627952
4-10000-refine-clever-nr-gen0.94700.160.800.760.717727
4-10000-add-clever-nr-gen0.91740.230.730.770.838825
4-10000-addjoin-clever-nr-gen0.95760.230.720.760.816238
4-10000-refadd-clever-nr-gen0.94780.230.700.760.857721
4-10000-delete-clever-nr-im80-gen0.93510.001.000.600.439646
4-10000-replace-clever-nr-im80-gen0.94550.001.000.640.478048
4-10000-refine-clever-nr-im80-gen0.93640.001.000.760.626460
4-10000-add-clever-nr-im80-gen0.91670.001.000.790.659351
4-10000-addjoin-clever-nr-im80-gen0.95690.001.000.800.676982
4-10000-refadd-clever-nr-im80-gen0.94750.001.000.860.767774

Analysis

Analyst: Jérôme Euzenat (INRIA) (2018-08-16)

Key points:

Further experiments: none


This file can be retrieved from URL https://sake.re/20180601-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20180601-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20180601-NOOR.zip