20180913-NOOR

Experiment design

Designer: Jérôme Euzenat (INRIA) (2018-09-13)

Revision of networks of ontologies with strenghening on longer span (4 agents; 10 runs; 20000 games; delete/replace/refine/add/addjoin/refadd)

Hypotheses: Strenghening with refadd, relaxation, expansion, generation reaches 100% recall

Experimental setting: Same as 20180828-NOOR on 20000 games (and only on those run involving expansion, generation and relaxation).

Experiment

Experimenter: Jérôme Euzenat (INRIA) (2018-09-13)

Date: 2018-09-13

Lazy lavender hash: 759ff097b96520c12aa84f3749927f9a22022e62

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Variation of: 20180828-NOOR

Parameters: params.sh

Command line (script.sh):

. params.sh

OUTPUT=${OUTPUT}${LABEL}

# bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-nothing java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=nothing -DsaveDir=input/expeRun2 -DsaveGames -DloadDir=input/expeRun2 -DloadEnv -DloadAgents
  
for op in ${OPS}
do
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-strgen java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -Dstrengthen=mostgeneral
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-strspc java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -Dstrengthen=mostspecific
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty-strgen java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -DstartEmpty -Dstrengthen=mostgeneral
  bash scripts/runexp.sh -p ${OUTPUT} -d ${DIRPREF}-${op}-clever-nr-im80-gen-empty-strspc java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} ${LOADOPT} -DrevisionModality=${op} -DexpandAlignments=clever -DnonRedundancy -DimmediateRatio=80 -Dgenerative -DstartEmpty -Dstrengthen=mostspecific
done

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: 24 * Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz with 20GB RAM / Linux ProxMox 2 / Linux 4.15.17-1-pve / Java Java(TM) SE Runtime Environment 1.8.0_151 with 4.33G max heap size

Note: This experiment has been set up to first generate the runs and then play them with different configuration (this experiment generated expeRun2). To strictly repeat it, it is necessary to reuse the same input and not generate a new one. Hence, the input is saved in the repository, the generating line above must be commented (including in Docker).

Input: Input required for reproducibility can be retrieved from: https://files.inria.fr/sakere/input/expeRun2.zip Then unzip expeRun2.zip -d input

Raw results

4-20000-add-clever-nr-im80-gen-empty-strgen.tsv 4-20000-add-clever-nr-im80-gen-empty-strgen.txt 4-20000-add-clever-nr-im80-gen-empty-strspc.tsv 4-20000-add-clever-nr-im80-gen-empty-strspc.txt 4-20000-add-clever-nr-im80-gen-strgen.tsv 4-20000-add-clever-nr-im80-gen-strgen.txt 4-20000-add-clever-nr-im80-gen-strspc.tsv 4-20000-add-clever-nr-im80-gen-strspc.txt 4-20000-addjoin-clever-nr-im80-gen-empty-strgen.tsv 4-20000-addjoin-clever-nr-im80-gen-empty-strgen.txt 4-20000-addjoin-clever-nr-im80-gen-empty-strspc.tsv 4-20000-addjoin-clever-nr-im80-gen-empty-strspc.txt 4-20000-addjoin-clever-nr-im80-gen-strgen.tsv 4-20000-addjoin-clever-nr-im80-gen-strgen.txt 4-20000-addjoin-clever-nr-im80-gen-strspc.tsv 4-20000-addjoin-clever-nr-im80-gen-strspc.txt 4-20000-delete-clever-nr-im80-gen-empty-strgen.tsv 4-20000-delete-clever-nr-im80-gen-empty-strgen.txt 4-20000-delete-clever-nr-im80-gen-empty-strspc.tsv 4-20000-delete-clever-nr-im80-gen-empty-strspc.txt 4-20000-delete-clever-nr-im80-gen-strgen.tsv 4-20000-delete-clever-nr-im80-gen-strgen.txt 4-20000-delete-clever-nr-im80-gen-strspc.tsv 4-20000-delete-clever-nr-im80-gen-strspc.txt 4-20000-nothing.tsv 4-20000-nothing.txt 4-20000-refadd-clever-nr-im80-gen-empty-strgen.tsv 4-20000-refadd-clever-nr-im80-gen-empty-strgen.txt 4-20000-refadd-clever-nr-im80-gen-empty-strspc.tsv 4-20000-refadd-clever-nr-im80-gen-empty-strspc.txt 4-20000-refadd-clever-nr-im80-gen-strgen.tsv 4-20000-refadd-clever-nr-im80-gen-strgen.txt 4-20000-refadd-clever-nr-im80-gen-strspc.tsv 4-20000-refadd-clever-nr-im80-gen-strspc.txt 4-20000-refine-clever-nr-im80-gen-empty-strgen.tsv 4-20000-refine-clever-nr-im80-gen-empty-strgen.txt 4-20000-refine-clever-nr-im80-gen-empty-strspc.tsv 4-20000-refine-clever-nr-im80-gen-empty-strspc.txt 4-20000-refine-clever-nr-im80-gen-strgen.tsv 4-20000-refine-clever-nr-im80-gen-strgen.txt 4-20000-refine-clever-nr-im80-gen-strspc.tsv 4-20000-refine-clever-nr-im80-gen-strspc.txt 4-20000-replace-clever-nr-im80-gen-empty-strgen.tsv 4-20000-replace-clever-nr-im80-gen-empty-strgen.txt 4-20000-replace-clever-nr-im80-gen-empty-strspc.tsv 4-20000-replace-clever-nr-im80-gen-empty-strspc.txt 4-20000-replace-clever-nr-im80-gen-strgen.tsv 4-20000-replace-clever-nr-im80-gen-strgen.txt 4-20000-replace-clever-nr-im80-gen-strspc.tsv 4-20000-replace-clever-nr-im80-gen-strspc.txt

Result exploration

test success
rate
network
size
incoherence
degree
semantic
precision
semantic
F-measure
semantic
recall
maximum
convergence
4-20000-nothing 0.26 54 0.34 0.11 0.20 0.89 1
strengthen=general (compare to 20180829-NOOR)
4-20000-delete-clever-nr-im80-gen-strgen 0.95 56 0.00 1.00 0.57 0.40 10236
4-20000-replace-clever-nr-im80-gen-strgen 0.95 55 0.00 1.00 0.56 0.39 9529
4-20000-refine-clever-nr-im80-gen-strgen 0.94 56 0.00 1.00 0.61 0.43 10572
4-20000-add-clever-nr-im80-gen-strgen 0.94 64 0.00 1.00 0.85 0.75 15173
4-20000-addjoin-clever-nr-im80-gen-strgen 0.97 67 0.00 1.00 0.88 0.78 7815
4-20000-refadd-clever-nr-im80-gen-strgen 0.96 84 0.00 1.00 0.99 0.98 6613
strengthen=general + startEmpty (compare to 20180829-NOOR)
4-20000-delete-clever-nr-im80-gen-empty-strgen 0.95 65 0.00 1.00 0.60 0.43 7608
4-20000-replace-clever-nr-im80-gen-empty-strgen 0.95 63 0.00 1.00 0.57 0.40 7509
4-20000-refine-clever-nr-im80-gen-empty-strgen 0.94 65 0.00 1.00 0.61 0.44 12513
4-20000-add-clever-nr-im80-gen-empty-strgen 0.95 74 0.00 1.00 0.88 0.79 11268
4-20000-addjoin-clever-nr-im80-gen-empty-strgen 0.97 75 0.00 1.00 0.89 0.81 7183
4-20000-refadd-clever-nr-im80-gen-empty-strgen 0.97 87 0.00 1.00 0.97 0.95 6022
strengthen=specific (compare to 20180828-NOOR)
4-20000-delete-clever-nr-im80-gen-empty-strspc 0.95 62 0.00 1.00 0.71 0.55 11197
4-20000-replace-clever-nr-im80-gen-strspc 0.95 56 0.00 1.00 0.70 0.54 10480
4-20000-refine-clever-nr-im80-gen-strspc 0.93 57 0.00 1.00 0.71 0.56 9199
4-20000-add-clever-nr-im80-gen-empty-strspc 0.94 73 0.00 1.00 0.88 0.78 13073
4-20000-addjoin-clever-nr-im80-gen-strspc 0.96 67 0.00 1.00 0.88 0.79 11567
4-20000-refadd-clever-nr-im80-gen-strspc 0.96 84 0.00 1.00 0.99 0.98 8851
strengthen=specific + startEmpty (compare to 20180828-NOOR)
4-20000-delete-clever-nr-im80-gen-strspc 0.95 56 0.00 1.00 0.70 0.54 10415
4-20000-replace-clever-nr-im80-gen-empty-strspc 0.95 64 0.00 1.00 0.73 0.57 7247
4-20000-refine-clever-nr-im80-gen-empty-strspc 0.93 65 0.00 1.00 0.74 0.58 12147
4-20000-add-clever-nr-im80-gen-strspc 0.93 65 0.00 1.00 0.87 0.77 13026
4-20000-addjoin-clever-nr-im80-gen-empty-strspc 0.96 76 0.00 1.00 0.90 0.82 7598
4-20000-refadd-clever-nr-im80-gen-empty-strspc 0.96 87 0.00 1.00 0.97 0.95 10562

We simply display the size and recall for the four kinds of experiments on 20000 games compared to those on 10000 games (see data sets to compare above). This experiment is in plain, the previous one (on 10000) in dashed.

Analysis

Analyst: Jérôme Euzenat (INRIA) (2018-09-18)

Key points:

Further experiments: none


This file can be retrieved from URL https://sake.re/20180913-NOOR

It is possible to check out the repository by cloning https://felapton.inrialpes.fr/cakes/20180913-NOOR.git

This experiment has been transferred from its initial location at https://gforge.inria.fr.

The original, unaltered associated zip file can be obtained from https://files.inria.fr/sakere/gforge/20180913-NOOR.zip