Experiment 20140204-NOOR¶

Invalid experiment: incorrect reference alignment (see 20140208-NOOR)

Experiment design¶

Date: 2014-02-04

Variation: 20140131-NOOR

Hypotheses: Experimenting 3 agents performing revision of a network of ontology.

Experimental setting: This the same experiment as the [20140131-NOOR]: 3 agents performing revision of a network of ontology. The NOOEnvironment and AlignmentAdjustingAgent have been rewritten for using the OWL API and HermiT (instead of Jena); computing the reference alignment; computing Syntactic F-measures (PRecEvaluation). AverageLogger has been provided to aggregate several runs together and compute the average.

Variables¶

controled variables: []

dependent variables: ['srate', 'inc', 'fmeas', 'conv']

Experiment¶

Date: 2014-02-04

Performer: Jérôme Euzenat (INRIA)

Lazy lavender hash: 769936317dba7a3a3d7294155ef333e12f5d905a

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor.

Execution environment: MacBook Pro 15, C2D 2.66GHz, 8GB - MacOS X 6.8 - Java 1.6.0 HotSpot

Parameter file: params.sh This parameter file has been generated post hoc. However, it is used for generating the skeleton of this notebook.

Executed command: There is no script.sh available for this test This was likely java -cp lib/lazylav/ll.jar fr.inria.exmo.lazylavender.engine.Monitor

Raw results¶

Raw results are not available for this experiment.

Result exploration¶

The following plot provides results using such features:

• In blue the average success rate (over 5 runs);
• In dotted, the minimum and maximum success rate (over the same 5 runs);
• In red, the average syntactic F-measure.

Analysis¶

Initial F-measure is .12 (likely corresponding to the 6 correct top-top correspondences), final is .15 (likely corresponding to the increase in precision once discarded all bad correspondences).

Further experiments:¶

With respect to F-measure, there are several things to be changed:

• Using SemPRecEvaluator (this requires to create the alignments through Ontowrap);
• Using a revision modality different from simply deleting the incorrect correspondences (this is already implemented).