Experiment 20180826-NOOR¶

Experiment design¶

Designer: Jérôme Euzenat (INRIA)

Date: 20180826

Hypotheses: Addjoin achieves the same as add but faster

Variation of: 20180305-NOOR

4 agents; 10 runs; 2000 games

Adaptation operators: add addjoin

Experimental setting: Same as [[20180305-NOOR]] after correction of expansion...

Variables¶

controled variables: ['revisionModality', 'syntactic']

dependent variables: ['srate', 'size', 'inc', 'prec', 'fmeas', 'rec', 'conv']

Experiment¶

Date: 20180826

Performer: Jérôme Euzenat (INRIA)

Lazy lavender hash: 759ff097b96520c12aa84f3749927f9a22022e62

Classpath: lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.

OS: stretch

Parameter file: params.sh

Executed command (script.sh):

Class used: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

Execution environment: This is the first time that an experiment is rerun using the Docker environment instead as being replayed directly. The Docker file generated for 20180305-NOOR is found in the zip file. It has been tweaked, in particular to run the experiments on 2000 runs and with non realistic size

Note: This experiment has been set up to first generate the runs and then play them with different configuration. To strictly repeat it, it is necessary to reuse the same input and not generate a new one. Hence, the input is saved in the repository, the generating line above must be commented (including in Docker).

Raw results¶

Analysis¶

Results without -Drealistic, i.e., commensurate with most experiments so far¶

In dashed semantic measures (sometimes, they are not relevant).

We observe that:

• addjoin converges faster;
• add and addjoin converge to the same size semantic and syntactic precision and recall.

Then results with -Drealistic as in 20180305-NOOR, i.e. starting with larger alignment size¶

The same observation has above can be made.

Discussion¶

Analyst: Jérôme Euzenat (INRIA) (2018-08-26)

Key points¶

This experiment fully confirms 20180305-NOOR, i.e., addjoin is now working properly, with the observation that:

• add and addjoin eventually achieve the same results (as shown by Size and Syntactic/Semantic Precision/Recall)
• addjoin converges faster

In addition, it shows that this is true with realistic and non realistic starting size and that the true convergence was indeed after 1000.

Further experiments¶

None