# 20180826-NOOR

## Experiment design
**Designer**: Jérôme Euzenat (INRIA) (2018-05-03)

[<img src='../imgs/icon8-docker-50.png' align='right' width='50px' />](../dockerfiles/stretch.dkr) [<img src='../imgs/icon8-blue-docker-50.png' align='right' width='50px' />](20180826-NOOR.dkr)
Revision of networks of ontologies with most specific strenghening (4 agents; 10 runs; 2000 games; add/addjoin)

**Hypotheses**:
Addjoin achieves the same as add but faster

**Experimental setting**: 
Same as [20180305-NOOR](../20180305-NOOR) after correction of expansion... (and likely realistic size estimation)

## Experiment
**Experimenter**: Jérôme Euzenat (INRIA) (2018-08-26)

**Date**: 20180826

**Lazy lavender hash**: 759ff097b96520c12aa84f3749927f9a22022e62

**Classpath**: `lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.2.3.jar:lib/slf4j/logback-core-1.2.3.jar:.`

**OS**: stretch

**Variation of**: [20180305-NOOR](../20180305-NOOR)

**Parameters**: [params.sh](params.sh)

**Command line** ([script.sh](script.sh)):
~~~~
. params.sh

# bash scripts/runexp.sh -d ${DIRPREF}-nothing java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=nothing -DsaveDir=expeRun -DsaveInit -DsaveParams -DsaveGames
bash scripts/runexp.sh -d ${DIRPREF}-add java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=add ${LOADOPT}
bash scripts/runexp.sh -d ${DIRPREF}-addjoin java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=addjoin ${LOADOPT}
bash scripts/runexp.sh -d ${DIRPREF}-add-syntactic java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=add -Dsyntactic=true ${LOADOPT} -DsaveDir=addRes -DsaveFinal
bash scripts/runexp.sh -d ${DIRPREF}-addjoin-syntactic java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -DrevisionModality=addjoin -Dsyntactic=true ${LOADOPT} -DsaveDir=addjoinRes -DsaveFinal

# bash scripts/runexp.sh -d ${DIRPREF}-real-nothing java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=nothing -DsaveDir=expeRun -DsaveInit -DsaveParams -DsaveGames
bash scripts/runexp.sh -d ${DIRPREF}-real-add java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=add ${LOADOPT}
bash scripts/runexp.sh -d ${DIRPREF}-real-addjoin java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=addjoin ${LOADOPT}
bash scripts/runexp.sh -d ${DIRPREF}-real-add-syntactic java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=add -Dsyntactic=true ${LOADOPT} -DsaveDir=addRes -DsaveFinal
bash scripts/runexp.sh -d ${DIRPREF}-real-addjoin-syntactic java -Dlog.level=INFO -cp ${JPATH} fr.inria.exmo.lazylavender.engine.Monitor ${OPT} -Drealistic -DrevisionModality=addjoin -Dsyntactic=true ${LOADOPT} -DsaveDir=addjoinRes -DsaveFinal
~~~~

**Class used**: 
NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor

**Execution environment**: 
This is the first time that an experiment is rerun using the Docker environment instead as being replayed directly.
The Docker file generated for [20180305-NOOR](../20180305-NOOR) is found in the zip file.
It has been tweaked, in particular to run the experiments on 2000 runs and with non realistic size

**Note**: This experiment has been set up to first generate the runs and then play them with different configuration. To strictly repeat it, it is necessary to reuse the same input and not generate a new one. Hence, the input is saved in the repository, the generating line above must be commented (including in Docker).

### Raw results

[4-2000-add-syntactic.tsv](results/4-2000-add-syntactic.tsv) 
[4-2000-add-syntactic.txt](results/4-2000-add-syntactic.txt) 
[4-2000-add.tsv](results/4-2000-add.tsv) 
[4-2000-add.txt](results/4-2000-add.txt) 
[4-2000-addjoin-syntactic.tsv](results/4-2000-addjoin-syntactic.tsv) 
[4-2000-addjoin-syntactic.txt](results/4-2000-addjoin-syntactic.txt) 
[4-2000-addjoin.tsv](results/4-2000-addjoin.tsv) 
[4-2000-addjoin.txt](results/4-2000-addjoin.txt) 
[4-2000-nothing.tsv](results/4-2000-nothing.tsv) 
[4-2000-nothing.txt](results/4-2000-nothing.txt) 
[4-2000-real-add-syntactic.tsv](results/4-2000-real-add-syntactic.tsv) 
[4-2000-real-add-syntactic.txt](results/4-2000-real-add-syntactic.txt) 
[4-2000-real-add.tsv](results/4-2000-real-add.tsv) 
[4-2000-real-add.txt](results/4-2000-real-add.txt) 
[4-2000-real-addjoin-syntactic.tsv](results/4-2000-real-addjoin-syntactic.tsv) 
[4-2000-real-addjoin-syntactic.txt](results/4-2000-real-addjoin-syntactic.txt) 
[4-2000-real-addjoin.tsv](results/4-2000-real-addjoin.tsv) 
[4-2000-real-addjoin.txt](results/4-2000-real-addjoin.txt) 
[4-2000-real-nothing.tsv](results/4-2000-real-nothing.tsv) 
[4-2000-real-nothing.txt](results/4-2000-real-nothing.txt) 

## Result exploration

Results without -Drealistic, i.e., commensurate with most experiments so far:

<img src='imgs/20180826-NOOR-4.png' align='center'/>


<img src='imgs/20180826-NOOR-5.png' align='center'/>


<img src='imgs/20180826-NOOR-6.png' align='center'/>


<img src='imgs/20180826-NOOR-7.png' align='center'/>


First results with -Drealistic as in [20180305-NOOR](../20180305-NOOR):

<img src='imgs/20180826-NOOR-0.png' align='center'/>


<img src='imgs/20180826-NOOR-1.png' align='center'/>


<img src='imgs/20180826-NOOR-2.png' align='center'/>


<img src='imgs/20180826-NOOR-3.png' align='center'/>


<center><table><tr>
<th>test</th>
<th>success<br />rate</th>
<th>network<br />size</th>
<th>incoherence<br />degree</th>
<th>semantic<br />precision</th>
<th>semantic<br />F-measure</th>
<th>semantic<br />recall</th>
<th>syntactic<br />precision</th>
<th>syntactic<br />F-measure</th>
<th>syntactic<br />recall</th>
<th>maximum<br />convergence</th>
</tr><tr>
<td style="border: 1px solid #CCC; font-weight: bold;">4-2000-nothing</td>
<td style="border: 1px solid #CCC;">0.26</td>
<td style="border: 1px solid #CCC;">54</td>
<td style="border: 1px solid #CCC;">0.36</td>
<td style="border: 1px solid #CCC;">0.11</td>
<td style="border: 1px solid #CCC;">0.20</td>
<td style="border: 1px solid #CCC;">0.89</td>
<td style="border: 1px solid #CCC;"></td>
<td style="border: 1px solid #CCC;"></td>
<td style="border: 1px solid #CCC;"></td>
<td style="border: 1px solid #CCC;">1</td>
</tr><tr>
<td style="border: 1px solid #CCC; font-weight: bold;">4-2000-add</td>
<td style="border: 1px solid #CCC;">0.91</td>
<td style="border: 1px solid #CCC;">32</td>
<td style="border: 1px solid #CCC;">0.14</td>
<td style="border: 1px solid #CCC;">0.80</td>
<td style="border: 1px solid #CCC;">0.50</td>
<td style="border: 1px solid #CCC;">0.36</td>
<td style="border: 1px solid #CCC;">0.65</td>
<td style="border: 1px solid #CCC;">0.35</td>
<td style="border: 1px solid #CCC;">0.24</td>
<td style="border: 1px solid #CCC;">1572</td>
</tr><tr>
<td style="border: 1px solid #CCC; font-weight: bold;">4-2000-addjoin</td>
<td style="border: 1px solid #CCC;">0.95</td>
<td style="border: 1px solid #CCC;">32</td>
<td style="border: 1px solid #CCC;">0.14</td>
<td style="border: 1px solid #CCC;">0.80</td>
<td style="border: 1px solid #CCC;">0.50</td>
<td style="border: 1px solid #CCC;">0.36</td>
<td style="border: 1px solid #CCC;">0.65</td>
<td style="border: 1px solid #CCC;">0.35</td>
<td style="border: 1px solid #CCC;">0.24</td>
<td style="border: 1px solid #CCC;">1361</td>
</tr><tr>
<td style="border: 1px solid #CCC; font-weight: bold;">4-2000-real-nothing</td>
<td style="border: 1px solid #CCC;">0.53</td>
<td style="border: 1px solid #CCC;">92</td>
<td style="border: 1px solid #CCC;">0.36</td>
<td style="border: 1px solid #CCC;">0.24</td>
<td style="border: 1px solid #CCC;">0.38</td>
<td style="border: 1px solid #CCC;">0.90</td>
<td style="border: 1px solid #CCC;"></td>
<td style="border: 1px solid #CCC;"></td>
<td style="border: 1px solid #CCC;"></td>
<td style="border: 1px solid #CCC;">1</td>
</tr><tr>
<td style="border: 1px solid #CCC; font-weight: bold;">4-2000-real-add</td>
<td style="border: 1px solid #CCC;">0.94</td>
<td style="border: 1px solid #CCC;">45</td>
<td style="border: 1px solid #CCC;">0.16</td>
<td style="border: 1px solid #CCC;">0.81</td>
<td style="border: 1px solid #CCC;">0.57</td>
<td style="border: 1px solid #CCC;">0.44</td>
<td style="border: 1px solid #CCC;">0.49</td>
<td style="border: 1px solid #CCC;">0.34</td>
<td style="border: 1px solid #CCC;">0.26</td>
<td style="border: 1px solid #CCC;">1261</td>
</tr><tr>
<td style="border: 1px solid #CCC; font-weight: bold;">4-2000-real-addjoin</td>
<td style="border: 1px solid #CCC;">0.97</td>
<td style="border: 1px solid #CCC;">45</td>
<td style="border: 1px solid #CCC;">0.16</td>
<td style="border: 1px solid #CCC;">0.81</td>
<td style="border: 1px solid #CCC;">0.57</td>
<td style="border: 1px solid #CCC;">0.44</td>
<td style="border: 1px solid #CCC;">0.49</td>
<td style="border: 1px solid #CCC;">0.34</td>
<td style="border: 1px solid #CCC;">0.26</td>
<td style="border: 1px solid #CCC;">1012</td>
</tr></table></center>

## Analysis

**Analyst**: Jérôme Euzenat (INRIA) (2018-08-26)


**Key points**:
This experiment fully confirms [20180305-NOOR](../20180305-NOOR), i.e., addjoin is now working properly, with the observation that:
* add and addjoin eventually achieve the same results (as shown by Size and Syntactic/Semantic Precision/Recall)
* addjoin converges faster
In addition, it shows that this is true with realistic and non realistic starting size and that the true convergence was indeed after 1000.

**Further experiments**:
None


----

This file can be retrieved from URL [https://sake.re/20180826-NOOR](https://sake.re/20180826-NOOR)
<!-- A zip of the whole experiment data can be found at [DOI]() -->
It is possible to check out the repository by cloning <span style="color: blue;">https://felapton.inrialpes.fr/cakes/20180826-NOOR.git</span>

This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)

The original, unaltered associated zip file can be obtained from [https://files.inria.fr/sakere/gforge/20180826-NOOR.zip](https://files.inria.fr/sakere/gforge/20180826-NOOR.zip)