# 20140207-NOOR

<span style="background-color: red;">Invalid experiment: incorrect reference alignment (see [20140208-NOOR](../20140208-NOOR))</span>

## Experiment design
[<img src='../imgs/icon8-docker-50.png' align='right' width='50px' />](../dockerfiles/wheezy.dkr) [<img src='../imgs/icon8-blue-docker-50.png' align='right' width='50px' />](20140207-NOOR.dkr)

**Hypothesis**: The network can achieve null incoherence degree. 

The network can revise better (in terms of F-measure) than LogMap and Alcomo. 

**Experimental setting**: Same as [20140204-NOOR](../20140204-NOOR) but it is possible to compare with LogMap and Alcomo. 

## Experiment
**Experimenter**: Jérôme Euzenat (INRIA)

**Date**: 2014-02-07 

**Lazy lavender hash**: 279920d1b24c5c799c7ba550402d1a6aabf2a11d 


**Parameters**: [params.sh](params.sh)

**Command line** ([script.sh](script.sh)):
~~~~
java -Xms500M -Xmx1G -Dlog.level=DEBUG -cp lib/lazylav/ll.jar:lib/slf4j/logback-classic-1.0.9.jar:lib/slf4j/logback-core-1.0.9.jar:. fr.inria.exmo.lazylavender.engine.Monitor -DreportIncoherence=1 
~~~~

**Class used**: NOOEnvironment, AlignmentAdjustingAgent, AlignmentRevisionExperiment, ActionLogger, AverageLogger, Monitor. 

**Execution environment**:
MacBook Pro 15, C2D 2.66GHz, 8GB - MacOS X 6.8 - Java 1.6.0 HotSpot

### Raw results

[20140207-NOOR.tsv](results/20140207-NOOR.tsv) 
[20140207-NOOR.txt](results/20140207-NOOR.txt) 

## Result exploration

<img src='imgs/20140207-NOOR.png' align='center'/>
 

- In blue the average success rate (over 5 runs); 
- In dotted, the minimum and maximum success rate (over the same 5 runs); 
- In red, the average syntactic F-measure (Alignment API). 
- In green, the average incoherence degree (Alcomo). 
- In black, the logMap and Alcomo baseline F-measure.

Average degrees: 

<center><table><tr>
</tr><tr>
<th>measure </th>
<th>F-measure </th>
<th>Incoherence degree</th>
</tr><tr>
<th>Initial </th>
<td>0.12 </td>
<td>0.33</td>
</tr><tr>
<th>LogMap </th>
<td>0.13 </td>
<td>0.18</td>
</tr><tr>
<th>Alcomo </th>
<td>0.13 </td>
<td>0.00</td>
</tr><tr>
<th>Final </th>
<td>0.15 </td>
<td>0.01</td>
</tr></table></center>

## Analysis
After, 100+ steps the network has reduced incoherence to nearly null. 

**Key points**:

F-measure passes above that of LogMap and Alcomo (simply because agents know exactly what to revise). 

F-measure remains low because: 

- All actions only remove correspondences, but above all 
- F-measure is purely syntactic 
- Alignments are not symmetric, then revision can only apply on half of them

Incoherence degree is non null because: 

- Alignments are not symmetric

**Further experiments**: 

- Make the matrix symmetric or implements "alignment" at each stage 
- Use semantic precision and recall.


----

This file can be retrieved from URL [https://sake.re/20140207-NOOR](https://sake.re/20140207-NOOR)
<!-- A zip of the whole experiment data can be found at [DOI]() -->
It is possible to check out the repository by cloning <span style="color: blue;">https://felapton.inrialpes.fr/cakes/20140207-NOOR.git</span>
This experiment has been transferred from its initial location at https://gforge.inria.fr (not available any more)