Sake.re: metadata

Besides the content of the script.sh most of the experiments are described through metadata. These metadata, in turn, are used for various purposes:

They will facilitate all tasks. Hence, it is worth devoting some time to fill them.

These metadata are entered in the params.sh file which only contains shell variable assignments. This may look as a strange way to express metadata. There exist many other way to do this, but none that really do a better job than the others, and every now and then a new one pops up. So let stick to this one for the moment. Beware that these variables cannot be assigned any values like in a shell, because they may be included in the Docker file, which only evaluates variables at build time.

These metadata can be classified into three kinds:

Environment metadata (E):
describes the environment required by the experiment (operating system and software versions, etc.);
Experiment metadata (X):
describes the parameters of the experiment (type of experiment, number of agents, type of collected output, etc.);
Documentation metadata (D):
provides information used for documenting the experiment (dates, performers, hypothesis tested, summary, etc.).
The two former categories are necessary for performing the experiment.

Here is a short and non exhaustive list of metadata (C=compulsory, O=optional):

The params.sh file may contain more variables which can used to document better the experiments.

Here is the file provided with the experiment template, which is fully functional:

# ENVIRONMENT

OSVERS=busternb

LOGBACK_VERSION=1.2.3

JPATH=lazylav/lib/lazylav/ll.jar:lib/slf4j/logback-classic-${LOGBACK_VERSION}.jar:lib/slf4j/logback-core-${LOGBACK_VERSION}.jar:.

LLHASH=b2093e6ce8e1d2aeddc5e08d79eb6a08d1932894

# EXPERIMENT

LABEL=YYYYMMDD-TYPE
NAME=${LABEL}

NBAGENTS=4
NBPOPS=4
SYNCRATE=250
NBITERATIONS=100
NBRUNS=3
OPS="delete replace refine add addjoin refadd"
OPT="-DnbAgents=${NBAGENTS} -DnbIterations=${NBITERATIONS} -DnbRuns=${NBRUNS} -DreportPrecRec" 

LOADOPT=
DIRPREF=${NBAGENTS}-${NBITERATIONS}

TOCOPY="params.sh script.sh notebook.ipynb logback.xml"

OUTPUT=results/

# METADATA

VARIATIONOF=20191218-NOOR
EXPE="Put a short description of the experiment"

HYPOTHESIS="A precise and short description of tested hypotheses"

DESIGNDATE=2020-12-23
DATE=2020-12-23

SETTING="Variation of ${VARIATIONOF} with a different setting (describe)"

CLASSES="Population, PopulatedARExperiment, PopulationAlignmentAdjustingAgent, NOOEnvironment, ActionLogger, AverageLogger, Monitor"

PERFORMER="Your name here"
DESIGNER=${PERFORMER}
EXPERIMENTER=${PERFORMER}
ANALYST=${PERFORMER}