Sake.re: Producing (new) experiments

See the prerequisite and be sure to be familiar with Sake.re.

Designing experiments

This starts by first cloning the experiment template:

$ git clone --recurse-submodules cakes@felapton.inrialpes.fr:YYYYMMDD-TYPE.git ${LABEL}
$ cd ${LABEL}
$ git remote remove origin

$ git submodule foreach 'git checkout master'
$ git commit -a -m "Upgraded submodule versions"

Then, edit the experiment descriptions params.sh, script.sh, notebook.ipynb and eventually the input directory.

Here you are on your own, this is where you can carefully design your own experiments. This is of course the main part of the work and it is fully dependent on your own needs.

• params.sh describes the experiment through a set of variables. See the metadata documentation.
• script.sh is a bash file calling Lazy lavender to process the experiment. See its thin documentation.
• notebook.ipynb is a python jupyter notebook description used to perform the analysis of the experiment. You must resort to Jupyter and Python documentation. You are encouraged to take advantage of the pySake analysis library that we maintain.
• input/ may contain any input you need. In particular, when running experiment on an existing sequence of events, it will go there, but it may also be sufficient to fetch it from a repository where it is shared.

Once this done, tag the repository with the DESIGNED label.

$ git commit -a -m "Explain the design"
$ git tag DESIGNED -m "Experiment design completed"

Running experiments

In case one uses lazy lavender, then compile it (beware that it is checkout at the desired hash):

$ ant -f lazylav/build.xml compileall

Then, process the experiment:

$ bash script.sh

It may be useful for later to generate the docker file (this actually can be done already at design time):

$ lazylav/scripts/gendkr.sh params.sh script.sh
$ git add ${LABEL}.dkr
$ git commit -a -m "Dockerfile generated"

If not too large, i.e. less than 1GB, results may be committed in the repository:

$ git add results      # please be sure that all files in result must be committed
$ git commit -a -m "Added experiment raw results"

Then tag the repository with the PERFORMED label.

$ git tag PERFORMED -m "Experiment performed"

Analysing experiments

You must have python and jupyter notebook installed. If not already installed, this can be achieved with:

$ python3 -m pip install notebook pipreqsnb nb-clean
$ python3 -m pip install jupyter_contrib_nbextensions
$ jupyter nbextension enable python-markdown/main
$ jupyter nbextension enable hide_input_all/main
$ jupyter nbextension enable collapsible_headings/main
$ jupyter nbextension enable livemdpreview/livemdpreview

$ python3 -m pip install configobj matplotlib numpy pandas scipy statsmodels

You can refine the notebook.ipynb notebook through jupyter, by running:

$ jupyter trust notebook.ipynb
$ jupyter notebook

Generating HTML file

Once the analysis completed, it is very useful to generate a HTML file from it, that can be published without jupyter. This is obtained by running:

$ bash lazylav/scripts/genHTML.sh ${LABEL}

Committing analysis

Before committing the notebook file, save it and strip it from its output so that git only tracks the input cells:

$ nb-clean clean -e notebook.ipynb

It may also be useful, and is advised, to record the set of python packages necessary to the experiment in a requirements.txt file. This is achieved by running:

$ pipreqs --scan-notebooks

Finally commit and tag:

$ git add notebook.ipynb requirements.txt index.html
$ git commit -a -m "analyzed"
$ git tag ANALYSED -m "Analysis completed"

If some byproducts of the analysis have to be committed, then this is preferably done into a analysis/ directory.

You are encouraged to publish your experiment results. If you have followed the instructuon above, you should be well prepared for that.