Sake.re: Producing (new) experiments

See the prerequisite and be sure to be familiar with Sake.re.

Designing experiments

This starts by first cloning the experiment template:

$ git clone --recurse-submodules cakes@felapton.inrialpes.fr:YYYYMMDD-TYPE.git ${LABEL}
$ cd ${LABEL}
$ git remote remove origin
It is advised to upgrade supporting software (Lazy lavender and pySake) by:
$ git submodule foreach 'git checkout master'
$ git commit -a -m "Upgraded submodule versions"

Then, edit the experiment descriptions params.sh, script.sh, notebook.ipynb and eventually the input directory.

Here you are on your own, this is where you can carefully design your own experiments. This is of course the main part of the work and it is fully dependent on your own needs.

Once this done, tag the repository with the DESIGNED label.

$ git commit -a -m "Explain the design"
$ git tag DESIGNED -m "Experiment design completed"

Running experiments

Compile lazy lavender:

$ ant -f lazylav/build.xml compileall
Process the experiment:
$ bash script.sh

It may be useful for later to generate the docker file (this actually can be done already at design time):

$ lazylav/scripts/gendkr.sh params.sh script.sh
$ git add ${LABEL}.dkr
$ git commit -a -m "Dockerfile generated"
The variables that have to be set in params.sh before running this are: LABEL, OSVERS, OUTPUT=results/, LLHASH, and LOGBACK_VERSION.

If not too large, i.e. less than 1GB, results may be committed in the repository:

$ git add results      # please be sure that all files in result must be committed
$ git commit -a -m "Added experiment raw results"

Then tag the repository with the PERFORMED label.

$ git tag PERFORMED -m "Experiment performed"

Analysing experiments

You must have python and jupyter notebook installed. If not already installed, this can be achieved with:

$ python3 -m pip install notebook pipreqsnb
$ python3 -m pip install jupyter_contrib_nbextensions
$ jupyter nbextension enable python-markdown/main
$ jupyter nbextension enable hide_input_all/main
$ jupyter nbextension enable collapsible_headings/main
$ jupyter nbextension enable livemdpreview/livemdpreview
Our notebooks using pySake also require the installation of various classical python libraries:
$ python3 -m pip install configobj matplotlib numpy pandas scipy 

You can refine the notebook.ipynb notebook through jupyter, by running:

$ jupyter trust notebook.ipynb
$ jupyter notebook
And changing it through the browser usually at the http://localhost:8888 address.

Generating HTML file

Once the analysis completed, it is very useful to generate a HTML file from it, that can be published without jupyter. This is obtained by running:

$ bash lazylav/scripts/genHTML.sh ${LABEL}
This generates a minimal index.html file suited for committing and publishing.

Committing analysis

Before committing the notebook file, strip it from its output so that git only tracks the input cells:

$ python3 pysake/scripts/stripvar.py notebook.ipynb > notebook.tmp.ipynb
$ jupyter nbconvert --ClearOutputPreprocessor.enabled=True notebook.tmp.ipynb --to=notebook --output=notebook.ipynb

It may also be useful, and is advised, to record the set of python packages necessary to the experiment in a requirements.txt file. This is achieved by running:

$ pipreqsnb .

Finally commit and tag:

$ git add notebook.ipynb requirements.txt index.html
$ git commit -a -m "analyzed"
$ git tag ANALYSED -m "Analysis completed"
Methodologically, preparing the analysis at design time, is even better.

If some byproducts of the analysis have to be committed, then this is preferably done into a analysis/ directory.


You are encouraged to publish your experiment results. If you have followed the instructuon above, you should be well prepared for that.