Documentation Documentation AKA Meta-Documentation

How to link from Sphinx to API documentation

Link from Sphinx to epydoc this way:

:api:`linkname <path-to-symbol>`

For example:

:api:`function <theano.compile.function>`

You can also use :api:\`path-to-symbol\` directly.

If you are lazy, you can try:

:api:`<function>`

However, if the link target is ambiguous, Sphinx will generate errors.

NB the :api: reference is special magic by Olivier, in ./doc/scripts/docgen.py.

How to add TODO comments in Sphinx documentation

To include a TODO comment in Sphinx documentation, use an indented block as follows:

.. TODO: This is a comment.
.. You have to put .. at the beginning of every line :(
.. These lines should all be indented.

It will not appear in the output generated.

How to write API documentation

API documentation is processed by epydoc and should be written in reStructuredText. See here for details about how to use reST with epydoc documentation.

Use ReST for API and Sphinx documentation

• ReST is standardized. epydoc is not. trac wiki-markup is not. This means that ReST can be cut-and-pasted between epydoc, code, other docs, and TRAC. This is a huge win!
• ReST is extensible: we can write our own roles and directives to automatically link to WIKI, for example.
• ReST has figure and table directives, and can be converted (using a standard tool) to latex documents.
• No text documentation has good support for math rendering, but ReST is closest: it has three renderer-specific solutions (render latex, use latex to build images for html, use itex2mml to generate MathML)

How to build documentation

Let’s say you are writing documentation, and want to see the epydoc and sphinx output before you push it.

The following technique uses Olivier’s code to insert :api: links:

cd Theano/
python ./doc/scripts/docgen.py

If you don’t want to rerun epydoc, do the following:

cd Theano/
python ./doc/scripts/docgen.py --rst

For more details:

$ python doc/scripts/docgen.py --help
Usage: doc/scripts/docgen.py [OPTIONS]
  -o <dir>: output the html files in the specified dir
  --rst: only compile the doc (requires sphinx)
  --nopdf: do not produce a PDF file from the doc, only HTML
  --epydoc: only compile the api documentation (requires epydoc)
  --help: this help

How documentation is built on deeplearning.net

The server that hosts the theano documentation runs a cron job roughly every 2 hours that fetches a fresh Theano install (clone, not just pull) and executes the docgen.py script. It then over-writes the previous docs with the newly generated ones.

Note that the server will most definitely use a different version of sphinx than yours so formatting could be slightly off, or even wrong. If you’re getting unxpected results and/or the auto-build of the documentation seems broken, please contact theano-dev@ (or Dumitru, or Fred directly).

In the future, we might go back to the system of auto-refresh on push (though that might increase the load of the server quite significantly).

pylint

Here is the autogenerated pylint output.

Pylint documentation is generated using pylintrc file: Theano/doc/pylintrc

You can see a list of all pylint messages.

The nightly build/tests process

The user lisa runs a cronjob on the computer boltzmann. This happens nightly. The cronjob executes the script ~/www/nightly_build/do_nightly_build_theano. This script performs a an “hg pull -u” of theano (and pylearn too). Then it executes something like

THEANO_FLAGS='unittests.rseed=<SEED>,mode=DEBUG_MODE' /usr/bin/nosetests --with-coverage --cover-package=theano --cover-package=pylearn

in the updated theano directory. The output is emailed automatically to one of the developers. This line is executed with a several fixed SEED values in DEBUG_MODE. It is executed in FAST_RUN and FAST_COMPILE with just one SEED value.

We can’t email directly to the mailing list as it is emailed as the root user. We haven’t yet found a way to have google groups accept email from a user without sending an email to it. As we don’t have root control from this domain, we won’t receive the email. We haven’t pursued the option of configuring the lisa account to send email from an official gmail account.

Also, to have the crontab executed, the ``lisa`` user must be logged into ``boltzmann``.

In a nutshell: Fred keeps a shell open on boltzmann at all times, gets the email, and forwards it to the mailing list when something interesting shows up in the report.

TO WRITE

There is other stuff to document here, e.g.:

• where the documentation is getting built, and epy.
• How epydoc and sphinx are integrated
• Explain the special magic in ./doc/scripts/docgen.py
• We also want examples of good documentation, to show people how to write sphinx,