Overview of the compilation pipeline¶
The purpose of this page is to explain each step of defining and compiling a Theano function.
Definition of the computation graph¶
By creating Theano Variables using
theano.tensor.dmatrix or by using
Theano functions such as
theano.tensor.log, the user builds a computation graph. The
structure of that graph and details about its components can be found
in the Graph Structures article.
Compilation of the computation graph¶
Once the user has built a computation graph, she can use
theano.function in order to make one or more functions that
operate on real data. function takes a list of input Variables as well as a list of output Variables that define a
precise subgraph corresponding to the function(s) we want to define,
compile that subgraph and produce a callable.
Here is an overview of the various steps that are done with the computation graph in the compilation phase:
Step 1 - Create a FunctionGraph¶
The subgraph given by the end user is wrapped in a structure called FunctionGraph. That structure defines several hooks on adding and removing (pruning) nodes as well as on modifying links between nodes (for example, modifying an input of an Apply node) (see the article about fg – Graph Container [doc TODO] for more information).
FunctionGraph provides a method to change the input of an Apply node from one Variable to another and a more high-level method to replace a Variable with another. This is the structure that Optimizers work on.
Some relevant Features are typically added to the FunctionGraph, namely to prevent any optimization from operating inplace on inputs declared as immutable.
Step 2 - Execute main Optimizer¶
Once the FunctionGraph is made, an optimizer is produced by
the mode passed to
function (the Mode basically has two
optimizer). That optimizer is
applied on the FunctionGraph using its optimize() method.
The optimizer is typically obtained through
Step 4 - Wrap the thunk in a pretty package¶
The thunk returned by the linker along with input and output
containers is unwieldy.
function hides that complexity away so
that it can be used like a normal function with arguments and return