Release Notes

Theano 0.10.0beta1 (9th of August, 2017)

This release contains a lot of bug fixes, improvements and new features to prepare the upcoming release candidate.

We recommend that every developer updates to this version.

Highlights:
  • Moved Python 3.* minimum supported version from 3.3 to 3.4
  • Replaced deprecated package nose-parameterized with up-to-date package parameterized for Theano requirements
  • Theano now internally uses sha256 instead of md5 to work on systems that forbide md5 for security reason
  • Removed old GPU backend theano.sandbox.cuda. New backend theano.gpuarray is now the official GPU backend
  • Support more debuggers for PdbBreakpoint
  • Scan improvements
    • Speed up Theano scan compilation and gradient computation
    • Added meaningful message when missing inputs to scan
  • Speed up graph toposort algorithm
  • Faster C compilation by massively using a new interface for op params
  • Faster optimization step
  • Documentation updated and more complete
  • Many bug fixes, crash fixes and warning improvements

A total of 65 people contributed to this release since 0.9.0, see list below.

Interface changes:
  • Merged duplicated diagonal functions into two ops: ExtractDiag (extract a diagonal to a vector), and AllocDiag (set a vector as a diagonal of an empty array)
  • Renamed MultinomialWOReplacementFromUniform to ChoiceFromUniform
  • Removed or deprecated Theano flags:
    • cublas.lib
    • cuda.enabled
    • enable_initial_driver_test
    • gpuarray.sync
    • home
    • lib.cnmem
    • nvcc.* flags
    • pycuda.init
  • Changed grad() method to L_op() in ops that need the outputs to compute gradient
Convolution updates:
  • Extended Theano flag dnn.enabled with new option no_check to help speed up cuDNN importation
  • Implemented separable convolutions
  • Implemented grouped convolutions
GPU:
  • Prevent GPU initialization when not required
  • Added disk caching option for kernels
  • Added method my_theano_function.sync_shared() to help synchronize GPU Theano functions
  • Added useful stats for GPU in profile mode
  • Added Cholesky op based on cusolver backend
  • Added GPU ops based on magma library: SVD, matrix inverse, QR, cholesky and eigh
  • Added GpuCublasTriangularSolve
  • Added atomic addition and exchange for long long values in GpuAdvancedIncSubtensor1_dev20
  • Support log gamma function for all non-complex types
  • Support GPU SoftMax in both OpenCL and CUDA
  • Support offset parameter k for GpuEye
  • CrossentropyCategorical1Hot and its gradient are now lifted to GPU
  • Better cuDNN support
    • Official support for v5.* and v6.*
    • Better support and loading on Windows and Mac
    • Support cuDNN v6 dilated convolutions
    • Support cuDNN v6 reductions
    • Added new Theano flags cuda.include_path, dnn.base_path and dnn.bin_path to help configure Theano when CUDA and cuDNN can not be found automatically.
  • Updated float16 support
    • Added documentation for GPU float16 ops
    • Support float16 for GpuGemmBatch
    • Started to use float32 precision for computations that don’t support float16 on GPU
New features:
  • Added a wrapper for Baidu’s CTC cost and gradient functions
  • Added scalar and elemwise CPU ops for modified Bessel function of order 0 and 1 from scipy.special.
  • Added Scaled Exponential Linear Unit (SELU) activation
  • Added sigmoid_binary_crossentropy function
  • Added tri-gamma function
  • Added modes half and full for Images2Neibs ops
  • Implemented gradient for AbstractBatchNormTrainGrad
  • Implemented gradient for matrix pseudoinverse op
  • Added new prop replace for ChoiceFromUniform op
  • Added new prop on_error for CPU Cholesky op
  • Added new Theano flag deterministic to help control how Theano optimize certain ops that have deterministic versions. Currently used for subtensor Ops only.
  • Added new Theano flag cycle_detection to speed-up optimization step by reducing time spending in inplace optimizations
  • Added new Theano flag check_stack_trace to help check the stack trace during optimization process
  • Added new Theano flag cmodule.debug to allow a debug mode for Theano C code. Currently used for cuDNN convolutions only.
Others:
  • Added deprecation warning for the softmax and logsoftmax vector case
  • Added a warning to announce that C++ compiler will become mandatory in next Theano release 0.11
Other more detailed changes:
  • Removed useless warning when profile is manually disabled
  • Added tests for abstract conv
  • Added options for disconnected_outputs to Rop
  • Removed theano/compat/six.py
  • Removed COp.get_op_params()
  • Support of list of strings for Op.c_support_code(), to help not duplicate support codes
  • Macro names provided for array properties are now standardized in both CPU and GPU C codes
  • Started to move C code files into separate folder c_code in every Theano module
  • Many improvements for Travis CI tests (with better splitting for faster testing)
  • Many improvements for Jenkins CI tests: daily testings on Mac and Windows in addition to Linux
Commiters since 0.9.0:
  • Frederic Bastien
  • Arnaud Bergeron
  • amrithasuresh
  • João Victor Tozatti Risso
  • Steven Bocco
  • Pascal Lamblin
  • Mohammed Affan
  • Reyhane Askari
  • Alexander Matyasko
  • Simon Lefrancois
  • Shawn Tan
  • Thomas George
  • Faruk Ahmed
  • Zhouhan LIN
  • Aleksandar Botev
  • jhelie
  • xiaoqie
  • Tegan Maharaj
  • Matt Graham
  • Cesar Laurent
  • Gabe Schwartz
  • Juan Camilo Gamboa Higuera
  • AndroidCloud
  • Saizheng Zhang
  • vipulraheja
  • Florian Bordes
  • Sina Honari
  • Vikram
  • erakra
  • Chiheb Trabelsi
  • Shubh Vachher
  • Daren Eiri
  • Gijs van Tulder
  • Laurent Dinh
  • Mohamed Ishmael Diwan Belghazi
  • mila
  • Jeff Donahue
  • Ramana Subramanyam
  • Bogdan Budescu
  • Ghislain Antony Vaillant
  • Jan Schlüter
  • Xavier Bouthillier
  • fo40225
  • Aarni Koskela
  • Adam Becker
  • Adam Geitgey
  • Adrian Keet
  • Adrian Seyboldt
  • Andrei Costinescu
  • Anmol Sahoo
  • Chong Wu
  • Holger Kohr
  • Jayanth Koushik
  • Jenkins
  • Lilian Besson
  • Lv Tao
  • Michael Manukyan
  • Murugesh Marvel
  • NALEPA
  • Ubuntu
  • Zotov Yuriy
  • dareneiri
  • lrast
  • morrme
  • yikang