Package pylearn :: Package io :: Module filetensor
[hide private]

Module filetensor

source code

Read and write the matrix file format described at http://www.cs.nyu.edu/~ylclab/data/norb-v1.0/index.html

The format is for dense tensors:

The number of dimensions and rank is slightly tricky:

For rank >= 3, the number of dimensions matches the rank exactly.


To Do: add complex type support

Classes [hide private]
arraylike
Provide an array-like interface to the filetensor in f.
Functions [hide private]
 
_prod(lst) source code
 
_read_int32(f)
unpack a 4-byte integer from the current position in file f
source code
 
_read_header(f, debug=False)
:returns: data type, element size, rank, shape, size
source code
 
read(f, subtensor=None, debug=False)
Load all or part of file 'f' into a numpy ndarray
source code
 
write(f, mat)
Write a numpy.ndarray to file.
source code
Variables [hide private]
  _magic_dtype = {507333713: ('float32', 4), 507333715: ('float6...
  _dtype_magic = {'float32': 507333713, 'float64': 507333715, 'i...

Imports: sys, numpy


Function Details [hide private]

read(f, subtensor=None, debug=False)

source code 
Load all or part of file 'f' into a numpy ndarray

@param f: file from which to read
@type f: file-like object

If subtensor is not None, it should be like the argument to
numpy.ndarray.__getitem__.  The following two expressions should return
equivalent ndarray objects, but the one on the left may be faster and more
memory efficient if the underlying file f is big.

    read(f, subtensor) <===> read(f)[*subtensor]

Support for subtensors is currently spotty, so check the code to see if your
particular type of subtensor is supported.

write(f, mat)

source code 

Write a numpy.ndarray to file.

Parameters:
  • f (file-like object) - file into which to write
  • mat (numpy ndarray or compatible) - array to write to file

Variables Details [hide private]

_magic_dtype

Value:
{507333713: ('float32', 4),
 507333715: ('float64', 8),
 507333716: ('int32', 4),
 507333717: ('uint8', 1),
 507333718: ('int16', 2)}

_dtype_magic

Value:
{'float32': 507333713,
 'float64': 507333715,
 'int16': 507333718,
 'int32': 507333716,
 'uint8': 507333717}