tf.contrib.distributions.bijectors.Softplus

View source on GitHub

Class Softplus

Bijector which computes Y = g(X) = Log[1 + exp(X)].

Inherits From: Bijector

The softplus Bijector has the following two useful properties:

• The domain is the positive real numbers
• softplus(x) approx x, for large x, so it does not overflow as easily as the Exp Bijector.

The optional nonzero hinge_softness parameter changes the transition at zero. With hinge_softness = c, the bijector is:

For large x >> 1, c * Log[1 + exp(x / c)] approx c * Log[exp(x / c)] = x, so the behavior for large x is the same as the standard softplus.

As c > 0 approaches 0 from the right, f_c(x) becomes less and less soft, approaching max(0, x).

• c = 1 is the default.
• c > 0 but small means f(x) approx ReLu(x) = max(0, x).
• c < 0 flips sign and reflects around the y-axis: f_{-c}(x) = -f_c(-x).

• c = 0 results in a non-bijective transformation and triggers an exception.

  Example Use:

# Create the Y=g(X)=softplus(X) transform which works only on Tensors with 1
# batch ndim and 2 event ndims (i.e., vector of matrices).
softplus = Softplus()
x = [[[1., 2],
      [3, 4]],
     [[5, 6],
      [7, 8]]]
log(1 + exp(x)) == softplus.forward(x)
log(exp(x) - 1) == softplus.inverse(x)

__init__

View source

__init__(
    hinge_softness=None,
    validate_args=False,
    name='softplus'
)

DEPRECATED FUNCTION

kwargs:

• hinge_softness: Nonzero floating point Tensor. Controls the softness of what would otherwise be a kink at the origin. Default is 1.0

Properties

dtype

dtype of Tensors transformable by this distribution.

forward_min_event_ndims

Returns the minimal number of dimensions bijector.forward operates on.

graph_parents

Returns this Bijector's graph_parents as a Python list.

hinge_softness

inverse_min_event_ndims

Returns the minimal number of dimensions bijector.inverse operates on.

is_constant_jacobian

Returns true iff the Jacobian matrix is not a function of x.

Returns:

• is_constant_jacobian: Python bool.

name

Returns the string name of this Bijector.

validate_args

Returns True if Tensor arguments will be validated.

Methods

tf.contrib.distributions.bijectors.Softplus.forward

View source

forward(
    x,
    name='forward'
)

Returns the forward Bijector evaluation, i.e., X = g(Y).

Args:

• x: Tensor. The input to the "forward" evaluation.
• name: The name to give this op.

Returns:

Tensor.

Raises:

• TypeError: if self.dtype is specified and x.dtype is not self.dtype.
• NotImplementedError: if _forward is not implemented.

tf.contrib.distributions.bijectors.Softplus.forward_event_shape

View source

forward_event_shape(input_shape)

Shape of a single sample from a single batch as a TensorShape.

Same meaning as forward_event_shape_tensor. May be only partially defined.

Args:

• input_shape: TensorShape indicating event-portion shape passed into forward function.

Returns:

• forward_event_shape_tensor: TensorShape indicating event-portion shape after applying forward. Possibly unknown.

tf.contrib.distributions.bijectors.Softplus.forward_event_shape_tensor

View source

forward_event_shape_tensor(
    input_shape,
    name='forward_event_shape_tensor'
)

Shape of a single sample from a single batch as an int32 1D Tensor.

Args:

• input_shape: Tensor, int32 vector indicating event-portion shape passed into forward function.
• name: name to give to the op

Returns:

• forward_event_shape_tensor: Tensor, int32 vector indicating event-portion shape after applying forward.

tf.contrib.distributions.bijectors.Softplus.forward_log_det_jacobian

View source

forward_log_det_jacobian(
    x,
    event_ndims,
    name='forward_log_det_jacobian'
)

Returns both the forward_log_det_jacobian.

Args:

• x: Tensor. The input to the "forward" Jacobian determinant evaluation.
• event_ndims: Number of dimensions in the probabilistic events being transformed. Must be greater than or equal to self.forward_min_event_ndims. The result is summed over the final dimensions to produce a scalar Jacobian determinant for each event, i.e. it has shape x.shape.ndims - event_ndims dimensions.
• name: The name to give this op.

Returns:

Tensor, if this bijector is injective. If not injective this is not implemented.

Raises:

• TypeError: if self.dtype is specified and y.dtype is not self.dtype.
• NotImplementedError: if neither _forward_log_det_jacobian nor {_inverse, _inverse_log_det_jacobian} are implemented, or this is a non-injective bijector.

tf.contrib.distributions.bijectors.Softplus.inverse

View source

inverse(
    y,
    name='inverse'
)

Returns the inverse Bijector evaluation, i.e., X = g^{-1}(Y).

Args:

• y: Tensor. The input to the "inverse" evaluation.
• name: The name to give this op.

Returns:

Tensor, if this bijector is injective. If not injective, returns the k-tuple containing the unique k points (x1, ..., xk) such that g(xi) = y.

Raises:

• TypeError: if self.dtype is specified and y.dtype is not self.dtype.
• NotImplementedError: if _inverse is not implemented.

tf.contrib.distributions.bijectors.Softplus.inverse_event_shape

View source

inverse_event_shape(output_shape)

Shape of a single sample from a single batch as a TensorShape.

Same meaning as inverse_event_shape_tensor. May be only partially defined.

Args:

• output_shape: TensorShape indicating event-portion shape passed into inverse function.

Returns:

• inverse_event_shape_tensor: TensorShape indicating event-portion shape after applying inverse. Possibly unknown.

tf.contrib.distributions.bijectors.Softplus.inverse_event_shape_tensor

View source

inverse_event_shape_tensor(
    output_shape,
    name='inverse_event_shape_tensor'
)

Shape of a single sample from a single batch as an int32 1D Tensor.

Args:

• output_shape: Tensor, int32 vector indicating event-portion shape passed into inverse function.
• name: name to give to the op

Returns:

• inverse_event_shape_tensor: Tensor, int32 vector indicating event-portion shape after applying inverse.

tf.contrib.distributions.bijectors.Softplus.inverse_log_det_jacobian

View source

inverse_log_det_jacobian(
    y,
    event_ndims,
    name='inverse_log_det_jacobian'
)

Returns the (log o det o Jacobian o inverse)(y).

Mathematically, returns: log(det(dX/dY))(Y). (Recall that: X=g^{-1}(Y).)

Note that forward_log_det_jacobian is the negative of this function, evaluated at g^{-1}(y).

Args:

• y: Tensor. The input to the "inverse" Jacobian determinant evaluation.
• event_ndims: Number of dimensions in the probabilistic events being transformed. Must be greater than or equal to self.inverse_min_event_ndims. The result is summed over the final dimensions to produce a scalar Jacobian determinant for each event, i.e. it has shape y.shape.ndims - event_ndims dimensions.
• name: The name to give this op.

Returns:

Tensor, if this bijector is injective. If not injective, returns the tuple of local log det Jacobians, log(det(Dg_i^{-1}(y))), where g_i is the restriction of g to the ith partition Di.

Raises:

• TypeError: if self.dtype is specified and y.dtype is not self.dtype.
• NotImplementedError: if _inverse_log_det_jacobian is not implemented.