Skip to content

6. Distributions

6.1 gaussian

This module contains the code for the Gaussian distribution.

6.1.1 GaussianDistribution(shape, mu_prior=0.0, std_prior=0.1, mu_init=0.0, rho_init=-7.0, **kwargs)

This is the class to implement a learnable Gaussian distribution.

Constructor for GaussianDistribution.

Parameters:

Name Type Description Default
shape tuple[int, ...]

The shape of the distribution.

 required
mu_prior float

The mean for the prior distribution.

 0.0
std_prior float

The standard deviation for the prior distribution.

 0.1
mu_init float

The initial mean for the distribution.

 0.0
rho_init float

The initial value for the rho parameter.

 -7.0
**kwargs Any

Additional keyword arguments.

 {}
Source code in illia/distributions/tf/gaussian.py 
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
def __init__(
    self,
    shape: tuple[int, ...],
    mu_prior: float = 0.0,
    std_prior: float = 0.1,
    mu_init: float = 0.0,
    rho_init: float = -7.0,
    **kwargs: Any,
) -> None:
    """
    Constructor for GaussianDistribution.

    Args:
        shape: The shape of the distribution.
        mu_prior: The mean for the prior distribution.
        std_prior: The standard deviation for the prior distribution.
        mu_init: The initial mean for the distribution.
        rho_init: The initial value for the rho parameter.
        **kwargs: Additional keyword arguments.
    """

    # Call super class constructor
    super().__init__(**kwargs)

    # Set parameters
    self.shape = shape
    self.mu_prior_value = mu_prior
    self.std_prior_value = std_prior
    self.mu_init = mu_init
    self.rho_init = rho_init

    # Call build method
    self.build(shape)

6.1.1.1 num_params property

This method computes the number of parameters of the distribution.

Returns:

Type Description
int

Number of parameters.

6.1.1.2 log_prob(x=None)

This method computes the log prob of the distribution.

Parameters:

Name Type Description Default
x Optional[Tensor]

Output already sampled. If no output is introduced, first we will sample a tensor from the current distribution.

 None

Returns:

Type Description
Tensor

Log prob calculated as a tensor. Dimensions: [].
Source code in illia/distributions/tf/gaussian.py 
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
def log_prob(self, x: Optional[tf.Tensor] = None) -> tf.Tensor:
    """
    This method computes the log prob of the distribution.

    Args:
        x: Output already sampled. If no output is introduced,
            first we will sample a tensor from the current
            distribution.

    Returns:
        Log prob calculated as a tensor. Dimensions: [].
    """

    # Sample if x is None
    if x is None:
        x = self.sample()

    # Define pi variable
    pi: tf.Tensor = tf.convert_to_tensor(math.pi)

    # Compute log priors
    log_prior = (
        -tf.math.log(tf.math.sqrt(2 * pi))
        - tf.math.log(self.std_prior)
        - (((x - self.mu_prior) ** 2) / (2 * self.std_prior**2))
        - 0.5
    )

    # Compute sigma
    sigma: tf.Tensor = tf.math.log1p(tf.math.exp(self.rho))

    # Compute log posteriors
    log_posteriors: tf.Tensor = (
        -tf.math.log(tf.math.sqrt(2 * pi))
        - tf.math.log(sigma)
        - (((x - self.mu) ** 2) / (2 * sigma**2))
        - 0.5
    )

    # Compute final log probs
    log_probs = tf.math.reduce_sum(log_posteriors) - tf.math.reduce_sum(log_prior)

    return log_probs

6.1.1.3 sample()

This method samples a tensor from the distribution.

Returns:

Type Description
Tensor

Sampled tensor. Dimensions: [*] (same ones as the mu and std parameters).
Source code in illia/distributions/tf/gaussian.py 
120
121
122
123
124
125
126
127
128
129
130
131
132
133
def sample(self) -> tf.Tensor:
    """
    This method samples a tensor from the distribution.

    Returns:
        Sampled tensor. Dimensions: [*] (same ones as the mu and
            std parameters).
    """

    # Sampling with reparametrization trick
    eps: tf.Tensor = tf.random.normal(shape=self.rho.shape)
    sigma: tf.Tensor = tf.math.log1p(tf.math.exp(self.rho))

    return self.mu + sigma * eps