Distributions¶

This tutorial demonstrates the functionality of the distributions module. It covers operations like sampling from distributions, calculating log probabilities, and checking the number of parameters.

Libraries¶

To get started, you'll need to import some essential libraries. The specific libraries you use will depend on the backend you've chosen, such as PyTorch, TensorFlow, or Jax. Additionally, you'll need to import NumPy.

In [1]:

import torch
import numpy as np

import torch import numpy as np

Functions¶

The compare_tensors function checks whether two tensors are close in value, given a relative and absolute tolerance. It prints the maximum absolute difference between the tensors and returns whether they are close.

In [2]:

def compare_tensors(a, b, rtol=1e-1, atol=1e-1, name=""):
    are_close = np.allclose(a, b, rtol=rtol, atol=atol)
    max_diff = np.max(np.abs(a - b))
    print(f"{name} are close: {are_close}")
    print(f"Max absolute difference for {name}: {max_diff}")
    return are_close

def compare_tensors(a, b, rtol=1e-1, atol=1e-1, name=""): are_close = np.allclose(a, b, rtol=rtol, atol=atol) max_diff = np.max(np.abs(a - b)) print(f"{name} are close: {are_close}") print(f"Max absolute difference for {name}: {max_diff}") return are_close

Random seeds¶

Set random seeds for reproducibility. This ensures that the results are consistent each time the code is executed.

In [3]:

np.random.seed(0)
torch.manual_seed(0)

np.random.seed(0) torch.manual_seed(0)

Out[3]:

<torch._C.Generator at 0x7ffaeffd93b0>

Illia¶

When setting the backend, we import the Illia library, which provides Bayesian module implementations. Note that backend selection requires a kernel restart and cannot be changed dynamically.

In [ ]:

import sys
import os

sys.path.append("/home/dani/Repositorios/illia/")
os.environ["ILLIA_BACKEND"] = "torch"

import illia

# Display available backends
print(f"Version: {illia.version()}, Backend: {illia.get_backend()}")

import sys import os sys.path.append("/home/dani/Repositorios/illia/") os.environ["ILLIA_BACKEND"] = "torch" import illia # Display available backends print(f"Version: {illia.version()}, Backend: {illia.get_backend()}")

Version: 0.0.1, Backend: torch

Test parameters and utilities¶

Define parameters and utilities that will be used in the tests.

In [5]:

shape = (3, 2)  # Shape of the distribution
mu_init = 0.0  # Initial mean
rho_init = -7.0  # Initial rho

shape = (3, 2) # Shape of the distribution mu_init = 0.0 # Initial mean rho_init = -7.0 # Initial rho

Initialize class distributions¶

Import and initialize Gaussian distribution classes for the various frameworks.

In [6]:

from illia.distributions import GaussianDistribution

# PyTorch
torch_dynamic_dist = GaussianDistribution(
    shape=shape, mu_init=mu_init, rho_init=rho_init
)

from illia.distributions import GaussianDistribution # PyTorch torch_dynamic_dist = GaussianDistribution( shape=shape, mu_init=mu_init, rho_init=rho_init )

Sampling¶

Sample from the distributions and compare the means and standard deviations.

In [7]:

print("Test 1: Sampling")

n_samples = 10000  # Number of samples
torch_samples = np.array(
    [torch_dynamic_dist.sample().detach().cpu().numpy() for _ in range(n_samples)]
)

# Compare means
torch_mean = np.mean(torch_samples, axis=0)
print(torch_mean)

# Compare standard deviations
torch_std = np.std(torch_samples, axis=0)
print(torch_std)

print("Test 1: Sampling") n_samples = 10000 # Number of samples torch_samples = np.array( [torch_dynamic_dist.sample().detach().cpu().numpy() for _ in range(n_samples)] ) # Compare means torch_mean = np.mean(torch_samples, axis=0) print(torch_mean) # Compare standard deviations torch_std = np.std(torch_samples, axis=0) print(torch_std)

Test 1: Sampling
[[ 0.04033715  0.08378547]
 [-0.07190923 -0.04033187]
 [-0.05966421  0.01821103]]
[[0.00082605 0.00077829]
 [0.0009529  0.00105996]
 [0.00100908 0.00098696]]

Log probabilities¶

Calculate and compare the log probabilities of a sample for both distributions.

In [8]:

print("\nTest 2: Log probability")

x = np.random.randn(*shape).astype(np.float32)
torch_log_prob = (
    torch_dynamic_dist.log_prob(torch.tensor(x, dtype=torch.float32))
    .detach()
    .cpu()
    .numpy()
)
print(torch_log_prob)

print("\nTest 2: Log probability") x = np.random.randn(*shape).astype(np.float32) torch_log_prob = ( torch_dynamic_dist.log_prob(torch.tensor(x, dtype=torch.float32)) .detach() .cpu() .numpy() ) print(torch_log_prob)

Test 2: Log probability
-7539210.5

Number of parameters¶

Check the number of parameters in both distributions to ensure they match.

In [9]:

print("\nTest 3: Number of parameters")
print("PyTorch num params:", torch_dynamic_dist.num_params())

print("\nTest 3: Number of parameters") print("PyTorch num params:", torch_dynamic_dist.num_params())

Test 3: Number of parameters
PyTorch num params: 6

In [ ]: