ActiveSubspace#
- class ActiveSubspace(posterior)[source]#
Bases:
objectIdentify the active subspace of the posterior for sensitivity analyses.
- Parameters:
posterior (Any)
- add_property(theta, emergent_property, model='resnet', hidden_features=100, num_blocks=2, dropout_probability=0.5, z_score_theta=True, z_score_property=True, embedding_net=Identity())[source]#
Add a property whose sensitivity is to be analysed.
To analyse the sensitivity of an emergent property, we train a neural network to predict the property from the parameter set \(\theta\). The hyperparameters of this neural network also have to be specified here.
- Parameters:
theta (Tensor) – Parameter sets \(\theta\) sampled from the posterior.
emergent_property (Tensor) – Tensor containing the values of the property given each parameter set \(\theta\).
model (str | Callable) – Neural network used to distinguish valid from bad samples. If it is a string, use a pre-configured network of the provided type (either mlp or resnet). Alternatively, a function that builds a custom neural network can be provided. The function will be called with the first batch of parameters (theta,), which can thus be used for shape inference and potentially for z-scoring. It needs to return a PyTorch nn.Module implementing the classifier.
hidden_features (int) – Number of hidden units of the classifier if model is a string.
num_blocks (int) – Number of hidden layers of the classifier if model is a string.
dropout_probability (float) – Dropout probability of the classifier if model is resnet.
z_score_theta (bool) – Whether to z-score the parameters \(\theta\) used to train the classifier.
z_score_property (bool) – Whether to z-score the property used to train the classifier.
embedding_net (Module) – Neural network used to encode the parameters before they are passed to the classifier.
- Returns:
ActiveSubspace to make the call chainable.
- Return type:
- train(training_batch_size=200, learning_rate=0.0005, validation_fraction=0.1, stop_after_epochs=20, max_num_epochs=2147483647, clip_max_norm=5.0)[source]#
Train a regression network to predict the specified property from \(\theta\).
- Parameters:
training_batch_size (int) – Training batch size.
learning_rate (float) – Learning rate for Adam optimizer.
validation_fraction (float) – The fraction of data to use for validation.
stop_after_epochs (int) – The number of epochs to wait for improvement on the validation set before terminating training.
max_num_epochs (int) – Maximum number of epochs to run. If reached, we stop training even when the validation loss is still decreasing. Otherwise, we train until validation loss increases (see also stop_after_epochs).
clip_max_norm (float | None) – Value at which to clip the total gradient norm in order to prevent exploding gradients. Use None for no clipping.
- Return type:
- find_directions(posterior_log_prob_as_property=False, norm_gradients_to_prior=True, num_monte_carlo_samples=1000)[source]#
Return eigenvectors and values corresponding to directions of sensitivity.
The directions of sensitivity are the directions along which a specific property changes in the fastest way. They will have the largest eigenvalues.
This computes the matrix: \(\mathbf{M} = \mathbb{E}_{p(\theta|x_o)}[\nabla_{\theta} f(\theta)^T \nabla_{\theta} f(\theta)]\) where \(f(\cdot)\) is the trained regression network. The expected value is approximated with a Monte-Carlo mean. Next, do an eigenvalue decomposition of the matrix \(\mathbf{M}\):
\(\mathbf{M} = \mathbf{Q} \mathbf{\Lambda} \mathbf{Q}^{-1}\)
We then return the eigenvectors and eigenvalues found by this decomposition. Eigenvectors with large eigenvalues are directions along which the property is sensitive to changes in the parameters \(\theta\) (active directions). Increases along these directions will increase the value of the property.
- Parameters:
posterior_log_prob_as_property (bool) – Whether to use the posterior log-probability the key property whose sensitivity is analysed. If False, one must have specified an emergent property and trained a regression network using .add_property().train(). If True, any previously specified property is ignored.
norm_gradients_to_prior (bool) – Whether to normalize each entry of the gradient by the standard deviation of the prior in each dimension. If set to False, the directions with the strongest eigenvalues might correspond to directions in which the prior is broad.
num_monte_carlo_samples (int) – Number of Monte Carlo samples that the average is based on. A larger value will make the results more accurate while requiring more compute time.
- Returns:
Eigenvectors and corresponding eigenvalues. They are sorted in ascending order. The column eigenvectors[:, j] is the eigenvector corresponding to the j-th eigenvalue.
- Return type:
- project(theta, num_dimensions)[source]#
Return \(\theta\) that were projected into the subspace.
To identify the dimensionality of the active subspace num_dimensions, Constantine et al. 2015 suggest to look at gaps in the eigenvalue spectrum.
Performs a linear projection. Also takes care of normalizing the data. The mean and standard deviation used for normalizing are the same as used to compute the eigenvectors and eigenvalues (mean and std of prior).