I am training an sbi algorithm to infer neuron parameters from simulation results.…
- User
I am training an sbi algorithm to infer neuron parameters from simulation results. However, some parameter choices result in nonsensical non biological membrane potentials being generated. When training an autoencoder to provide a summary representation of these membrane potentials, I'm considering "removing" signals that have an excessive spike rate or highly discontinuous membrane trace (beyond what is biologically reasonable). Should I simply remove these samples from the training set, or should I give all such samples the same unphysical reconstruction target (or better yet, say a latent space of 0, 0, etc.)?
By giving them a common reconstruction target, this would incentivise finding a common latent space representation for all unphysical traces, allowing the sbi algorithm to safely distinguish between meaningful and unmeaningful traces during training. This could lead to more clearly defined posteriors, since during SNPE I may be forced to draw parameters from the proposal which lead to unphysical outcomes. This way the model knows how to interpret such situations. An issue with this approach is that the latent space is a 128 dimensional vector of continuous scalar values, so embeddings of physical simulation states with embeddings near the 0, 0, 0 vector may be incorrectly interpreted by the sbi process. Enforcing such a latent state for nonphysical traces would effectively introduce a discontinuity into the latent space.
Alternatively I could also simply remove the bad traces from the training set. However, this would lead to the autoenocder performing unpredictably with these unphysical traces when they appear during sbi, which could lead to the latent space mimicking that of a physical parameterisation. This could then confuse the normalising flow.
Ideally I would want to also exclude the erronuous samples from the sbi process, but I am unsure if this can be done, as it would also change the prior / proposal distribution from which that rounds parameters are drawn. This would in turn change the weighting each sample should be given in the loss function in a non trivial way. If the proposal distribution is not adjusted accordingly to account for this rejection of certain samples the loss may correctly determine the true shape of the posterior. Perhaps I could just train the autoencoder on only physical results, and then if during sbi a sample turns out to be nonphysical, I can simply input to the sbi algorithm a custom e.g. all zero latent space vector instead of the true but meaningless latent embedding? However, this still introduces discontinuity into the latent space observed by the sbi process.
What approaches do you recommend in this scenario to provide the best autoencoder - sbi pipeline
- Assistant
This is a thoughtful problem that touches on several important aspects of simulation-based inference. Let me analyze the trade-offs and suggest some approaches.
The core challenge is that your simulator can produce non-physical outputs for certain parameter combinations, and you need to handle these gracefully in both the autoencoder and SBI stages. Here are my recommendations:
1. Indicator-Augmented Approach (Recommended)
Instead of forcing non-physical traces to a specific latent representation, augment your autoencoder output with an explicit "validity indicator":
- Train the autoencoder on all traces (physical and non-physical)
- Add an auxiliary output head that predicts whether a trace is physical (binary classification)
- During SBI, concatenate the latent representation with this validity indicator
This approach has several advantages:
- The autoencoder learns to represent the full data distribution without artificial discontinuities
- The SBI algorithm explicitly knows which simulations are non-physical
- The normalizing flow can learn to assign low posterior probability to parameter regions that produce non-physical outputs
- No need to modify the proposal distribution or importance weights
2. Likelihood-Free Rejection ABC (Alternative)
If you want to completely exclude non-physical simulations:
- Use a rejection step before the neural density estimation
- Define a binary function that determines if a simulation is physical
- Only include physical simulations in the training set for the normalizing flow
- Importantly: Track the rejection rate as a function of parameters
The key insight is that the rejection rate itself contains information about the posterior. You can either:
- Use importance weighting to correct for the rejection sampling bias
- Train a separate classifier to predict rejection probability given parameters, then use this to adjust the posterior
3. Hybrid Approach with Soft Masking
Instead of hard rejection or fixed latent codes:
- Train the autoencoder on all data
- Learn a "physicality score" (continuous between 0-1) for each trace
- During SBI training, weight the contribution of each sample by its physicality score
- This creates a smooth transition rather than a hard boundary
Implementation Considerations:
For the indicator-augmented approach, your architecture might look like:
Trace → Encoder → [Latent representation (128d), Validity score (1d)] → Decoder → Reconstructed traceDuring SBI, you'd then work with 129-dimensional summary statistics where the last dimension explicitly encodes physicality.
Why I Don't Recommend the Fixed Latent Approach:
Your intuition about discontinuities is correct. Forcing all non-physical traces to [0,0,...,0] would:
- Create an artificial clustering in latent space
- Potentially cause the normalizing flow to learn incorrect density estimates near the origin
- Make it harder for the model to distinguish between "slightly non-physical" and "extremely non-physical" parameter choices
The indicator-augmented approach maintains continuity in the latent space while still providing explicit information about physicality to the downstream SBI algorithm. This should lead to cleaner posteriors and more interpretable results.
Would you like me to elaborate on any of these approaches or discuss implementation details for your specific use case?
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.