Describe how you can derive an estimate if you don't have any data
Inference Without Empirical Data:
deriving estimates without empirical data involves using prior knowledge or assumptions.
For instance, some models can provide initial parameter estimates.
These are not derived from empirical data but rather from knowledge or assumptions about the system's behavior.
Example:
In Bayesian inference, including MAP, the role of prior knowledge is crucial.
When empirical data is sparse or absent, the choice of prior can significantly influence the estimates.
This approach allows for the integration of external knowledge into the estimation process, which can be particularly useful in the absence of data.
What is backpropagation and what do you need to perform it?
Backpropagation is the algorithm used to calculate the gradient of the loss function with respect to the weights in a neural network.
To perform it, we need
a neural network and
a loss function
activation function that is differentiable
backpropagation process
a forward pass
inputs are passed through the network to generate predictions.
crucial for computing the output of the network,
which is then compared to the true labels to calculate the loss.
backward pass
the calculation of gradients of the loss function with respect to each weight in the network by applying the chain rule.
involves computing partial derivatives at each layer starting from the output layer back to the input layer, hence the term "backpropagation."
the gradients are then summed up and an optimization algorithm, typically stochastic gradient descent, will update the weights of the neural network in the direction of the negative gradient
Last changed10 months ago