glossary-aez-notes

• Continuous probability distribution used as a pathological example which is a symmetric distribution about zero which does not have a well defined mean. The tails decay as \(1 / x^{2}\).

• An estimator of a parameter, \(\theta\), is said to be consistent if it converges to \(\theta\) as the sample size goes to infinity.
• An estimator can be unbiased but not consistent and it can be biased and consistent.

• A buzzword used to refer to the phenomenon that large collections of data tend to attract attention and additional data.

• Wikipedia says: "In information theory, the Hamming distance between two strings of equal length is the number of positions at which the corresponding symbols are different."

• See leverage.

• A continuous shrinkage prior which is a normal distribution where the scale parameter is drawn from a half-Cauchy distribution.

• Parameters that specify the parameters of the prior distribution.

• A random variable is infinitely divisible if for every natural number \(n\) there exists \(n\) IID random variables whos sum has the same same distribution.

• An older term used to describe the posterior distribution in Bayesian statistics.

• In regression, leverage is a measure of how unusual the explanatory values of a particular datum are. Hat-values are one measure of leverage. Leverage is important because points with high leverage have the potential to have a strong effect on estimates.

• The joint probability distribution of the sample viewed as a function of the parameters treating the sample as a fixed quantity¹.

• It is simple to construct a hypothesis test in which the Bayesian and frequentist approaches give different results. Typically, the frequentist test will reject and the Bayesian one will not. When this occurs it is said to be a Lindley paradox.

• Categorical data with an ordering, for example the data collected on a Likert scale.

• In regression, an outlier is a point that has a response value that is inconsistent with what one might expect given the rest of the data. This is not to be confused with an unusual point which is one for which both the response and predictor values are atypical.

• Probability distributions representing the beliefs about parameters held before seeing the data.
• Examples include reference priors, and the trilogy: diffuse priors, weakly informative and informative priors which span the spectrum of informativeness.

• The process of constructing a prior distribution based upon existing knowledge, potentially by interviewing domain experts. There is a large literature attempting to find best practises for eliciting prior distributions. In some settings it is very important, in others it is not worth the large amount of effort required to consult experts.

• When carrying out a Bayesian analysis, this is the process of checking that the implications of the prior distribution make sense. As with prior elicitation there are a lot of methods available for this².

• A type of non-informative prior distribution developed by Berger and Bernardo which aims to maximise the KL-divergence when averaged over potential data generated by the prior predictive distribution³.

• A prior distribution which concentrates substantial prior probability on a particular value of the parameter, typically at a value of zero; this is useful for variable selection.
• See the spike-and-slab prior and the horseshoe prior.

• A shrinkage prior which is a mixture of two distributions, one with a small variance about zero, the spike, and another with a large variance, the slab. For example a normal distribution with a point mass at zero.

Given a set \(X\), a binary relation, \(\leq\), is a total order iff it is

1. reflexive
2. transitive
3. anti-symmetric (meaning \(\forall a b\) if \(a\leq b\) and \(b \leq a\) then \(a=b\).)
4. total (meaning \(\forall a b\) either \(a\leq b\) or \(b \leq a\).)

• When to worry and how to Avoid the Misuse of Bayesian Statistics from 10.1037/met0000065.