Frequentest rationale

• Decision theory
• A procedure \(\delta(x)\)
• Define a loss function \(l(\theta, \delta)\) to measure performance
• Report \((\delta(x), R_\delta(\theta))\) where \[ R_\delta(\theta) = E_\theta L[\theta, \delta(X)] \]

Justification

Under some mild conditions, \[ \lim_{m\to \infty} \frac{1}{m} \sum_{i=1}^{m} L(\theta_0, \delta(X_i)) = R_\delta(\theta_0). \]

\(R_\delta(\theta_0)\) measures the long run performances of \(\delta\) for \(\theta_0\).

However, in practice, it is not quite useful.

• \(\theta_0\) is unknown
• \(\delta\) could be applied to different problems, different \(\theta_i\)

Formal definition

• Consider an infinite sequence of problemes: \(X_i \sim P_{\theta_i}\)

• Neyman (1967) defined the following frequentest measure \(\bar R_\delta\) to measure the performance of \(\delta\) \[ \limsup_{m\to \infty} \frac{1}{m} \sum_{i=1}^{m} L(\theta_i, \delta(X_i)) \le \bar R_\delta \]
  • it is free of \(\theta\)
  • Compare to the Bayes risk \[ r_\delta = \int R_\delta(\theta) \pi(\theta) d\theta \]

• (Berger, 1985) “practicing” frequentist haves quite differently from the “formal” frequentist.

Potential problems of frequentist approach

Example 1

if \(x=1\), how much evidence/confidence to support \(P_0\) or \(P_1\)?

Example 2

Supposer \(X\sim N(\mu, 1)\) and it is desired to test \[ H_1: \mu \le -2 \text{ vs } H_1: \mu >2. \]

• Consider the test: reject \(H_0\) if \(x \ge 0\).
• \(\alpha = \sup_{\mu\le -2} P_\mu(X\le 0) = 0.0228\).

• If \(x=0\), a frequentist will say “\(H_0\) is rejected with error 0.0228”.

• misleading?

• even worse, this frequentist statement will be valid as long as \(x\ge0\).

Fiducial inference

• Originated from Fisher (Fisher, 1935, 1930, 1933)
• Was considered “one great failure” of Fisher (Zabell, 1992)

• Fisher once said

  I don’t understand yet what fiducial probability does. We shall have to live with it a long time before we know what it’s doing for us. But it should not be ignored just because we don’t yet have a clear interpretation.

Fiducial in action

Suppose we have - \(F(x|\theta) = P(X\le x | \theta)\)

• Fisher argued that \(R(\theta) = 1- F(x|\theta)\) such that

\[ r(\theta|x) \propto - \frac{\partial F(x|\theta)}{\partial\theta} \]

Reactions from the community

• fiducial inference was first proposed by Fisher in 1930

• has never gained widespread acceptance

• the original fiducial argument was radically different from its later versions

• fiducial inference never actually developed during Fisher’s lifetime

• dozens of example of non-uniqueness and non-existence

Break with Neyman

• Neyman introduced confidence interval in 1934, claiming to have generalized fiducial probability

• Interestingly, Fisher was one of the discussants

• Fisher disputed the idea of “confidence”

• “confidence” is a purely frequentist concept

• “confidence” is known to omit, or suppress, part of the information supplied by the sample

• In a 1935 JRSS paper, Fisher shapely criticized Neyman

“Dr. Neyman had been somewhat unwise in his choice of topics”

• In 1941, Neyman published a paper “Fiducial argument and the theory of confidence interval”

• During the first 2 decdeas of dispute, Fisher almost never referred directly to Neyman in print

After many years

• Tsui and Weerahandi (1989) and Weerahandi (1993) proposed generalized inference and generalized \(p\)-value.

• Hannig et al. (2006) noted the relationship between generalized inference and fiducial inference.

• Hannig (2009) termed the new approach GFI or generalized fiducial inference

• what is GFI?

• switching principle

Generalized fiducial inference

• Consider the following data generating / structural / auxiliary equation \[ Y = G(U, \theta) \]

• \(U\) is a random variable whose distribution if free of \(\theta\).

• \(\theta\) is a fixed parameter

• Example: if \(Y\sim N(\theta, 1)\)
  • \(Y = \theta + Z\), \(Z \sim N(0,1)\)
  • \(Y = \theta + \Phi^{-1}(U)\), \(U \sim U(0,1)\)

Definition

• Suppose \(Y=y\)

• the generalized fiducial distribution of \(\theta\) is defined as the conditional distribution of \(Q(Y, U^*)\) given \(y = G(U^*, \theta)\), i.e.,

where \(U^*\) have the same distribution of \(U\).

• naive generation method
• generate \(U^*\)
• if exists \(\theta\) st \(y = G(U^*, \theta)\), proceed, otherwise, go to 1
• compute \(\theta = Q(y, U^*)\)

Examples

• My favorite example: \[ Y = \theta + Z, \ Z \sim N(0,1) \]

• The generalized fiducial distribution of \(\theta\) is

\[ Q(y, Z^*) | \{\exists \theta, y = \theta + Z^*\} \]

which is \(y-Z^* \sim N(y, 1)\)

• Slightly more difficult example

\[ \begin{align*} Y_1 = \theta + Z_1\\ Y_2 = \theta + Z_2 \end{align*} \]

Given \(Y=y\) the fiducial distribution is

\[ Q({\boldsymbol{y}}, {\boldsymbol{Z}}^*) | \{y_1 - Z_1^* = y_2 - Z_2^*\} \]

which is \(N(\bar y, 1)\).

Closer look and Borel paradox

• \(\{y_1 - Z_1^* = y_2 - Z_2^*\}\) is of probability 0

• lead to Borel paradox - conditioning on sets of probability 0 may not be well defined

• example, if \(X\) and \(Y\) are i.i.d. standard normal, what is the distribution of \(Y\) given \(Y=X\).

Closed form of the weak limit

Under some regularity conditions,

\[ \begin{equation*} r(\theta|y) \propto J(y,\theta) f(y,\theta) , \end{equation*} \] where \(f(y,\theta)\) is the likelihood and the function \(J(y,\theta)\) is

\[ \begin{equation*} J(y,\theta)= \sum_{\substack{{\boldsymbol{i}}=(i_1,\ldots,i_p) \\ 1\leq i_1<\cdots<i_p\leq n}}\left|\det\left(\left.\frac{\partial}{\partial\theta} G(u,\theta)\right|_{u=Q(y,\theta)}\right)_{\boldsymbol{i}}\right|. \end{equation*} \]

• data-dependent prior
• depends on \(G\)
• for linear regression model, this coincides with Jeffreys prior.

Two theorems

Theorem 1 (Bernstein–von Mises theorem of GFI): let

\[r^{*}\left(s|y\right)=n^{-1/2} r(n^{-1/2}s+\hat{\theta} | y),\]

we have \[ \int_{\mathbb{R}^{p}}\left|r^{*}\left(s|y\right)-\frac{\sqrt{det\left|I\left({\theta}_{0}\right)\right|}}{\sqrt{2\pi}}e^{-s^{T}I\left({\theta}_{0}\right)s/2}\right|\, ds\stackrel{P_{\boldsymbol{\theta}_{0}}}{\rightarrow}0. \]

Theorem 2 (Consistency of confidence sets):

• Suppose \(C_n(y_n)\) is a open sets of fiducial probability \(1-\alpha\), i.e. \(R_n(C_n(y_n)) = 1-\alpha\).

Under some regularity conditions,

\[ P(\theta_0 \in C_n(Y_n)) \to 1 - \alpha \]

End quote

• Maybe Fisher’s biggest blunder will become a big hit in the 21st century! (Efron, 1998)

References