Renewal Process: Intuition & The Renewal Argument

Perspective 1: Why Not Calculate Directly? (The Computational Challenge of Convolution)

We represent the system in two complementary ways:

1. Micro: The single inter-arrival interval $X_i$.
2. Macro: The time of the $n$-th occurrence $S_n = \sum X_i$.

If we want to find the distribution of $N(t)$ (e.g., $P(N(t)=k)$), by definition, this is equivalent to:

$$P(N(t)=k) = P(S_k \le t) - P(S_{k+1} \le t)$$

The Core Difficulty: $S_n$ is the sum of $n$ random variables. In probability theory, finding the “distribution of a sum” necessitates Convolution.

• $X_1$ has distribution $F$.
• $X_1+X_2$ has distribution $F * F$ (2-fold convolution).
• $S_n$ has distribution $F^{(n)}$ ($n$-fold convolution of $F$).

Thus, $P(N(t)=k) = F^{(k)}(t) - F^{(k+1)}(t)$.

Motivation: While this formula is theoretically rigorous, in practice, convolution is analytically intractable! Apart from the exponential distribution (Poisson process) and the normal distribution, finding an analytical solution for $F^{(n)}$ is nearly impossible. This is why we cannot rely solely on the distribution of $N(t)$, but must study its expectation $m(t)$ and asymptotic properties (Limit Theorems).

Perspective 2: The Philosophy of “Rebirth” (The Renewal Argument)

This is the soul of this chapter and the origin of the integral equations (Renewal Equations).

In a Poisson process, because the exponential distribution possesses the “memoryless property,” if we observe the system at any time $t$, the remaining waiting time is still exponentially distributed. Calculation is straightforward.

However, in a general Renewal Process, $X_i$ usually lacks the memoryless property. If we observe at time $t$, the waiting time for the next renewal depends on the time elapsed since the last occurrence. This dependency creates significant complexity.

Solution: Since analyzing the process at an arbitrary time is difficult, we return to the origin!

We use the “First Renewal” ($X_1$) as a pivot:

1. At the instant $X_1$ occurs, the entire process “Renews”.
2. The sequence of events following $X_1$ follows statistically identical laws to the process starting from $0$, merely shifted in time.

High-Level Logic (Conditioning on first arrival):

• We seek the mean count $m(t) = E[N(t)]$ over time $t$.
• Instead of calculating directly, we condition on the time of the first renewal $x$:
  • If $x > t$: The first event has not occurred, so $N(t)=0$.
  • If $x \le t$: One count is contributed. For the remaining time $t-x$, the process generates an expected $E[N(t-x)]$ counts (due to the renewal property).

$$E[N(t) | X_1 = x] = \begin{cases} 0 & x > t \\ 1 + E[N(t-x)] & x \le t \end{cases}$$

• This leads to the foundational Renewal Equation:

$$m(t) = \int_0^t (1 + m(t-x)) dF(x) = F(t) + \int_0^t m(t-x) dF(x)$$

Summary: The core of this perspective is decomposing a complex problem into “1 + a smaller instance of the same problem” via the “First Renewal”. This structure gives rise to the integral equations characterizing the process.

Perspective 3: From “Exact” to “Asymptotic” (Limit Theory)

Since the exact distribution of $N(t)$ (via convolution) is intractable, and the exact solution to the Renewal Equation is often complex, what is the objective?

Motivation: We are often less concerned with the exact value of $N(t)$ at a specific $t$, and more interested in “How the system behaves in the long run.”

This leads to a hierarchy of theorems:

1. Feller (LLN level): What is the long-term average rate?

   • Answer: $N(t)/t \to 1/\mu$. This aligns with intuition (Law of Large Numbers).

2. Central Limit Theorem level: What is the fluctuation range of $N(t)$?

   • Answer: For large $t$, $N(t)$ is approximately normally distributed.

3. Blackwell/Key Renewal (Local level): Is the system stationary locally at infinity?

   • Answer: Yes. The renewal density tends to a constant $1/\mu$ regardless of the observation time.

Conceptual Framework Update:

1. Input: $F$ (Micro mechanism)
2. Challenge: Direct summation ($S_n$) leads to computational intractability (convolution), complicated by the lack of memorylessness.
3. Tool 1 (Structural): Duality of $N(t) \leftrightarrow S_n$ ($N(t) \ge n \iff S_n \le t$). Used to establish the theoretical foundation.
4. Tool 2 (Computational): Conditioning on $X_1$ (Renewal Equation). This is the primary method for derivations. We use the “Process Restart” property to formulate equations.
5. Goal (Asymptotic): Since exact values are limited, we employ Limit Theorems (Feller, Blackwell) to describe long-term stable behavior.

One-Sentence Summary: The central theme involves addressing the complexity arising from the absence of memorylessness. To bypass this, we condition on the first renewal to establish recursive relationships, ultimately relying on limit theorems to describe the system’s asymptotic behavior without performing explicit convolutions.