Markov Processes

• J_n describes the n-th jump

The mean is 1/ƛ

e^{-𝛌t} * (𝛌t)^k / k!

Mean = variance = 𝛌

An increment is a change from Xt+h-Xt

It means that P(Xt+h-Xt = k) only depends on h and not t.

Definition 1:

• Si are iid Exp(𝛌) and Yn = n

Definition 3:

If we know that between s and s+t there is exactly one jump, then that jump is uniformly distributed on that interval.

If a jump Yi can have either one of two properties, lets say red with probability p and white with probability (1-p), then we have

A set of points 𝓥⊂ℝ^d must fulfil the following two properties:

to be a Poisson point process

A compound Poisson Process is an extension of the normal PP, where the jumps Yi are iid but not always 1. This means that

A birth process is very similar to a Poisson process, but the parameters qi may change with each jump.

Explosion Time

𝛇 < ∞ :<=> explodes

𝛇 = ∞ :<=> doesn’t explode

When P(T > t+s | T > s) = P(T > t)

Basically you determine P(Tk ≥ t , Tj > Tk ∀j) by integrating over all s from t to infty. This will give you exactly what you want because you can pull the two probabilities apart in the integral.

We need to show that wherever we start the Poisson Process, lets say we restart at time t=10, that it is independent of anything that happened beforehand and that X_10+s, s≥0 is PP(𝛌) as well.

A stopping time is a random variable T, where {T≤t} only depends on the times Xs where s≤t.

For any stopping time, if T < infty, then

X{t+T} - X{T} is also a PP(lambda) and independent of anything that happened before.

A Q-Matrix is a matrix that fulfils the following conditions:

• qij ≥ 0 for all i≠j

• 0 ≤ -qii < ∞ for all i

• ∑qij = 0 for one row. (summing over j)

In the jump matrix ∏ for the matrix Q, the qij entry gives the probabilities to jump from state i to state j.

Hence, 𝛑ij = qij / qii if qii >0, else its just 0. For 𝛑ii, that is always 0 unless all other entries in the row are 0, then its 1 and we always stay there.

P(t) = exp(Qt)

• I is finite

• X0 = i and i is recurrent

• sup qij < ∞

A process where you can jump up and down at specific rates.

A matrix is irreducible when there exists a path from each to every index in the matrix.

A communicating class is a group of indices where the above holds

A point is recurrent if qi = 0 or qi >0 and Pi(Ti<∞) = 1.

It is transient if qi > 0 and Pi(Ti < ∞) < 1.

Positive Recurrent

This means qi = 0 or qi > 0 and Ei[Ti] < ∞

Null Recurrent

This means qi > 0 and Ei[Ti] = ∞

𝛌 is a probability distribution if ∑𝛌i = 1

This means that 𝛌Q = 0

Where mi = Ei[Ti]

𝛌i = 1/(qi*mi) where mi = Ei[Ti]

This is the distribution for Pi(Xt = i) for t-> \infty if Q is irreducible and recurrent

traffic intensity is ρ= 𝛌 / 𝛍 where 𝛌 is the rate of jumping upward and 𝛍 the rate of going downward

When for all T>0 the time reverse process is also a Markov process.

This is equivalent to 𝛌 and Q being in detailed balance and Q = Q hat

For a migration process with N particles and J nodes, then the state space is

That is a function 𝜙j which satisfies:

• 𝜙j(0) = 0

• 𝜙j(nj) > 0 if nj > 0

• if there are nj particles at Node j, service is provided at rate qj * 𝜙j(nj)

The rate of moving from one particle from node j to node k with the current state being n.

We add one more line as if the exterior was a node and just denote that with Δ.

If the state Mn only depends on the events X0…Xn

If E|Mn| < ∞

• Mn is adapted, integrable and E[Mn+1-Mn | Fn] = 0