CONTINUOUS-TIME MARKOV CHAINS

QR-Code Scanner In VS .NETUsing Barcode Control SDK for VS .NET Control to generate, create, read, scan barcode image in Visual Studio .NET applications.

that Theorem 441 remains valid with the same matrix A(z) For the particular case of the unloader problem, we nd that (449) reduces to the polynomial equation ( + z) ( z2 + ( + + )z) + z = 0 Special case of linear birth-death rates Suppose that the transition rates s , s , s and s have the special form s = b1 (m s) + c1 s, s = b 1 (m s) + c 1 s (4410)

Drawing QR Code In VS .NETUsing Barcode creator for VS .NET Control to generate, create Denso QR Bar Code image in .NET framework applications.

s = a0 (m s) and s = d0 s

Read QR Code 2d Barcode In Visual Studio .NETUsing Barcode reader for Visual Studio .NET Control to read, scan read, scan image in VS .NET applications.

for constants a0 , b1 , b 1 , c1 , c 1 and d0 Then the numerical problem of computing the roots of det A(z) = 0 can be circumvented The decay factor in (442) is then the unique solution of the equation B(x) + C(x) x[A(1) + B(1) + C(1) + D(1)] + on the interval (0,1), where A(x) = a0 x, B(x) = b1 + b 1 x 2 , C(x) = c1 + c 1 x 2 , D(x) = d0 x, F (x) = [A(1) + B(1) C(1) D(1)]x + C(x) B(x) In a more general context this result has been proved in Adan and Resing (1999) It also follows from this reference that Assumption 421 holds when d0 (b 1 b1 ) + a0 (c 1 c1 ) > 0 The condition (4410) is satis ed for several interesting queueing models For example, take a queueing model with m traf c sources which act independently of each other Each traf c source is alternately on and off, where the ontimes and off-times have exponential distributions with respective means 1/ and 1/ The successive on- and off-times are assumed to be independent of each other During on-periods a source generates service requests according to a Poisson process with rate There is a single server to handle the service requests and the server can handle only one request at a time The service times of the requests are independent random variables that have a common exponential distribution with mean 1/ This queueing problem can be modelled as a continuous-time Markov chain whose state space is given by (441) with i denoting the number of service requests in the system and s denoting the number of sources that are on This Markov chain has the property (4410) with s = s, s = , s = (m s) and s = s F (x)2 + 4A(x)D(x) = 0

Bar Code Encoder In VS .NETUsing Barcode drawer for VS .NET Control to generate, create barcode image in Visual Studio .NET applications.

TRANSIENT STATE PROBABILITIES

Barcode Decoder In VS .NETUsing Barcode decoder for .NET framework Control to read, scan read, scan image in Visual Studio .NET applications.

In many practical situations one is not interested in the long-run behaviour of a stochastic system but in its transient behaviour A typical example concerns airport runway operations The demand pro le for runway operations shows considerable

Generating QR In Visual C#Using Barcode creator for Visual Studio .NET Control to generate, create QR Code ISO/IEC18004 image in VS .NET applications.

TRANSIENT STATE PROBABILITIES

Print QR Code 2d Barcode In Visual Studio .NETUsing Barcode maker for ASP.NET Control to generate, create QR-Code image in ASP.NET applications.

variation over time with peaks at certain hours of the day Equilibrium models are of no use in this kind of situation The computation of transient solutions for Markov systems is a very important issue that arises in numerous problems in queueing, inventory and reliability In this section we discuss two basic methods for the computation of the transient state probabilities of a continuous-time Markov chain The next section deals with the computation of the transient distribution of the cumulative reward in a continuous-time Markov chain with a reward structure The transient probabilities of a continuous-time Markov chain {X(t), t 0} are de ned by pij (t) = P {X(t) = j | X(0) = i}, i, j I and t > 0 In Section 451 we discuss the method of linear differential equations The probabilistic method of uniformization will be discussed in Section 452 In Section 453 we show that the computation of rst passage time probabilities can be reduced to the computation of transient state probabilities by introducing an absorbing state 451 The Method of Linear Differential Equations This basic approach has a solid backing by tools from numerical analysis We rst prove the following theorem Theorem 451 (Kolmogoroff s forward differential equations) Suppose that the continuous-time Markov chain {X(t), t 0} satis es Assumption 412 Then for any i I , pij (t) =

QR Code Creator In Visual Basic .NETUsing Barcode creator for .NET framework Control to generate, create QR Code image in .NET framework applications.

qkj pik (t) j pij (t),

Data Matrix ECC200 Generator In .NET FrameworkUsing Barcode maker for Visual Studio .NET Control to generate, create Data Matrix image in Visual Studio .NET applications.

UPC-A Creation In Visual Studio .NETUsing Barcode creation for Visual Studio .NET Control to generate, create UPCA image in .NET framework applications.

ITF Creator In .NET FrameworkUsing Barcode creator for .NET Control to generate, create Uniform Symbology Specification ITF image in .NET applications.

UPC-A Supplement 5 Creator In Visual Studio .NETUsing Barcode drawer for ASP.NET Control to generate, create GTIN - 12 image in ASP.NET applications.

ECC200 Generation In .NET FrameworkUsing Barcode creator for ASP.NET Control to generate, create Data Matrix ECC200 image in ASP.NET applications.

Scan Barcode In Visual Studio .NETUsing Barcode recognizer for .NET Control to read, scan read, scan image in .NET applications.

Code 128B Encoder In JavaUsing Barcode creator for Java Control to generate, create USS Code 128 image in Java applications.