, r r(j ) )(1 j ) , r r(k) Fkj )qkj

Denso QR Bar Code Decoder In .NETUsing Barcode Control SDK for Visual Studio .NET Control to generate, create, read, scan barcode image in VS .NET applications.

fk (

QR Code JIS X 0510 Maker In .NETUsing Barcode encoder for Visual Studio .NET Control to generate, create QR Code JIS X 0510 image in .NET framework applications.

for = 0, , , (t/ ) and r = 0, , , (x/ ) (for ease assume that x and t are multiples of ) For any j I , the boundary conditions are fj (0, r) = j / , 0, r = 0, otherwise,

Reading Quick Response Code In .NETUsing Barcode recognizer for .NET framework Control to read, scan read, scan image in Visual Studio .NET applications.

178 and

Bar Code Generation In VS .NETUsing Barcode printer for .NET Control to generate, create bar code image in .NET framework applications.

CONTINUOUS-TIME MARKOV CHAINS

Bar Code Scanner In VS .NETUsing Barcode recognizer for VS .NET Control to read, scan read, scan image in .NET applications.

fj ( , r) = 0 for any 0 when r < 0 Using the simple-minded approximation

QR Printer In Visual C#Using Barcode printer for .NET framework Control to generate, create QR-Code image in Visual Studio .NET applications.

x 0 x/ 1

QR Code 2d Barcode Generation In VS .NETUsing Barcode generator for ASP.NET Control to generate, create QR Code image in ASP.NET applications.

fj (t, y) dy

Quick Response Code Generator In Visual Basic .NETUsing Barcode encoder for Visual Studio .NET Control to generate, create QR Code image in .NET applications.

fj (t,

Generating Bar Code In Visual Studio .NETUsing Barcode printer for .NET Control to generate, create barcode image in Visual Studio .NET applications.

the desired probability P {R(t) x} is approximated by

Encoding EAN / UCC - 13 In VS .NETUsing Barcode drawer for .NET Control to generate, create GTIN - 13 image in .NET framework applications.

x/ 1

Code 128A Printer In .NETUsing Barcode printer for Visual Studio .NET Control to generate, create Code 128A image in VS .NET applications.

P {R(t) x}

Encoding MSI Plessey In VS .NETUsing Barcode printer for Visual Studio .NET Control to generate, create MSI Plessey image in Visual Studio .NET applications.

j I =0

ANSI/AIM Code 128 Creator In Visual C#Using Barcode maker for .NET Control to generate, create Code 128 Code Set B image in Visual Studio .NET applications.

fj (t,

Creating Barcode In VS .NETUsing Barcode maker for ASP.NET Control to generate, create barcode image in ASP.NET applications.

(462)

Generate Bar Code In C#Using Barcode creation for .NET framework Control to generate, create barcode image in .NET framework applications.

For xed x and t, the computational effort of the algorithm is proportional to 1/ 2 and so it quadruples when is halved Hence the computation time of the algorithm will become very large when the probability P {R(t) x} is desired at high accuracy and there are many states Another drawback of the discretization algorithm is that no estimate is available for the discretization error Fortunately, both dif culties can be partially overcome Let

Encoding Code 128 Code Set B In Visual Studio .NETUsing Barcode generation for ASP.NET Control to generate, create Code 128 Code Set B image in ASP.NET applications.

x/ 1

Code 128C Printer In JavaUsing Barcode drawer for Java Control to generate, create USS Code 128 image in Java applications.

P( ) =

Making Bar Code In .NET FrameworkUsing Barcode maker for ASP.NET Control to generate, create barcode image in ASP.NET applications.

j I =0

Code 128A Decoder In VS .NETUsing Barcode recognizer for .NET Control to read, scan read, scan image in VS .NET applications.

fj (t,

Data Matrix Generation In VS .NETUsing Barcode creator for ASP.NET Control to generate, create Data Matrix ECC200 image in ASP.NET applications.

be the rst-order estimate for P {R(t) x} and let the error term e( ) = P ( ) P {R(t) x} The following remarkable result was empirically found: e( ) P (2 ) P ( ) when is not too large Thus the rst-order approximation P ( ) to P {R(t) x} is much improved when it is replaced by P ( ) = P ( ) [P (2 ) P ( )] Example 453 (continued) The Hubble telescope problem What is the probability distribution of the number of repair missions that will be prepared in the next 10 years when currently all six gyroscopes are in perfect condition To consider this question we impose the following reward structure on the continuous-time Markov chain that is described in Figure 451 (with the states sleep 2 and sleep 1 numbered as the states 7 and 8) The reward rates r(j ) and the lump rewards Fjk are taken as r(j ) = 0 for all j, F27 = F18 = 1 and the other Fjk = 0 (463)

EXERCISES

Then the cumulative reward variable R(t) represents the number of repair missions that will be prepared up to time t Note that in this particular case the stochastic variable R(t) has a discrete distribution rather than a continuous distribution However, the discretization algorithm also applies to the case of a reward variable R(t) with a non-continuous distribution For the numerical example with = 01, = 100 and = 5 we found that P {R(t) > k} has the respective values 06099, 00636 and 00012 for k = 0, 1 and 2 (accurate to four decimal places with = 1/256)

EXERCISES

41 A familiar sight in Middle East street scenes are the so-called sheroots A sheroot is a seven-seat cab that drives from a xed stand in a town to another town A sheroot leaves as soon as all seven seats are occupied by passengers Consider a sheroot stand which has room for only one sheroot Potential passengers arrive at the stand according to a Poisson process at rate If upon arrival a potential customer nds no sheroot present and seven other customers already waiting, the customer goes elsewhere for transport; otherwise, the customer waits until a sheroot departs After a sheroot leaves the stand, it takes an exponential time with mean 1/ until a new sheroot becomes available Formulate a continuous-time Markov chain model for the situation at the sheroot stand Specify the state variable(s) and the transition rate diagram 42 In a certain city there are two emergency units, 1 and 2, that cooperate in responding to accident alarms The alarms come into a central dispatcher who sends one emergency unit to each alarm The city is divided in two districts, 1 and 2 The emergency unit i is the rst-due unit for response area i for i = 1, 2 An alarm coming in when only one of the emergency units is available is handled by the idle unit If both units are not available, the alarm is settled by some unit from outside the city Alarms from the districts 1 and 2 arrive at the central dispatcher according to independent Poisson processes with respective rates 1 and 2 The amount of time needed to serve an alarm from district j by unit i has an exponential distribution with mean 1/ ij The service times include travel times Formulate a continuous-time Markov chain model to analyse the availability of the emergency units Specify the state variable(s) and the transition rate diagram 43 An assembly line for a certain product has two stations in series Each station has only room for a single unit of the product If the assembly of a unit is completed at station 1, it is forwarded immediately to station 2 provided station 2 is idle; otherwise the unit remains in station 1 until station 2 becomes free Units for assembly arrive at station 1 according to a Poisson process with rate , but a newly arriving unit is only accepted by station 1 when no other unit is present in station 1 Each unit rejected is handled elsewhere The assembly times at stations 1 and 2 are exponentially distributed with respective means 1/ 1 and 1/ 2 Formulate a continuous-time Markov chain to analyse the situation at both stations Specify the state variable(s) and the transition rate diagram 44 Cars arrive at a gasoline station according to a Poisson process with an average of 10 customers per hour A car enters the station only if less than four other cars are present The gasoline station has only one pump The amount of time required to serve a car has an exponential distribution with a mean of four minutes (a) Formulate a continuous-time Markov chain to analyse the situation of the gasoline station Specify the state diagram (b) Solve the equilibrium equations