232 Continuous-time Markov chains

QR Code Generator In Visual Studio .NETUsing Barcode generation for .NET Control to generate, create QR Code ISO/IEC18004 image in Visual Studio .NET applications.

We now examine how the previous framework applies to continuous-time Markov chains (CTMC) Following [13], let Y = {Y (t), t 0} be a CTMC evolving in Y up to some stopping time T = inf{t 0 : Y (t) }, where Y The initial distribution is 0 and the jump rate from y to z, for z = y, is ay,z Let ay = z=y ay,z be the departure rate from y The aim is to estimate E[X], where X = h(Y ) is a function of the entire sample path of the CTMC up to its stopping time T A sample path for this chain is determined uniquely by the sequence (Y0 , V0 , Y1 , V1 , , Y , V ) where Yj is the j th visited state of the chain, Vj the

QR Code Recognizer In Visual Studio .NETUsing Barcode reader for .NET framework Control to read, scan read, scan image in Visual Studio .NET applications.

IMPORTANCE SAMPLING

Bar Code Creation In .NETUsing Barcode printer for .NET Control to generate, create barcode image in .NET framework applications.

time spent in that state, and is the index of the jump that corresponds to the stopping time (the rst jump that hits ) Therefore h(Y ) can be re-expressed as h (Y0 , V0 , Y1 , V1 , , Yn , Vn ), and a sample path (y0 , v0 , y1 , v1 , , yn , vn ) has density (or likelihood)

Read Bar Code In .NET FrameworkUsing Barcode reader for .NET Control to read, scan read, scan image in Visual Studio .NET applications.

p(y0 , v0 , , yn , vn ) =

QR Code Creator In Visual C#.NETUsing Barcode printer for VS .NET Control to generate, create QR Code 2d barcode image in .NET framework applications.

j =0 n 1

Paint QR Code In VS .NETUsing Barcode drawer for ASP.NET Control to generate, create QR Code ISO/IEC18004 image in ASP.NET applications.

ayj ,yj +1 ayj

QR Printer In Visual Basic .NETUsing Barcode printer for VS .NET Control to generate, create QR-Code image in Visual Studio .NET applications.

ayj exp[ ayj vj ]

UCC.EAN - 128 Maker In .NET FrameworkUsing Barcode encoder for .NET Control to generate, create GTIN - 128 image in Visual Studio .NET applications.

j =0

Code 128B Creator In .NET FrameworkUsing Barcode generation for .NET Control to generate, create Code 128A image in .NET framework applications.

j =0

Creating ECC200 In .NETUsing Barcode drawer for VS .NET Control to generate, create Data Matrix image in .NET framework applications.

ayj ,yj +1 exp

Generating EAN13 In .NET FrameworkUsing Barcode generation for .NET framework Control to generate, create EAN-13 image in VS .NET applications.

n j =0

Print Uniform Symbology Specification Code 93 In .NET FrameworkUsing Barcode encoder for VS .NET Control to generate, create USS-93 image in Visual Studio .NET applications.

ayj vj ,

GS1 - 13 Generator In Visual Studio .NETUsing Barcode printer for ASP.NET Control to generate, create GTIN - 13 image in ASP.NET applications.

each term ayj ,yj +1 /ayj being the probability of moving from yj to yj +1 and ayj exp[ ayj vj ] the density for leaving yj after a sojourn time vj Then we have E[X] =

Recognizing Barcode In JavaUsing Barcode scanner for Java Control to read, scan read, scan image in Java applications.

y0 ,,yn 0

DataMatrix Encoder In Visual C#.NETUsing Barcode drawer for .NET framework Control to generate, create DataMatrix image in .NET framework applications.

h (y0 , v0 , , yn , vn )p(y0 , v0 , , yn , vn )dv0 dvn

Encoding DataMatrix In JavaUsing Barcode printer for Java Control to generate, create DataMatrix image in Java applications.

Suppose that the cost function has the form

Code 39 Creator In Visual C#Using Barcode encoder for VS .NET Control to generate, create Code 3 of 9 image in .NET framework applications.

X = h(Y ) =

UCC - 12 Generator In JavaUsing Barcode generator for Java Control to generate, create UPC Symbol image in Java applications.

j =1

Recognizing ANSI/AIM Code 128 In Visual Studio .NETUsing Barcode decoder for .NET framework Control to read, scan read, scan image in Visual Studio .NET applications.

c (Yj 1 , Vj 1 , Yj )

Bar Code Encoder In JavaUsing Barcode encoder for Java Control to generate, create bar code image in Java applications.

where c : Y [0, ) Y [0, ) In this case, a standard technique that always reduces the variance, and often reduces the computations as well, is to replace the estimator X by

Xcmc = E[X | Y0 , , Y ] =

j =1

c(Yj 1 , Yj ),

where c(Yj 1 , Yj ) = E[c (Yj 1 , Vj 1 , Yj ) | Yj 1 , Yj ] [9] In other words, we would never generate the sojourn times We are now back in our previous DTMC setting and the zero-variance transition probabilities are given again by (24) Consider now the case of a xed time horizon T , which therefore no longer has the form T = inf{t 0 : Y (t) } We then have two options: either we again reformulate the process as a DTMC, or retain a CTMC formulation In the rst case, we can rede ne the state as (Yj , Rj ) at step j , where Rj is the remaining clock time (until we reach time T ), as in [6] Then the zero-variance scheme is the same as for the DTMC setting if we replace the state Yj there by (Yj , Rj ), and if we rede ne We then have a non-denumerable state space, so the sums must be replaced by combinations of sums and integrals In this context of a nite time horizon, effective IS schemes will typically use non-exponential (often far from exponential) sojourn time distributions This means that we will no longer have a CTMC under IS Assume now that we want to stick with a

IMPORTANCE SAMPLING

CTMC formulation, and that we restrict ourselves to choosing a Markovian IS measure with new initial distribution 0 and new generator A such that 0 (y) > 0 (or ay,z > 0) whenever 0 (y) > 0 (or ay,z > 0) Let be the index j of the rst jump to a state Yj = Y (tj ) at time tj such that tj T Then, similarly to the discrete-time case, it can be shown that, provided is a stopping time with nite expectation under ( 0 , A), E[X] = E[XL ], with L the likelihood ratio given by 0 (Y0 ) L = 0 (Y0 )

1 j =0