Method of Maximum Likelihood

Get qr code 2d barcode in .netusing barcode creation for .net framework control to generate, create qr code 2d barcode image in .net framework applications.

One of the best methods of obtaining a point estimator of a parameter is the method of maximum likelihood. This technique was developed in the 1920s by a famous British statistician, Sir R. A. Fisher. As the name implies, the estimator will be the value of the parameter that maximizes the likelihood function.

Visual .net qr codes recognizer for .netUsing Barcode decoder for .NET Control to read, scan read, scan image in .NET applications.

7-3 METHODS OF POINT ESTIMATION

Barcode implement with .netuse .net framework bar code development tocreate barcode for .net

De nition Suppose that X is a random variable with probability distribution f (x; ), where is a single unknown parameter. Let x1, x2, p , xn be the observed values in a random sample of size n. Then the likelihood function of the sample is L1 2 f 1x1; 2 f 1x2; 2 p f 1xn ; 2 (7-5)

reading bar code in .netUsing Barcode decoder for VS .NET Control to read, scan read, scan image in VS .NET applications.

Note that the likelihood function is now a function of only the unknown parameter . The maximum likelihood estimator of is the value of that maximizes the likelihood function L( ).

In the case of a discrete random variable, the interpretation of the likelihood function is clear. The likelihood function of the sample L( ) is just the probability P1X1 x1, X2 x2, p , Xn xn 2

Control qr-codes image with .netusing an asp.net form toincoporate qr for asp.net web,windows application

That is, L( ) is just the probability of obtaining the sample values x1, x2, p , xn. Therefore, in the discrete case, the maximum likelihood estimator is an estimator that maximizes the probability of occurrence of the sample values. EXAMPLE 7-6 Let X be a Bernoulli random variable. The probability mass function is f 1x; p2 e px 11 0, p2 1 x, x 0, 1 otherwise

QR Code ISO/IEC18004 generating for vb.netusing barcode development for .net vs 2010 control to generate, create qr code image in .net vs 2010 applications.

where p is the parameter to be estimated. The likelihood function of a random sample of size n is L1 p2 px1 11

n i 1

Barcode integrating with .netusing visual studio .net toconnect bar code on asp.net web,windows application

p2 1

Barcode barcode library with .netuse vs .net crystal barcode printer toconnect barcode in .net

x1 x2

Make ean13 for .netusing barcode maker for .net crystal control to generate, create ean-13 image in .net crystal applications.

xi q p 11

Royal Mail Barcode integrating with .netgenerate, create rm4scc none on .net projects

p 11

.Net Winforms qrcode writer with .netusing barcode generation for .net winforms control to generate, create qr code iso/iec18004 image in .net winforms applications.

p2 1 p

VS .NET barcode implementation on .net c#generate, create barcode none in visual c# projects

n a xi i 1

DataMatrix generation for excel spreadsheetsuse office excel barcode data matrix drawer torender data matrix barcodes in office excel

x2 p

p2 1

pxn 11 p2

.NET Windows Forms Crystal pdf-417 2d barcode development with visual c#.netusing .net winforms crystal tocreate pdf 417 on asp.net web,windows application

p2 1

QR Code ISO/IEC18004 generation with excel spreadsheetsuse excel qr code iso/iec18004 integrated tobuild qr code iso/iec18004 for excel

n a xi i 1

Control pdf417 size for excel barcode pdf417 size in excel

We observe that if p maximizes L( p), p also maximizes ln L( p). Therefore,

ln L1 p2 Now

a a xi b ln p

a xi b ln 11

d ln L1 p2 dp

a xi

an 1

a xi b

Equating this to zero and solving for p yields p likelihood estimator of p is P

11 n2 g i

xi . Therefore, the maximum

1 n n ia Xi 1

CHAPTER 7 POINT ESTIMATION OF PARAMETERS

Suppose that this estimator was applied to the following situation: n items are selected at random from a production line, and each item is judged as either defective (in which case n we set xi 1) or nondefective (in which case we set xi 0). Then g i 1 xi is the number of defective units in the sample, and p is the sample proportion defective. The parameter p is the population proportion defective; and it seems intuitively quite reasonable to use p as an estimate of p. Although the interpretation of the likelihood function given above is con ned to the discrete random variable case, the method of maximum likelihood can easily be extended to a continuous distribution. We now give two examples of maximum likelihood estimation for continuous distributions.

EXAMPLE 7-7

Let X be normally distributed with unknown and known variance function of a random sample of size n, say X1, X2, p , Xn, is L1 2 Now ln L1 2 and d ln L1 2 d 1

1 12

. The likelihood

22 12 22

2 n2

11 2 22

1xi 1

1n 22 ln12

a 1xi

a 1xi

Equating this last result to zero and solving for

yields

a Xi

Thus the sample mean is the maximum likelihood estimator of . Notice that this is identical to the moment estimator.

EXAMPLE 7-8

Let X be exponentially distributed with parameter . The likelihood function of a random sample of size n, say X1, X2, p , Xn, is L1 2 The log likelihood is ln L1 2

n n n xi n

xi 1

n ln

a xi

7-3 METHODS OF POINT ESTIMATION

Now d ln L1 2 d n

a xi

and upon equating this last result to zero we obtain

n a Xi

Thus the maximum likelihood estimator of this is the same as the moment estimator.

is the reciprocal of the sample mean. Notice that

It is easy to illustrate graphically just how the method of maximum likelihood works. Figure 7-3(a) plots the log of the likelihood function for the exponential parameter from Example 7-8, using the n 8 observations on failure time given following Example 7-3. We 0.0462. From Example 7-8, we know that this is a found that the estimate of was maximum likelihood estimate. Figure 7-3(a) shows clearly that the log likelihood function is maximized at a value of that is approximately equal to 0.0462. Notice that the log likelihood function is relatively at in the region of the maximum. This implies that the parameter is not estimated very precisely. If the parameter were estimated precisely, the log likelihood function would be very peaked at the maximum value. The sample size here is relatively small, and this has led to the imprecision in estimation. This is illustrated in Fig. 7-3(b) where we have plotted the difference in log likelihoods for the maximum value, assuming that the sample sizes were n 8, 20, and 40 but that the sample average time to failure remained constant at x 21.65. Notice how much steeper the log likelihood is for n 20 in comparsion to n 8, and for n 40 in comparison to both smaller sample sizes. The method of maximum likelihood can be used in situations where there are several unknown parameters, say, 1, 2, p , k to estimate. In such cases, the likelihood function is a function of the k unknown parameters 1, 2, p , k, and the maximum likelihood estimators 5 i 6 would be found by equating the k partial derivatives L1 1, 2, p , k 2 i, i 1, 2, p , k to zero and solving the resulting system of equations.