Multinomial Models for Nominal Responses

The outcome of a response variable might be one of a restricted set of possible values. If there are only two possible outcomes, such as a yes or no answer to a question, these responses are called binary responses. If there are multiple outcomes, then they are called polytomous responses. Some examples include the degree of a disease (mild, medium, severe), preferred districts to live in a city, and so on. When the response variable is nominal, there is no natural order among the response variable categories. Nominal response models explain and predict the probability that an observation is in each category of a categorical response variable.

A nominal response model is one of several natural extensions of the binary logit model and is also called a multinomial logit model. The multinomial logit model explains the relative risk of being in one category versus being in the reference category, k, using a linear combination of predictor variables. Consequently, the probability of each outcome is expressed as a nonlinear function of p predictor variables. The 'interactions','on' name-value pair argument in mnrfit corresponds to this multinomial model with separate intercept and slopes among categories. mnrfit uses the default logit link function for multinomial models. You cannot specify a different link function for multinomial responses.

The multinomial logit model is

$\begin{array}{l}\mathrm{ln}\left(\frac{{\pi }_{1}}{{\pi }_{k}}\right)={\alpha }_{1}+{\beta }_{11}{X}_{1}+{\beta }_{12}{X}_{2}+\cdots +{\beta }_{1p}{X}_{p},\\ \mathrm{ln}\left(\frac{{\pi }_{2}}{{\pi }_{k}}\right)={\alpha }_{2}+{\beta }_{21}{X}_{1}+{\beta }_{22}{X}_{2}+\cdots +{\beta }_{2p}{X}_{p},\\ \text{ }\text{\hspace{0.17em}}⋮\\ \mathrm{ln}\left(\frac{{\pi }_{k-1}}{{\pi }_{k}}\right)={\alpha }_{\left(k-1\right)}+{\beta }_{\left(k-1\right)1}{X}_{1}+{\beta }_{\left(k-1\right)2}{X}_{2}+\cdots +{\beta }_{\left(k-1\right)p}{X}_{p},\end{array}$

where πj = P(y = j) is the probability of an outcome being in category j, k is the number of response categories, and p is the number of predictor variables. Theoretically, any category can be the reference category, but mnrfit chooses the last one, k, as the reference category. Thus, mnrfit assumes the coefficients of the kth category are zero. The total of j – 1 equations are solved simultaneously to estimate the coefficients. mnrfit uses the iteratively weighted least squares algorithm to find the maximum likelihood estimates.

The coefficients in the model express the effects of the predictor variables on the relative risk or the log odds of being in category j versus the reference category, here k. For example, the coefficient β23 indicates that the probability of the response variable being in category 2 compared to the probability of being in category k increases exp(β23) times for each unit increase in X3, given all else is held constant. Or it indicates that the relative log odds of the response variable being category 2 versus in category k increases β23 times with a one-unit increase in X3, given all else equal.

Based on the nominal response model, and the assumption that the coefficients for the last category are zero, the probability of being in each category is

${\pi }_{j}=P\left(y=j\right)=\frac{{e}^{{\alpha }_{j}+\sum _{l=1}^{p}{\beta }_{jl}{x}_{l}}}{1+\sum _{j=1}^{k-1}{e}^{{\alpha }_{j}+\sum _{l=1}^{p}{\beta }_{jl}{x}_{l}}},\text{ }j=1,\cdots ,k-1.$

The probability of the kth category becomes

${\pi }_{k}=P\left(y=k\right)=\frac{1}{1+\sum _{j=1}^{k-1}{e}^{{\alpha }_{j}+\sum _{l=1}^{p}{\beta }_{jl}{x}_{l}}},$

which is simply equal to 1 – π1π2 – ... – πk–1.

After estimating the model coefficients using mnrfit, you can estimate the category probabilities or the number in each category using mnrval (the default name-value pair is 'type','category'). This function accepts the coefficient estimates and the model statistics mnrfit returns and estimates the categorical probabilities or the number in each category and their confidence bounds. You can also specify the cumulative or conditional probabilities or numbers to estimate using the 'type' name-value pair argument in mnrval.

 McCullagh, P., and J. A. Nelder. Generalized Linear Models. New York: Chapman & Hall, 1990.

 Long, J. S. Regression Models for Categorical and Limited Dependent Variables. Sage Publications, 1997.

 Dobson, A. J., and A. G. Barnett. An Introduction to Generalized Linear Models. Chapman and Hall/CRC. Taylor & Francis Group, 2008.