A Discussion on Non-Hierarchical Regression Models

There is a certain point in many trainings where we talk about the advice to respect "model hierarchy" in the context of multiple linear regression models. Model hierarchy means: If there is a term of higher order included in the model, all corresponding terms of lower order should be in the model, too.

If you estimated a model $Y  = b_0 + b_1 X_1 + b_2 X_2 + b_{12} X_1 \cdot X_2 + \epsilon$ and see that $b_{12}$ is significant different from 0 the main effects $X_1$ and $X_2$ should stay in the model in any case. No matter if they are significant themselves or not.

In the following series of articles I want to give an explanation why we give this recommendation.

In the first article I will present the problems that occur if the intercept term drops out of the model. Of course this is the most simple kind of violating the model hierarchy. I give a motivation why we should prefer:

$$ y = b + b_1 x_1 + \epsilon $$

in nearly all cases over:

$$ y = b_1 x_1 + \epsilon $$

The second article will explain the difficulties that occur whenever you estimate a non-hierarchical model by removing terms of lower order, while keeping the corresponding terms of higher order. This includes not only 2 (and more)-factor-interactions but polynomial terms as well. So our general advice applies to models like

$$ y = b + b_1 x_1 + b_{11} x_1^2 + \epsilon$$

even though.

Of course there are exceptions from these general rules. The last article will give some examples, where one might use non-hierarchical models.

Keine Kommentare:

Kommentar veröffentlichen