Positive correlation, negative coefficient
The variables , and are standardised (mean 0, variance 1), with
Regress on and together (population least squares). Find the coefficient on .
Fractions, decimals with at least 3 significant digits, and expressions such as sqrt(2)/2, e^-1 or pi/4.
The normal equations for standardised variables are , where is the correlation matrix of and holds their correlations with .
Solve the 2 by 2 system directly.
h
s Shown after a correct answer. Revealing it first marks the problem as seen, not solved.
Idea. In the joint regression, the coefficient on measures what says about once is known. Here is mostly a noisy copy of , and it correlates with less than that copy would, so its coefficient turns negative.
Step 1: the normal equations.
Step 2: solve. and .
Why the sign flips. Through alone, would be expected to correlate with at . Its actual correlation, , is lower, so the part of not explained by is negatively related to . In general the sign of is the sign of .
The lesson. A coefficient in a multiple regression is not a correlation. A signal with a positive correlation to the target can get a negative weight when combined with a stronger, correlated signal.