Beacon Hill Institute Fails to Show that the Index Works

Beacon Hill Institute Fails to Show that the Index Works

To its credit, Beacon Hill Institute (BHI) tests its Competitiveness Index against real world results. However, because of the problematic selection of variables described above, their test stacks the deck in favor of BHI’s index and proves nothing more than the obvious fact that higher income predicts higher income.

BHI attempts to measure the accuracy of the index through a statistical analysis to see if differences in per capita income across states are related to their BHI competitiveness score. Based on this, BHI reports that a one point increase for a state on their index score (which would be a quite sizeable increase since the scores range from 3.17 to 8.10) is associated with higher per capita income of $2,429.

But it should not be surprising that the BHI index correlates with higher per capita income; after all the variables in the index include the percent of households with cell phones, bank deposits per capita, percent of population without health insurance, the infant mortality rate, the prevalence of high-paid occupations, the prevalence of scientists and engineers, the unemployment rate. Again, this amounts to stacking the deck in favor of the index.

A better test of the usefulness of the Competitiveness Index is to see if it actually predicts which states experience growth in incomes. Beacon Hill, after all, claims that the index measures the extent to which states have the “policies and conditions that ensure and sustain a high level of per capita income and its continued growth.”

We did just this and found no relationship between a better BHI score and income growth. To test the claim, we constructed statistical analyses to determine whether a higher score on the 2006 Competitiveness Index was associated with higher growth in per capita income or median family income from 2007 through 2014. We first tested the index alone, and then tested the index while controlling for the state’s economic structure in 2006. In no case was there evidence that states scoring higher in the 2006 index experienced higher income growth over the next eight years.1 This is illustrated in the chart below, where states are arrayed along the horizontal axis from the lowest scoring to the highest scoring state. (The outlier is North Dakota, with a high growth rate driven entirely by the oil and gas industry.) Growth in income between 2007 and 2014 clearly shows no trend upward or downward as a state’s index improves.

image023

The results are the same if we use only the Government and Fiscal Policy sub-index in the statistical analysis, that being the area where public policy has most effect. Once again, in no case does this sub-index help explain why some states experienced greater growth in per capita income or median family income from 2007 to 2014.

1. We used regression analysis. The coefficient on the BHI index was not statistically significant at the 10 percent level in any of the four regressions. The economic structure variable measures the growth that would be predicted from 2007 to 2014 on the basis of the sectoral composition of the state economy in 2006 if each sector grew at the national rate for that sector from 2007 to 2014. Controlling for structure thus tests whether the BHI index is measuring something that causes states to grow faster than predicted, increasing their share of the national economy.