Statistical agencies everywhere are reporting solutions to the challenges of compiling reliable and informative economic indicators during an unprecedented global pandemic. Conducting the surveys that economic data releases are made of in a world where businesses are shuttered by coronavirus containment policies is not easy. Sample sizes have been reduced, and site visits are not being conducted to find out why. If a national statistical office calls a company and no one answers the phone, the establishment is excluded from the data.
This is a problem that we think is leading to an underestimation of the actual decline in economic activity. If a business with 100 employees fails during a lockdown, no one answers the phone when the national statistical office calls to collect employment data. So the loss of 100 jobs is not recorded. Consider the May retail sales report for Australia: If ABS calls a retail outlet to ask about activity in May and the store is closed—if no one answers the phone—then the establishment is excluded from that month’s survey result. However, the right answer should be that this outlet is reporting zero sales. The survey result will overstate actual sales because the sample under-represents outlets that have failed.
Statisticians whose job it is to sample populations call this a sampling error—a change in inferences derived from a sample of a population that is due to a systematic change in the composition of the sample rather than a change in the sample itself. If failed companies are dropped from the sample of retail establishments surveyed, estimates of aggregate retail sales from that sample become biased toward sales at outlets strong enough to survive the pandemic. If a store is closed entirely, its entry in the survey should be a zero. Instead, it is excluded from both the numerator and the denominator of the calculation of the sales statistics.
So when you read the latest economic reports, remember that they likely tell a less woeful tale about economy than is accurate.