Earthquake hazard maps are used worldwide to predict future damage and plan mitigation strategies. However, in recent years, highly destructive earthquakes (2008 Wenchuan, China, M7.9; 2010 Haiti, M 7.0; 2011 Tohoku, Japan, M9.1) have occurred in areas mapped as having lower hazard than nearby areas. The question is whether these reflect the limitations of the mapping methods or rare events that should not be used to judge the maps as unsuccessful. Because of the limited seismic record available and limited understanding of earthquake mechanics, these maps depend dramatically on difficult to assess parameters and hence on the mapmakers' preconceptions, resulting in large uncertainties. For example, the M9.1 March 2010 earthquake along the Tohoku coast of northeast Japan generated a much larger tsunami than considered in the hazard planning. This view arose because of the absence of such large earthquakes in the seismological record there, which was consistent with a model based on the convergence rate and age of the subducting lithosphere, which predicted at most a low M 8 earthquake. Although this model was invalidated by the 2004 Sumatra earthquake and tsunami, the revised ideas were too recent to be incorporated in hazard mitigation.

Such failures indicate the need to objectively test how well hazard maps work by comparing their predictions and those of null hypotheses based on random regional seismicity to earthquakes that actually occurred after they were published, and use test results to assess maps' uncertainties and improve them. Such testing is common and useful in other fields. Weather forecasts, which are conceptually similar to earthquake hazard mapping, are routinely evaluated to assess how well their predictions matched what actually occurred. Forecasts are also tested to see if they do better than using the average of that date in previous years, or by assuming that today's weather will be the same as yesterday's. Over the years, this process has produced measurable improvements in forecasting methods and results, and yielded much better assessment of uncertainties. Another analogy is the trend to evidence-based medicine, which assesses how well commonly used treatments work, often with surprising results.