Comments I wrote October 17, 2004 about the NASA public release. It was written for a small group of colleagues, which explains its "free" style. I did not have time to carefully read through all relevant publications and check all the claims of the announcement, but I hope my remarks would be useful. -----------------------YYK: Begin of included message------------------------- RE: Rundle-Tiampo prediction http://www.nasa.gov/vision/earth/environment/0930_earthquake.html http://quakesim.jpl.nasa.gov/scorecard.html http://quakesim.jpl.nasa.gov/ScoreCard_New_Composite_Oct_6_2004.pdf I got a request from a Japanese reporter to discuss the recent NASA claim of successful earthquake prediction in California. Generally, I do not interact with reporters, but I decided to briefly look at the NASA materials and report to you my impressions. Year 2004 seems to be the year of earthquake prediction: Keilis-Borok, Parkfield, NASA (Rundle-Tiampo), and more to come? 1. First, I would like to say that to be taken seriously an earthquake forecast should be quantitatively probabilistic. In 1654 Blaise Pascal and Pierre de Fermat exchanged letters in which they founded the quantitative probability theory (see, for example, Franklin, 2001, pp. 306ff). Now 350 years later, any earthquake forecast without direct use of probability has a medieval flavor. This is perhaps the reason the general public and media reporters are so attracted to yes/no forecasts. Although Rundle-Tiampo forecast calculates probabilities of earthquake occurrence, in their announcements they use the antiquated binary language. 2. Rundle-Tiampo forecast applies methods developed in statistical physics, meteorology and other disciplines, to calculate earthquake probability. The problem with these techniques is that they are created for quasi-Gaussian continuous processes. Earthquake process is not continuous, and much better approximation for it is a multidimensional stochastic point process. If one tries to apply continuous stochastic processes to analyze earthquake occurrence, empty intervals would always present a problem. Moreover, even in non-empty space-time intervals, the distribution of earthquake numbers is discrete for small numbers and highly non-Gaussian, heavy-tailed for large earthquake numbers. The negative-binomial distribution is a better approximation (Kagan and Jackson, 2000) than the Poisson or Gaussian distribution. It is quite possible that the Rundle-Tiampo method has some predictive skills, perhaps because it somehow uses long-term earthquake clustering to evaluate future seismic hazard. But I doubt that by using an inappropriate model of earthquake occurrence, one can achieve a high prediction performance. 3. Any forecast should be accompanied by the calculation of a probability of success if one uses a null hypothesis. This may present a problem since, as we have seen recently the formulation of the null hypothesis is not an easy task (see Kagan and Jackson, 2000 and http://moho.ess.ucla.edu/~kagan/sjg.pdf). Rundle et al. (2002, p. 2519) perform two forecast tests. The first one with epicenters uniformly random scattered over S. California is not a good one, since any forecast which predicts earthquakes in places where they occurred previously would strongly outperform such a model. The second test is better: the 1992-2001 forecast is compared to spatial distribution of seismicity in 1932-1991. However, the actual forecast is produced on the basis of seismic history in 1978-1991. Therefore, if long-term earthquake clustering exists in California (and elsewhere) this forecast would be successful simply because 1978-1991 earthquake distribution has a better predictive power for 1992-2001 seismicity than that of 1932-1991. 4. There is a large difference in the outcome whether a forecast is retrospective or prospective. In retrospective prediction various biases either intentional or unpremeditated are unavoidable. It is next to impossible to correct for these systematic effects. The forecast presented in Rundle et al. (2002) is retrospective. Although the authors claim to have no adjustable parameters, in reality various magnitude thresholds, time and space limits, selection of catalogs are degrees of freedom. To make it at least quasi-prospective, one needs to apply the technique to the new data without any substantial modification. 5. In the 2004/10/1 NASA public release, Rundle et al. claim to have predicted 14/15 out of 15/16 earthquakes since 2001 (the time of their work -- Rundle et al., 2002 -- was completed), or 11 since the paper publication. I see several problems with this announcement: (a) I could not find in the relevant publications that the predicted events should occur not only in "seismic hotspots" but could be up 11 km away (see http://quakesim.jpl.nasa.gov/scorecard.html -- "within margin of error +/- 11 km"). If this extended limit has been applied after the publication, it makes the forecast retroactive. Moreover, in its testing, the null hypothesis results should be calculated with the same "margin of error". (b) As far as I can judge, the prediction map in the announcement is significantly different from that published by Rundle et al. (2002, Fig. 4), and the authors in effect say this. For example, in the original map, I do not see any "hotspot" near the future 2003 San Simeon earthquake. In general, such maps should be used only as an illustration tool, the forecast itself must be documented as a quantitative file (table). The same seems to be true for the 2004 San Clemente Island earthquake. The map is not an appropriate tool to check whether other earthquakes, claimed to be successfully predicted, are actually within hazardous areas. (c) In summary, I doubt that the proposed earthquake forecast method has a significant predictive skill, especially the "amazing success" the NASA announcement claims. REFERENCES: Franklin, J., 2001. The Science of Conjecture: Evidence and Probability before Pascal, J. Hopkins Univ. Press, Baltimore, pp. 497. Kagan, Y. Y., and D. D. Jackson, 2000. Probabilistic forecasting of earthquakes, (Leon Knopoff's Festschrift), Geophys. J. Int., 143, 438-453. Rundle, JB, Tiampo KF, Klein W, Martins JSS, 2002. Self-organization in leaky threshold systems: The influence of near-mean field dynamics and its implications for earthquakes, neurobiology, and forecasting, Proc. Nat. Acad. Sci. USA, 99, Suppl. 1, 2514-2521. ------------------------YYK: End of included message-------------------------- NASA.TXT;1 14/18 18-NOV-2004 18:41:19.89